Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Apr 1.
Published in final edited form as: Contemp Clin Trials. 2020 Mar 5;91:105968. doi: 10.1016/j.cct.2020.105968

Development of a Core Outcome Set for Pediatric Critical Care Outcomes Research

Ericka L Fink 1, Jessica M Jarvis 2, Aline B Maddux 3, Neethi Pinto 4, Patrick Galyean 5, Lenora M Olson 6, Susan Zickmund 7, Melissa Ringwood 6, Samuel Sorenson 6, J Michael Dean 6, Joseph A Carcillo 1, Robert A Berg 8, Athena Zuppa 8, Murray M Pollack 9, Kathleen L Meert 10, Mark W Hall 11, Anil Sapru 12, Patrick S McQuillen 13, Peter M Mourani 3, R Scott Watson 14; the Pediatric Acute Lung Injury and Sepsis Investigators (PALISI) Long-term Outcomes Subgroup Investigators, and the Eunice Kennedy Shriver National Institute of Child Health and Human Development Collaborative Pediatric Critical Care Research Network (CPCCRN)
PMCID: PMC7313715  NIHMSID: NIHMS1585821  PMID: 32147572

Abstract

Background:

Pediatric Intensive Care Unit (PICU) teams provide care for critically ill children with diverse and often complex medical and surgical conditions. Researchers often lack guidance on an approach to select the best outcomes when evaluating this critically ill population. Studies would be enhanced by incorporating multi-stakeholder preferences to better evaluate clinical care. This manuscript outlines the methodology currently being used to develop a PICU Core Outcome Set (COS). This PICU COS utilizes mixed methods, an inclusive stakeholder approach, and a modified Delphi consensus process that will serve as a resource for PICU research programs.

Methods:

A Scoping Review of the PICU literature evaluating outcomes after pediatric critical illness, a qualitative study interviewing PICU survivors and their parents, and other relevant literature will serve to inform a modified, international Delphi consensus process. The Delphi process will derive a set of minimum domains for evaluation of outcomes of critically ill children and their families. Delphi respondents include researchers, multidisciplinary clinicians, families and former patients, research funding agencies, payors, and advocates. Consensus meetings will refine and finalize the domains of the COS, outline a battery instruments for use in future studies, and prepare for extensive dissemination for broad implementation.

Discussion:

The PICU COS will be a guideline resource for investigators to assure that outcomes most important to all stakeholders are considered in PICU clinical research in addition to those deemed most important to individual scientists.

Trial registration:

COMET database (http://www.comet-initiative.org/, Record ID 1131, 01/01/18).

Keywords: Pediatrics, Core Outcomes Set, Clinical Research, Critical Illness, Morbidity

Background

Approximately 480,000 children and young adults < 20 years old are admitted to pediatric intensive care units (PICUs) at a cost of $8 billion annually in the United States alone. Mortality has decreased to 2–4% in high-resource countries, however this increased survival is accompanied by an increased in those discharged with new morbidity (1–4). Survival from critical illness is frequently accompanied by new or worsened impairments in physical, psychological, cognitive, and/or social domains for both the child and their family that impact recovery and important life functions for all, characterized as “Post Intensive Care Syndrome - Pediatrics” (PICS-p) (5,6).

A Scoping Review of PICU literature found that evidence for innovative approaches towards improving outcomes is based on studies that typically focus on short term, physiological or mortality outcomes rather than patient-centered long term outcomes (7). Furthermore, there is great heterogeneity in outcome measures selected for study among any given condition. For example, a recent systematic review evaluating characteristics of PICS-p was unable to perform a quantitative analysis of the 19 studies identified due to heterogeneity in outcome instruments used (8). Similarly, a Scoping Review examining child functional outcomes and physical impairments in PICU survivors found that 11 different instruments were used among 25 separate studies (9). Multicenter, randomized controlled trials in pediatric critical illness are expensive and complex to manage. It is crucial to ensure that researchers are selecting appropriate, comparable instruments to evaluate long-term outcomes. A Core outcome Set can facilitate this objective (10).

A Core Outcome Set (COS) is defined as “a patient outcome, health-related condition, or aspects of health that relevant stakeholders agree are essential to assess in all clinical research studies evaluating outcomes” (11). The Core Outcomes Measures in Effectiveness Trials (COMET) guidelines recommend using multiple methods to ensure a COS is informed by all relevant stakeholders (12). Inclusion of various viewpoints in research has the potential to make study findings ultimately more “useful, reliable, and relevant to patients, healthcare professionals, and others making decisions regarding healthcare provision” (13,14). Unfortunately, scant evidence exists about which outcomes are most important to critically ill and injured children, their families and caregivers and research priorities are often determined solely by investigators.

Development of a COS to guide selection of outcome domains is one step towards assuring that the most highly valued domains are recognized. After defining core outcome domains, the next step is the identification of feasible and reliable measurement instruments (termed core outcome measurement sets) to evaluate the core outcome domains (15). Together, the core outcome domains and the core outcome measurement sets constitute the two components of the COS. A COS is meant to serve as a minimum set of outcome domains and instruments and does not exclude other domains or instruments that may be important for a particular study. Utilizing a COS minimizes bias and heterogeneity in choice and reporting of outcomes, which can improve the quality of systematic reviews, clinical trials and implementation of new innovations (16).

Core outcome sets have been developed for adult respiratory failure and adult and pediatric cardiac arrest, but there are no COS to guide research in the general PICU population post discharge (11,17,18). This article describes the methods of developing a COS for the PICU population post discharge. The objective of this project is to develop a multi-stakeholder-informed PICU COS. The intended scope of this COS is to recommend a minimum set of outcome domains and instruments that should be evaluated in PICU patients and potentially in their families/caregivers in all clinical research studies pertaining to outcomes after pediatric critical illness. This project is currently underway and the tense in this manuscript reflects whether that portion of the work has been done is or forthcoming.

Methods

Overview

This project utilizes a mixed-methods and consensus methodology as per the Core Outcomes Measures in Effectiveness Trials (COMET) guidelines (12) and the minimum quality standards recommended by the Core Outcome Set-STAndards for Development (COS-STAD) (Table 1) (14). Reporting of PICU COS results will follow COS-STAR (Core Outcome Set-STAndards for Reporting) guidelines (19). This project is registered on the COMET database (http://www.comet-initiative.org/studies/details/1131) and funded by the National Institute of Child Health and Development’s Collaborative Pediatric Critical Care Research Network (CPCCRN) and in partnership with the Pediatric Acute Lung Injury and Sepsis Investigators (PALISI) Outcomes Subgroup. The PICU COS Steering Committee includes this study’s investigators, researchers and clinicians with attention to diverse expertise and geographical representation, a family member, and a program manager from CPCCRN’s data coordinating center.

Table 1.

Stakeholder groups who will be invited to participate in the Delphi consensus process.

Stakeholder group Criteria
RESEARCH
 Clinical researchers Lead researchers in pediatric critical care
 Research personnel Research study coordinators and other personnel
 Funding agency Program officer focused on pediatric critical care research programs
CLINICAL
 Pediatric critical care physicians Clinicians caring for children with critical illness or injury
 Outcomes specialists Includes neuropsychologists and other specialists with expertise in outcomes-specific diagnoses and recovery
 Rehabilitation physician Specialists in pediatric rehabilitation medicine
 PICU nurses, advanced practice providers Nurses who care for children with critical illness or injury
 Complex care pediatrician Pediatricians caring for medically complex children with frequent need for critical care
 Allied health clinicians Clinical and research leaders in physical, occupational, and speech/language
 Supportive/palliative care providers Physicians, nurses, and other providers focused on care
of children with life threatening and/or chronic conditions
FAMILY/ADVOCACY
 Pediatric critical care advocates Non-profit agency focused on promotion of pediatric critical care
 Family members/adult survivors Parent/guardian of a child who survived a critical illness; adults who as a child survived critical illness
 Payors Insurance program administrators for pediatric critical care patients
Open in a new tab

Scope of the PICU-COS

Specific components of the PICU COS development includes the following: 1) Scoping Review of outcome domains and instruments previously reported in PICU-related studies to evaluate post-discharge outcomes of PICU survivors or families of PICU patients (completed),2) Qualitative study of PICU survivors (14–18 years) beliefs on important outcomes (completed);3) Modified, international Delphi consensus process with multi-stakeholder respondents and consensus to finalize the PICU COS and recommended instruments; and 5) Dissemination, implementation, and assessment of uptake (Figure 1)(15,20–22).

Figure 1.

Figure 1.

Open in a new tab

PICU-Core Outcomes Set study design.

PICU, pediatric intensive care unit; COS, core outcomes set

1Merritt et al. PCCM 2018, Pasek et al. Crit Care Nurse 2019, Turnbull et al. Crit Care Med 2017, Gershon et al. Journal of Applied Measurement 2010.

The PICU COS serves a heterogeneous population. Children admitted to PICUs have a wide range of ages, conditions, severities of illness, comorbidities, living situations, geographical locations, quality of healthcare resources and access to care. It is unlikely that a single instrument will effectively assess each of the COS outcome domains. Thus, this COS focuses on the evidence to support recommendations for outcome domains. Specific outcome measures or instruments and timing of measurements will be subsequently recommended by the Steering Committee using a consensus process once the COS domains are finalized.

Scoping Review of domains and instruments used in PICU outcomes research

A Scoping Review broadly examines the body of literature about a topic without evaluation of methodology, calculation of risk of bias, or determination of effectiveness (23). We completed a Scoping Review of outcome domains and instruments previously studied in survivors and families of children who experienced critical illness to inform the domains and instruments that will be included in the Delphi consensus process. Our methods reflected the methodology used in a prior Scoping Review to delineate domains and instruments employed to study outcomes in survivors of critically-ill adults with respiratory failure (24).

Search criteria

We included peer-reviewed studies evaluating pediatric critical care survivors or families and caregivers of critically ill children published in English between 1990 and 2017. We used search strategies including a combination of keywords and controlled vocabulary for the concepts of “intensive care” and “critical care/illness” combined with “outcome assessment,” “health status,” “functional status,” “quality of life,” “anxiety,” “depression,” “mental health,” and “follow-up.” Additionally, each domain had its own set of search terms developed. The search included PubMed, EMBASE, PsycINFO, Cumulative Index of Nursing and Allied Health Literature, and the Cochrane Controlled Trials Registry. We excluded articles in which there were no post-discharge outcomes assessed, only survival was assessed, the study evaluated only the psychometric properties of an instrument, the article evaluated the outcome of a technical procedure/condition without report of relationship to ICU care, only one subject was included, or the majority of the study population was older than 18 years old, preterm infants, neonates, or had not been definitively admitted to an ICU.

Article review methods

To determine if an article met the inclusion and exclusion criteria, article abstracts were uploaded into Covidence (Melbourne, Australia), a web-based software platform, and reviewed by the PALISI Outcomes Subgroup. The PALISI Outcomes Subgroup is comprised of PICU researchers with specific expertise or interest in the study of outcomes after pediatric critical illness. Two members of the PALISI Outcomes Subgroup independently reviewed each abstract to identify those that should be excluded. A third reviewer resolved discrepant votes. Next, for articles that did not obviously meet exclusion criteria at the abstract review stage, the full manuscripts were uploaded into Covidence. Two members of the PALISI Outcomes Subgroup independently reviewed each manuscript to assess for inclusion. Discrepancies were reviewed by the Steering Committee or a third reviewer from the PALISI Outcomes Subgroup. The included articles comprised the Scoping Review article set.

Data collection

Each article of the Scoping Review article set was independently reviewed by two members of the PALISI Outcomes Subgroup to extract the post-discharge outcome instruments used. We also collected additional data related to the study design, age and disease characteristics of the study population, enrollment, mode and timing of outcome assessment, and follow-up rates. The dual extractions were compared and discrepancies resolved between the two reviewers. A Subgroup leader or Steering Committee member assisted with discrepancy resolution when a consensus was not achieved between the two reviewers. We a priori identified the following outcome domains as those that were commonly reported in pediatric critical care outcomes literature: overall health, physical, cognitive, emotional, social, health-related quality of life and family. Due to the significant overlap of social health with the other domains, the instruments measuring social health were distributed to alternative domains most relevant to their properties. We divided the most frequently used outcome instruments amongst these domains to facilitate data collection. If an instrument had multiple versions, we collapsed the instruments into an umbrella version (e.g. editions of the Vineland Adaptive Behavior Scale). Additionally, we collected all other domains and instruments measured including those not represented by the 7 categories above to ensure complete capture of the domains and instruments measured. A complete set of domains and instruments derived from the Scoping Review article set was determined based on review of the extracted data.

Qualitative interviews with caregivers and teenagers who experienced critical illness

We conducted semi-structured interviews to explore the caregiver and critically ill or injured child’s perceptions of health issues or outcomes necessary to include in studies that evaluate child and family outcomes after they are discharged from the PICU.

Study population

We included parents or legal guardians (any age) whose child (aged 2 weeks to 18 years) survived an unplanned PICU admission and the child was living at the time of the interview and children (age 14–18 years) who survived an unplanned PICU admission. These participants had consented previously to a CPCCRN study that allows for re-contact and were discharged from a PICU stay of greater than 48-hours at a CPCCRN center 6 to 48 months prior. Only participants who were English speaking were included because the qualitative researchers were unable to support non-English speaking families. Purposeful sampling was used to maximize diversity in recruitment of either dyads (a primary caregiver and one age-eligible child) or a primary caregiver only from each of the seven CPCCRN sites. We recruited caregivers and children with diverse characteristics to allow for a broad exploration of outcomes important to families. Participants were identified and contacted for possible enrollment by the CPCCRN research coordinator, informed consent and assent obtained, and the interview scheduled.

Data collection

Semi-structured telephone interviews were conducted by an experienced qualitative research team. We chose telephone interviews due to the wide geographical variation of the CPCCRN sites. The interview guide was developed and piloted with two potential participants to ensure that interviews would develop participant-centered data that followed the caregivers’ and child’s way of disclosing their experience. Interviews lasted approximately one hour and were audio recorded and transcribed verbatim.

Data analysis

Interview transcripts were coded and analyzed using the qualitative research software ATLAS.ti (Scientific Software, Berlin Germany) based on the qualitative coding philosophy developed by Crabtree and Miller (25). This coding and analysis process is consistent with constant comparison, which required sorting and comparing data to discern key themes. First, open codes were developed that represent key concepts that closely categorized participants’ language. Next, focused codes were identified that represent key concepts evident across transcripts and were examined to develop emerging themes that represented the most salient aspects of the health issues or outcomes for the participants collectively. Emerging themes and health issues or outcomes were reviewed and refined by the research team for inclusion in the Delphi consensus process.

Modified, international Delphi consensus process

Leaders of the Scoping Review and Qualitative Study will collate and draft a list of outcomes domains from several sources, including those described above - scoping review, qualitative study, and other relevant sources (e.g., preliminary data, other published COS focused on critical care) - for inclusion in the Delphi consensus process. Outcomes will be paired with lay definitions which will undergo review by the CPCCRN’s Family Network Collaborative to assure clarity. This collaborative includes families with children who had critical illness at participating network centers who were recruited to ensure family stakeholder involvement in network initiatives. An international Steering Committee is being formed and will convene on a webinar to conduct a consensus conference to finalize domains and associated lay definitions. The CPCCRN’s Data Coordinating Center will conduct testing of the Delphi software obtained from the COMET initiative prior to beginning the Delphi consensus process.

The Delphi consensus process allows for equal weighting of responses to assure all stakeholder viewpoints are valued and it will be conducted on a web-based platform to allow for anonymity (26). Two Delphi rounds are anticipated to reach consensus but more may be necessary.

Stakeholders

There is no minimum or optimal number of Delphi participants. Thus, we will approach recruitment to assure experienced, diverse stakeholder participation, resulting in over 300 participants. We will seek to recruit a balanced representation (no less than 15% representation in a group) from three stakeholder groups: research, clinical, and family/advocacy (Table 1). We will include English-speaking multinational stakeholders for all stakeholder groups. We will include an international panel of respondents to ensure the generalizability of our results and facilitate broad acceptance and implementation. The Steering Committee will execute the recruitment process and assure appropriate representation. This will include use of existing networks and contacts, authors of the scoping review manuscripts, and people recommended by local, national, or international associations. In the case of non-family stakeholders, we will seek to recruit participants with at least 3–5 years of professional work experience, fluent in English, and committed to completing the study. In addition, we will seek diversity in age, sex, geographical location, and race/ethnicity. Potential Delphi participants will be sent a short introductory email with specific information about the project and their responsibilities. They will be asked for their consent and desire to participate. Stakeholders will be asked to vote based on their own perspective, and for those who are representative of a stakeholder organization (hospital, funding agency, advocacy group), they will be asked to represent the perspective of their organization. We will request that the stakeholder declare/deny any real or perceived conflict of interest with the process and update any conflict of interest information at each Delphi round. We will record stakeholder demographic information and assign each consented participant a unique identifier to track responses. Members of the Steering Committee will also be invited to participate in the Delphi consensus process as they represent leaders in pediatric critical care outcomes and families of children with critical illness.

Modified Delphi Consensus Methods

Voting for Rounds 1 and 2 will occur using the web-based COMET Delphi Manager software. Panel members will be asked to vote utilizing the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Scale, consisting of a 9-point scale, divided into the following categories: “not important for inclusion” (consisting of scores 1–3), “important but not critical for inclusion” (scores 4–6), “critical for inclusion” (scores 7–9), plus a category for “unable to score” (score 10), if the panel member is unable to respond. Members will remain part of the panel throughout the entire Delphi process, even if they answer only a portion for any round or skip a round(s). Panel members will be asked to complete each round within an appropriate time frame. Each non-respondent will receive a personalized email, phone call, or text reminder with a weblink to DelphiManager at least once per week during the response period as per regulatory permissions. Panel member response rate will be calculated as the total number of respondents who completed each round as a percentage of those for whom an email invitation was sent.

Round 1

Outcome domains will be randomized into four different orders (i.e., Domain Order A, B, C, and D). Panel members in each stakeholder category will be randomly assigned to Domain Order A, B, C or D and provide a GRADE score (1–10) for each domain. In addition to the pre-specified domains, respondents have the option of proposing novel domains. Next, a one-time collection of demographic and contact information including: sex, age, country and state/province/region; years of formal education and clinical training (if any); previous involvement in research or clinical work with critically ill children and young adults (asked as 2 separate questions); area of expertise (physical, cognitive, mental health), if any, will be collected. For each member, we will manually code for their designated role in the Delphi (i.e., one of three stakeholder groups) based on the recruitment process.

Round 2

The Steering Committee will review results from Round 1 and consider whether domains that neared but did not achieve consensus should be included in Round 2. Text-based suggestions for core outcome domains in Round 1 will be reviewed by the Steering Committee to ensure that they represent a new contribution and, if so, they will be provided as a new domain for inclusion in Round 2. Based on the COMET guidelines, the a priori criteria for an outcome domain from Round 1 to be included in Round 2 requires it achieve >70% of responses rating ≥7 (indicating “critical” for inclusion) AND <15% of response rating <3 (indicating “not important” for inclusion). Prior to Round 2, members will be provided aggregate responses from the first round (including new domains) for all stakeholders and by group, along with their own response from Round 1, and then the respondents will complete Round 2 of the Delphi. For a Core Outcome domain to be considered in the final panel, it will need to achieve >70% of responses rating ≥7 (indicating “critical” for inclusion) AND <15% of response rating <3 (indicating “not important” for inclusion). Additional rounds may be held if there is a lack of consensus.

Steering Committee consensus meetings

The final COS panel will be discussed and finalized by the Steering Committee via webinar. The committee will review results and consider whether outcomes that neared but did not achieve consensus should be included in the COS. The Steering Committee will also discuss and make recommendations for measurement instruments and approaches (e.g., qualitative and quantitative) and timing of measurements for each of the final COS domains.

Data analysis plan and reporting of consensus process and data.

We will report panel member characteristics. Specifically, the total number of stakeholders invited to participate in the panel, proportion from each stakeholder group, and participation in each round. Each domain voted upon will be analyzed based on the total number of respondents who answered the question (i.e., the denominator will include total number of panel members who scored each question 1–9). Panel members who partially completed responses to a round will only be included as part of the denominator for the domains and questions they answered. We will provide the comprehensive list of domains determined to be part of the COS (as well as all those that were considered). Domains will be accompanied by measures of central tendency and distribution of score in addition to the number/percentage above the consensus threshold (as defined above).

Dissemination and implementation

The primary method of attaining widespread PICU COS usage is a thoughtful approach to Delphi stakeholder selection and broad collaboration. Inclusion of international representatives in all stakeholder groups will strongly contribute to this aim. We are seeking to recruit members of key stakeholder groups with a high likelihood of disseminating and implementing the COS.

Second, the PICU COS is registered on the COMET website, the registration number will be included in all PICU COS publications. Steering Committee members (and other stakeholders) will strategically plan presentations and manuscripts to assure a broad stakeholder audience. In addition, fact sheets and infographics will be approved for use by the CPCCRN Family Network Collaborative and potentially other patient advocate and interested groups. The PICU COS and associated publications will be linked for ease of download for use on COMET, and on websites including CPCCRN.org. We will also lead a social media campaign to disseminate the COS.

Discussion

The PICU COS will serve as an important resource for research programs that seek to improve outcomes for children with critical illness or injury. This mixed-methods and inclusive approach to stakeholder recruitment was based on international guidelines. A strong dissemination and implementation plan is vital to ensure broad use of the PICU COS. We aim to develop an efficient process to monitor future use of the PICU COS in research proposals, grants, and publications. Lastly, we encourage re-evaluation of the COS every 5–10 years to improve future research based on modifications to critical care delivery and available instruments to measure long-term outcomes.

Trial Status

Protocol version 1.02, 7/30/19. Work on the project aims began on 01/01/18 and is expected to be complete by 4/1/20. The data collection for the scoping review and qualitative study are complete. The Delphi consensus participants are currently being identified and recruited.

Acknowledgements

We acknowledge Professor Dale Needham, Dr. Paula Williamson, the members of the International Steering Committee members: Jhuma Sankar, MD (All India Institute of Medical Sciences), Hennie Knoester, MD, PhD (Centrum Universiteit van Amsterdam), Debbie Long, RN, PhD (Children’s Health Queensland Hospital), Ruth Grosskreuz, MD., CCRP (Children’s Hospital of Colorado), Werther Brunow de Carvalho, MD, PhD (University of São Paulo), Maria del Pilar Arias Lopez, MD (Hospital de Niños Dr. Ricardo Gutiérrez, Ciudad Autónoma de Buenos Aires, Argentina), Beth Slomine, PhD (Kennedy Krieger Institute), Jan Hau Lee, MBBS, MD (KK Women’s and Children’s Hospital), Karen Choong, MB, Bch, MSc (McMaster University, Canada), Martha A.Q. Curley, PhD, RN, FAAN (Perelman), Warwick Butt, MD, PhD (The Alfred Hospital, Melbourne), Andrew Argent, MD and Brenda Morrow, PhD, PT (University of Cape Town), Joseph Manning, PhD (University of Nottingham), Amy Houtrow, MD, MPH, PhD (University of Pittsburgh), Brenda Morrow, PhD, PT (University of Capetown), Demet Demirkol, MD (Ankara University), and the PALISI-Outcomes subgroup members: Laura Loftis and Brian Rissmiller, MD (Baylor College of Medicine), Julie Menzies, PhD, MSc, B(Nurs), RNC (Birmingham Children’s Hospital), Robert J. Graham, MD (Boston Children’s Hospital), Elizabeth Pace, MD, Steve L. Shein, MD, and Katherine N. Slain, DO (Case Western Reserve University), Shilpa Shah, DO (Children’s Hospital Los Angeles), Alicia M. Alcamo, MD, MPH, Vinay Nadkarni, MD, MS, and Alexis Topjian, MD, MSCE (Children’s Hospital of Philadelphia), Lindy Moake, APRN, MSN, PCCNP (Children’s Medical Center, Dallas, TX), Mekela Whyte-Nesfield, MD (Children’s National Hospital), Lauren Yagiela, MD, MS (Children’s Hospital of Michigan, Detroit, MI), Madiha Raees, MD (Children’s Hospital of Pittsburgh), AM Iqbal O’Meara, MD (Virginia Commonwealth University), Andy Geneslaw, MD (Columbia University Irving Medical Center), Marcy Singleton, ARNP (Dartmouth-Hitchcock Medical Center), J. Dean Jarvis, BSN, MBA, RN, CCRP, Mary McNally, BSRT, RRT, and Sholeen Nett, MD, PhD (Dartmouth-Hitchcock Medical Center, Children’s Hospital at Dartmouth-Hitchcock), Stefanie G. Ames, MD MS (David Geffen School of Medicine at University of California-Los Angeles), LeeAnn M. Christie, MSN, RN (Dell Children’s Medical Center), Peter M. Luckett, MD (Univers ity of Texas Southwestern Medical Center), Sarah Murphy, MD and Jane E. Whitney, MD MSCE (Harvard Medical School), Mara Leimanis, PhD (Michigan State University), Samer Abu-Sultaneh, MD (Indiana University School of Medicine), Sapna R. Kudchadkar, MD, PhD (Johns Hopkins University School of Medicine), Simon Li, MD, MPH (New York Medical College), Karen Choong, MB, FRCP(C), MSc (McMaster University), Jennifer A Muszynski, MD, MPH (Nationwide Children’s Hospital), Catherine Madurski, MD (Nemours Children’s Health System), Meredith Bone, MD, MSCI, Sabrina Derrington, MD, MA, FAAP, Denise M. Goodman MD MS, FCCM, and Kelly Michelson, MD, MPH, FCCM, FAAP (Northwestern University Feinberg School of Medicine), Joseph C. Manning, PhD, MNursSci, RN ( University of Nottingham), Debbie Long RN PhD (Queensland Children’s Hospital), Reinis Balmaks, MD, PhD (Riga Stradins University), Sara K VandenBranden, MD (Rush University Medical Center), Maureen A. Madden MSN, RN, CPNP-AC, CCRN, FCCM (Rutgers Robert Wood Johnson Medical School), Katherine Biagas, MD, FCCM, FAAP (Renaissance School of Medicine), Manzilat Akande, MD, MPH (The Children’s Hospital of Oklahoma), Haifa Mtaweh, MD ( University of Toronto), Andrew Prout, MD, MPH (University at Buffalo), Madhura Hallman, MD, MPH (University of Alabama at Birmingham), Jay M. Hunter, DNP, APRN, CPNP-AC, CCRN, CPN (University of California San Francisco School of Nursing), Michele Loi, MD (University of Colorado School of Medicine), Danielle Van Damme, DNP, CPNP-AC (University of Louisville,), Kristen Smith, MD, MS (University of Michigan School of Medicine), Julia A. Heneghan, MD (University of Minnesota Masonic Children’s Hospital), Idris Evans, MD, MSc (University of Pittsburgh Medical Center), Anoopindar Bhalla, MD, MSCI (University of Southern California, Children’s Hospital Los Angeles), Kevin Hummel, MD and Ben White, MD, MA (University of Utah), Jonna Clark, MD, MA, Mike Cronin, MD, MPH, Leslie A. Dervan, MD, MS, Reid WD Farris, MD MS, Jane (Lin) Di Gennaro, MD, Elizabeth Y. Killien, MD, MPH, Katie R. Nielsen, MD MPH, Monique Radman, MD, MAS, Joan S Roberts, and Jerry Zimmerman, MD, PhD, FCCM (University of Washington School of Medicine), Alan G. Woodruff, MD (Wake Forest University School of Medicine, Brenner Children’s Hospital), Mary Hartman, MD and John C. Lin, MD (Washington University in St. Louis), and Chani Traube, MD (Weill Cornell Medical Center).

Funding. Funding for this project was provided by the Eunice Kennedy Shriver National Institute of Child Health and Human Development Collaborative Pediatric Critical Care Research Network. It was approved by the CPCCRN Steering Committee and funded by grant number U01-HD49934.

List of Abbreviations

COMET:

Core Outcome Measures in Effectiveness Trials

COS:

Core outcome set

COSMIN:

COnsensus-based Standards for the selection of health Measurement INstruments

COS-STAD:

Core Outcome Set – STAndards for Development

COS-STAR:

Core Outcome Set – STAndards for Reporting

GRADE:

Grading of Recommendations Assessment, Development, and Evaluation

PICU:

Pediatric intensive care unit

RCT:

Randomized controlled trial

Footnotes

Declarations

Ethics approval and consent to participate. This project was approved by the University of Utah Institutional Review Board. Qualitative interviews required telephone-sourced informed consent, and participants who completed an interview received an incentive payment of $30. Delphi participants will respond affirmatively via email to consent to participation.

Consent for publication. Not applicable

Availability of data and material. The dataset supporting the conclusions of this article will be made available in the Eunice Kennedy Shriver National Institute of Child Health and Human Development Data and Specimen Hub (DASH) repository, https://dash.nichd.nih.gov/.

Competing interests. The authors declare not competing interests.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • 1.Atkins DL, Everson-Stewart S, Sears GK, Daya M, Osmond MH, Warden CR, et al. Epidemiology and Outcomes From Out-of-Hospital Cardiac Arrest in Children: The Resuscitation Outcomes Consortium Epistry–Cardiac Arrest. Circulation. 2009;119(11):1484–91. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Girotra S, Spertus JA, Li Y, Berg RA, Nadkarni VM, Chan PS, et al. Survival Trends in Pediatric In-Hospital Cardiac Arrests: An Analysis From Get With The Guidelines– Resuscitation. Circ Cardiovasc Qual Outcomes. 2013;6(1):42–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Moreau JF, Fink EL, Hartman ME, Angus DC, Bell MJ, Linde-Zwirble WT, et al. Hospitalizations of Children With Neurologic Disorders in the United States: Pediatr Crit Care Med. 2013;14(8):801–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Typpo KV, Petersen NJ, Hallman DM, Markovitz BP, Mariscalco MM. Day 1 multiple organ dysfunction syndrome is associated with poor functional outcome and mortality in the pediatric intensive care unit: Pediatr Crit Care Med. 2009;10(5):562–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Watson RS. Life after Critical Illness in Children—Toward an Understanding of Pediatric Post-intensive Care Syndrome. J Pediatr. 2018;198:16–24. [DOI] [PubMed] [Google Scholar]
  • 6.Manning JC, Pinto NP, Rennick JE, Colville G, Curley MAQ. Conceptualizing Post Intensive Care Syndrome in Children—The PICS-p Framework. Pediatr Crit Care Med. 2018;19(4):298–300. [DOI] [PubMed] [Google Scholar]
  • 7.Duffett M, Choong K, Hartling L, Menon K, Thabane L, Cook DJ. Randomized controlled trials in pediatric critical care: a scoping review. Crit Care. 2013;17(5):R256. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Herrup EA, Wieczorek B, Kudchadkar SR. Characteristics of postintensive care syndrome in survivors of pediatric critical illness: A systematic review. World J Crit Care Med. 2017;6(2):124–124. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Ong C, Lee JH, Leow MKS, Puthucheary ZA. Functional outcomes and physical impairments in pediatric critical care survivors: A scoping review. Pediatr Crit Care Med. 2016;17(5):13. [DOI] [PubMed] [Google Scholar]
  • 10.Duffett M, Choong K, Foster J, Meade M, Menon K, Parker M, et al. High-Quality Randomized Controlled Trials in Pediatric Critical Care: A Survey of Barriers and Facilitators*. Pediatr Crit Care Med. 2017. May;18(5):405–13. [DOI] [PubMed] [Google Scholar]
  • 11.Turnbull AE, Sepulveda KA, Dinglas VD, Chessare CM, Bingham CO, Needham DM. Core Domains for Clinical Research in Acute Respiratory Failure Survivors: An International Modified Delphi Consensus Study. Crit Care Med. 2017;45(6):1001–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Williamson PR, Altman DG, Bagley H, Barnes KL, Blazeby JM, Brookes ST, et al. The COMET Handbook: version 1.0. Trials. 2017;18(S3):280. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Snyder C, Jensen R, Segal J, Wu A. Patient-reported Outcomes (PROs): Putting the patient perspective in patient-centered outcomes research. Med Care. 2013;51(Supplement 3): S73–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Kirkham JJ, Davis K, Altman DG, Blazeby JM, Clarke M, Tunis S, et al. Core Outcome Set-STAndards for Development: The COS-STAD recommendations. PLOS Med. 2017;14(11):e1002447. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Needham DM, Sepulveda KA, Dinglas VD, Chessare CM, Friedman LA, Bingham CO, et al. Core Outcome Measures for Clinical Research in Acute Respiratory Failure Survivors. An International Modified Delphi Consensus Study. Am J Respir Crit Care Med. 2017. November;196(9):1122–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Williamson PR, Altman DG, Blazeby JM, Clarke M, Devane D, Gargon E, et al. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012;13(1):132. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Haywood K, Whitehead L, Nadkarni VM, Achana F, Beesems S, Böttiger BW, et al. COSCA (Core Outcome Set for Cardiac Arrest) in Adults: An Advisory Statement From the International Liaison Committee on Resuscitation. Resuscitation. 2018. June;127:147–63. [DOI] [PubMed] [Google Scholar]
  • 18.Blackwood B, Ringrow S, Clarke M, Marshall JC, Connolly B, Rose L, et al. A Core Outcome Set for Critical Care Ventilation Trials: Crit Care Med. 2019. October;47(10):1324–31. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Kirkham J, Gorst S, Altman D, Blazeby J, Clarke M, Devane D, et al. COS-STAR: a reporting guideline for studies developing core outcome sets (protocol). Trials. 2015;16(1):373. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Pasek TA, Burns C, Treble-Barna A, Ortiz-Aguayo R, Kochanek PM, Houtrow AJ, et al. Important Outcomes for Parents of Critically Ill Children. Crit Care Nurse. 2019. June;39(3):74–9. [DOI] [PubMed] [Google Scholar]
  • 21.Merritt C, Menon K, Agus MSD, Choong K, McNally D, O’Hearn K, et al. Beyond survival: Pediatric critical care interventional trial outcomes measure preferences of families and healthcare professionals. Pediatr Crit Care Med. 2018;19(2):e105–11. [DOI] [PubMed] [Google Scholar]
  • 22.Gershon RC, Rothrock N, Hanrahan R, Bass M, Cella D. The Use of PROMIS and Assessment Center to Deliver Patient-Reported Outcome Measures in Clinical Research. 2013;15. [PMC free article] [PubMed] [Google Scholar]
  • 23.Colquhoun HL, Levac D, O’Brien KK, Straus S, Tricco AC, Perrier L, et al. Scoping reviews: time for clarity in definition, methods, and reporting. J Clin Epidemiol. 67(12):1291–4. [DOI] [PubMed] [Google Scholar]
  • 24.Turnbull AE, Rabiee A, Davis WE, Nasser MF, Venna VR, Lolitha R, et al. Outcome measurement in ICU survivorship research from 1970–2013: A scoping review of 425 publications. Crit Care Med. 2017;44(7):1267–77. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Miller W, Crabtree B. Doing qualitative research in primary care: Multiple strategies In: Crabtree B, editor. Primary care research: A multi typology and qualitative road map. Newbury Park, CA: SAGE publications; 1992. p. 3–28. [Google Scholar]
  • 26.Diamond IR, Grant RC, Feldman BM, Pencharz PB, Ling SC, Moore AM, et al. Defining consensus: A systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol. 2014. April;67(4):401–9. [DOI] [PubMed] [Google Scholar]

RESOURCES