Core outcome set in paediatric sepsis in low- and middle-income countries: a study protocol

Abstract

Introduction

Sepsis is the leading cause of death in children worldwide and has recently been declared a major global health issue. New interventions and a concerted effort to enhance our understanding of sepsis are required to address the huge burden of disease, especially in low- and middle-income countries (LMIC) where it is highest. An opportunity therefore exists to ensure that ongoing research in this area is relevant to all stakeholders and is of consistently high quality. One method to address these issues is through the development of a core outcome set (COS).

Methods and analysis

This study protocol outlines the phases in the development of a core outcome set for paediatric sepsis in LMIC. The first step involves performing a systematic review of all outcomes reported in the research of paediatric sepsis in low middle-income countries. A three-stage international Delphi process will then invite a broad range of participants to score each generated outcome for inclusion into the COS. This will include an initial two-step online survey and finally, a face-to-face consensus meeting where each outcome will be reviewed, voted on and ratified for inclusion into the COS.

Ethics and dissemination

No core outcome sets exist for clinical trials in paediatric sepsis. This COS will serve to not only highlight the heavy burden of paediatric sepsis in this setting and aid collaboration and participation between all stakeholders, but to promote ongoing essential high quality and relevant research into the topic. A COS in paediatric sepsis in LMIC will advocate for a common language and facilitate interpretation of findings from a variety of settings. A waiver for ethics approval has been granted by University of British Columbia Children’s and Women’s Research Ethics Board.

Strengths and limitations of this study.

• With the continued high burden of disease in low- and middle-income countries, a core outcome set (COS) for use in paediatric sepsis research in a resource-limited setting is required to focus efforts and improve research consistency, relevance and quality in this area.

• The protocol uses a well-established process involving a thorough systematic review and an all-encompassing three-step Delphi survey involving clinicians, researchers, nurses, patients and parents/guardians.

• The feasibility of an e-survey is limited in regions with poor internet access, so the use of a local facilitator will ensure patients and the parent/guardian have equal input into the COS development.

Introduction

Sepsis is the leading cause of death in children, with the highest-burden in those countries least resourced to address it. Now recognised as a major global health issue, in 2017 the WHO adopted a resolution with the aim of reducing the human and economic burden of sepsis.¹ Severe paediatric sepsis has a worldwide prevalence of 8%² and a mortality of 9% to 20%.³ Severe sepsis and septic shock has a mortality of 20% to 40% within developing countries.^{4 5} It is thought to be the final common pathway to multiorgan failure and death from a number of infectious diseases,⁶ including the top four causes of childhood mortality as reported by the WHO.

With the high prevalence of disease, new interventions and treatments are essential, as is a concerted effort to better understand the genesis of sepsis. Analysis and comparison of trials involving paediatric sepsis in low- and middle-income countries (LMIC) are problematic however due to the significant heterogeneity and inconsistency in reporting outcomes.^{7 8} A systematic review into primary outcome measures in paediatric septic shock trials demonstrated that the use of mortality alone had significant limitations and suggested more long-term outcomes should be explored.⁴ A standardised way to evaluate trial outcomes is therefore imperative.

One method that is being increasingly used to address these issues is a minimum core outcome set (COS).⁷ These are defined as a minimum set of outcomes that should be consistently measured and reported in clinical trials for a specific clinical area.⁷ All stakeholders have a role in developing the COS. The use of COS not only aims to improve consistency across trials allowing accurate comparison, including potential meta-analysis, but to limit selective reporting^{9 10} and improve the quality and relevance of clinical research.¹¹

A number of sepsis COS have been developed (http://www.comet-initiative.org/studies/details/1317?result=true), including one ongoing in Paediatric Critical Care Medicine research (http://www.comet-initiative.org/studies/details/1131?result=true). No core outcome set currently exists for paediatric sepsis however, and very few are region specific. Limited funding and resources, a high sepsis burden of varying infectious aetiologies and a large proportion of children with significant comorbidities determine the differences in outcomes seen in LMIC. These factors are in contrast to high-income countries (HIC) and hence necessitate a region-specific COS to better understand these disparities and craft context-specific solutions. As an example total annual expenditure on health in many parts of sub-Saharan Africa is under US$25 per capita and often less than 3% of the GDP (gross domestic product)¹² and the number of intensive care unit beds as a percentage of hospital beds is approximately 1.5% in LMIC compared with 2.5% to 9% in HIC.¹³ Recognising the huge discrepancy in resources between regions is essential as is the potential financial impact. In 2018, 11.7% of the world’s population spent over 10% of their annual income on healthcare, leaving an estimated 100 million people impoverished by out-of-pocket spending.¹⁴ A region-specific COS will ensure relevance and meaning to patients and clinicians alike. By advocating a common language between stakeholders, interpretation of findings from a variety of diverse settings will become simpler and attention can focus on research into the essential outcomes. Potential barriers to development and dissemination of a COS in this setting may be hindered by a lack of access to research capabilities and limited implementation feasibility.

Aims and objectives

This study aims to develop a COS for use in clinical trials involving paediatric sepsis in LMIC. The specific study objectives are to identify outcomes previously reported in trials of paediatric sepsis in LMIC and to prioritise these outcomes using clinicians, researchers, patients and parents.

The protocol outlines our methods used to establish a COS and aims to raise awareness of paediatric sepsis in LMIC and aid collaboration and participation between all stakeholders. The protocol has been developed using the Core Outcome Set- STAndards for Development (COS-STAD) guidance for developing core outcome sets (15, online supplementary file 1) and the Core Outcome Set - STAndardised Protocol Items (COS-STAP) guidance for a protocol for a core outcome set development study (16, online supplementary file 2). A stepwise approach to COS development will be undertaken as suggested by The Core Outcome Measures in Effectiveness Trials (COMET) initiative and the Outcome Measures in Rheumatology initiative.¹⁷

Methods and analysis

Our COS development plan has been registered with the COMET initiative (http://www.comet-initiative.org/studies/details/1400?result=true). As per published recommendations¹⁸ and previous core outcome sets,^{11 19–21} a multistep process will be used to develop an international consensus for paediatric sepsis in low- and middle-countries COS (figure 1):

Figure 1.

Timeline of core outcome set development. LMIC, low- and middle-income countries.

1. Systematic review of outcomes currently reported in the research of paediatric sepsis in low- and middle-income countries.

2. Three round Delphi process:

   1. Two round e-Delphi survey to prioritise outcomes.

   2. Consensus meeting of global experts, clinicians, parent/guardians, patients and nurses and e-survey participants to ratify COS.

When complete, the Core Outcome Set Standards for Reporting (COS-STAR)²² will be applied.

Scope of core outcome set

This COS will be purely focussed on clinical outcomes in paediatric sepsis clinical trials in LMIC, and not involve those outcomes that are specific to premature infants or certain conditions, such as meningococcal septicaemia. The population will include those from newborn up to under 18 years of age.

Identification of existing knowledge

One prior systematic review in 2017 evaluated all paediatric randomised controlled trials of patients with septic shock. This however included high-income countries and was intensive care specific. Neonates were also excluded. Mortality was the most frequent primary outcome while long-term patient-centred outcomes were rarely used.⁴

Systematic review

A systematic review will be performed to identify both morbidity and mortality outcomes reported in existing studies involving paediatric sepsis in low- and middle-income countries.

Types of studies, participants and interventions

All forms of published studies will be included. Subjects will be children under the age of 18 with sepsis. Those involving a high proportion (>50%) of premature patients will be excluded. Studies undertaken in a low- and middle-income country, as defined by the World Bank,²³ and those describing a clinical outcome that is measured systematically across the population studied will be included. Outcome will be defined to be a measurement or observation used to capture and assess the effect of treatment such as assessment of side effects (risk) or effectiveness (benefits).²⁴ All interventions related to paediatric sepsis in this setting will be applicable. Sepsis, severe sepsis or septic shock will be defined by either the International Consensus Conference on Pediatric Sepsis Definitions, American College of Chest Physicians/Society of Critical Care Medicine consensus criteria, sepsis-relevant International Classification of Diseases (ICD)−9/ICD-10 codes), Integrated Management of Childhood Illness or clearly self-defined (eg, positive blood culture with one or more symptoms). Only articles in English from 1^st January 1994 to 1^st September 2019 will be assessed.

Search methods for identification of studies and study eligibility

The search strategy will be applied to PubMed, Embase, Scopus, Cochrane Central Register of Controlled Trials and the WHO International Clinical Trials Platform (ICTRP). One review author (GW) will independently screen the abstracts returned from the search strategy and any studies not meeting inclusion criteria will be excluded. Assessment of methodological quality will not be undertaken as we are evaluating outcomes only. The electronic search strategy was developed as per the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.²⁵ Details are outlined in online supplementary files 3,4.

Data extraction, analysis and presentation

One review author (GW) will perform data extraction using a standardised form. A second reviewer (SM) will provide validation in cases of doubt. The following data will be retrieved: author details, year and journal of publication, sepsis definition, reported outcomes and outcome definition(s). The data will be synthesised and presented in a descriptive table. The outcomes will then be condensed into a list and placed into one of the 38 domains in the outcome taxonomy.²⁶ Care will be taken to avoid ambiguity in the language used in the list of outcomes, with the final list of outcomes being reviewed by multiple healthcare professionals from around the world prior to commencement of the Delphi.

A list of authors will be compiled and invited to participate.

Records will be managed in EndNote X9 reference software (Clarivate Analytics, Boston, Massachusetts).

Delphi process

A two-round international e-Delphi survey involving an electronic-based questionnaire populated by outcomes from the literature review will then be performed to rank the outcomes.

Selection of panel members

A minimum of 50 participants for the Delphi panel will be involved. Nurses, clinicians and researchers will be recruited from around the world, with an aim to have all from low- and middle-income countries. This will ensure a direct relevance to the population of children in most need, and give a voice to those rarely heard, a need previously identified in COS development.²⁷ These participants will be identified from prior research papers on paediatric sepsis in LMIC, current involvement in ongoing studies and previous collaborations with members of the research team. The planned start date for the initial online survey is 5^th January 2020. The second round will occur 6 weeks after participants completed round 1, and therefore occur in March 2020. The survey and list of outcomes will be translated into French and Spanish.

Delphi round 1

Participants will initially be asked to complete brief questions related to their profession and experience. A list of outcomes generated from the systematic review will then be presented, and the participants required to rank each one on a scale from 1 to 9.²⁸ A score of 1 to 3 indicates an unimportant outcome that should not be included, scores of 4 to 6 demonstrate an important but not critical outcome and scores of 7 to 9 imply an outcome that is essential to inclusion.^19–21 A survey reminder will be sent out after 7 days and then again after 2 weeks. All outcomes will be carried on to the second round. There will be a free-text option for participants to suggest additional outcomes.

At this point, approximately 10 patients and their parents or guardian will be identified by clinicians known to team members and approached after discharge. The questionnaire will be paper-based, and outcome measures simplified into broad domains for the patients to score with the use of a local facilitator, due to the limited access to internet, possible illiteracy and non-English speaking. The patient survey will generate the relevant domains of key interest, and be incorporated into the outcomes for round 2 of the Delphi. The patient and parent/guardian generated outcomes and associated scores will be displayed to the remaining stakeholders in round 2. The specific granular outcomes will be defined by the formal Delphi

Delphi round 2

In the second phase of the electronic questionnaire, the first round scores for each outcome and number of prior respondents will be fed back to the participants. With this knowledge, including the patient scored outcomes, each outcome will be rescored as described previously in Delphi round 1.

Once again, a survey reminder will be sent out after 7 days and then again after 2 weeks.

Analysis of outcomes

Descriptive statistics will be used to analyse the responses from both round 1 and 2, including quantitative (absolute values, percentages) and qualitatively (suggestions given by participants) measures. The denominator will reflect only those outcomes scored. Free-text answers will be reviewed by the authorship team to evaluate for uncaptured outcomes in the first-round questionnaire and added if deemed appropriate to the second Delphi round questionnaire.

Our definition of consensus will follow that of previous COS publications.¹⁸ Analysis of the second round outcomes for which ≥70% of panellists scored it 7 to 9 and fewer than 15% of panellists scored it 1 to 3 will have met criteria for inclusion in the consensus meeting discussion. Outcomes for which ≥70% of panellists scored it 1 to 3, and fewer than 15% of panellists scored it 7 to 9 will be defined to have met consensus for exclusion.¹⁸ Those outcomes not meeting criteria will be defined as lack of consensus.

Consensus meeting

A face-to-face consensus meeting will be held after completion of the e-Delphi survey to finalise the outcomes for inclusion in the COS. Invites will be sent to those that have completed both rounds of the Delphi survey. All stakeholder groups will be represented, including local patients, parents, nurses and clinicians. The exact final meeting format will depend on the location and the number of participants able to attend, but most likely occur in June 2020. All outcomes from round 2 of the e-Delphi survey will be presented, including the results of the patient survey. Both the aggregate score for each outcome and the individual score for each stakeholder group will be displayed in order to ensure any important differences are highlighted and discussed. Participants will then anonymously vote for each outcome for inclusion and exclusion in the finalised COS using a format similar to that of the Delphi survey. Consensus for inclusion will be as before: if ≥70% of panellists vote in favour and fewer than 15% of panellists vote against. At this time, we will also enquire of the participants as to the practicality, feasibility and cost-effectiveness of the finalised core outcome set and the ease as to which it could be implemented. A formal feasibility matrix will be incorporated to inform this process so as to ensure that feasibility is assessed in a standardised way. Those outcomes that are deemed feasible by ≥70% of panellists, will meet consensus for inclusion into the COS.

Ethics and dissemination

All participants involved will be asked for their consent before undertaking the Delphi survey, and all procedures will be conducted according to the Declaration of Helsinki. Involvement in the survey will be completely voluntary and the responses anonymised. A waiver for ethics approval has been granted by the University of British Columbia Children’s and Women’s Research Ethics Board.

Implementation, dissemination and updating the COS

On completion of the consensus meeting, we will draft a COS guideline including an explanation of our methods using the COS-STAR template.²² It will be submitted to a high impact journal and presented at international meetings. We will seek endorsement and dissemination by major international societies and journals with an interest in paediatric sepsis and hope that it can be used as a standard in LMIC data sets for paediatric sepsis.

The next step will be to determine how best to measure the core outcomes using the COSMIN-COMET²⁹ guidance. Those attending the consensus meeting and prior Delphi participants will be invited to form a group to develop this further.

Discussion

At present, no core outcome sets exist for clinical trials in paediatric sepsis in LMIC. Outcomes from HIC cannot be reliably extrapolated to this setting.³⁰ This region-specific COS aims to reduce heterogeneity that currently exists in this area, limit reporting bias and improve the quality and relevance of outcomes published. It will also hopefully serve to highlight the heavy burden of paediatric sepsis in this setting and aid collaboration and participation between all stakeholders, encouraging ongoing essential high-quality research into the topic and facilitate further understanding of this complex disease.