Protocol for the development of a core outcome set for neonatal sepsis (NESCOS)

Petek Eylul Taneri; Jamie J Kirkham; Eleanor J Molloy; Linda Biesty; Richard A Polin; James L Wynn; Barbara J Stoll; Niranjan Kissoon; Kondwani Kawaza; Mandy Daly; Aoife Branagan; Lívia Nagy Bonnard; Eric Giannoni; Tobias Strunk; Magdalena Ohaja; Kenneth Mugabe; Denise Suguitani; Fiona Quirke; Declan Devane

doi:10.1371/journal.pone.0295325

. 2023 Dec 5;18(12):e0295325. doi: 10.1371/journal.pone.0295325

Protocol for the development of a core outcome set for neonatal sepsis (NESCOS)

Petek Eylul Taneri ^1,^2,^*, Jamie J Kirkham ³, Eleanor J Molloy ^4,⁵, Linda Biesty ^2,⁶, Richard A Polin ⁷, James L Wynn ⁸, Barbara J Stoll ^9,^10,¹¹, Niranjan Kissoon ¹², Kondwani Kawaza ¹³, Mandy Daly ¹⁴, Aoife Branagan ^4,⁵, Lívia Nagy Bonnard ¹⁵, Eric Giannoni ¹⁶, Tobias Strunk ¹⁷, Magdalena Ohaja ², Kenneth Mugabe ¹⁸, Denise Suguitani ¹⁹, Fiona Quirke ^1,²⁰, Declan Devane ^1,^2,⁶

Editor: Andrew Harding²¹

¹HRB-Trials Methodology Research Network, University of Galway, Galway, Ireland

²School of Nursing and Midwifery, University of Galway, Galway, Ireland

³Centre for Biostatistics, Manchester Academic Health Science Centre, The University of Manchester, Manchester, United Kingdom

⁴Department of Neonatology, The Coombe Hospital, Dublin, Ireland

⁵Department of Paediatrics and Child Health, Trinity College Dublin, Trinity Research in Childhood Centre (TRiCC), Neonatology, Children’s Health Ireland, Dublin, Ireland

⁶Evidence Synthesis Ireland & Cochrane Ireland, University of Galway, Galway, Ireland

⁷Department of Paediatrics, College of Physicians and Surgeons, Columbia University, New York, New York, United States of America

⁸Department of Paediatrics, College of Medicine, University of Florida, Gainesville, Florida, United States of America

⁹China Medical Board, China

¹⁰Emory University School of Medicine, Atlanta, GA, United States of America

¹¹McGovern Medical School of the University of Texas Health Science Center at Houston, Houston, Texas, United States of America

¹²Department of Paediatrics, College of Medicine, University of British Columbia, Vancouver, Canada

¹³Department of Paediatrics and Child Health, Kamuzu University of Health Sciences, Blantyre, Malawi

¹⁴Education and Research, Irish Neonatal Health Alliance, Bray, Ireland

¹⁵Patient Advocacy, Melletted a helyem Egyesület, Budapest, Hungary

¹⁶Clinic of Neonatology, Department Mother-Woman-Child, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland

¹⁷Neonatal Directorate, Child and Adolescent Health Service, Wesfarmers’ Centre for Vaccines and Infectious Diseases, Telethon Kids Institute, University of Western Australia, Perth, Australia

¹⁸Mbale Regional Referral Hospital, Busitema University Faculty of Health Sciences, Mbale, Uganda

¹⁹Brazilian Parents of Preemies’ Association, Brazil

²⁰School of Medicine, University of Limerick, Limerick, Ireland

²¹Lancaster University, UNITED KINGDOM

Competing Interests: The authors have declared that no competing interests exist.

^✉

* E-mail: petekeylul.taneri@universityofgalway.ie

Roles

Petek Eylul Taneri: Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Writing – original draft, Writing – review & editing

Jamie J Kirkham: Conceptualization, Methodology, Writing – review & editing

Eleanor J Molloy: Conceptualization, Methodology, Writing – review & editing

Linda Biesty: Conceptualization, Investigation, Methodology, Writing – original draft, Writing – review & editing

Richard A Polin: Conceptualization, Writing – review & editing

James L Wynn: Conceptualization, Writing – review & editing

Barbara J Stoll: Conceptualization, Writing – review & editing

Niranjan Kissoon: Conceptualization, Writing – review & editing

Kondwani Kawaza: Conceptualization, Writing – review & editing

Mandy Daly: Conceptualization, Writing – review & editing

Aoife Branagan: Conceptualization, Writing – review & editing

Lívia Nagy Bonnard: Conceptualization, Writing – review & editing

Eric Giannoni: Conceptualization, Writing – review & editing

Tobias Strunk: Conceptualization, Writing – review & editing

Magdalena Ohaja: Conceptualization, Writing – review & editing

Kenneth Mugabe: Conceptualization, Writing – review & editing

Denise Suguitani: Conceptualization, Writing – review & editing

Fiona Quirke: Conceptualization, Writing – review & editing

Declan Devane: Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Writing – original draft, Writing – review & editing

Andrew Harding: Editor

PMCID: PMC10697588 PMID: 38051733

Abstract

Neonatal sepsis is a serious public health problem; however, there is substantial heterogeneity in the outcomes measured and reported in research evaluating the effectiveness of the treatments. Therefore, we aim to develop a Core Outcome Set (COS) for studies evaluating the effectiveness of treatments for neonatal sepsis. Since a systematic review of key outcomes from randomised trials of therapeutic interventions in neonatal sepsis was published recently, we will complement this with a qualitative systematic review of the key outcomes of neonatal sepsis identified by parents, other family members, parent representatives, healthcare providers, policymakers, and researchers. We will interpret the outcomes of both studies using a previously established framework. Stakeholders across three different groups i.e., (1) researchers, (2) healthcare providers, and (3) patients’ parents/family members and parent representatives will rate the importance of the outcomes in an online Real-Time Delphi Survey. Afterwards, consensus meetings will be held to agree on the final COS through online discussions with key stakeholders. This COS is expected to minimize outcome heterogeneity in measurements and publications, improve comparability and synthesis, and decrease research waste.

Introduction

Sepsis is a major global public health problem. Children account for more than half of all sepsis cases, most of whom are neonates [1]. The global incidence of neonatal sepsis has been estimated at 2824 (95% CI 1892 to 4194) cases per 100,000 live births, with a mortality rate of 17.6% (95% CI 10.3% to 28.6%) and a higher burden in low-middle income countries [2]. In addition to the increased risk of early death, neonatal sepsis may cause long-term neurodevelopmental delay [3] and neurodevelopmental impairments such as cerebral palsy and neurosensory deficits [4]. Neonatal sepsis is also associated with impaired physical growth, increased risk of bronchopulmonary dysplasia and multiple organ failure [3].

There is significant variation in definitions of neonatal sepsis [5], clinical signs of neonatal sepsis [6], and antibiotic use [7]. The lack of a universally accepted definition for neonatal sepsis inhibits efforts to improve accurate diagnostic and prognostic testing methods for this vulnerable population [8]. In addition, there is substantial heterogeneity in the outcomes measured and reported in studies evaluating the effectiveness of treatments for neonatal sepsis, with Henry et al. identifying 88 outcomes across 90 studies [9]. This heterogeneity in outcomes limits the ability to compare, contrast, and synthesise [10] the findings of individual studies and contributes to research waste.

Furthermore, the challenges experienced by those who use research to inform decisions about their health care are exacerbated further by outcome-reporting bias [11]. Outcomes in which the intervention has a statistically significant effect are more likely to be fully reported [12]. Developing a Core Outcome Set (COS) for treatments for neonatal sepsis could help minimise these current shortcomings in reporting challenges.

A COS is ‘the minimum that should be measured and reported in all clinical trials of a specific condition and could also be suitable for use in other types of research and clinical audit’ [13]. The COS should always be measured, collected, and reported. Researchers may measure and report outcomes additional to the COS, but the COS should be reported in its entirety [11]. The advantages of the widespread use of a COS includes increased outcome consistency across trials, a likely decrease in selective reporting, and increased opportunity for a study to contribute to syntheses across outcomes included in the COS [14]. Importantly, using a COS helps ensure that the outcomes measured and reported are those that matter to stakeholders, including patients and/or their carers [15].

This study aims to develop a COS for studies evaluating the effectiveness of treatments for neonatal sepsis.

Materials and methods

Overview

The University of Galway, Research Ethics Committee provided ethical approval for this project (Reference Number: 2022.10.002). During the Delphi survey and consensus meeting phases, participants were provided with detailed project information via participant information leaflets. They were then cordially invited to partake in the study, contingent upon their agreement as evidenced by completing the standardized consent forms. These forms, which included a thorough briefing, were utilized to obtain written consent from the participants. Integral to our adherence to ethical standards, these documents were incorporated into our submission to the ethical committee and were subsequently granted approval.

This protocol has been prepared according to COS-STAP Statement recommendations [16]. We are not aware of any published COS evaluating the effectiveness of interventions for treating neonatal sepsis. The protocol of this study has been registered with the Core Outcome Measures in Effectiveness Trials (COMET) initiative (https://www.comet-initiative.org/Studies/Details/2118).

This study will be conducted in four stages (Fig 1):

Qualitative systematic review
Delphi survey
Consensus meetings
Dissemination and implementation

Stage 1: Qualitative systematic review

Recently, Henry et al. published a systematic review of core outcomes from randomised trials of therapeutic interventions in neonatal sepsis. They reported 88 unique outcomes from 90 included studies [9].

The outcomes chosen and reported by researchers or clinicians alone may not be important and/or relevant for other stakeholders, such as patients, caregivers, or other decision-makers [17]. The outcomes considered during the consensus process should represent the views of all relevant stakeholders. Therefore, it has been advised that COS developers consider different approaches to establishing an initial list of outcomes for prioritisation, such as collecting data from patient interviews or analysing qualitative research studies focusing on patients’ opinions [18]. Based on this recommendation, several qualitative systematic reviews have been published to support the COS development processes in, for example, female chronic pelvic pain [17], type 2 diabetes [18], and neonatal care [19].

Our proposed qualitative systematic review was registered with The International Prospective Register of Systematic Reviews (PROSPERO ID: CRD42022344485). The review will be conducted and reported following the guidelines of the ENTREQ statement and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [19, 20].

Review/synthesis question: What are the key outcomes of neonatal sepsis identified by parents, other family members, parent representatives, healthcare providers, policymakers, and researchers that should be included in a ’long list’ of outcomes in a Delphi Survey as part of the development of a COS on treatments for neonatal sepsis?

Inclusion criteria

The PerSPEcTiF framework was used to structure the description of inclusion criteria [21] as per qualitative review guidelines [22, 23] (Table 1).

Table 1. The PerSPEcTiF question formulation framework.

Per	S	P	E	(C)	Ti	F
Perspective	Setting	Phenomenon of interest/ Problem	Environment	Comparison (optional)	Time/ Timing	Findings
Parents, other family members, parent representatives, healthcare providers, policymakers, researchers	Any setting	The outcomes of treatments for neonatal sepsis are important to parents, other family members, parent representatives, healthcare providers, policymakers, and researchers that should be included in a ’long list’ of outcomes contributing to the COS	High, middle and low-income countries	-	Anytime following diagnosis of sepsis	All outcomes regarding the phenomenon of interest

Open in a new tab

We will include studies with a qualitative study design such as ethnographic, grounded theory, historical, and case study; those that used qualitative data-collection methods, including observations, textual or visual analysis, and interviews (individual or group). We will also include mixed-method studies with a qualitative component where participant experiences are reported separately. We will exclude abstracts, randomised trials, clinical trials, intervention studies, cross-sectional studies, case-control studies, prospective and retrospective cohort studies, letters to the editor, conference proceedings, and systematic reviews with or without meta-analyses.

Search strategy

With no date restrictions, a comprehensive literature search will be carried out using MEDLINE, EMBASE, CINAHL, and PsycINFO databases. The complete search strategy is presented in S1 File.

The reference lists of included studies will be hand-searched to identify other potentially relevant studies [24–27]. Findings of the literature search will be transferred to Endnote, and after deduplication, abstract and title screening will be done via Rayyan [28].

Study selection

The titles and abstracts will be screened by two reviewers independently using the eligibility criteria, and discrepancies will be resolved through discussion with a third review author where necessary. For all studies that appear to fulfil the eligibility criteria based on the abstract, two authors will examine full-text articles independently. Discrepancies will be resolved again through discussion with a third review author where necessary.

Quality assessment

Two reviewers will independently assess the quality of the included studies using the Critical Appraisal Skills Programme (CASP) tool [29]. While studies will not be excluded based on this assessment of methodological limitations, the assessment will be reported in the final write-up of our systematic review. The results will be reported in the manuscript to inform readers about the methodological quality of the included studies.

Data extraction

One review author will extract the data using a pre-piloted data extraction form to obtain study characteristics from qualifying studies. One additional review author will double‐check the data extraction performed by the first review author and verify that all relevant data are extracted. Author information, publication year, study design, number of participants, data collection and analysis methods, and text excerpts relevant to outcomes will be noted on the form.

Data analysis

Drawing on the principles of thematic synthesis [30], extracted data related to outcomes will be coded line by line, and similarities will be identified. Codes will be grouped to define distinct, descriptive themes related to outcomes of therapeutic interventions in neonatal sepsis. Two reviewers will review and interpret the outcomes noted in the themes and will map these to a framework of outcome domains presented by Webbe et. al’s “Core outcomes in neonatology: development of a core outcome set for neonatal research.” [31]. The following domain headings will be used to guide this mapping: survival, respiratory, cardiovascular, gastrointestinal, neurological, genitourinary, skin, surgical, development (gross motor/fine motor/cognitive/special senses/speech and social), psychosocial, healthcare utilisation, outcomes related to parents, outcomes related to healthcare workers, general outcomes and miscellaneous. If the identified outcomes do not map to any domains, the reviewers will develop additional relevant domains.

The outcomes determined at the end of this phase will be brought forward to Stage 2.

Stage 2: Delphi survey

Delphi studies were used first in the 1960s by Dalkey and Helmer of the Rand Corporation to obtain consensus on the views of a group of experts [32]. Traditionally, the Delphi procedure helps develops consensus among experts by conducting controlled feedback on the findings of multiple survey rounds (usually) or interviews [33]. It can foster collective ownership and promote unity among people with different points of view [34]. The traditional Delphi technique is used extensively in health care to achieve consensus on various topics such as mental health [35–37], cancer [38, 39], and primary health care [40, 41].

The traditional Delphi technique’s recurring and multiple feedback nature necessitates a significant amount of time, which increases the risk of participant attrition across rounds [42]. In 2006, Gordon & Pease introduced the Real-Time Delphi (RTD) as an alternative to multi-round Delphi surveys, aiming to improve the process’s efficiency and shorten the required time. In this method, participants can view the group’s responses for each question in real time and the total number of responses simultaneously [43]. RTD is round-less; however, in light of changing feedback, participants can be recommended to revisit and re-rate questions. The responses participants will see on visiting an RTD survey reflect those who have participated in the survey up to that point rather than reflecting all survey respondents after each round as in traditional Delphi.

The number of papers using RTD is increasing. The RTD has been used to develop consensus across a range of settings, including economics, [43, 44] transportation [44], education [45, 46], religion [47], professional development [48], and sustainability [49]. RTD has been used in healthcare studies that aim to develop a medication adherence technologies repository [50] and determine the negative impacts of substance use disorders among people with HIV in the United States [51]. Several researchers have incorporated an RTD approach into the COS development processes [52, 53].

A comparison of real-time and traditional Delphi approaches for the future of the logistics industry in the year 2025 showed no statistically significant differences in participants’ projections on the future of logistics between the two studies [42]. A study comparing online Delphi and Real-Time Delphi surveys suggested that the RTD may reduce survey time and expenses and increase tester convergence and consensus. However, the study is small (n = 12) and lacks clarity on experimental methods [54]. A randomised trial comparing a three-round with a Real-Time Delphi approach on outcomes prioritised as part of a core outcome set development is currently awaiting reporting [52].

Participants

We aim to include at least 100 stakeholders across three different stakeholder groups i.e., (1) researchers, (2) healthcare providers, and (3) patients’ parents/family members and parent representatives. We plan to include participants from high, middle, and low-income countries. Participants will be sought through professional associations and groups representing different stakeholders. Through the encouragement of the participants to forward the invitation, especially to stakeholders from low-income countries, snowball sampling will also be encouraged.

Procedures

Previously, 88 outcomes were identified from 90 randomised trials [9]. Those will be mapped to Webbe et al.’s framework [31]. If a domain and/or outcome cannot be mapped, a new domain/outcome will be created. This will be combined with the domains and outcomes identified in Stage 1. Duplicate domains and outcomes will be removed, leaving a list of unique domains and outcomes for presentation as the ‘long list’ outcomes for use in the Delphi survey.

We will develop a survey in which participants will be invited to rate the importance of each outcome. Participants will rate the importance of each outcome for inclusion in the COS on a 9-point Likert scale (i.e., 1–3 limited importance, 4–6 important but not critical, and 7–9 critical) [11].

Once a participant rates an outcome, they will be presented with ‘real-time’ feedback on the portion of participants from each stakeholder group and all participants’ overall rating for each point on the 9-point scale for each outcome. Participants will then consider their ratings based on real-time feedback on responses from other participant groups and the group overall.

Before the survey goes live, we want to ensure that the survey is populated with feedback responses representative of the stakeholder groups. For this, we will recruit at least 5 participants from each stakeholder group (researchers, healthcare providers, patients’ parents/family members and parent representatives). These participants will also be included in the final survey numbers. This first group of participants will be invited to revisit the survey when it is live, to see stakeholder responses and amend their rating for each outcome if they wish.

Once the main survey goes live, participants will have a 3-week window during which they will be invited to re-visit the survey and review responses if they wish. They can choose to retain their original rating for each outcome or alter their rating based on how others have responded. Based on a comparison of software-based tools for RTD [55, 56], we will use Surveylet (https://calibrum.com) platform for our RTD.

At the end of the three weeks, outcomes that scored 7–9 by 70%, and 1–3 by less than 15% of participants across all stakeholder groups will be brought forward to the consensus meetings. When 50% or fewer participants score 7–9 in each stakeholder group that outcome will be excluded. This consensus definition was created to ensure that a measure could not achieve consensus if a minority stakeholder group mostly rated it as ‘limited importance’ and was used previously during the development of other COSs [57–59].

Stage 3: Consensus meetings

The consensus meetings aim to agree on the final COS through online discussions with key stakeholders.

Participants

Consensus meetings will be conducted with an international panel of stakeholders representing patients’ parents/family members and parent representatives, researchers, and healthcare providers from high, low, and middle-income countries.

Schedule

We will hold two independent online consensus meetings, including at least three people from each stakeholder group, to make it feasible for the participants living in different time zones. The unique outcomes identified for inclusion in the COS from each of the two consensus meetings will be discussed at a third, final online consensus meeting attended by stakeholders representing patients’ parents/family members and parent representatives, researchers, and healthcare providers from high, low, and middle-income countries.

During the meetings, the outcomes emerging from the RTD will be presented to the participants, along with the voting patterns of the stakeholder groups for each outcome. An experienced facilitator who will not participate in voting will chair each consensus meeting. There will be anonymous, computerised voting after a discussion of each outcome. If at least 80% of participants, including at least one representative from each stakeholder group, vote in favour of an outcome, it will be included in the COS.

Stage 4: Dissemination and implementation strategy

Once the consensus meeting is over and the final COS has been approved, a paper will be written, published, and distributed in accordance with the COS-STAP recommendations [16]. The final COS will be made available through the COMET database. We will present our results at national and international conferences to encourage researchers and clinicians to use the COS. We will target the audience/key stakeholders (e.g., all participants in the survey, known maternity and neonatal care researchers, the Cochrane Pregnancy and Childbirth Group, and the Cochrane Neonatal Group.

Discussion

Despite advancements in the quality of neonatal care, sepsis-related morbidity and mortality are growing concerns, particularly in low-middle-income countries [60]. Currently, there is inconsistency in the reporting of outcomes in studies on treatments of neonatal sepsis. A COS can help standardize outcomes measured and reported in studies and enable comparisons between and synthesis across them [61]. By conducting a systematic review, RTD survey, and consensus meetings, we will develop a COS for studies evaluating the effects of treatments for neonatal sepsis. We anticipate that this COS will minimise heterogeneity in outcomes measured and reported, enable better comparison and synthesis, and reduce research waste.

Study status

The study is ongoing, in Stage 1 (systematic review).

Supporting information

S1 Checklist. PRISMA-P (Preferred Reporting Items for Systematic review and Meta-Analysis Protocols) 2015 checklist: Recommended items to address in a systematic review protocol*.

(DOC)

Click here for additional data file.^{(83.5KB, doc)}

S1 File. NESCOS search strategy.

The complete search strategy of the qualitative systematic review of the NESCOS project.

(DOCX)

Click here for additional data file.^{(25.8KB, docx)}

Data Availability

No datasets were generated or analysed during the current study. All relevant data from this study will be made available upon study completion.

Funding Statement

This research was funded by the Health Research Board (HRB, Ireland) through funding to the HRB Irish Network for Children’s Clinical Trials (In4Kids) [Grant number: CTN-2021-007] Funders did not play any role in the study design, data collection (when applicable) and analysis, decision to publish, or preparation of the manuscript.

References

1.Rudd KE, Johnson SC, Agesa KM, Shackelford KA, Tsoi D, Kievlan DR, et al. Global, regional, and national sepsis incidence and mortality, 1990–2017: analysis for the Global Burden of Disease Study. Lancet. 2020;395(10219):200–11. doi: 10.1016/S0140-6736(19)32989-7 ; PubMed Central PMCID: PMC6970225. [DOI] [PMC free article] [PubMed] [Google Scholar]
2.Fleischmann C, Reichert F, Cassini A, Horner R, Harder T, Markwart R, et al. Global incidence and mortality of neonatal sepsis: a systematic review and meta-analysis. Arch Dis Child. 2021. Epub 20210122. doi: 10.1136/archdischild-2020-320217 ; PubMed Central PMCID: PMC8311109. [DOI] [PMC free article] [PubMed] [Google Scholar]
3.Bakhuizen SE, de Haan TR, Teune MJ, van Wassenaer-Leemhuis AG, van der Heyden JL, van der Ham DP, et al. Meta-analysis shows that infants who have suffered neonatal sepsis face an increased risk of mortality and severe complications. Acta Paediatr. 2014;103(12):1211–8. Epub 2014/07/31. doi: 10.1111/apa.12764 . [DOI] [PubMed] [Google Scholar]
4.Cai S, Thompson DK, Anderson PJ, Yang JY. Short- and Long-Term Neurodevelopmental Outcomes of Very Preterm Infants with Neonatal Sepsis: A Systematic Review and Meta-Analysis. Children (Basel). 2019;6(12). Epub 2019/12/07. doi: 10.3390/children6120131 ; PubMed Central PMCID: PMC6956113. [DOI] [PMC free article] [PubMed] [Google Scholar]
5.McGovern M, Giannoni E, Kuester H, Turner MA, van den Hoogen A, Bliss JM, et al. Challenges in developing a consensus definition of neonatal sepsis. Pediatr Res. 2020;88(1):14–26. Epub 2020/03/04. doi: 10.1038/s41390-020-0785-x . [DOI] [PubMed] [Google Scholar]
6.Wynn JL, Wong HR, Shanley TP, Bizzarro MJ, Saiman L, Polin RA. Time for a neonatal-specific consensus definition for sepsis. Pediatr Crit Care Med. 2014;15(6):523–8. Epub 2014/04/23. doi: 10.1097/PCC.0000000000000157 ; PubMed Central PMCID: PMC4087075. [DOI] [PMC free article] [PubMed] [Google Scholar]
7.McMullan B, Cooper C, Spotswood N, James R, Jones C, Konecny P, et al. Antibiotic prescribing in neonatal sepsis: an Australian nationwide survey. BMJ Paediatr Open. 2020;4(1):e000643. Epub 2020/04/02. doi: 10.1136/bmjpo-2020-000643 ; PubMed Central PMCID: PMC7101048. [DOI] [PMC free article] [PubMed] [Google Scholar]
8.Wynn JL. Defining neonatal sepsis. Curr Opin Pediatr. 2016;28(2):135–40. Epub 2016/01/15. doi: 10.1097/MOP.0000000000000315 ; PubMed Central PMCID: PMC4786443. [DOI] [PMC free article] [PubMed] [Google Scholar]
9.Henry CJ, Semova G, Barnes E, Cotter I, Devers T, Rafaee A, et al. Neonatal sepsis: a systematic review of core outcomes from randomised clinical trials. Pediatr Res. 2022;91(4):735–42. Epub 2022/01/09. doi: 10.1038/s41390-021-01883-y ; PubMed Central PMCID: PMC9064797. [DOI] [PMC free article] [PubMed] [Google Scholar]
10.Bellucci C, Hughes K, Toomey E, Williamson PR, Matvienko-Sikar K. A survey of knowledge, perceptions and use of core outcome sets among clinical trialists. Trials. 2021;22(1):1–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
11.Williamson PR, Altman DG, Bagley H, Barnes KL, Blazeby JM, Brookes ST, et al. The COMET Handbook: version 1.0. Trials. 2017;18(Suppl 3):280. Epub 2017/07/07. doi: 10.1186/s13063-017-1978-4 ; PubMed Central PMCID: PMC5499094. [DOI] [PMC free article] [PubMed] [Google Scholar]
12.Higgins JPT, Savović J, Page MJ, Elbers RG, Sterne JAC. Assessing risk of bias in a randomized trial. In: Higgins J, Thomas J, editors. Cochrane Handbook for Systematic Reviews of Interventions. Version 6.4, 2023. [Google Scholar]
13.Clarke M. Standardising outcomes for clinical trials and systematic reviews. Trials. 2007;8:39. Epub 2007/11/28. doi: 10.1186/1745-6215-8-39 ; PubMed Central PMCID: PMC2169261. [DOI] [PMC free article] [PubMed] [Google Scholar]
14.Saldanha IJ, Lindsley KB, Money S, Kimmel HJ, Smith BT, Dickersin K. Outcome choice and definition in systematic reviews leads to few eligible studies included in meta-analyses: a case study. BMC Med Res Methodol. 2020;20(1):30. Epub 2020/02/13. doi: 10.1186/s12874-020-0898-2 ; PubMed Central PMCID: PMC7014938. [DOI] [PMC free article] [PubMed] [Google Scholar]
15.Williamson PR, de Avila Oliveira R, Clarke M, Gorst SL, Hughes K, Kirkham JJ, et al. Assessing the relevance and uptake of core outcome sets (an agreed minimum collection of outcomes to measure in research studies) in Cochrane systematic reviews: a review. BMJ Open. 2020;10(9):e036562. Epub 2020/09/09. doi: 10.1136/bmjopen-2019-036562 ; PubMed Central PMCID: PMC7476465. [DOI] [PMC free article] [PubMed] [Google Scholar]
16.Kirkham JJ, Gorst S, Altman DG, Blazeby JM, Clarke M, Tunis S, et al. Core Outcome Set-STAndardised Protocol Items: the COS-STAP Statement. Trials. 2019;20(1):116. Epub 2019/02/13. doi: 10.1186/s13063-019-3230-x ; PubMed Central PMCID: PMC6371434. [DOI] [PMC free article] [PubMed] [Google Scholar]
17.Keeley T, Williamson P, Callery P, Jones LL, Mathers J, Jones J, et al. The use of qualitative methods to inform Delphi surveys in core outcome set development. Trials. 2016;17(1):230. Epub 2016/05/05. doi: 10.1186/s13063-016-1356-7 ; PubMed Central PMCID: PMC4855446. [DOI] [PMC free article] [PubMed] [Google Scholar]
18.Kirkham JJ, Davis K, Altman DG, Blazeby JM, Clarke M, Tunis S, et al. Core Outcome Set-STAndards for Development: The COS-STAD recommendations. PLoS Med. 2017;14(11):e1002447. Epub 20171116. doi: 10.1371/journal.pmed.1002447 ; PubMed Central PMCID: PMC5689835. [DOI] [PMC free article] [PubMed] [Google Scholar]
19.Tong A, Flemming K, McInnes E, Oliver S, Craig J. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC medical research methodology. 2012;12(1):1–8. doi: 10.1186/1471-2288-12-181 [DOI] [PMC free article] [PubMed] [Google Scholar]
20.Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–9, W64. Epub 20090720. doi: 10.7326/0003-4819-151-4-200908180-00135 . [DOI] [PubMed] [Google Scholar]
21.Booth A, Noyes J, Flemming K, Moore G, Tunçalp Ö, Shakibazadeh E. Formulating questions to explore complex interventions within qualitative evidence synthesis. BMJ global health. 2019;4(Suppl 1):e001107. doi: 10.1136/bmjgh-2018-001107 [DOI] [PMC free article] [PubMed] [Google Scholar]
22.Flemming K, Noyes J. Qualitative evidence synthesis: Where are we at? International Journal of Qualitative Methods. 2021;20:1609406921993276. [Google Scholar]
23.Noyes J, Booth A, Cargo M, Flemming K, Harden A, Harris J, et al. Qualitative evidence. Cochrane Handbook for Systematic Reviews of Interventions version 63: Cochrane; 2022. [Google Scholar]
24.Pearson M, Moxham T, Ashton K. Effectiveness of search strategies for qualitative research about barriers and facilitators of program delivery. Evaluation & the health professions. 2011;34(3):297–308. doi: 10.1177/0163278710388029 [DOI] [PubMed] [Google Scholar]
25.Papaioannou D, Sutton A, Carroll C, Booth A, Wong R. Literature searching for social science systematic reviews: consideration of a range of search techniques. Health Information & Libraries Journal. 2010;27(2):114–22. [DOI] [PubMed] [Google Scholar]
26.Evans D. Database searches for qualitative research. J Med Libr Assoc. 2002;90(3):290–3. ; PubMed Central PMCID: PMC116400. [PMC free article] [PubMed] [Google Scholar]
27.Booth A, Harris J, Croot E, Springett J, Campbell F, Wilkins E. Towards a methodology for cluster searching to provide conceptual and contextual “richness” for systematic reviews of complex interventions: case study (CLUSTER). BMC medical research methodology. 2013;13(1):1–14. doi: 10.1186/1471-2288-13-118 [DOI] [PMC free article] [PubMed] [Google Scholar]
28.Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web and mobile app for systematic reviews. Systematic reviews. 2016;5(1):1–10. doi: 10.1186/s13643-016-0384-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
29.Critical Appraisal Skills Programme. CASP for Qualitative Studies Checklist 2018. Available from: https://casp-uk.net/wp-content/uploads/2018/03/CASP-Qualitative-Checklist-2018_fillable_form.pdf.
30.Thomas J, Harden A., Newman M. Synthesis: combining results systematically and appropriately. An Introduction to Systematic Reviews Sage Publications; 2012. [Google Scholar]
31.Webbe JWH, Duffy JMN, Afonso E, Al-Muzaffar I, Brunton G, Greenough A, et al. Core outcomes in neonatology: development of a core outcome set for neonatal research. Arch Dis Child Fetal Neonatal Ed. 2020;105(4):425–31. Epub 20191115. doi: 10.1136/archdischild-2019-317501 ; PubMed Central PMCID: PMC7363790. [DOI] [PMC free article] [PubMed] [Google Scholar]
32.Williamson K. Research methods for students, academics and professionals: Information management and systems: Elsevier; 2002. [Google Scholar]
33.Sproull NL. Handbook of research methods: A guide for practitioners and students in the social sciences: Scarecrow press; 2002. [Google Scholar]
34.Thangaratinam S, Redman CW. The delphi technique. The obstetrician & gynaecologist. 2005;7(2):120–5. [Google Scholar]
35.Cleverley K, McCann E, O’Brien D, Davies J, Bennett K, Brennenstuhl S, et al. Prioritizing core components of successful transitions from child to adult mental health care: a national Delphi survey with youth, caregivers, and health professionals. European Child & Adolescent Psychiatry. 2021:1–14. doi: 10.1007/s00787-021-01806-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
36.Hultsjö S, Berterö C, Arvidsson H, Hjelm K. Core components in the care of immigrants with psychoses: A Delphi survey of patients, families, and health‐care staff. International journal of mental health nursing. 2011;20(3):174–84. doi: 10.1111/j.1447-0349.2010.00720.x [DOI] [PubMed] [Google Scholar]
37.Campbell S, Shield T, Rogers A, Gask L. How do stakeholder groups vary in a Delphi technique about primary mental health care and what factors influence their ratings? BMJ Quality & Safety. 2004;13(6):428–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
38.Lambert SD, Ould Brahim L, Morrison M, Girgis A, Yaffe M, Belzile E, et al. Priorities for caregiver research in cancer care: an international Delphi survey of caregivers, clinicians, managers, and researchers. Supportive Care in Cancer. 2019;27(3):805–17. doi: 10.1007/s00520-018-4314-y [DOI] [PMC free article] [PubMed] [Google Scholar]
39.Efstathiou N, Ameen J, Coll A-M. Healthcare providers’ priorities for cancer care: a Delphi study in Greece. European Journal of Oncology Nursing. 2007;11(2):141–50. doi: 10.1016/j.ejon.2006.06.005 [DOI] [PubMed] [Google Scholar]
40.Santaguida P, Dolovich L, Oliver D, Lamarche L, Gilsing A, Griffith LE, et al. Protocol for a Delphi consensus exercise to identify a core set of criteria for selecting health related outcome measures (HROM) to be used in primary health care. BMC family practice. 2018;19(1):1–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
41.Gharibi F, Tabrizi JS. Development of an accreditation model for health education and promotion programs in the Iranian primary healthcare system: a Delphi study. Health Promotion Perspectives. 2018;8(2):155. doi: 10.15171/hpp.2018.20 [DOI] [PMC free article] [PubMed] [Google Scholar]
42.Gnatzy T, Warth J, von der Gracht H, Darkow IL. Validating an innovative real-time Delphi approach—A methodological comparison between real-time and conventional Delphi studies. Technol Forecast Soc. 2011;78(9):1681–94. doi: 10.1016/j.techfore.2011.04.006 WOS:000297564200016. [DOI] [Google Scholar]
43.Gordon T, Pease A. RT Delphi: An efficient, "round-less" almost real time Delphi method. Technol Forecast Soc. 2006;73(4):321–33. doi: 10.1016/j.techfore.2005.09.005 WOS:000236821400001. [DOI] [Google Scholar]
44.Meyer T, von der Gracht HA, Hartmann E. Technology foresight for sustainable road freight transportation: Insights from a global real‐time Delphi study. Futures & Foresight Science. 2022;4(1):e101. [Google Scholar]
45.Sunderji N, Waddell A. Using real-time Delphi to develop a consensus on competencies. Med Educ. 2015;49(11):1151–2. doi: 10.1111/medu.12851 WOS:000363514600026. [DOI] [PubMed] [Google Scholar]
46.Genova JP. Futuring in high performing, high poverty, and high minority schools: A real-time Delphi study: St. John’s University (New York), School of Education and Human Services; 2018. [Google Scholar]
47.Gary JE. Outlook 2020: Results from a Real-Time Delphi Survey of Global Pentecostal Leaders. Pneuma. 2012;34(3):383–414. [Google Scholar]
48.Gary JE, von der Gracht HA. The future of foresight professionals: Results from a global Delphi study. Futures. 2015;71:132–45. doi: 10.1016/j.futures.2015.03.005 WOS:000361407800012. [DOI] [Google Scholar]
49.Serra M, Antonio N, Henriques C, Afonso CM. Promoting Sustainability through Regional Food and Wine Pairing. Sustainability. 2021;13(24):13759. [Google Scholar]
50.Makovec UN, Goetzinger C, Ribaut J, Barnestein-Fonseca P, Haupenthal F, Herdeiro MT, et al. Developing a medication adherence technologies repository: proposed structure and protocol for an online real-time Delphi study. BMJ open. 2022;12(4):e059674. doi: 10.1136/bmjopen-2021-059674 [DOI] [PMC free article] [PubMed] [Google Scholar]
51.Garner BR, Gotham HJ, Knudsen HK, Zulkiewicz BA, Tueller SJ, Berzofsky M, et al. The prevalence and negative impacts of substance use disorders among people with HIV in the United States: A real-time delphi survey of key stakeholders. AIDS and Behavior. 2022;26(4):1183–96. doi: 10.1007/s10461-021-03473-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
52.Quirke FA, Healy P, Bhraonáin EN, Daly M, Biesty L, Hurley T, et al. Multi-Round compared to real-time Delphi for consensus in core outcome set (COS) development: a randomised trial. Trials. 2021;22(1):1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
53.Dube K, Ayebare E, Bakhbakhi D, Bedwell C, Chandriah S, Chaudhry N, et al. Protocol for the development of a meta-core outcome set for stillbirth prevention and bereavement care following stillbirth. medRxiv. [Preprint]. 2022. [cited 2023 August 22]. Available from: https://www.medrxiv.org/content/10.1101/2022.10.13.22281030v1 [Google Scholar]
54.Hsieh CH, Tzeng FM, Wu CG, Kao JS, Lai YY. The Comparison of Online Delphi and Real-Time Delphi. 2011 Proceedings of Picmet 11: Technology Management in the Energy-Smart World (Picmet). 2011. WOS:000298384400042. [Google Scholar]
55.Aengenheyster S, Cuhls K, Gerhold L, Heiskanen-Schüttler M, Huck J, Muszynska M. Real-Time Delphi in practice—A comparative analysis of existing software-based tools. Technol Forecast Soc. 2017;118:15–27. [Google Scholar]
56.Varndell W, Fry M, Elliott D. Applying real-time Delphi methods: development of a pain management survey in emergency nursing. BMC nursing. 2021;20(1):1–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
57.Needham DM, Sepulveda KA, Dinglas VD, Chessare CM, Friedman LA, Bingham III CO, et al. Core outcome measures for clinical research in acute respiratory failure survivors. An international modified Delphi consensus study. American journal of respiratory and critical care medicine. 2017;196(9):1122–30. doi: 10.1164/rccm.201702-0372OC [DOI] [PMC free article] [PubMed] [Google Scholar]
58.Prinsen CA, Vohra S, Rose MR, Boers M, Tugwell P, Clarke M, et al. How to select outcome measurement instruments for outcomes included in a “Core Outcome Set”–a practical guideline. Trials. 2016;17(1):1–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
59.Bartlett SJ, Hewlett S, Bingham CO, Woodworth TG, Alten R, Pohl C, et al. Identifying core domains to assess flare in rheumatoid arthritis: an OMERACT international patient and provider combined Delphi consensus. Annals of the rheumatic diseases. 2012;71(11):1855–60. doi: 10.1136/annrheumdis-2011-201201 [DOI] [PubMed] [Google Scholar]
60.Satar M, Ozlu F. Neonatal sepsis: a continuing disease burden. Turk J Pediatr. 2012;54(5):449–57. Epub 2013/02/23. . [PubMed] [Google Scholar]
61.Molloy EJ, Wynn JL, Bliss J, Koenig JM, Keij FM, McGovern M, et al. Neonatal sepsis: need for consensus definition, collaboration and core outcomes. Pediatr Res. 2020;88(1):2–4. Epub 2020/03/21. doi: 10.1038/s41390-020-0850-5 . [DOI] [PubMed] [Google Scholar]

PLoS One. doi: 10.1371/journal.pone.0295325.r001

Decision Letter 0

Andrew Harding

21 Aug 2023

PONE-D-23-05983Protocol for the development of a Core Outcome Set for Neonatal Sepsis (NESCOS)PLOS ONE

Dear Dr. Taneri,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Given the manuscript reports a robust and well established study design, and the manuscript is well written, as academic editor I am happy to accept your submission without undertaking a formal peer review process. However, you will notice that I am flagging it as a minor revision and I ask that some extremely minor formatting issues are addressed before I can proceed to formally accept the manuscript in the system. For example, I did notice a minor issue - one of the citations (44) in the reference list is incomplete and displayed an error message. I hope you are pleased with this outcome and I look forward to seeing and processing the revised manuscript.

Please submit your revised manuscript by Aug 31 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Andrew Harding, PhD

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

3. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2023 Dec 5;18(12):e0295325. doi: 10.1371/journal.pone.0295325.r002

Author response to Decision Letter 0

30 Aug 2023

Dear editor,

Thank you for your valuable suggestions, which we have now incorporated into the submission. Kindly find below the specific changes we've made as per your recommendations.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

Changes we made: We carefully reviewed the PLOS ONE style templates and made the necessary adjustments to ensure that our manuscript meets the criteria set by the journal.

Changes we made: We added captions for the Supporting Information file at the end of the manuscript and revised the in-text citations to align with the provided guidelines. We took into account the guidelines while making these updates.

Changes we made: We thoroughly reviewed the reference list to confirm its completeness and accuracy. There are no instances of retracted papers within the list. It's important to note that we made no additions or removals of references; our sole focus was on rectifying the reference list itself.

Kind regards,

Petek Eylul Taneri

Attachment

Submitted filename: Response to reviewers.docx

Click here for additional data file.^{(14.6KB, docx)}

PLoS One. doi: 10.1371/journal.pone.0295325.r003

Decision Letter 1

Andrew Harding

10 Nov 2023

PONE-D-23-05983R1Protocol for the development of a Core Outcome Set for Neonatal Sepsis (NESCOS)PLOS ONE

Dear Dr. Taneri,

During our internal assessment of the manuscript, we noted that the consent statement is missing. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified (1) whether consent will be informed and (2) what type you will obtain (for instance, written or verbal, and if verbal, how it will be documented and witnessed). If the need for consent was waived by the ethics committee, please include this information.

Please submit your revised manuscript by Dec 25 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

We look forward to receiving your revised manuscript.

Kind regards,

Johanna Pruller, PhD

Associate Editor

PLOS ONE

on behalf of

Andrew Harding, PhD

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

PLoS One. 2023 Dec 5;18(12):e0295325. doi: 10.1371/journal.pone.0295325.r004

Author response to Decision Letter 1

14 Nov 2023

Response to Reviewers

Thank you for your valuable suggestions.

1. During our internal assessment of the manuscript, we noted that the consent statement is missing. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified (1) whether consent will be informed and (2) what type you will obtain (for instance, written or verbal, and if verbal, how it will be documented and witnessed). If the need for consent was waived by the ethics committee, please include this information.

Changes we made: We carefully revised our submission, and we added this information to our manuscript and online submission information:

“During the Delphi survey and consensus meeting phases, participants were provided with detailed project information via participant information leaflets. They were then cordially invited to partake in the study, contingent upon their agreement as evidenced by completing the standardized consent forms. These forms, which included a thorough briefing, were utilized to obtain written consent from the participants. Integral to our adherence to ethical standards, these documents were incorporated into our submission to the ethical committee and were subsequently granted approval.”

PLoS One. doi: 10.1371/journal.pone.0295325.r005

Decision Letter 2

Andrew Harding

21 Nov 2023

Protocol for the development of a Core Outcome Set for Neonatal Sepsis (NESCOS)

PONE-D-23-05983R2

Dear Dr. Taneri,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Andrew Harding, PhD

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Thank you for the minor changes to your protocol, and we can now accept your manuscript for publication. Good luck with publishing the rest of the work associated with your project.

Reviewers' comments:

PLoS One. doi: 10.1371/journal.pone.0295325.r006

Acceptance letter

Andrew Harding

24 Nov 2023

PONE-D-23-05983R2

Protocol for the development of a core outcome set for neonatal sepsis (NESCOS)

Dear Dr. Taneri:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at customercare@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Andrew Harding

Academic Editor

PLOS ONE

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

S1 Checklist. PRISMA-P (Preferred Reporting Items for Systematic review and Meta-Analysis Protocols) 2015 checklist: Recommended items to address in a systematic review protocol*.

(DOC)

Click here for additional data file.^{(83.5KB, doc)}

S1 File. NESCOS search strategy.

The complete search strategy of the qualitative systematic review of the NESCOS project.

(DOCX)

Click here for additional data file.^{(25.8KB, docx)}

Attachment

Submitted filename: Response to reviewers.docx

Click here for additional data file.^{(14.6KB, docx)}

Data Availability Statement

No datasets were generated or analysed during the current study. All relevant data from this study will be made available upon study completion.

[pone.0295325.ref001] 1.Rudd KE, Johnson SC, Agesa KM, Shackelford KA, Tsoi D, Kievlan DR, et al. Global, regional, and national sepsis incidence and mortality, 1990–2017: analysis for the Global Burden of Disease Study. Lancet. 2020;395(10219):200–11. doi: 10.1016/S0140-6736(19)32989-7 ; PubMed Central PMCID: PMC6970225. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref002] 2.Fleischmann C, Reichert F, Cassini A, Horner R, Harder T, Markwart R, et al. Global incidence and mortality of neonatal sepsis: a systematic review and meta-analysis. Arch Dis Child. 2021. Epub 20210122. doi: 10.1136/archdischild-2020-320217 ; PubMed Central PMCID: PMC8311109. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref003] 3.Bakhuizen SE, de Haan TR, Teune MJ, van Wassenaer-Leemhuis AG, van der Heyden JL, van der Ham DP, et al. Meta-analysis shows that infants who have suffered neonatal sepsis face an increased risk of mortality and severe complications. Acta Paediatr. 2014;103(12):1211–8. Epub 2014/07/31. doi: 10.1111/apa.12764 . [DOI] [PubMed] [Google Scholar]

[pone.0295325.ref004] 4.Cai S, Thompson DK, Anderson PJ, Yang JY. Short- and Long-Term Neurodevelopmental Outcomes of Very Preterm Infants with Neonatal Sepsis: A Systematic Review and Meta-Analysis. Children (Basel). 2019;6(12). Epub 2019/12/07. doi: 10.3390/children6120131 ; PubMed Central PMCID: PMC6956113. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref005] 5.McGovern M, Giannoni E, Kuester H, Turner MA, van den Hoogen A, Bliss JM, et al. Challenges in developing a consensus definition of neonatal sepsis. Pediatr Res. 2020;88(1):14–26. Epub 2020/03/04. doi: 10.1038/s41390-020-0785-x . [DOI] [PubMed] [Google Scholar]

[pone.0295325.ref006] 6.Wynn JL, Wong HR, Shanley TP, Bizzarro MJ, Saiman L, Polin RA. Time for a neonatal-specific consensus definition for sepsis. Pediatr Crit Care Med. 2014;15(6):523–8. Epub 2014/04/23. doi: 10.1097/PCC.0000000000000157 ; PubMed Central PMCID: PMC4087075. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref007] 7.McMullan B, Cooper C, Spotswood N, James R, Jones C, Konecny P, et al. Antibiotic prescribing in neonatal sepsis: an Australian nationwide survey. BMJ Paediatr Open. 2020;4(1):e000643. Epub 2020/04/02. doi: 10.1136/bmjpo-2020-000643 ; PubMed Central PMCID: PMC7101048. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref008] 8.Wynn JL. Defining neonatal sepsis. Curr Opin Pediatr. 2016;28(2):135–40. Epub 2016/01/15. doi: 10.1097/MOP.0000000000000315 ; PubMed Central PMCID: PMC4786443. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref009] 9.Henry CJ, Semova G, Barnes E, Cotter I, Devers T, Rafaee A, et al. Neonatal sepsis: a systematic review of core outcomes from randomised clinical trials. Pediatr Res. 2022;91(4):735–42. Epub 2022/01/09. doi: 10.1038/s41390-021-01883-y ; PubMed Central PMCID: PMC9064797. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref010] 10.Bellucci C, Hughes K, Toomey E, Williamson PR, Matvienko-Sikar K. A survey of knowledge, perceptions and use of core outcome sets among clinical trialists. Trials. 2021;22(1):1–13. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref011] 11.Williamson PR, Altman DG, Bagley H, Barnes KL, Blazeby JM, Brookes ST, et al. The COMET Handbook: version 1.0. Trials. 2017;18(Suppl 3):280. Epub 2017/07/07. doi: 10.1186/s13063-017-1978-4 ; PubMed Central PMCID: PMC5499094. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref012] 12.Higgins JPT, Savović J, Page MJ, Elbers RG, Sterne JAC. Assessing risk of bias in a randomized trial. In: Higgins J, Thomas J, editors. Cochrane Handbook for Systematic Reviews of Interventions. Version 6.4, 2023. [Google Scholar]

[pone.0295325.ref013] 13.Clarke M. Standardising outcomes for clinical trials and systematic reviews. Trials. 2007;8:39. Epub 2007/11/28. doi: 10.1186/1745-6215-8-39 ; PubMed Central PMCID: PMC2169261. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref014] 14.Saldanha IJ, Lindsley KB, Money S, Kimmel HJ, Smith BT, Dickersin K. Outcome choice and definition in systematic reviews leads to few eligible studies included in meta-analyses: a case study. BMC Med Res Methodol. 2020;20(1):30. Epub 2020/02/13. doi: 10.1186/s12874-020-0898-2 ; PubMed Central PMCID: PMC7014938. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref015] 15.Williamson PR, de Avila Oliveira R, Clarke M, Gorst SL, Hughes K, Kirkham JJ, et al. Assessing the relevance and uptake of core outcome sets (an agreed minimum collection of outcomes to measure in research studies) in Cochrane systematic reviews: a review. BMJ Open. 2020;10(9):e036562. Epub 2020/09/09. doi: 10.1136/bmjopen-2019-036562 ; PubMed Central PMCID: PMC7476465. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref016] 16.Kirkham JJ, Gorst S, Altman DG, Blazeby JM, Clarke M, Tunis S, et al. Core Outcome Set-STAndardised Protocol Items: the COS-STAP Statement. Trials. 2019;20(1):116. Epub 2019/02/13. doi: 10.1186/s13063-019-3230-x ; PubMed Central PMCID: PMC6371434. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref017] 17.Keeley T, Williamson P, Callery P, Jones LL, Mathers J, Jones J, et al. The use of qualitative methods to inform Delphi surveys in core outcome set development. Trials. 2016;17(1):230. Epub 2016/05/05. doi: 10.1186/s13063-016-1356-7 ; PubMed Central PMCID: PMC4855446. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref018] 18.Kirkham JJ, Davis K, Altman DG, Blazeby JM, Clarke M, Tunis S, et al. Core Outcome Set-STAndards for Development: The COS-STAD recommendations. PLoS Med. 2017;14(11):e1002447. Epub 20171116. doi: 10.1371/journal.pmed.1002447 ; PubMed Central PMCID: PMC5689835. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref019] 19.Tong A, Flemming K, McInnes E, Oliver S, Craig J. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC medical research methodology. 2012;12(1):1–8. doi: 10.1186/1471-2288-12-181 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref020] 20.Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–9, W64. Epub 20090720. doi: 10.7326/0003-4819-151-4-200908180-00135 . [DOI] [PubMed] [Google Scholar]

[pone.0295325.ref021] 21.Booth A, Noyes J, Flemming K, Moore G, Tunçalp Ö, Shakibazadeh E. Formulating questions to explore complex interventions within qualitative evidence synthesis. BMJ global health. 2019;4(Suppl 1):e001107. doi: 10.1136/bmjgh-2018-001107 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref022] 22.Flemming K, Noyes J. Qualitative evidence synthesis: Where are we at? International Journal of Qualitative Methods. 2021;20:1609406921993276. [Google Scholar]

[pone.0295325.ref023] 23.Noyes J, Booth A, Cargo M, Flemming K, Harden A, Harris J, et al. Qualitative evidence. Cochrane Handbook for Systematic Reviews of Interventions version 63: Cochrane; 2022. [Google Scholar]

[pone.0295325.ref024] 24.Pearson M, Moxham T, Ashton K. Effectiveness of search strategies for qualitative research about barriers and facilitators of program delivery. Evaluation & the health professions. 2011;34(3):297–308. doi: 10.1177/0163278710388029 [DOI] [PubMed] [Google Scholar]

[pone.0295325.ref025] 25.Papaioannou D, Sutton A, Carroll C, Booth A, Wong R. Literature searching for social science systematic reviews: consideration of a range of search techniques. Health Information & Libraries Journal. 2010;27(2):114–22. [DOI] [PubMed] [Google Scholar]

[pone.0295325.ref026] 26.Evans D. Database searches for qualitative research. J Med Libr Assoc. 2002;90(3):290–3. ; PubMed Central PMCID: PMC116400. [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref027] 27.Booth A, Harris J, Croot E, Springett J, Campbell F, Wilkins E. Towards a methodology for cluster searching to provide conceptual and contextual “richness” for systematic reviews of complex interventions: case study (CLUSTER). BMC medical research methodology. 2013;13(1):1–14. doi: 10.1186/1471-2288-13-118 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref028] 28.Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web and mobile app for systematic reviews. Systematic reviews. 2016;5(1):1–10. doi: 10.1186/s13643-016-0384-4 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref029] 29.Critical Appraisal Skills Programme. CASP for Qualitative Studies Checklist 2018. Available from: https://casp-uk.net/wp-content/uploads/2018/03/CASP-Qualitative-Checklist-2018_fillable_form.pdf.

[pone.0295325.ref030] 30.Thomas J, Harden A., Newman M. Synthesis: combining results systematically and appropriately. An Introduction to Systematic Reviews Sage Publications; 2012. [Google Scholar]

[pone.0295325.ref031] 31.Webbe JWH, Duffy JMN, Afonso E, Al-Muzaffar I, Brunton G, Greenough A, et al. Core outcomes in neonatology: development of a core outcome set for neonatal research. Arch Dis Child Fetal Neonatal Ed. 2020;105(4):425–31. Epub 20191115. doi: 10.1136/archdischild-2019-317501 ; PubMed Central PMCID: PMC7363790. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref032] 32.Williamson K. Research methods for students, academics and professionals: Information management and systems: Elsevier; 2002. [Google Scholar]

[pone.0295325.ref033] 33.Sproull NL. Handbook of research methods: A guide for practitioners and students in the social sciences: Scarecrow press; 2002. [Google Scholar]

[pone.0295325.ref034] 34.Thangaratinam S, Redman CW. The delphi technique. The obstetrician & gynaecologist. 2005;7(2):120–5. [Google Scholar]

[pone.0295325.ref035] 35.Cleverley K, McCann E, O’Brien D, Davies J, Bennett K, Brennenstuhl S, et al. Prioritizing core components of successful transitions from child to adult mental health care: a national Delphi survey with youth, caregivers, and health professionals. European Child & Adolescent Psychiatry. 2021:1–14. doi: 10.1007/s00787-021-01806-6 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref036] 36.Hultsjö S, Berterö C, Arvidsson H, Hjelm K. Core components in the care of immigrants with psychoses: A Delphi survey of patients, families, and health‐care staff. International journal of mental health nursing. 2011;20(3):174–84. doi: 10.1111/j.1447-0349.2010.00720.x [DOI] [PubMed] [Google Scholar]

[pone.0295325.ref037] 37.Campbell S, Shield T, Rogers A, Gask L. How do stakeholder groups vary in a Delphi technique about primary mental health care and what factors influence their ratings? BMJ Quality & Safety. 2004;13(6):428–34. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref038] 38.Lambert SD, Ould Brahim L, Morrison M, Girgis A, Yaffe M, Belzile E, et al. Priorities for caregiver research in cancer care: an international Delphi survey of caregivers, clinicians, managers, and researchers. Supportive Care in Cancer. 2019;27(3):805–17. doi: 10.1007/s00520-018-4314-y [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref039] 39.Efstathiou N, Ameen J, Coll A-M. Healthcare providers’ priorities for cancer care: a Delphi study in Greece. European Journal of Oncology Nursing. 2007;11(2):141–50. doi: 10.1016/j.ejon.2006.06.005 [DOI] [PubMed] [Google Scholar]

[pone.0295325.ref040] 40.Santaguida P, Dolovich L, Oliver D, Lamarche L, Gilsing A, Griffith LE, et al. Protocol for a Delphi consensus exercise to identify a core set of criteria for selecting health related outcome measures (HROM) to be used in primary health care. BMC family practice. 2018;19(1):1–14. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref041] 41.Gharibi F, Tabrizi JS. Development of an accreditation model for health education and promotion programs in the Iranian primary healthcare system: a Delphi study. Health Promotion Perspectives. 2018;8(2):155. doi: 10.15171/hpp.2018.20 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref042] 42.Gnatzy T, Warth J, von der Gracht H, Darkow IL. Validating an innovative real-time Delphi approach—A methodological comparison between real-time and conventional Delphi studies. Technol Forecast Soc. 2011;78(9):1681–94. doi: 10.1016/j.techfore.2011.04.006 WOS:000297564200016. [DOI] [Google Scholar]

[pone.0295325.ref043] 43.Gordon T, Pease A. RT Delphi: An efficient, "round-less" almost real time Delphi method. Technol Forecast Soc. 2006;73(4):321–33. doi: 10.1016/j.techfore.2005.09.005 WOS:000236821400001. [DOI] [Google Scholar]

[pone.0295325.ref044] 44.Meyer T, von der Gracht HA, Hartmann E. Technology foresight for sustainable road freight transportation: Insights from a global real‐time Delphi study. Futures & Foresight Science. 2022;4(1):e101. [Google Scholar]

[pone.0295325.ref045] 45.Sunderji N, Waddell A. Using real-time Delphi to develop a consensus on competencies. Med Educ. 2015;49(11):1151–2. doi: 10.1111/medu.12851 WOS:000363514600026. [DOI] [PubMed] [Google Scholar]

[pone.0295325.ref046] 46.Genova JP. Futuring in high performing, high poverty, and high minority schools: A real-time Delphi study: St. John’s University (New York), School of Education and Human Services; 2018. [Google Scholar]

[pone.0295325.ref047] 47.Gary JE. Outlook 2020: Results from a Real-Time Delphi Survey of Global Pentecostal Leaders. Pneuma. 2012;34(3):383–414. [Google Scholar]

[pone.0295325.ref048] 48.Gary JE, von der Gracht HA. The future of foresight professionals: Results from a global Delphi study. Futures. 2015;71:132–45. doi: 10.1016/j.futures.2015.03.005 WOS:000361407800012. [DOI] [Google Scholar]

[pone.0295325.ref049] 49.Serra M, Antonio N, Henriques C, Afonso CM. Promoting Sustainability through Regional Food and Wine Pairing. Sustainability. 2021;13(24):13759. [Google Scholar]

[pone.0295325.ref050] 50.Makovec UN, Goetzinger C, Ribaut J, Barnestein-Fonseca P, Haupenthal F, Herdeiro MT, et al. Developing a medication adherence technologies repository: proposed structure and protocol for an online real-time Delphi study. BMJ open. 2022;12(4):e059674. doi: 10.1136/bmjopen-2021-059674 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref051] 51.Garner BR, Gotham HJ, Knudsen HK, Zulkiewicz BA, Tueller SJ, Berzofsky M, et al. The prevalence and negative impacts of substance use disorders among people with HIV in the United States: A real-time delphi survey of key stakeholders. AIDS and Behavior. 2022;26(4):1183–96. doi: 10.1007/s10461-021-03473-9 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref052] 52.Quirke FA, Healy P, Bhraonáin EN, Daly M, Biesty L, Hurley T, et al. Multi-Round compared to real-time Delphi for consensus in core outcome set (COS) development: a randomised trial. Trials. 2021;22(1):1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref053] 53.Dube K, Ayebare E, Bakhbakhi D, Bedwell C, Chandriah S, Chaudhry N, et al. Protocol for the development of a meta-core outcome set for stillbirth prevention and bereavement care following stillbirth. medRxiv. [Preprint]. 2022. [cited 2023 August 22]. Available from: https://www.medrxiv.org/content/10.1101/2022.10.13.22281030v1 [Google Scholar]

[pone.0295325.ref054] 54.Hsieh CH, Tzeng FM, Wu CG, Kao JS, Lai YY. The Comparison of Online Delphi and Real-Time Delphi. 2011 Proceedings of Picmet 11: Technology Management in the Energy-Smart World (Picmet). 2011. WOS:000298384400042. [Google Scholar]

[pone.0295325.ref055] 55.Aengenheyster S, Cuhls K, Gerhold L, Heiskanen-Schüttler M, Huck J, Muszynska M. Real-Time Delphi in practice—A comparative analysis of existing software-based tools. Technol Forecast Soc. 2017;118:15–27. [Google Scholar]

[pone.0295325.ref056] 56.Varndell W, Fry M, Elliott D. Applying real-time Delphi methods: development of a pain management survey in emergency nursing. BMC nursing. 2021;20(1):1–11. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref057] 57.Needham DM, Sepulveda KA, Dinglas VD, Chessare CM, Friedman LA, Bingham III CO, et al. Core outcome measures for clinical research in acute respiratory failure survivors. An international modified Delphi consensus study. American journal of respiratory and critical care medicine. 2017;196(9):1122–30. doi: 10.1164/rccm.201702-0372OC [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref058] 58.Prinsen CA, Vohra S, Rose MR, Boers M, Tugwell P, Clarke M, et al. How to select outcome measurement instruments for outcomes included in a “Core Outcome Set”–a practical guideline. Trials. 2016;17(1):1–10. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0295325.ref059] 59.Bartlett SJ, Hewlett S, Bingham CO, Woodworth TG, Alten R, Pohl C, et al. Identifying core domains to assess flare in rheumatoid arthritis: an OMERACT international patient and provider combined Delphi consensus. Annals of the rheumatic diseases. 2012;71(11):1855–60. doi: 10.1136/annrheumdis-2011-201201 [DOI] [PubMed] [Google Scholar]

[pone.0295325.ref060] 60.Satar M, Ozlu F. Neonatal sepsis: a continuing disease burden. Turk J Pediatr. 2012;54(5):449–57. Epub 2013/02/23. . [PubMed] [Google Scholar]

[pone.0295325.ref061] 61.Molloy EJ, Wynn JL, Bliss J, Koenig JM, Keij FM, McGovern M, et al. Neonatal sepsis: need for consensus definition, collaboration and core outcomes. Pediatr Res. 2020;88(1):2–4. Epub 2020/03/21. doi: 10.1038/s41390-020-0850-5 . [DOI] [PubMed] [Google Scholar]

PERMALINK

Protocol for the development of a core outcome set for neonatal sepsis (NESCOS)

Petek Eylul Taneri

Jamie J Kirkham

Eleanor J Molloy

Linda Biesty

Richard A Polin

James L Wynn

Barbara J Stoll

Niranjan Kissoon

Kondwani Kawaza

Mandy Daly

Aoife Branagan

Lívia Nagy Bonnard

Eric Giannoni

Tobias Strunk

Magdalena Ohaja

Kenneth Mugabe

Denise Suguitani

Fiona Quirke

Declan Devane

Roles

Abstract

Introduction

Materials and methods

Overview

Fig 1. Schematic of COS development.

Stage 1: Qualitative systematic review

Inclusion criteria

Table 1. The PerSPEcTiF question formulation framework.

Search strategy

Study selection

Quality assessment

Data extraction

Data analysis

Stage 2: Delphi survey

Participants

Procedures

Stage 3: Consensus meetings

Participants

Schedule

Stage 4: Dissemination and implementation strategy

Discussion

Study status

Supporting information

Data Availability

Funding Statement

References

Decision Letter 0

Andrew Harding

Roles

Author response to Decision Letter 0

Decision Letter 1

Andrew Harding

Roles

Author response to Decision Letter 1

Decision Letter 2

Andrew Harding

Roles

Acceptance letter

Andrew Harding

Roles

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases