Automated digital technologies for supporting sepsis prediction in children: a scoping review protocol

Abstract

Introduction

While there have been several literature reviews on the performance of digital sepsis prediction technologies and clinical decision-support algorithms for adults, there remains a knowledge gap in examining the development of automated technologies for sepsis prediction in children. This scoping review will critically analyse the current evidence on the design and performance of automated digital technologies to predict paediatric sepsis, to advance their development and integration within clinical settings.

Methods and analysis

This scoping review will follow Arksey and O’Malley’s framework, conducted between February and December 2022. We will further develop the protocol using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews. We plan to search the following databases: Association of Computing Machinery (ACM) Digital Library, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Embase, Google Scholar, Institute of Electric and Electronic Engineers (IEEE), PubMed, Scopus and Web of Science. Studies will be included on children >90 days postnatal to <21 years old, predicted to have or be at risk of developing sepsis by a digitalised model or algorithm designed for a clinical setting. Two independent reviewers will complete the abstract and full-text screening and the data extraction. Thematic analysis will be used to develop overarching concepts and present the narrative findings with quantitative results and descriptive statistics displayed in data tables.

Ethics and dissemination

Ethics approval for this scoping review study of the available literature is not required. We anticipate that the scoping review will identify the current evidence and design characteristics of digital prediction technologies for the timely and accurate prediction of paediatric sepsis and factors influencing clinical integration. We plan to disseminate the preliminary findings from this review at national and international research conferences in global and digital health, gathering critical feedback from multidisciplinary stakeholders.

Scoping review registration

https://osf.io/veqha/?view_only=f560d4892d7c459ea4cff6dcdfacb086

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• This review is a rigorous approach to collectively synthesising current research on automated paediatric sepsis prediction technologies, critically examining the relationships between their design, performance and clinical integration to identify sociotechnical challenges and research gaps.

• The chosen review strategy will comprehensively evaluate the vast literature across various study types and research disciplines by a multidisciplinary research team.

• The review will exclude digital prediction technologies for paediatric sepsis treatment decisions and is limited to peer-reviewed literature written in the English language with a full-text version available.

• Articles focusing on age cohorts <90 days postnatal or >21 years old will be excluded due to significant differences in sepsis aetiology and clinical presentation.

Introduction

Globally, it is estimated that there were a total of 25.2 million cases of sepsis in children (<19) in 2017, imposing significant healthcare and societal burden.¹ Healthcare costs for severe paediatric sepsis hospitalisations reached approximately US$7.31 billion in the USA in 2016, accounting for almost 20% of total paediatric hospitalisation costs.² However, about 85% of global sepsis cases and 84.8% of sepsis-related deaths among all age groups occur in low–middle-income countries, specifically those in sub-Saharan Africa and South-East Asia.¹ Annual global mortality rates for children (<5) are approximately 2.9 million (table 1).³

Table 1.

Differences between global neonatal, paediatric and adult sepsis³

	Neonatal (<90 days)	Paediatric (<5 years)	Adult (>20 years)
Annual cases (mil)	1.3–3.9	20.3	23.7
Annual mortality (mil)	0.4–0.7	2.9	7.7

Early recognition of sepsis in children is challenging. Unlike adult sepsis, children have different sepsis aetiologies.³ For example, children commonly develop sepsis from pneumonia, diarrhoea, meningitis or viral infections, where abdominal or genitourinary sources are more common in adults.⁴ Differences in aetiology can also be found between childhood and neonatal sepsis, with early-onset neonatal sepsis having a distinct microbial pattern.⁵ Recognising sepsis in children is also significantly more challenging due to maturation-based differences in physiology (including immune system response), limitations in the communication of symptoms and diagnostic modalities.^{4 6 7} Sepsis can lead to life-altering organ dysfunction if not identified quickly in children,⁶ where mortality rates are reduced two-fold if treated within the first hour.⁴ Recognition of sepsis is confounded by the age-based symptom variations within children, such as their differences in blood pressure response, serum lactate levels⁴ and commonalities among other childhood conditions and syndromes like Kawasaki syndrome or bronchiolitis.⁸ This milieu of complex information combined with significant time pressure provides a significant cognitive burden for healthcare professionals to promptly identify the onset of deterioration that can lead to this serious medical condition.

In 2020, updated Paediatric Sepsis Survival guidelines were published calling for the integration of screening standards in healthcare facilities to support rapid identification of sepsis in children⁹ and provide the appropriate antimicrobial therapy at the proper time.^{5 9} Established screening tools such as the Paediatric Early Warning Score may support the timeliness of detecting clinical deterioration in children that can lead to sepsis.¹⁰ Recently, adaptations to the Sequential Organ Assessment Score (SOFA) for paediatric patients and neonates have shown promise in identifying children at risk for mortality with sepsis¹¹; however, it is controversial whether these scores provide value in low-resource environments.^12–14 Development and implementation of algorithms such as the Sepsis Prediction and Optimisation Therapy that can analyse electronic health data in real time to provide a rule-based approach to initiate a physical sepsis screen have also been reported.¹⁵ With the call from the WHO to improve sepsis identification and the potential for data-driven and knowledge-based technologies,^{3 16} digital prediction technologies are becoming more advanced using mathematical, statistical and machine learning techniques to support sepsis prediction using clinical information, symptoms, biomarkers and other signs at the bedside.^17–20 While recent reviews have explored the literature on the effectiveness of digital technologies for adult and neonate sepsis prediction,17 18 21–25 there is currently no review on the design and implementation of these predictive technologies for children. Considering the pathophysiology and aetiology for paediatric sepsis are different from that seen in adults and neonates,²⁶ combined with the lack of widely accessible digital technologies for children compared with adults,²⁷ it is critically important to review the literature on this age cohort.

Prior reviews on sepsis prediction technologies

Recent narrative reviews discuss machine learning-based technologies for adult and paediatric sepsis.^{19 28 29} However, their eligibility criteria focus primarily on adults, with only two¹⁹ or three²⁸ articles on children. One review excluded digital technologies that were not based on ‘modern’ machine learning models,²⁰ and one involved a broad search on infectious disease prediction beyond sepsis.²⁸ Others have also limited their investigations to PubMed/Medline, excluding engineering databases, which may provide greater insight into the design characteristics of digital technologies,19 25 30 31 or they focus exclusively on US hospitals.²⁹

Many systematic and scoping reviews have been rigorous in their search strategy but similar to the identified narrative reviews, report on screening tools and technologies for adult patients while excluding children23 24 27 32–35 and the engineering disciplines.18 22 36–38 Currently published protocols plan to exclude data-driven algorithms³⁷ or only include literature on the application of machine learning,³⁹ which may not capture all research on certain relevant technologies. While there have been systematic reviews on the performance of neonatal sepsis prediction and recognition technologies providing insight into their capabilities,^{18 22} none focus on the specifics of paediatric sepsis.

Current systematic reviews that include the paediatric literature as part of their search strategy are not strictly focused on this patient population,^{21 27 36} having only identified one³⁶ or three^{21 27} related articles specific to children. Other reviews broadly examine early warning systems for paediatric clinical deterioration.^{40 41} We have not identified any systematic or scoping reviews that comprehensively scope the literature on digital paediatric sepsis prediction technology. While one identified protocol aims to capture strategies for early recognition of paediatric sepsis from clinical deterioration, the focus of the review is general strategy effectiveness and does not explicitly include engineering databases, which would describe technical design aspects.³⁸

Purpose of the study

Given the limitations of recent literature reviews and the lack of reviews focused on paediatric sepsis, it is necessary to synthesise the current research describing the development and evaluation of automated sepsis prediction technologies for this underrepresented age cohort. The scoping review defined by this protocol will identify and summarise the existing literature on the design characteristics, performance and integration of automated sepsis prediction technologies in paediatric contexts. The scoping review, a methodology focusing on answering broader research questions through a systematic search and presenting tabular findings along with a narrative integration,⁴² was identified as the best approach for this study. We anticipate that the rigorous methodology will warrant a meaningful summary about the current development of digital technologies for sepsis prediction that can inform future research towards improving their performance and evidence-based clinical implementation to ultimately improve the lives of children globally.

Methods and analysis

The reviewers on this scoping review consist of a multidisciplinary team of engineers, a health researcher/biomedical engineering research librarian, a psychology student and a paediatric clinician. Our methodology will be guided by the framework developed by Arksey and O’Mally,⁴² which iterates through six steps: (1) identifying the research questions; (2) searching for relevant studies; (3) selecting the studies; (4) charting the data; (5) collating, summarising and reporting the results and (6) consulting with stakeholders to inform or validate findings. The sixth step is optional, and we will modify this step to consult with experts specifically around finding technologies used in hospital or industry settings. Levac’s recommendations for independent full-text reviews by at least two reviewers will also be followed.⁴³ This study protocol will follow the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for a scoping review (PRISMA-ScR)⁴⁴ with any gaps being filled by the PRISMA-extension for protocols.⁴⁵ This protocol has been registered on the Open Science Framework (https://osf.io/nh6qz/?view_only=8c840412a2a44117ac16fdf76e06abd6).

Step 1: identifying the research questions

The research questions were developed through an initial search of the literature on automated digital technologies for paediatric sepsis recognition and gaps identified in current systematic and narrative reviews in the neonatal and adult context. The Joanna Briggs Institute recommendations of the Population, Concept and Context model were followed,⁴⁶ maintaining a broad scope for understanding the existing evidence on paediatric sepsis prediction technologies with respect to their current performance, identified outcome measures and existing research gaps:

1. How do the design characteristics of automated paediatric sepsis prediction technologies for healthcare facilities (eg, the recognition task, type, method, demographics and indicators) influence their performance?

2. What are the impacts of clinically implemented automated paediatric sepsis prediction technologies on decision-making and patient outcome measures?

3. What challenges and research gaps (eg, evidence, practical knowledge, population, theoretical, methodological) exist for improving the sociotechnical integration of knowledge-based algorithms and data-driven models for predicting paediatric sepsis in healthcare facilities?

Step 2: identifying relevant studies

We will conduct a comprehensive scoping review that includes a multidisciplinary group of scholarly databases: Association of Computing Machinery (ACM) Digital Library, Cumulative Index to Nursing and Allied Health Literature (CINAHL, Embase, Google Scholar, Institute of Electric and Electronic Engineers (IEEE), PubMed, Scopus and Web of Science. Articles will further be identified using the snowballing technique,⁴⁷ to identify relevant literature among the references and citations of articles included for the full review. We will also hand-search for reports on the design, validation and implementation of commercial digital technologies for sepsis prediction, which may be approved by governing bodies such as Health Canada (health-products.canada.ca/mdall-limh/), the Food and Drug Administration (accessdata.fda.gov/scripts/cdrh/cfdocs/cfrl/textsearch.cfm) and the European Union Medical Device Regulation (ec.europa.eu/tools/eudamed/#/screen/home).

Guided by a University of Waterloo biomedical engineering research librarian, we developed a comprehensive search strategy for each database. The approach employs keywords, medical subject headings (MeSH), key concept subject headings and Boolean terms broken down into the following parts: the recognition algorithm or model, type of digital technology, health condition, alert type, implementation or validation factors and patient population. A sample search strategy for PubMed is presented in table 2.

Table 2.

Sample search strategy and results

Database	Search terms	Results	Date
PubMed	(“decision support”(All Fields] OR “decision-support”(All Fields] OR “early warning score”(MeSH Terms] OR “early warning score”(All Fields] OR “smart system*“(All Fields] OR “electronic alert*“(All Fields] OR “artificial intelligence”(All Fields] OR “artificial intelligence”(MeSH Terms] “machine learning”(All Fields] OR “deep learning”(All Fields] OR “neural network*“(All Fields] OR “support vector machine”(All Fields] OR “hidden markov model”(All Fields] OR “statistical learning”(All Fields] OR “predictive function”(All Fields] OR “algorithm”(All Fields] OR “algorithms”(MeSH Terms] OR “automat*“(All Fields] OR “comput*“(All Fields] OR “decision making, computer assisted”(MeSH Terms] OR “electronic*“(All Fields] OR “representation learning”(All Fields] OR “conformal prediction”(All Fields] OR “random forest”(All Fields] OR “naïve bayes”(All Fields] OR “regression” OR “regression analysis”(MeSH Terms] OR “gradient boosting”(All Fields] OR “artificial learning”(All Fields] OR “machine intelligence”(All Fields] OR “probabilistic network*“(All Fields] OR “knowledge representation”(All Fields] OR “bayesian learning”(All Fields] OR “expert system*“(All Fields] OR “technology assisted”(All Fields] OR “computer assisted”(All Fields] OR “statistical”(All Fields] OR “mathematical”(All Fields)) AND (“system”(All Fields] OR “tool”(All Fields] OR “alert*“(All Fields] OR “technology”(All Fields] OR “software”(All Fields] OR “model*“(All Fields] OR “engine”(All Fields] OR “approach*“(All Fields] OR “algorithm”(All Fields] OR “platform”(All Fields] OR “method*“(All Fields] OR “scor*“(All Fields] OR “device”(All Fields)) AND (“sepsis”(All Fields] OR “sepsis”(MeSH Terms] OR “septic shock”(All Fields] OR “systemic inflammatory response syndrome”(All Fields] OR “acute deterioration”(All Fields] OR “patient deterioration”(All Fields] OR “clinical deterioration”(MeSH Terms] OR “clinical deterioration”(All Fields] OR “severe infection”(All Fields] OR “severe bacterial infection”(All Fields] OR “bacterial infections”(MeSH Terms] OR “febrile illness”(All Fields] OR “non-malarial febrile illness”(All Fields] OR “bacteremia”(All Fields)) AND (“diagnos*“(All Fields] OR “detect*“(All Fields] OR “predict*“(All Fields] OR “prognosticate”(All Fields] OR “identif*“(All Fields] OR “infer*“(All Fields] OR “warn*“(All Fields] OR “alert*“(All Fields] OR “recog*“(All Fields] OR “screen*“(All Fields] OR “monitor*“(All Fields] OR “assess*“(All Fields] OR “surveillance”(All Fields] OR “classif*“(All Fields]) AND (“evaluat*“(All Fields] OR “implement*“(All Fields] OR “perform*“(All Fields] OR “design”(All Fields] OR “validat*“(All Fields] OR “usability”(All Fields] OR “effectiveness”(All Fields] OR “efficiency”(All Fields] OR “satisfaction”(All Fields] OR “safety”(All Fields] OR “acceptance”(All Fields] OR “clinical value”(All Fields] OR “interpret*“(All Fields] OR “perception”(All Fields] OR “perspective”(All Fields] OR “opinion”(All Fields] OR “error”(All Fields]) AND (“child*“(All Fields] OR “paediatric”(All Fields] OR “pediatric”(All Fields] OR “pediatrics”(MeSH Terms] OR “toddler*“(All Fields] OR “teen*“(All Fields] OR “youth*“(All Fields] OR “adolescen*“(All Fields] OR “adolescent”(MeSH Terms] OR “infan*“(All Fields] OR “infant”(MeSH Terms] OR “school age*“(All Fields] OR “PICU”(All Fields]) LIMIT TO: [Text Availability): Full text, [Language): English, [Species): Human	15 531	02/15/2022

The search results will be imported to Mendeley’s reference management software for future referencing and organisation (Mendeley). A systematic review management software, Covidence (Veritas Health Innovation), will be used to identify and merge duplicate articles. A sample of 20 abstracts will be initially screened by two reviewers (RT and JG), ensuring that the inclusion–exclusion requirements are robust in capturing relevant articles related to the design and evaluation of automated prediction technologies for paediatric sepsis. Both reviewers will also ensure that the data extraction items capture valuable and appropriate study details from the articles included in the full-text review, which will be shared with the research team.

Step 3: study selection

Inclusion criteria

The proposed review will include articles that meet the following inclusion criteria:

• The article is written in English.

• The article is a peer-reviewed journal article, full conference proceeding or research published on a commercially available digital technology, which may be approved by a medical device regulatory body.

• Following the American Academy of Pediatrics’ definition for late adolescence, more than the majority of data reported will include children aged >90 days postnatal to <21 years old.⁴⁸

• The article is about an automated data-driven or knowledge-based approach towards paediatric sepsis prediction in a healthcare setting, including sepsis risk, severe sepsis, septic shock or sepsis mortality risk.

• The digital technology is evaluated for its performance through validation testing, experiments or an observational study.

• There is no specification for publication years.

Exclusion criteria

Screened articles that fit within the following categories will be excluded from this review: Commentaries, dissertations, editorials, books and book chapters, lectures and addresses, study protocols, review articles and articles inaccessible for full-text review after using library resources. Articles that describe digital technologies informing sepsis treatment strategy selection are outside the scope of this review, because this study is focused on technologies supporting clinical decision-making and screening that occurs before fluid resuscitation or antibiotic selection for confirmed sepsis patients. Digital technologies developed for at-home use are also outside the scope of this review, as the context of the protocol is to review the evidence on automated sepsis prediction technologies in regulated healthcare settings.

Selection process

This review will follow the reporting checklist in the PRISMA-ScR, provided by Tricco et al.⁴⁴ First, all relevant articles will be imported into Covidence. Second, two reviewers (RT and JG) will independently perform the title and abstract screening using the developed eligibility criteria by classifying them as ‘yes’, ‘no’ or ‘maybe’. Any article classified as ‘yes’ or ‘maybe’ by RT or JG will be included in the full-text review during this stage by adding them to an Excel spreadsheet for access by all authors. If a full-text article cannot be accessed, the reviewers will seek assistance from library services at the institution or directly contact the article’s corresponding author. Third, two investigators (RT and JG) will independently perform the full-text screening for eligibility using the listed inclusion–exclusion criteria. A third member of the research team will resolve any disagreements on eligibility that occur during the full-text review. After the full-text review, an inter-rater agreement will be calculated using Cohen’s kappa coefficient (κ) statistic.

The first step in identifying relevant studies was performed on 15 February 2022. The planned end date for completing the full-text screening and analysis is 30 December 2022. We have maintained search alerts for potentially eligible articles to ensure our review remains updated before dissemination through publication.

Step 4: charting the data

The data extraction form will be developed in Covidence and exported to Excel to capture the relevant information from each article. Two reviewers (RT and JG) will individually extract the relevant data from a sample of eligible articles screened for inclusion in the full-text review to ensure consistency of recording data. Any disagreements on extracted data will be resolved through discussion between the reviewers. The form will be iteratively updated until the authors reach a consensus on the relevant data to extract. We will begin by pulling the following type of data into the form, with additional data included as we screen more articles:

• Article information: author(s), year published, city, country, discipline(s).

• Prediction task: the definition of sepsis being identified and the use context for recognition in paediatrics.

• Prediction task type:

  • Alerting automation that provides a notification that a patient has met the objective sepsis recognition criteria.

  • Decision support automation that provides assistance in the diagnosis of sepsis.

  • Data automation that collects clinically relevant cues and information on behalf of the user(s), which may be used in combination with alerting and decision support.

• Prediction method:

  • Data-driven methods that use retrospective data sets to build a statistical or machine learning-based model.

  • Knowledge-based methods that use consensus criteria to build an algorithm with threshold-based criteria.

• Participant demographics: age cohort, number of participants.

• Prediction indicators: vital signs, biomarkers, sociodemographics, prior treatments, medical history.

• Prediction interface: audible alert, dialogue box, provided information.

• Validation measures:

  • Reported number of true positives, false positives and false negatives.

  • Reported sensitivity and specificity.

  • Time to accurate sepsis recognition by the technology and/or the clinician.

  • Measured or expected impact on clinical decisions and patient outcomes.

  • Generalisability of the digital technology in the context of bias, fairness and appropriateness.^{49 50}

Step 5: collating, summarising and reporting the results

The extracted data will be synthesised within tables that summarise the current digital technology landscape in predicting paediatric sepsis, including characteristics that describe their performance and the sociotechnical factors of their integration by healthcare providers on patient outcomes. Within summary tables, we will present the current approaches towards model and algorithm development for automated sepsis prediction technologies, including the predictive indicators, the prediction timing objective and how they interface with clinicians. Quantitative performance and implementation measures such as sensitivity and specificity, and the impacts on intervention timing will also be reported in data tables, including calculations of precision, recall and F1 score, when possible.

We will then perform a thematic analysis to identify concepts related to our research questions. This analysis will be presented as a narrative, including an organisation of themes on the identified design characteristics of automated prediction technologies integrated within clinical contexts. The purpose of the analysis will be to identify the types of research gaps that exist for knowledge-based algorithms and data-driven models to improve sociotechnical integration (ie, supporting clinical decision-making) and patient outcomes. Challenges with bias, fairness and appropriateness will also be qualitatively examined with respect to potential generalisability barriers. Diagrams will be developed for the identified relationships and themes among the design characteristics of the automated technologies for paediatric sepsis prediction and their influence on system performance and implementation throughout time to visually highlight the opportunities for future investigations.

Step 6: methodological quality appraisal

We will consult with experts in automated paediatric sepsis prediction technologies for this review to identify those applied in clinical settings. While critical appraisal of the identified articles is not mandatory in the scoping review methodology, we will consult with stakeholders to inform and validate our findings.

Patient and public involvement

There were no patients or public involvement in the development of this protocol.

Ethics and dissemination

Approval from an ethics review committee is not required for this study because it is a scoping review of previously published literature. Once the review is completed, we plan to disseminate the preliminary findings at national and international research conferences in global and digital health to gather critical feedback from researchers and the public. The finalised results from the review will be submitted for publication in an open access peer-reviewed journal.

Discussion

This scoping review will provide a comprehensive and structured understanding of the automated digital technologies that have been developed to support the timely prediction of paediatric sepsis. At a high-level, the results will focus on design characteristics, performance validation and current sociotechnical integration factors, which will be analysed thematically and reported in data summary tables, indicating how the development of these technologies is evolving throughout time. It is anticipated that the outcomes will reveal the current challenges in developing and implementing clinically meaningful digital prediction technologies for paediatric sepsis across various clinical environments. Furthermore, the results are expected to identify critical research aspects requiring further investigation.

Compared with previous articles, this scoping review focuses on the complexities of paediatric sepsis, with a methodological strength in taking a comprehensive and systematic approach that will provide an overview of the evidence in this digital technology landscape. Inherent in the approach of a scoping review is the limitation of its objective: to summarise the literature and identify meaningful gaps for further research. As this study will include articles with various study designs, it does not aim to answer specific questions about recommending the use or application of certain sepsis prediction technologies for paediatrics. With the results of the pilot search (table 2), this review is also limited in its scope, where non-English articles or articles without a full-text version will not be included. Finally, digital technologies informing treatment strategies for sepsis and studies looking at age cohorts <90 days postnatal or >21 years old will be excluded because of significant differences in sepsis aetiology and clinical presentation, while capturing literature from geographic areas that provide paediatric healthcare services to this age range. We plan to adequately convey the overall strengths and limitations once the full-text review is completed, including any deviations from the protocol, in the final review.

In conclusion, by mapping the attributes of paediatric sepsis prediction technologies to outcomes related to clinical integration and performance, we anticipate that our results will highlight critical research gaps among the medical, engineering and computer science disciplines. The results may inform research on identifying relevant predictive indicators best suited for the design of digital technologies in specific use contexts and environments, improvements towards model development for sepsis prediction and factors supporting the optimal workflow integration of digital prediction systems by clinicians. Ultimately, this review will be critical for advancing knowledge to improve sepsis prediction for children globally.

Ethics statements

Patient consent for publication

Not applicable.