Abstract
Several triage systems have been developed, but little is known about their performance in low-resource settings. Evaluating and comparing novel triage systems to existing triage scales provides essential information about their added value, reliability, safety, and effectiveness before adoption. This study included children aged < 15 years who presented to the emergency departments of two public hospitals in Kenya between February and December 2021. We compared the performance of Emergency Triage Assessment and Treatment (ETAT) guidelines and Smart Triage (ST) models (ST model with independent triggers, and recalibrated ST model with independent triggers) in categorizing children into emergency, priority, and non-urgent triage categories. Sankey diagrams were used to visualize the distribution of children into similar or different triage categories by ETAT and ST models. Sensitivity, specificity, negative and positive predictive values for mortality and admission were calculated. 5618 children were enrolled, and the majority (3113, 55.4%) were aged between one and five years of age. Overall admission and mortality rates were 7% and 0.9%, respectively. ETAT classified 513 (9.2%) children into the emergency category compared to 1163 (20.8%) and 1161 (20.7%) by the ST model with independent triggers and recalibrated model with independent triggers, respectively. ETAT categorized 3089 (55.1%) children as non-urgent compared to 2097 (37.4%) and 2617 (46.7%) for the respective ST models. ETAT classified 191/395 (48.4%) admitted patients as emergencies compared to more than half by all the ST models. ETAT and ST models classified 25/49 (51%) and 39/49 (79.6%) deceased children as emergencies. Sensitivity for admission and mortality was 48.4% and 51% for ETAT and 74.9% and 79.6% for the ST models, respectively. Smart Triage shows potential for identifying critically ill children in low-resource settings, particularly when combined with independent triggers and performs comparably to ETAT. Evaluation of Smart Triage in other contexts and comparison to other triage systems is required.
Author summary
Prioritizing children according to the level of severity of illness in the outpatient department is crucial to ensure very sick children are identified and receive life-saving treatment while those with less severe symptoms can safely wait in the queue. Appropriate triage prevents avoidable paediatric mortality. As new triage systems are developed, it is essential to evaluate their performance before being used by healthcare professionals to manage patients. In this study, we compared a newly developed triage algorithm, Smart Triage, to the World Health Organization’s Emergency Triage Assessment and Treatment (ETAT) guidelines. Here, we highlight how participants were categorised into emergency, priority, and non-urgent categories by both triage systems. We also assessed changes in triage categorization by comparing the Smart Triage model with independent emergency and priority triggers (with and without site specific recalibration) with ETAT. Our study shows that Smart Triage had comparable performance to ETAT, and it can be used to triage children in resource-limited settings. Smart Triage can be integrated into a digital device allowing frontline healthcare workers to rapidly triage children presenting to the outpatient department and recognize very sick children faster, so that they can be treated in a timely manner.
Introduction
Overcrowding is a major global problem in many emergency departments (ED) [1,2]. This is compounded by lack of validated triage systems that can help health workers distinguish between low and high-priority patients, along with poor adherence to existing triage protocols [3]. Triage is a vital component of effective and efficient emergency care for children presenting to health facilities with varying severity of illness. Triage categorizes patients according to the severity of their illness and designates a level of urgency that includes non-urgent, priority, and emergency cases that require immediate medical attention [4].
Rapid triage of critically ill patients can reduce the waiting time for children needing life-saving treatment by distinguishing them from non-urgent patients who can safely wait in the queue for assessment and treatment by health workers [5,6]. However, in resource-limited settings, triage remains underused, especially in paediatric emergency care, owing to significant barriers such as inadequate staffing, the complexity of guidelines, social and organizational context, and lack of capacity [7–9]. Consequently, the sickest children are not prioritized, resulting in delayed care and inefficient resource utilization [10].
Several paediatric triage systems have been developed and adopted in high and low-income settings to help frontline health workers improve triage accuracy [11–15]. One of the most widely implemented triage systems is the World Health Organization’s (WHO) Emergency Triage Assessment and Treatment (ETAT) guidelines, which are recommended for use in resource-constrained settings [16, 17]. The ETAT system provides a systematic and objective approach to triaging children using clinical signs to identify emergency, priority, and non-urgent cases. However, the implementation of ETAT in clinical practice has faced a myriad of challenges, including a high turnover of trained staff, lengthy and high-intensity training, and limited resources to support implementation [18,19].
A new paediatric triage model, Smart Triage, was recently developed based on the data collected from a hospital in Uganda [20]. This is a logistic regression model based on nine variables that were selected from over 100 demographic, vital sign measurements, symptoms, and socio-demographic variables collected upon arrival at the hospital. This model provides the probability of admission. The prediction model addresses some of the challenges associated with the implementation of ETAT by utilizing a limited number of predictors to identify critically ill children and can be used by frontline health workers with limited training and expertise. Moreover, the model can be integrated into digital health platforms to minimize the need for memorization of triage protocols and training of triage staff. To facilitate its use in clinical settings, the triage algorithm incorporates a set of independent emergency and priority triggers based on ETAT triage guidelines [20,21]. These triggers include rare emergency conditions and ensure that children who would have been misclassified by the model are safely assigned to either emergency or priority categories [16,20].
It is imperative to evaluate the performance of new triage systems and, if possible, to compare them with existing systems before clinical adoption. Assessment of a triage system’s performance should be based on its ability to differentiate between low and high acuity patients as they present to the ED, thus minimizing misclassification of patients. Misclassification of low acuity patients into the high acuity category misuses scarce resources and increases the waiting time for more urgent patients. On the other hand, misclassification of high acuity patients into a low acuity category may result in delayed assessment, diagnosis, and treatment, and potentially poorer outcomes [4,22]. Compared with existing paper-based guidelines, electronic triage systems provide valuable information to policymakers and health workers regarding the safety, reliability, and effectiveness of a triage system within a particular setting. Furthermore, potential research gaps and areas of improvement can be identified during the evaluation.
The purpose of this study was to compare the performance of ETAT to Smart Triage in classifying children into triage categories using prospectively collected paediatric data from two Kenyan public hospitals. This comparison can provide validation and performance trade-offs of the new triage model compared with the ETAT algorithm and inform its utility in low-resource settings.
Materials and methods
Study design
This was a secondary analysis that utilised baseline data of a multisite clinical study that aimed at developing and exploring the use of a paediatric rapid sepsis trigger (PRST) tool. The study was carried out at the emergency department (ED) of Mbagathi County Hospital and Kiambu County Referral Hospital in Kenya between January 2021 and December 2021. The detailed design and methods of the primary study have been described elsewhere [23].
Study setting
Mbagathi County Hospital and Kiambu County Referral Hospital are first-level referral hospitals located in Nairobi and Kiambu counties, respectively. The outpatient departments in both hospitals serve approximately 20,000 children annually. Each ED is managed by a qualified nurse who triages children and an additional nurse who administers treatment in the emergency room. In addition, one or two clinical officers (equivalent to physician assistants) provide consultation and decide on the appropriate management of the children. Children triaged as emergency cases by hospital staff are transferred directly to the emergency room, while the rest wait in the queue. Both hospitals admit children to a paediatric ward where provision of care is led by a paediatrician and the clinical team consists of medical officers, nurses, and medical and clinical officer interns. Children who are critically ill and require intensive care are referred to a tertiary hospital for specialized care.
Population, eligibility, and study procedures
All children aged < 15 years who presented to the ED with an acute illness were eligible to participate. Children scheduled for immunization, elective surgery, wound dressing changes, or clinical review appointments were excluded. Children presenting to the emergency department with an acute illness on weekdays between 8 am and 5 pm were screened for eligibility by study timekeepers and given a sticker showing their arrival time. A systematic sampling method based on a 30-minute time cut-off was adopted. Research nurses assessed patients’ eligibility and determined whether they met the criteria for inclusion in the study. If the first patient in the time cut-off was ineligible, the next patient in the same cut-off was evaluated based on their order of arrival at the hospital. Adherence to the sampling procedure was monitored by a trained nurse who acted as the study coordinator. Written informed consent was obtained from the caregiver or parent of the first eligible patient at each time cutoff, and assent was obtained from children aged > 13 years. Informed consent and data collection for patients in need of emergency care were deferred, and obtained after the child was stable, to avoid delays.
The study nurses performed clinical examinations and collected data using a password-protected custom-built android application installed on a Samsung Galaxy A8 tablet. Heart rate (HR) and blood oxygen saturation (SpO2) were recorded using a Masimo iSpO2 pulse oximeter connected to the tablet, and respiratory rate was measured using a version of RRate [24] built directly into the Android data collection application. The children were then reviewed by a qualified clinician who independently decided on the appropriate management. The study nurses recorded hospital outcomes including patient disposition from the hospital records and uploaded the data to a secure REDCap database [25] hosted on the KEMRI Wellcome Trust Research Programme (KWTRP) server. Children who were sent home from the ED on the day of enrolment and those who were admitted were followed up via a telephone call seven days after the initial visit or after discharge from the hospital to ascertain the outcome.
Sample size
The sample size for this study was not calculated since it was a secondary analysis of the data collected in the baseline phase of the main study. However, the primary study had a minimum sample size of 833 for this phase [23].
Triage systems
ETAT
This triage system uses clinical signs to assign a triage category depending on the level of illness severity. Frontline health workers identify emergency signs using an “ABCD” method, where A and B symbolize airway and breathing problems, C represents circulation, convulsions, and coma, while D denotes severe dehydration. Children presenting to the hospital with life-threatening problems that require immediate life-saving treatment are assigned to the emergency category, while children requiring urgent review by the health worker (from a set of clinical signs) are assigned a priority category. All other children are classified as non-urgent and can safely wait in the queue (Table 1).
Table 1. Clinical signs and categories of the ETAT triage system and how they were mapped to our dataset.
| Presenting clinical signs used in ETAT triage | Presenting clinical signs available in the study data | Intervention |
|---|---|---|
| Emergency | Requires immediate treatment | |
| Obstructed/absent breathing | Not available in the baseline dataset | |
| Central cyanosis | Cyanosis | |
| Severe respiratory distress | O2 saturation < 90% OR cyanosis OR grunting OR stridor. | |
|
Circulation Capillary refill >3 seconds AND Weak and fast (or absent) pulse AND Cool skin |
Circulation Capillary refill >3 seconds AND (Weak central pulse OR Weak radial pulse) AND Cool skin |
|
| Convulsions | Convulsions (now) | |
| Coma | Not alert (based on AVPU scale) | |
|
Severe dehydration Diarrhoea plus any two positive signs (Lethargy, sunken eyes, unable to drink or drinks poorly, slow skin pinch) |
Severe dehydration Diarrhoea plus any two positive signs (sunken eyes, can’t sit or drink, slow skin pinch) |
|
| Priority | Requires prompt assessment | |
| Tiny infant (age < 2 months) | Tiny infant (age < 2 months) | |
| Temperature ≥37.5°C | Temperature ≥37.5°C | |
| Trauma | Trauma | |
| Severe pallor | Pallor | |
| Severe pain | Severe pain | |
| Poisoning | Poisoning | |
| Respiratory distress | Chest indrawing OR fast breathing (age < 2 months = RR > 60 breaths per minute age 2–11 months = RR > 50 breaths per minute age ≥ 12 months = RR > 40 breaths per minute) |
|
| Urgent referral | Urgent referral | |
| Restless, continuously irritable or lethargic | Irritable | |
| Malnutrition—visible severe wasting | Visible severe wasting | |
| Oedema of both feet or face | Oedema | |
| Burns | Burns | |
| Non-urgent | Waits in the queue | |
| A child without any of the above signs. | A child without any of the above signs | |
Smart Triage
This logistic regression model incorporates nine predictors (five continuous and four categorical variables) implemented in a digital device [22]. The predictors are age, temperature, heart rate, transformed oxygen saturation, mid-upper arm circumference (MUAC), difficulty breathing, pallor, oedema, and parental concern. The selection of variables to be included in the model’s development was informed by guidelines established by the Pediatric Sepsis Predictor Standardization (PS2) working group through a rigorous process involving a modified Delphi method and systematic review [26]. The model uses low-risk and high-risk thresholds to categorize children into three triage categories (emergency, priority, and non-urgent). The original model has a low-risk threshold of 8% and high-risk threshold of 40%. These thresholds were selected to ensure the model had a high specificity and sensitivity of at least 90%, thus ensuring the detection of high-risk cases and identification of emergency cases, ultimately prioritizing these cases and minimizing delays in providing critical treatment [20]. This model was previously recalibrated using the recalibration in the large method using data from Mbagathi County Hospital, resulting in new thresholds of 2.6% and 13% for low-risk and high-risk thresholds, respectively (manuscript under review). This recalibration method involved re-estimating the intercept of the original model without changing the coefficients of the predictor variables [27]. These thresholds were selected to ensure that the model had sensitivity > 80% for identifying high-risk patients. Furthermore, independent emergency and priority triggers (Table 2) were included in a mobile application for the Smart Triage model to allow for its safe clinical implementation. The recalibrated model with independent triggers is currently being implemented and evaluated at the Mbagathi County Hospital in Kenya.
Table 2. Independent emergency and priority triggers included in the Smart Triage.
| Emergency triggers | Priority triggers |
|---|---|
| • Unresponsive • Convulsion • Shock (Cool hands with Capillary refill > 3 sec or Weak and fast pulse) • Major trauma • Severe pain • Not breathing • Obstructed breathing • Central cyanosis • Dehydration (At least 2 of sunken eyes, skin pinch taking longer than 2 seconds, or lethargy) • Heart Rate (HR < 45 bpm) • Oxygen Saturation (SpO2 < 90%) |
• Trauma or other injury • Burn • Poisoning • Urgent referral • Difficulty breathing (such as Not eating or drinking due to respiratory problems, Chest indrawing, Accessory muscle use, or Head nodding) • Irritable • Respiratory Rate (RR > 60 bpm) • Temperature (Temp > 40°C or Temp < 35°C) • MUAC < 115mm |
Data analysis
Using data collected from children at Mbagathi County Hospital and Kiambu County Referral Hospital, demographic characteristics were summarized as frequencies, percentages, proportions, medians, and corresponding interquartile ranges (IQR). A classification table was used to compare the distribution of participants into three triage categories according to the ETAT guidelines: the ST model with independent triggers, and the recalibrated ST model with independent triggers. The change in the classification of the participants into different triage categories was visualized using Sankey diagrams. Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) with 95% confidence interval (CI) for admission and mortality were assessed by creating two groups from each triage system: high acuity (emergency) and low acuity (priority and non-urgent). Smart Triage categories were calculated in Microsoft Excel 2016 (Microsoft, Richmond WA) and then data transferred to R 4.3.0 (R Foundation for Statistical Computing, Vienna, Austria) for final statistical analysis.
Ethics statement
The study was approved by the Kenya Medical Research Institute (KEMRI) Scientific and Ethics Review Unit (SERU/3958) and Institutional Review Boards at the University of British Columbia in Canada (ID: H19-02398; H20-00484).
Results
Participant characteristics
A total of 5920 children (Fig 1) were evaluated for eligibility between January 2021 and December 2021, of whom 5618 (94.9%) were enrolled in the study. A total 3041 (54.1%) participants were male, and the median age was 20.7 months (IQR 9.0–42.0). Of the enrolled participants, 383 (6.8%) were admitted on the day of enrolment and 11 (0.2%) were readmitted within 48 hours after discharge from the ED (Fig 1). The median length of hospital stay was 6 days (IQR 3–8) (Table 3). Cough was the most common presenting complaint among the participants (1647, 29.4%), while pneumonia was the primary reason for admission (220, 57.4%) (Table 3). Overall, 49 (0.9%) participants died during the study period (Table 3).
Fig 1. Participant flow diagram.
Table 3. Participant characteristics.
| Patient characteristics | n (%) |
|---|---|
| Enrolled | 5618 |
| Gender | |
| Male | 3041 (54.1) |
| Female | 2577 (45.9) |
| Age | |
| Age in months (Median, IQR) | 20.7 (9.0–42.0) |
| < 1 month | 182 (3.2) |
| 1–12 months | 1653 (29.4) |
| 1–5 years | 3113 (55.4) |
| 5–12 years | 605 (10.8) |
| >12 years | 65 (1.2) |
| Duration of symptoms in days (Median, IQR) | 3 (2–5) |
| Length of hospital stay in days (Median, IQR) | 6 (3–8) |
| Patient disposition | |
| Discharged from the outpatient department | 5222 (93) |
| Admitted on the day of enrolment | 383 (6.8) |
| Missing hospital outcome | 13 (0.2) |
| Mortality during the study period | 49 (0.9) |
| Primary presenting complaint a | |
| Cough | 1647 (29.4) |
| Fever | 903 (16.1) |
| Nasal Congestion | 489 (8.7) |
| Difficulty breathing | 376 (6.7) |
| Diarrhea | 321 (5.7) |
| Vomiting | 298 (5.3) |
| Abdominal pain | 241 (4.3) |
| Skin rash | 224 (4.0) |
| Trauma | 217 (3.9) |
| Swelling | 148 (2.6) |
| Other (e.g., headache, feeding poorly, jaundice, constipation, runny nose etc.) | 752 (13.4) |
| Admission diagnosis b | |
| Pneumonia | 220 (57.4) |
| Dehydration | 30 (7.8) |
| Jaundice | 27 (7) |
| Convulsions | 19 (5) |
| Neonatal sepsis | 13 (3.4) |
| Meningitis/encephalitis or other central nervous system (CNS) infection | 12 (3.1) |
| Malnutrition | 10 (2.6) |
| Malaria | 6 (1.6) |
| Gastroenteritis/Diarrhea | 6 (1.6) |
| Septicaemia | 6 (1.6) |
| Bronchiolitis | 4 (1.0) |
| Any skin or soft tissue infection | 4 (1.0) |
| Reactive Airway Disease/Asthma | 1 (0.3) |
| Other (e.g., intestinal obstruction, hepatomegaly etc.) | 25 (6.5) |
a Percentage is based on the total number of participants enrolled. b Percentage is based on the total number of admissions.
Triage assignment
Of the total number of participants enrolled, 513 (9.2%), 1163 (20.8%), and 1161 (20.7%) were identified as emergency cases by the ETAT, ST model with independent triggers, and recalibrated ST model with independent triggers, respectively (Table 4). ETAT categorized the majority of children, 3089 (55.1%), into the non-urgent triage category, unlike the ST model with independent triggers, and recalibrated ST model with independent triggers, which assigned 2097 (37.4%) and 2617 (46.7%) children, respectively, into the non-urgent category. The ST model with independent triggers had a higher proportion of children classified into the priority category, 2373 (42.3%) compared to ETAT and the recalibrated ST model with independent triggers, which only classified 2003 (35.7%) and 1827 (32.6%) children into this category, respectively (Table 4). The ST-only model assigned the highest proportion of children, 2373 (42.3%), to priority category compared to the other triage scales (S1 Table). The emergency category had the highest proportion of children who were admitted in all triage systems. For children who died, both the ST model with independent triggers and the recalibrated ST model with independent triggers identified 39 (79.6%) as emergency cases, whereas ETAT and the ST- only model identified 25 (51.0%) as such (S1 Table).
Table 4. Distribution of participants and outcomes by triage system.
| Triage system | ETAT | Smart Triage model with independent triggers | Recalibrated Smart Triage model with independent triggers | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Emergency | Priority | Non-urgent | Emergency | Priority | Non-urgent | Emergency | Priority | Non-urgent | |
| Participants n (%) |
513 (9.2) |
2003 (35.7) |
3089 (55.1) | 1163 (20.8) |
2345 (41.8) | 2097 (37.4) | 1161 (20.7) |
1827 (32.6) | 2617 (46.7) |
| Admission distribution n (%) |
191(37.2) | 168(8.4) | 36(1.2) | 296(25.5) | 83(3.5) | 16(0.8) | 296(25.5) | 74(4.1) | 25(1.0) |
| Mortality on the day of enrolment n (%) |
4(0.7) | 2(0.1) | 0(0) | 5(0.4) | 1(0) | 0(0) | 5(0.4) | 1(0) | 0(0) |
| In-hospital mortality n (%) |
13(2.5) | 8(0.4) | 0(0) | 18(1.5) | 3(0.1) | 0(0) | 18(1.6) | 3(0.2) | 0(0) |
| Mortality during follow-up n (%) |
8(1.6) | 12(0.6) | 2(0.1) | 16(1.4) | 6(0.3) | 0(0) | 16(1.4) | 6(0.3) | 0(0) |
Change in participant classification
Overall
Of the participants classified as non-urgent by ETAT, 20.4%, and 12.6% were reclassified as priority by the ST model with independent triggers, and recalibrated ST model with independent triggers, respectively (Fig 2). Interestingly, 656 (11.7%) of the children who had been categorized as priority by ETAT were classified as emergency by both ST models with independent triggers. ETAT identified 54 (1%) children as non-urgent, but ST models with independent triggers classified them as emergency (Fig 2). Only 7 (0.1%) and 10 (0.2%) children were classified as emergency by ETAT and were reclassified as non-urgent by the ST model with independent triggers, and recalibrated ST model with independent triggers, respectively. A total of 199 (3.6%) and 278 (5%) children who were classified as priority by ETAT and were categorized as non-urgent by the ST model with independent triggers, and recalibrated ST model with independent triggers, respectively. Additionally, 55 (1%) and 52 (0.9%) children who were classified as emergency by ETAT were categorized as priority by the ST model with independent triggers, and recalibrated ST model with independent triggers, respectively (Fig 2). A total of 3488 (62.2%) and 3849 (68.7%) children were classified into the same triage category by ETAT and ST model with independent triggers, and recalibrated ST model with independent triggers, respectively (Fig 2).
Fig 2. All children.
Change in the classification of children presenting to the emergency department by triage systems.
Admission
Of ETAT’s priority cases, 102/395 (25.8%) of the admitted patients were classified as emergency by both ST models with independent triggers, respectively (Fig 3). A total of 20 (5.1%) and 15 (3.8%) children who were admitted were classified as non-urgent by ETAT were categorized as priority by the ST model with independent triggers, and recalibrated ST model with independent triggers, respectively. Neither the ST model with independent triggers nor the recalibrated ST model with independent triggers categorized any of the ETAT emergency cases as priority. Overall, the majority of admitted patients were classified into the same triage category using the ETAT and ST models (Fig 3). ETAT demonstrated a sensitivity of 48.4% (95% CI: 43.5% - 53.3%) for determining hospital admission while both ST models with independent triggers achieved a sensitivity of 74.9% (95% CI: 70.4% - 79%) (Table 5).
Fig 3. Admitted children.
Change in the classification by triage systems for admitted children.
Table 5. Diagnostic tests for hospital admission.
| ETAT | Smart Triage model with independent triggers | Recalibrated Smart Triage model with independent triggers | |
|---|---|---|---|
| Sensitivity (95% CI) | 48.4% (43.5% - 53.3%) | 74.9% (70.4% - 79%) | 74.9% (70.4% - 79%) |
| Specificity (95% CI) | 93.8% (93.1% - 94.4%) | 83.4% (82.3% - 84.3%) | 83.4% (82.3% - 84.3%) |
| PPV (95% CI) | 37.2% (33.2% - 41.5%) | 25.5% (23% - 28%) | 25.5% (23% - 28%) |
| NPV (95% CI) | 96% (95.4% - 96.5%) | 97.8% (97.3% - 98.2%) | 97.8% (97.3% - 98.2%) |
Mortality
The ST models with independent triggers categorized 34 of the 49 deaths (69.4%) into the same triage categories at ETAT (Fig 4). The addition of independent triggers to the Smart Triage model improved the classification of mortality cases by ensuring that all ETAT emergencies were all still emergencies. Of the 49 patients who died, 13 (26.5%) were categorized as priority by ETAT and emergency by the ST models with independent triggers (Fig 4). The change in the classification of ETAT non-urgent patients into other triage categories was similar among ST models with independent triggers (Fig 4). The sensitivity for mortality was 51% (95% CI: 37.5% - 64.4%) for ETAT and 79.6% (95% CI: 66.4% - 88.5%) for ST models with independent triggers (Table 6).
Fig 4. Mortality.
The change in the classification by triage system for children who died.
Table 6. Diagnostic tests for mortality.
| ETAT | Smart Triage model with independent triggers | Recalibrated Smart Triage model with independent triggers | |
|---|---|---|---|
| Sensitivity (95% CI) | 51.0% (37.5% -64.4%) | 79.6% (66.4% - 88.5%) | 79.6% (66.4% - 88.5%) |
| Specificity (95% CI) | 91.2% (90.4% -91.9%) | 79.8% (78.7% - 80.8%) | 79.8% (78.7% - 80.8%) |
| PPV (95% CI) | 4.9% (3.3% - 7.1%) | 3.4% (2.5% - 4.6%) | 3.4% (2.5% - 4.6%) |
| NPV (95% CI) | 99.5% (99.2% - 99.7%) | 99.8% (99.6% - 99.9%) | 99.8% (99.6% - 99.9%) |
Discussion
In this study, we compared the ETAT guidelines commonly used in Kenyan public hospitals and the newly developed Smart Triage model based on their ability to classify children presenting to the ED into emergency, priority, and non-urgent categories. The major finding based on the selected model thresholds, was a shift in participants classification from non-urgent to priority to emergency when moving from ETAT to the ST- only model. The magnitude of this shift was larger when independent triggers were added. The emergency category consistently had the highest proportion of children who were admitted or died across all triage systems.
The purpose of paediatric triage is to distinguish children who require urgent intervention by the clinical team from those who can safely wait in the queue when they present to the emergency department. As expected in any triage system, the number of emergency cases should be fewer than priority and non-urgent cases. This was the case in our study, where the proportion of children who were classified as emergency was lower than those assigned priority and non-urgent categories by all triage systems. Assignment of fewer patients in the high acuity category is particularly important to minimize pressure on the already overwhelmed ED. Additionally, the emergency classification expedites clinical review and intervention for children in need of urgent care, thereby improving patient outcomes and ensuring the appropriate use of meager resources.
The Smart Triage models caused a shift in the distribution of children across the three triage categories, with more children assigned priority category by the ST model with independent triggers. Additionally, all ST models classified more children into the emergency category compared to ETAT. This was intentionally done with the selection of risk thresholds used to classify patients into these categories, to maximize the models’ sensitivity and specificity, and to reduce misclassification during triage [20]. Furthermore, as the Smart Triage algorithm is a continuous probability, the proportion of cases in the emergency and priority groups determined by the algorithm can be adjusted to meet the needs of the local context and resources.
Since there is no absolute measure of acuity, admission and mortality have typically been employed in research to assess how effectively a triage system performs [28,29]. Similarly, we used these outcomes as proxies for illness severity. An ideal triage system should categorize children in need of admission or at a high risk of mortality in the high-acuity category. Our study findings showed that ETAT and Smart Triage models accurately identified critically ill children, with most admissions and deaths assigned to the emergency category. In addition, incorporation of independent triggers improved the Smart Triage model’s ability to classify mortality cases. The ST models with independent triggers assigned all mortality cases to either emergency or priority category compared to ETAT which had some mortality cases assigned to the non-urgent category.
Smart Triage offers a major advantage over ETAT in that it can robustly combine a range of continuous and categorical variables in a prediction algorithm that would not be possible for even the most highly trained clinician. The algorithm can also be tuned to optimize the trade-offs clinicians perform in clinical practice. In the current implementation, a high specificity was selected to avoid missing critically ill children, especially those in the early stages of critical illness that have not reached the criteria for priority or emergency status using ETAT. This is reflected in the increased number of children in the priority and emergency categories. However, this must be balanced by including too many children in the emergency categories and slowing down access to care for critically ill children with obvious danger signs. Implementing Smart Triage can reduce reliance on memorization, manual processes, and paperwork, common in low-resource healthcare settings. This can lower administrative expenses and allocate resources to other crucial aspects of healthcare delivery. Smart Triage is designed with a simplified approach that utilizes only a few predictors and reduces the need for extensive training. The simultaneous integration of multiple predictors, which on their own would not increase priority, allows for identifying children with early signs of deterioration. Moreover, training resources, such as guides and illustrations, are integrated into the system to facilitate training and provide support during the triage process.
The incorporation of independent emergency and priority triggers into the Smart Triage models improved the categorization of admissions and mortality cases, as evidenced by the higher agreement between the Smart Triage models with independent triggers and the ETAT classification. Most of these triggers are used in ETAT to classify patients into emergency and priority categories and can explain the reason for the improvement in the categorization of admission and mortality cases. Therefore, this suggests that the inclusion of independent triggers as an additional criterion in paediatric triage systems that rely on prediction models is essential to avoid missing critically ill children. The prediction algorithm is not necessary in those children with a single symptom or sign that indicates a critically ill child. A similar argument has been proposed when comparing early warning scores to a single extreme observation [30].
Limitations
This study has some limitations. Enrollment in the study took place on weekdays between 8 am and 5 pm. The distribution of children in different triage categories may differ from that at other times of the day and on weekends. We used proxies for critical illness such as admission due to lack of a gold standard for patient acuity which could be influenced by other factors, rather than clinical indications alone. Increase in the number of emergency cases when using Smart Triage may suggest over-triaging by the triage system. However, the number of emergency and priority cases can be adjusted according to the hospital’s capacity, as the Smart Triage algorithm uses different thresholds to classify patients into triage categories. Data on the variable "obstructed/absent breathing" was not collected during the baseline phase of the primary study, hence not available for this analysis. Even though this is a rare presentation, it may have led to fewer patients being classified as emergency by ETAT. Lastly, the study focused on evaluating the triage systems’ ability to classify children into triage categories and did not assess other important factors, such as resource utilization.
Conclusion
Smart Triage compares well with ETAT and has potential as an efficient system for triaging children and identifying children in need of urgent care, especially when integrated with independent triggers. The Smart Triage algorithm increased the number of children in the priority and emergency groups, but marginally reduced the number of non-urgent children who were admitted or died. The addition of independent triggers to the Smart Triage model further improved the classification of children at high risk of admission and death. Further research is recommended to confirm these findings in other settings, such as primary care, and to compare Smart Triage with other triage systems as well as evaluate its impact on resource utilization.
Supporting information
(DOCX)
Acknowledgments
We are grateful to all study participants and the administration of Mbagathi County Hospital and Kiambu County Referral Hospital, Charly Huxford, Bella Hwang, Katija Pallot, Metrine Saisi, Elizabeth Isinde, Nancy Githinji, and Caroline Ndolo for their contributions and support during the study period. We highly appreciate the study nurses and clerks: Angela Wanjiru, Anastasia Gathigia, Felix Kimani, Mercy Mutuku, Emmah Kinyanjui, Esther Muthoni, Faith Wairimu, Sidney Kipkorir, Verah Karasi, Victor Achiro, Kevin Bosek, Brian Ochieng and John Mboya for their support in collecting the data.
Data Availability
We are unable to make the data publicly available, owing to the sensitive nature of clinical data. Access to the de-identified data is granted on a case-by-case basis and will require the signing of a data sharing agreement. Therefore, the dataset and data dictionary used in this analysis are available upon request from the KEMRI Wellcome Trust Research Programme Data Governance Committee. Applications to access the data should be directed to dgc@kemri-wellcome.org.
Funding Statement
This research was funded by the Wellcome Trust (grant code: 215695/B/19/Z), Grand Challenges Canada (grant code: 2008-35944), the BC Children’s Hospital Foundation and Mining4Life. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
References
- 1.Hoot NR, Aronsky D. Systematic Review of Emergency Department Crowding: Causes, Effects, and Solutions. Ann Emerg Med. 2008;52(2). doi: 10.1016/j.annemergmed.2008.03.014 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Bernstein SL, Aronsky D, Duseja R, Epstein S, Handel D, Hwang U, et al. The effect of emergency department crowding on clinically oriented outcomes. Academic Emergency Medicine. 2009;16(1):1–10. doi: 10.1111/j.1553-2712.2008.00295.x [DOI] [PubMed] [Google Scholar]
- 3.Janssen MA, van Achterberg T, Adriaansen MJ, Kampshoff CS, Mintjes-de Groot J. Adherence to the guideline “Triage in emergency departments”: A survey of Dutch emergency departments. J Clin Nurs. 2011;20(17–18):2458–68. doi: 10.1111/j.1365-2702.2011.03698.x [DOI] [PubMed] [Google Scholar]
- 4.Zachariasse JM, Van Der Hagen V, Seiger N, Mackway-Jones K, Van Veen M, Moll HA. Performance of triage systems in emergency care: A systematic review and meta-analysis. BMJ Open. 2019;9(5):1–9. doi: 10.1136/bmjopen-2018-026471 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Cheema B, Stephen C, Westwood A. Paediatric triage in South Africa. SAJCH South African Journal of Child Health. 2013;7(2):43–5. [Google Scholar]
- 6.Van Veen M, Moll HA. Reliability and validity of triage systems in paediatric emergency care. Scand J Trauma Resusc Emerg Med. 2009;17(1):1–8. doi: 10.1186/1757-7241-17-38 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Hategeka C, Shoveller J, Tuyisenge L, Kenyon C, Cechetto DF, Lynd LD. Pediatric emergency care capacity in a low-resource setting: An assessment of district hospitals in Rwanda. PLoS One. 2017;12(3):1–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Irimu GW, Gathara D, Zurovac D, Kihara H, Maina C, Mwangi J, et al. Performance of Health Workers in the Management of Seriously Sick Children at a Kenyan Tertiary Hospital: Before and after a Training Intervention. PLoS One. 2012;7(7):1–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Baker T. Critical care in low-income countries. Tropical Medicine and International Health. 2009;14(2):143–8. doi: 10.1111/j.1365-3156.2008.02202.x [DOI] [PubMed] [Google Scholar]
- 10.Hitchcock M, Gillespie B, Crilly J, Chaboyer W. Triage: An investigation of the process and potential vulnerabilities. J Adv Nurs. 2014;70(7):1532–41. doi: 10.1111/jan.12304 [DOI] [PubMed] [Google Scholar]
- 11.Mackway-Jones K, Marsden J, Windle J. Emergency triage: Manchester triage group. John Wiley & Sons; 2013. [Google Scholar]
- 12.Warren DW, Jarvis A, LeBlanc L, Gravel J. Canadian Association of Emergency Physicians; National Emergency Nurses Affiliation; Association des Médecins d’Urgence du Québec; Canadian Paediatric Society; Society of Rural Physicians of Canada. Revisions to the Canadian Triage and Acuity Scale paedia. CJEM. 2008;10(3):224–43. [PubMed] [Google Scholar]
- 13.Australasian College for Emergency Medicine. Guideline on The Implementation of The ATS. Australasian College for Emergency Medicine. 2016;1–8. [Google Scholar]
- 14.Gottschalk SB, Wood D, Devries S, Wallis LA, Bruijns S. The cape triage score: A new triage system South Africa. Proposal from the cape triage group. Emergency Medicine Journal. 2006;23(2):149–53. doi: 10.1136/emj.2005.028332 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Kenya Paediatric Association. Emergency Triage Assessment and Treatment—Plus (ETAT+). 2013; [Google Scholar]
- 16.WHO. Guideline: Updates on Paediatric Emergency Triage, Assessment and Treatment: Care of Critically-Ill Children. In Geneva: World Health Organization; 2016. [PubMed] [Google Scholar]
- 17.Duke T. New WHO guidelines on emergency triage assessment and treatment. The Lancet. 2016;387:721–4. doi: 10.1016/S0140-6736(16)00148-3 [DOI] [PubMed] [Google Scholar]
- 18.King C, Dube A, Zadutsa B, Banda L, Langton J, Desmond N, et al. Paediatric Emergency Triage, Assessment and Treatment (ETAT)–preparedness for implementation at primary care facilities in Malawi. Glob Health Action. 2021;14(1). doi: 10.1080/16549716.2021.1989807 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Hategeka C, Mwai L, Tuyisenge L. Implementing the Emergency Triage, Assessment and Treatment plus admission care (ETAT+) clinical practice guidelines to improve quality of hospital care in Rwandan district hospitals: healthcare workers’ perspectives on relevance and challenges. BMC Health Serv Res. 2017;17(1):256. doi: 10.1186/s12913-017-2193-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Mawji A, Li E, Dunsmuir D, Komugisha C, Novakowski SK, Wiens MO, et al. Smart triage: Development of a rapid pediatric triage algorithm for use in low-and-middle income countries. Front Pediatr. 2022;10(November):1–9. doi: 10.3389/fped.2022.976870 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Mawji A, Li E, Chandna A, Kortz T, Akech S, Wiens MO, et al. Common data elements for predictors of pediatric sepsis: A framework to standardize data collection. PLoS One [Internet]. 2021;16(6 June):1–12. Available from: doi: 10.1371/journal.pone.0253051 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Zachariasse JM, Nieboer D, Oostenbrink R, Moll HA, Steyerberg EW. Multiple performance measures are needed to evaluate triage systems in the emergency department. J Clin Epidemiol. 2018;94:27–34. doi: 10.1016/j.jclinepi.2017.11.004 [DOI] [PubMed] [Google Scholar]
- 23.Mawji A, Li E, Komugisha C, Akech S, Dunsmuir D, Wiens MO, et al. Smart triage: Triage and management of sepsis in children using the point-of-care Pediatric Rapid Sepsis Trigger (PRST) tool. BMC Health Serv Res. 2020;20(1):1–13. doi: 10.1186/s12913-020-05344-w [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Gan H, Karlen W, Dunsmuir D, Zhou G, Chiu M, Dumont GA, et al. The Performance of a Mobile Phone Respiratory Rate Counter Compared to the WHO ARI Timer. J Healthc Eng. 2015;6(4):691–703. doi: 10.1260/2040-2295.6.4.691 [DOI] [PubMed] [Google Scholar]
- 25.Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009. Apr;42(2):377–81. doi: 10.1016/j.jbi.2008.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Fung JST, Akech S, Kissoon N, Wiens MO, English M, Mark Ansermino J. Determining predictors of sepsis at triage among children under 5 years of age in resource-limited settings: A modified Delphi process. PLoS One. 2019;14(1):1–14. doi: 10.1371/journal.pone.0211274 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Vergouwe Y, Nieboer D, Oostenbrink R, Debray TPA, Murray GD, Kattan MW, et al. A closed testing procedure to select an appropriate method for updating prediction models. Stat Med. 2017. Dec 10;36(28):4529–39. doi: 10.1002/sim.7179 [DOI] [PubMed] [Google Scholar]
- 28.Magalhães-Barbosa MC, Robaina JR, Prata-Barbosa A, Lopes CDS. Reliability of triage systems for paediatric emergency care: A systematic review. Emergency Medicine Journal. 2019;36(4):231–8. doi: 10.1136/emermed-2018-207781 [DOI] [PubMed] [Google Scholar]
- 29.Marombwa NR, Sawe HR, George U, Kilindimo SS, Lucumay NJ, Mjema KM, et al. Performance characteristics of a local triage tool and internationally validated tools among under-fives presenting to an urban emergency department in Tanzania. BMC Pediatr. 2019;19(1):4–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Murali M, Inada-Kim M. Early warning scores: the case for aggregate vs. single extreme parameter activation to detect patient deterioration. Anaesthesia. 2023;803–6. doi: 10.1111/anae.16048 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
(DOCX)
Data Availability Statement
We are unable to make the data publicly available, owing to the sensitive nature of clinical data. Access to the de-identified data is granted on a case-by-case basis and will require the signing of a data sharing agreement. Therefore, the dataset and data dictionary used in this analysis are available upon request from the KEMRI Wellcome Trust Research Programme Data Governance Committee. Applications to access the data should be directed to dgc@kemri-wellcome.org.