Cardiac Children Hospital Early Warning ScoreVersus the Inadequate Oxygen Delivery Index for the Detection of Early Warning Signs of Deterioration

OBJECTIVES:

To assess the utility of the Cardiac Children’s Hospital Early Warning Score (C-CHEWS) in the early detection of deterioration.

DESIGN:

Single-center longitudinal pilot study.

SETTING:

Pediatric cardiac ICU (PCICU), Aga Khan University.

INTERVENTIONS:

C-CHEWS and Inadequate Oxygen Delivery (IDO₂) Index calculation every 2 hours.

PATIENTS:

A total of 60 children (0 d to 18 yr old).

MEASUREMENTS AND MAIN RESULTS:

A single-center longitudinal pilot study was conducted at PCICU. All postoperative extubated patients were assessed and scored between 0 and 11, and these scores were then correlated with the IDO₂ index data available from the T3 platform. Adverse events were defined as a need for cardiopulmonary resuscitation, or reintubation, and death. A total of 920 C-CHEWS and IDO₂ scores were analyzed on 60 patients during the study period. There were 36 males and 24 females, and the median age of the study population was 34 months (interquartile range, 9.0–72.0 mo). Fourteen patients (23.3%) developed adverse events; these included 9 reintubations and 5 cardiopulmonary arrests, resulting in 2 deaths. The area under the curve (AUC) for C-CHEWS scores fell in an acceptable range of 0.956 (95% CI, 0.869–0.992), suggesting an optimal accuracy for identifying early warning signs of cardiopulmonary arrest. Whereas, IDO₂ showed no discriminatory power to detect the adverse events with an AUC of 0.522 (95% CI, 0.389–0.652).

CONCLUSIONS:

The C-CHEWS tool provides a standardized assessment and approach to deteriorating congenital cardiac surgery patients in recognizing early postoperative deterioration.

KEY POINTS

Question: The C-CHEWS can detect early warning signs of deterioration in children post congenital cardiac surgery.

Findings: This single-center longitudinal pilot study suggests that C-CHEWS system has an optimal accuracy (AUC, 0.956 [95% CI, 0.869–0.992]) for identifying early warning signs in children post congenital cardiac surgery. The lead time for an elevated C-CHEWS score (≥2) was 10.0 hours (IQR, 3.5–14.5 hr) and for a score of ≥5 was 3 hours (IQR, 0.5–6 hr).

Meaning: The C-CHEWS provides a standardized assessment and approach to identify early warning signs of cardiopulmonary arrest in children post congenital cardiac surgery.

In the recent few years, there have been major advancements in medical technology, and still, cardiac arrest (CA) presents a major challenge. Children with congenital cardiac lesions have a higher incidence of inpatient CA than other hospitalized pediatric populations (1). According to the Society of Thoracic Surgeons database, the rate of CA in postoperative patients is 2.6%, with a survival rate of only 50.6% (2). Furthermore, CA is associated with significant morbidity due to ischemic brain injury and poor neurological outcomes (3). Studies have shown an increase in survival to discharge rate by up to 16% by implementing educational and community-based initiatives to improve recognition of early signs of CA (4). This lays special emphasis on the importance of early recognition of such adverse events to improve morbidity and mortality rates associated with CA.

McLellan and Connor (5) developed the Cardiac Children’s Hospital Early Warning Score (C-CHEWS) to provide a standardized assessment tool to identify clinical deterioration and pick early warning signs in children with heart diseases. C-CHEWS is a bedside scoring system consisting of three components: neurological, cardiovascular, and respiratory. Each component is scored between 0 and 3, and patients receive an additional “1” point for staff concerns and/or family concerns. Scores in each component are summed up to get a total score and interpreted based on three levels of color-coding; green (score 0–2; continue routine care), yellow (score 3–4; close monitoring), and red (score ≥5; attending physician notification and rapid response team activation) (Supplemental Fig. 1, http://links.lww.com/CCX/B118).

The T3 analytical platform is commercially available, Food and Drug Administration (FDA)-approved software that has a risk analytics algorithm known as the inadequate oxygen delivery (IDO₂) Index. The IDO₂ algorithm uses 10 physiologic values in the full dataset captured from the bedside monitor and laboratory values that are directly streamed into the T3 platform to compute the IDO₂ index in real time. The index provides a continuous risk estimation of IDO₂ and detection of subtle changes in the patient’s condition (6). Futterman et al (7) followed IDO₂ scores of 601 neonates postcardiopulmonary bypass and found that high IDO₂ scores over a 120-minute monitoring period are associated with an increased risk of CA.

Considering Pakistan’s resource-limited setting, most hospitals cannot afford advanced technology like T3. Indeed, there is also a significant shortage of trained personnel, with a limited number of bedside staff even in high acuity areas such as pediatric cardiac ICU (PCICU). Therefore, there is a dire need to implement a system that is both cost-effective and efficient in the prevention and early recognition of CA, thereby leading to improved survival. Our primary aim was to assess the utility of C-CHEWS in the identification of early warning signs of deterioration in the pediatric population post congenital cardiac surgery in the PCICU. Our secondary aim was to assess the concordance between C-CHEWS scores and IDO₂ to predict early warning signs of cardiopulmonary arrest in children after congenital cardiac surgery.

MATERIAL AND METHODS

We performed a single-center longitudinal pilot study at a four-bedded dedicated PCICU for children undergoing congenital cardiac surgery at Aga Khan University Hospital, Karachi Pakistan. This was started as a quality improvement project, and as per institutional policy, no institutional review board approval was necessary (thus no number was assigned) because it did not fall under the board’s guidelines as human subject research. All children (0 d to 18 yr old) who underwent congenital cardiac surgery and were admitted to the PCICU were eligible for inclusion in the study. Intubated patients and children whose parents did not give consent were excluded from the study. Data were collected between June 2019 and March 2021.

Data Collection

Permission was taken from McLellan et al.(5) to use the C-CHEWS scoring system, and two senior nurses trained 16 bedside nurses, on calculating C-CHEWS (Supplemental Fig. 2, http://links.lww.com/CCX/B118) and recording concomitant IDO₂ scores from the T3 software. Our PCICU beds have display screens for T3, and the IDO₂ scores can be easily captured by logging into the hospital system. C-CHEWS and IDO₂ scores were recorded every 2 hours on a structured questionnaire.

The questionnaire had details about patient anthropometrics (including age, weight, height, and body surface area [BSA]), diagnosis, type of procedure performed, and a 24-hour spreadsheet to record C-CHEWS and IDO₂ scores. C-CHEWS is normally done every 4 hours per McLellan et al (5), but since PCICU is a high dependency unit, we collected C-CHEWS and IDO₂ scores every 2 hours, starting immediately after extubating and continued till discharge/transfer from PCICU. If a patient’s C-CHEWS scores were in yellow (3, 4) or red (≥5), the frequency of C-CHEWS and IDO₂ was increased to once every hour. Adverse events were defined as a need for reintubation, cardiopulmonary resuscitation (CPR), or death.

Statistical Analysis

Data were analyzed on Medcalc analytical software Version 19.2 (MedCalc Software bv, Ostend, Belgium). The distribution of all continuous variables was assessed through the Shapiro-Wilk test. Mean and sd or median and interquartile ranges (IQRs) are presented according to their distribution. The area under the curve (AUC), sensitivity, specificity, positive predictive values (PPVs), and negative predictive values (NPVs) were computed for overall C-CHEWS and IDO₂ scores on an ordinal scale. Cut points greater than 3, 4, and 5 for C-CHEWS and 30, 40, and 50 for IDO₂ scores, respectively, were computed. Multivariable logistic regression, adjusted for demographics, weight, anesthesia time, bypass time, ICU stay, and the residual cardiac lesion were performed to assess the predictors of the outcome. Crude and adjusted odds ratios (ORs) were presented with a 95% confidence interval. A p value of < 0.05 was considered significant.

RESULTS

Demographics

Sixty patients were followed throughout their PCICU stay, and a total of 920 C-CHEWS and IDO₂ scores were recorded and analyzed. There were 36 males and 24 females with a median age of 34 months (IQR, 9.0–72.0 mo) and a BSA of 0.47 ± 0.21 m² (Table 1). The most common defect was ventricular septal defect 23 (38.3%), followed by Tetralogy of Fallot: 17 (28.3%), atrial septal defect: 8 (13.3%), dextrotransposition of great arteries with or without a ventricular septal defect: 6 (10%), severe right ventricular outflow tract obstruction: 1 (1.67%), atrioventricular septal defect: 2 (3.3%), mitral valve repair: 1 (1.67%), and aortopulmonary window with patent foramen ovale: 2 (3.3%). There was no statistically significant difference in basic demographics between the children who had an adverse event compared to those who did not (Table 1).

TABLE 1.

Comparison of Demographics, and Procedure Performed Based on the Occurrence of Adverse Events, n = 60

Variable	Occurrence of Adverse Eventa		p
	Yes, 14 (23.3%)	No, 46 (76.7%)
Age, mo, median (interquartile range)	24 (5.0–59.0)	35 (10.5–79.5)	0.280
Gender, n (%)
Female	6 (10.0)	18 (30.0)	0.803
Male	8 (13.3)	28 (46.7)
Weight (kg)b	8.18 ± 4.57	11.53 ± 7.37	0.114
Height (cm)b	76.43 ± 20.13	87.78 ± 20.13	0.158
BSA (m2)	0.41 ± 0.17	0.52 ± 0.25	0.121
PreOp Sr. Creatinine, median (IQR)	0.3 (0.2–0.6)	0.3 (0.2–0.4)	0.668c
Prematurity, n (%)	2 (14.3)	1 (2.2)	0.069
Procedure, n (%)	14(23.3)	46 (76.7)	0.078d
VSD	4 (17.4)	19 (82.6)
Atrial septal defect	0	8 (100)
Tetralogy of Fallot	4 (23.5)	13 (76.5)
Arterial switch + VSD	4 (66.7)	2 (33.3)
Severe right ventricular outflow tract obstruction	0	1 (100)
Atrioventricular septal defect	1 (50)	1 (50)
Mitral valve repair	0	1 (100)
Aortopulmonary window type 3	1 (50)	1 (50)

VSD = ventricular septal defect.

^aAdverse events: reintubation, cardiopulmonary resuscitation, or death.

^bMean ± sd.

^cMann-Whitney test.

^dFisher exact test.

PCICU Stay

Fourteen (23.3%) patients developed an adverse event, which included 9 reintubations and 5 CA leading to initiation of CPR. Out of the 9 reintubated patients, 7 patients were extubated and subsequently moved to a step-down unit, whereas 2 patients expired. Out of the 5 CPR cases, 2 patients were successfully revived and intubated, and 3 expired.

The number of adverse events was highest in the arterial switch group: 4 (66.7%), followed by an atrioventricular septal defect 1 (50%), and aortopulmonary window type 3: 1 (50%). The overall difference in the occurrence of events with regard to the type of procedure performed was statistically insignificant, with a p value of 0.078 (Table 1). During the PCICU stay, the median C-CHEWS scores were significantly higher in those who had an adverse event, 4.03 (IQR, 2.39–5.15) versus 2.77 (IQR, 2.06–3.39), p = 0.003. IDO₂ scores also showed similar trends (9.17 [IQR, 0.00–27.00]) versus (6.64 [0.00–24.87]), but there was no statistically significant difference between the groups (Table 2).

TABLE 2.

Comparison of Perioperative and Postoperative Patient Characteristics, n = 60

Variable	Occurrence of Adverse Eventa		p
	Yes	No
Anesthesia time (hr), median (IQR)	5.0 (4.8–6.0)	4.5 (3.8–5.0)	0.009b
Bypass time (min), median (IQR)	150 (82.5–133.8)	70.0 (45.0–100.0)	0.009b
ICU stay (d), median (IQR)	9.0 (7.0–16.5)	2.5 (2.0–4.0)	< 0.001b
Noninvasive ventilation, n (%)	13 (92.9)	21 (45.7)	0.002
Residual cardiac lesion, n (%)	6 (42.9)	21 (45.7)	0.854
Cardiac Children’s Hospital Early Warning Score, median (IQR)	4.03 (2.39–5.15)	2.77 (2.06–3.39)	0.003
Inadequate Oxygen Delivery Scores, median (IQR)	9.17 (0.00–27.0)	6.64 (0.0–24.87)	0.944

IQR = interquartile range.

^aAdverse events: reintubation, cardiopulmonary resuscitation, or death.

^bMann-Whitney test.

Median anesthesia time (5.0 hr [IQR, 4.8–6.0 hr] vs 4.5 hr [IQR, 3.8–5.0 hr]; p = 0.009) and cardiopulmonary bypass time (150 min [82.5–133.8 min] vs 70 min [45.0–100.0 min]; p = 0.009) were significantly greater in those patients who had an adverse event. Furthermore, children who had an adverse event had longer PCICU stays (median 9.0 d [7.0–16.5 d] vs 2.5 d [2.0–4.0 d]; p < 0.001), and almost all were in need of noninvasive ventilation (13 [92.9%] vs 21 [45.7%]; p = 0.002) (Table 2). Crude odds of anesthesia time for the adverse event group were 2.27 (95% CI, 1.19–4.33) times; similarly, for the bypass time, it was 1.02 (95% CI, 1.00–1.04; p = 0.012) (Table 3). However, after adjusting for adjusted for demographics, weight, anesthesia time, bypass time, ICU stay, and residual cardiac lesion, they were not found to be statistically significant. As expected, prolonged ICU stay was an independent risk factor for an adverse outcome; after adjustments, it changed from OR 1.98 (95% CI, 1.33–2.94) to OR 3.58 (95% CI, 1.29–9.98); p = 0.015 (Table 3).

TABLE 3.

Crude and Adjusted Odds Ratio (95% CI) Adverse Events, n = 56

Variables	Crude OR (95% CI)	p	Adjusted OR (95% CI)	p
Age (mo)	0.99 (0.97–1.01)	0.252	1.00 (0.95–1.06)	0.864
Gender	0.86 (0.25–2.88)	0.803	15.16 (0.23–985.99)	0.202
Weight	0.91 (0.82–1.02)	0.122	1.08 (0.79–1.49)	0.622
Anesthesia time (min)	2.27 (1.19–4.33)	0.012	1.64 (0.29–9.26)	0.577
Bypass time (min)	1.02 (1.00–1.04)	0.012	0.95 (0.89–1.02)	0.169
ICU stay (d)	1.98 (1.33–2.94)	0.001	3.58 (1.29–9.98)	0.015
Residual cardiac lesion	0.89 (0.27–2.99)	0.854	1.37 (0.1–19.19)	0.815

QR = odds ratio.

The time of elevated C-CHEWS scores before the adverse event was calculated to assess the potential lead time for clinical intervention. The lead time for an elevated C-CHEWS score (≥2) was 10.0 hours (IQR, 3.5–14.5 hr) and, for a score of ≥5, was 3 hours (IQR, 0.5–6 hr)

Sensitivity and Specificity

For a score >2 (green zone), the sensitivity of C-CHEWS was 100% (95% CI, 76.8–100.0) and specificity was 23.9% (95% CI, 12.6–38.8). The PPV was 28.6 (95% CI, 25.4–32.0), and the NPV was 100.0. For a score of >3 (yellow zone), the sensitivity was 100.0% (95% CI, 76.8–100.0), the specificity was 63.0% (95% CI, 47.5–76.8), PPV was 45.2 (95% CI, 36.1–54.6), and NPV was 100.0. For scores >5, which entered the red zone, the sensitivity was 42.86% (95% CI, 17.7–71.1), the specificity was 100.0% (95% CI, 92.3–100.0), PPV was 100.0, and the NPV was 85.2% (95% CI, 78.5–90.1) (Supplemental Table 1, http://links.lww.com/CCX/B118).

Whereas for an IDO₂ score of >20, the sensitivity was 42.9% (95% CI 17.7–71.1.0), the specificity was 67.4% (95% CI, 52.0–80.5), the PPV was 28.6 (95% CI, 16.1–45.5), and the NPV was 79.5 (95% CI, 70.2–86.4). For IDO₂ score of >40, the sensitivity was 28.57% (95% CI, 8.4–58.1), specificity was 93.5% (95% CI, 82.1–98.6), PPV was 57.1 (95% CI, 25.3–84.0), and the NPV was 81.1 (95% CI, 75.4–85.8) (Supplemental Table 2, http://links.lww.com/CCX/B118). Overall, a tradeoff is evident between sensitivity and specificity as scores progress for both C-CHEWS and IDO₂ measurements.

Comparison of Area Under the Receiving Operating Characteristic curve

Using an overall score as an ordinal variable, the C-CHEWS tool had a higher AUC of 0.956 (95% CI, 0.869–0.992) compared with the IDO₂ AUC 0.522 (95% CI, 0.389–0.652), and the difference between the two curves was significant 0.434 (95% CI, 0.231–0.637); p < 0.001 (Table 4). In our study population, a C-CHEWS score of greater than 4 was predictive of an adverse event, and the corresponding IDO₂ score was >40 (Fig. 1). The sensitivity of C-CHEWS was also higher at 64.3% (95% CI, 35.1–87.2) versus 28.7% (95% CI, 8.4–58.1); however, specificity was comparable 95.7% (95% CI, 85.2–99.5) versus 93.5% (95% CI, 82.1–98.6).

TABLE 4.

Receiver Operating Characteristic Curve Analysis of Ordinal Children’s Hospital Early Warning and Inadequate Oxygen Delivery Scores Against the Adverse Events, n = 60

Variable	Cardiac Children’s Hospital Early Warning Score	Inadequate Oxygen Delivery
Sensitivity (95% CI)	64.3 (35.1–87.2)	28.68. (8.4–58.1)
Specificity (95% CI)	95.7 (85.2–99.5)	93.5 (82.1–98.6)
Youden index J	0.793	0.293
Associated criterion (cutoff)	> 4.0	> 40.0
The area under the curve (95% CI)	0.956 (0.869–0.992)	0.522 (0.389–0.652)
Difference of AUCs	0.434 (0.231–0.637)
p (difference of AUCs)	< 0.001

AUC = area under the curve.

Figure 1.

The area under the receiving operating characteristic curve of Cardiac Children’s Hospital Early Warning Score (C-CHEWS) and inadequate oxygen delivery (IDO₂).

DISCUSSION

In this study, we have assessed the utility of C-CHEWS in the identification of early warning signs to predict adverse events in children undergoing congenital cardiac surgery. To our knowledge, this is the first study that has explored the implementation of the C-CHEWS algorithm in the immediate postoperative time period and correlated it with IDO₂. We intended to see how C-CHEWS performs in our PCICU where the rate of adverse events is high compared with the developed world, and we also intended to investigate whether this can be successfully implemented in the low-resource setting, where patient-to-nurse ratios are frequently higher than the developed world. Our study shows that the C-CHEWS has excellent accuracy for identifying early warning signs for CA in the critically ill pediatric population, and it can be used in a resource-limited setting where IDO₂ is not available. Our findings are comparable with an earlier study conducted by McLellan et al (8) on the validation of C-CHEWS, with an important difference: their critical score for predicting an adverse event is equal to or greater than 5, whereas our calculated critical score for PCICU is 4 (8). This difference can be due to the different hospital settings. McLellan (8) conducted her study in the cardiac ward where most patients are stable. Our study setting was PCICU where most patients are critically ill and more prone to adverse events; hence, the team should be vigilant at a lower C-CHEWS score. Furthermore, the sensitivity estimated in our sample was comparable with McLellan and Connor (5) (64.3% [95% CI, 35.1–87.2]) versus (75.6% [95% CI, 70.8–79.9]); however, specificity reported in our study was higher compared with McLellan and Connor (5) (95.73% [95% CI, 85.2–99.5] vs (67.8% [95% CI, 64.4–71.1]). This could be because of the frequency of the scoring in our setting where it was being done every 2 hours by a highly trained ICU nurse.

IDO₂ is a model-based risk analytic algorithm that continues to improve as more data are available (an inherent characteristic of machine learning algorithms). More research into the contribution of different variables for IDO₂ scores and determining different cutoffs will help shed more light on this. IDO₂ is an automated system and is a decision support tool that can help overcome the limitations of less trained and stretch human resources. C-CHEWS, on the other hand, is a low-cost tool; however, it does require more intense staff training and implementation effort, which can be a challenge in low-resource settings (9). The generation of these scores is highly dependent on nursing expertise in making clinical assessments; the use of such systems in settings with high-risk patients would require intense training of bedside nurses to pick subtle changes in patient condition. C-CHEWS also requires a sustained effort to continue training new coming staff and subsequent quality checks to make sure scores are accurately calculated. Our results show that C-CHEWS can accurately predict adverse events, so implementation of an algorithm with clear guidelines would help in the early recognition of warning signs by less experienced providers. These providers can then alert the attending and rapid response team who may not always be available in the PCICU. These findings could be groundbreaking and could help improve overall survival until discharge, especially in low-income hospital settings.

C-CHEWS scoring system has multiple domains including cardiovascular, behavioral, neurological, and respiratory, and incorporates staff and patient attendant concerns. With the help of C-CHEWS, a particular domain (cardiovascular, behavioral/neurological, and respiratory) can be identified, which contributes to a higher C-CHEWS score and initiates targeted therapy (5). Additionally, such systems, which incorporate clinical assessment of bedside caretakers in the detection of early signs, can be immensely helpful in improving effective communication between nurses and the treating physicians as these scores would alert the caretakers even on minor changes in the patient’s condition (10, 11). It also helps to improve the clinical judgment of the bedside nursing staff as scoring systems like these require them to quantify their clinical judgment to generate these early warning scores. C-CHEWS is a useful tool to implement where more sophisticated tools like T3 and IDO₂ are not available.

Another crucial point highlighted in our study was the overall risks of having adverse events. Although the type of congenital defect was not found to be a risk factor, the anesthesia time and bypass time do indicate a longer and more likely a complicated surgery. This is similar to other studies and helps in identifying areas needing further improvement and attention (12, 13).

The results should be interpreted considering some limitations. Since this study is single-centered and the data analyzed are from patients admitted to one institution, findings can be influenced by local practice patterns and may not be generalizable to other populations. The sample size is also small drawn for pilot purposes; however, this is a good representation of cases performed in our center and other centers in Pakistan. Our study was not powered to accurately compare the performance of the two tools we compared (C-CHEWS and IDO₂). Keeping in mind that this was a pilot study, doing a multicenter study as a next step would be the best approach to validate the findings of our study followed by the implementation of C-CHEWS.

CONCLUSIONS

The C-CHEWS tool provides a standardized assessment and approach to deteriorating cardiac surgery patients in recognizing early signs of CA and other adverse events compared with the IDO₂ system. This can have far-reaching implications in a low-resource setting where these advanced monitoring systems such as T3 are not available. Implementation of such a system in a unit with stretched resources can be beneficial to improving overall morbidity and mortality in patients who have undergone congenital cardiac surgery.

Supplementary Material

cc9-5-e0833-s001.pdf ^{(972.5KB, pdf)}