Abstract
Objective:
To develop a prognostic model for predicting mortality at time of extracorporeal membrane oxygenation (ECMO) initiation for children which is important for determining center-specific risk-adjusted outcomes.
Design:
Multivariable logistic regression using a large national cohort of pediatric ECMO patients.
Setting:
The intensive care units of the eight tertiary care children’s hospitals of the Collaborative Pediatric Critical Care Research Network
Patients:
514 children (< 19 years), enrolled with an initial ECMO run for any indication between January 2012 and September 2014.
Interventions:
None
Measurements and Main Results:
A total of 514 first ECMO runs were analyzed with an overall mortality of 45% (n=232). Weighted logistic regression was used for model selection and internal validation was performed using cross validation. The variables included in the Pediatric ECMO Prediction (PEP) model were age (pre-term neonate, full-term neonate, infant, child, and adolescent), indication for ECMO (extracorporeal cardiopulmonary resuscition, cardiac, or respiratory), meconium aspiration, congenital diaphragmatic hernia, documented blood stream infection, arterial blood pH, partial thromboplastin time, and international normalized ratio. The highest risk of mortality was associated with the presence of a documented blood stream infection (OR 5.26; CI 1.90–14.57) followed by extracorporeal cardiopulmonary resuscitation (OR = 4.36; CI 2.23–8.51). The c-statistic was 0.75 (95% CI, 0.70–0.80).
Conclusions:
The PEP model represents a model for predicting in-hospital mortality among children receiving ECMO support for any indication. Consequently, it holds promise as the first comprehensive pediatric ECMO risk stratification model which is important for benchmarking ECMO outcomes across many centers.
Keywords: Extracorporeal membrane oxygenation, Risk assessment, Risk adjustment, Pediatric, Decision Support, Predictive score model
INTRODUCTION
Indications for extracorporeal membrane oxygenation (ECMO) include a “reversible condition with a high predicted mortality rate if conventional management is continued” (1). However, the lack of randomized controlled trials and prognostic prediction models complicate prediction of reversibility of the condition and mortality when ECMO is used. Moreover, mortality is influenced by an increasing complexity of primary diagnoses and comorbid conditions, as well as practice variation (2–5). Discriminating the patient-related risk factors from center-specific practice variation is important for performance benchmarking, observational research, quality improvement, and anticipating mortality across similar patient groups (6–13).
Prediction models have been developed but the target populations of existing models are limited to neonates and pediatric patients with respiratory failure treated with ECMO and these models were developed using only variables captured in the Extracorporeal Life Support Organization (ELSO) registry (2, 14–16). The c-statistics of these models range from 0.69 to 0.78 with the best performance achieved in the neonatal respiratory failure population (2, 14–16). Only 35% (712/2060) of all patients age 14 days to 18 years in the 2016 ELSO registry are represented by current models and of those only 28% (571/2060) had sufficient data to calculate a score. The percentage of patients for which a score could be applied for the PIPER, Neo-RESCUERS Ped-RESCUERS and P-PREP models were 6%, 8%, 27%, and 31% respectively demonstrating that many pediatric patients cannot be risk-adjusted using current models (2, 14–16) .Furthermore, prognostic models for pediatric patients who receive extracorporeal cardiopulmonary resuscitation (eCPR) or require cardiac ECMO do not exist (17). In short, we lack an omnibus risk stratification model that can be applied to both neonates and pediatric patients without being limited to either cardiac or respiratory ECMO and also include those who receive eCPR.
Our aim was to develop and internally validate a comprehensive prognostic model to predict in-hospital mortality for all patients <19 years of age who received ECMO for any reason, including eCPR, by using pre-ECMO data available in the Bleeding and Thrombosis on ECMO (BATE) dataset (18). We hypothesized that a Pediatric ECMO Prediction (PEP) model could be developed that would be useful for the purposes of risk-adjusting to determine center-specific outcomes.
MATERIALS AND METHODS
The prognostic model was developed and internally validated using data originally collected for the BATE study. Permission was requested and granted to obtain access to the BATE dataset, which included 514 first ECMO run data from subjects birth to <19 years enrolled at the eight pediatric hospitals affiliated with the Eunice Kennedy Shriver National Institute of Child Health and Human Development Collaborative Pediatric Critical Care Research Network between December 2012 and September 2014. Our primary outcome was in-hospital mortality in pediatric patients supported with ECMO.
The study was designed following the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis statement (19) with waiver of informed consent by the Institutional Review Board for every clinical site and the Data Coordinating Center at the University of Utah. All data were collected by trained research coordinators via direct observation, discussion with bedside clinicians, and review of medical records.
Candidate variables
Potential predictors of mortality were limited to the pre-ECMO variables available in the BATE dataset with missingness <10% after imputation. Supplemental Table 1 illustrates the schema used for categorizing diagnostic groups within the BATE dataset which were adapted using previous publications (2–4, 14–16, 20–23). All time-stamped variables were collected within 12 hours prior to ECMO cannulation, and the single data point most proximal and prior to ECMO was used for analysis (see Supplementary Table 2).
Variables defined
Using clinical judgement at the time of cannulation indications for ECMO were categorized as respiratory, cardiac, or eCPR which included any subject who went onto ECMO during CPR. ECMO mode was categorized as venoarterial (VA) for any mode initially using an arterial cannula and veno-venous (VV) for any mode initially without an arterial cannula.
Patients ≤30 days of actual age at time of ECMO cannulation were categorized as neonates, with neonates born at <37 weeks gestation defined as preterm; >30 days to < 1 year as infants, ≥1 year to <12 years as children, and ≥12 years to < 19 years as adolescents. Body habitus was determined using weight-for-length percentiles for patients <2 years old and body mass index (BMI)-for-age percentiles for all others, according to the U.S. Center for Disease Control and Prevention (24). Obesity and underweight cutoffs were defined as greater than or equal to the 95th percentile or less than the 5th percentile, respectively.
Acute diagnoses were the primary conditions requiring ICU admission. Chronic diagnoses were those given to a patient prior to the current hospitalization that were not the primary reason for ICU admission.
Infections were all based on pre-ECMO data and were classified as ‘documented’ (culture or PCR testing confirmed the presence of a pathogen) or ‘suspected’ (patient treated with antibiotics for presumed infection despite culture/PCR never performed or negative). Documented infections were further categorized by pathogen and system (blood, respiratory or other).
The BATE protocol defined organ failure prior to ECMO separately from the indication for ECMO. Cardiovascular failure included those cannulated to VA ECMO and/or receiving vasoactive infusions at the time of cannulation. Hepatic failure was defined as an international normalized ratio (INR) ≥ 2. To determine acute kidney injury, we presumed baseline creatinine as the median creatinine for age provided the patient did not have known pre-existing renal injury or failure, because the first measured serum creatinine in critically ill children may be higher than baseline (25). As described by Zappitelli et al., we then calculated the change in creatinine from the presumed baseline and assigned acute kidney injury (AKI) using the Kidney Disease Improving Global Outcomes (KDIGO) consensus definition of AKI (26–28). All patients using renal replacement therapy at baseline or prior to ECMO were analyzed as having ‘stageII+ AKI.’ Neurologic failure was defined as seizures (clinical or EEG), intracranial hemorrhage, cerebrovascular accident, or stroke.
The Vasoactive-Inotropic Score (VIS) was calculated at the time of ECMO cannulation (29, 30). Vital sign ranges were categorized based on prior studies (see Supplemental Table 3) (31–34).
Development and Internal Validation of the PEP Model
Pre-ECMO measurements and characteristics were evaluated as predictors of in-hospital mortality using univariable logistic regression. In order to fully utilize the variables available to us in the multivariable model, multiple imputation was used to impute missing data. Imputation was performed using a sequence of regression models implemented in IVEware (Imputation and Variance Estimation Software, Version 0.3, Ann Arbor, Michigan). After imputation, we then excluded any variables that remained missing for >10% of subjects (e.g., vital signs such as blood pressure and heart rate were excluded since they were missing for the 14% of subjects who received eCPR, and imputation of these values would be inappropriate for these subjects). All candidate variables with missingness after imputation of <10% were then entered into a bidirectional stepwise selection process with a criterion of p < 0.05 to enter and stay in the final model. The interaction between each variable and the indication for ECMO was considered to allow appropriate modeling in case the relationship between predictors and in-hospital mortality varies by indication. Weighted logistic regression was used for model selection with weights defined uniformly as 1/10 to account for the 10 imputed datasets being used (35). After the variables to be included in the final model were selected, multiple logistic regression was performed separately for each of the ten imputed datasets. Results were combined, accounting for the amount of imputed data and appropriately inflating the variance of the estimators, using the MIANALYZE procedure in SAS 9.4 (SAS Institute; Cary, NC) (36).
The prediction equation for the probability of in-hospital mortality was defined as the logistic function of the log-odds of mortality, as estimated by the model. In order to assess the predictive performance of the model, the c-statistic was generated. Model calibration was tested using the Hosmer-Lemeshow goodness of fit test. The model was internally validated using cross validation. Due to the modest sample size, leave-one-out cross validation was used to accurately estimate the c-statistic without the bias that would otherwise occur when using the same dataset to develop and validate a model. The c-statistic for each imputed dataset was obtained, and the results were combined with the MIANALYZE procedure in order to obtain accurate confidence intervals for the c-statistic from the imputed data.
RESULTS
A total of 514 first ECMO runs were analyzed with an overall mortality of 45% (n=232) and a neonatal mortality of 42%. The median duration of ECMO was 5 days (IQR 2.7, 9.4]). The median time from hospital admission to cannulation was 2 days (IQR 0.4, 7.5]).
Pre-ECMO characteristics of survivors and non-survivors in the BATE dataset are found in Supplemental Table 3. Survival did not differ significantly by sex, race, or ECMO center. The most common acute and chronic diagnoses were congenital cardiovascular disease (38%), respiratory distress/failure (33%), and congenital anomaly/chromosomal defect (23%). Documented infections of any type were found in 16% of subjects. Of the 27 patients with a documented blood stream infection (D-BSI), two were fungal.
Pre-ECMO supportive therapies and laboratory findings by vital status are displayed in Tables 1 and 2. Respiratory failure was the most common indication for ECMO (46%); cardiac failure and eCPR respectively accounted for 40% and 14% of the total. Overall mortality for VV ECMO was significantly lower than for VA ECMO (30% vs 48%; p = 0.003).
Table 1:
Pre-extracorporeal membrane oxygenation supportive therapies by vital status
| Vital status at hospital discharge | |||
|---|---|---|---|
| Supportive therapy | Dead (n = 232) | Alive (n = 282) | Overall (n = 514) |
| Primary ECMO indication, n (%) | |||
| Respiratory | 83 (36%) | 154 (55%) | 237 (46%) |
| Cardiac | 105 (45%) | 102 (36%) | 207 (40%) |
| eCPR | 44 (19%) | 26 (9%) | 70 (14%) |
| Operation in the prior 24 hours, n (%) | 89 (38%) | 89 (32%) | 178 (35%) |
| CPB in the prior 24 hours, n (%) | 78 (34%) | 70 (25%) | 148 (29%) |
| ECMO directly from CPB, n (%) | 38 (16%) | 37 (13%) | 75 (15%) |
| Heparin bolus for cannulation, n (%) | 179 (77%) | 235 (83%) | 414 (81%) |
| Initial mode of ECMO, n (%) | |||
| VA | 207 (89%) | 224 (79%) | 431 (84%) |
| VV | 25 (11%) | 58 (21%) | 83 (16%) |
| Type of Pump, n (%) | |||
| Centrifuge | 164 (71%) | 170 (60%) | 334 (65%) |
| Roller head | 68 (29%) | 112 (40%) | 180 (35%) |
| Vasoactive inotropic score,a n (%) | |||
| None or low | 139 (60%) | 180 (64%) | 319 (62%) |
| High | 93 (40%) | 102 (36%) | 195 (38%) |
| Vasoactive bolus, n (%) | 50 (22%) | 44 (16%) | 94 (18%) |
| Respiratory support | 191 (82%) | 235 (83%) | 426 (83%) |
| Ventilation, n (%) | |||
| None | 30 (14%) | 38 (14%) | 68 (14%) |
| Conventional | 135 (62%) | 146 (54%) | 281 (57%) |
| High frequency | 54 (25%) | 89 (33%) | 143 (29%) |
| Settings, median [IQR] | |||
| Ventilator rate (bpm) | 28.0 [20.0, 35.0] | 27.0 [20.0, 35.0] | 28.0 [20.0, 35.0] |
| PIP (cmH2O) | 26.0 [21.0, 32.0] | 25.0 [21.0, 30.0] | 26.0 [21.0, 31.0] |
| Exhaled tidal volume (mL) | 41.0 [28.0, 110.0] | 43.9 [27.0, 90.0] | 43.0 [28.0, 100.0] |
| Set tidal volume (mL) | 60.0 [35.0, 150.0] | 43.0 [30.0, 90.0] | 50.0 [30.0, 104.0] |
| PEEP (cmH2O) | 6.0 [5.0, 8.0] | 6.0 [5.0, 8.0] | 6.0 [5.0, 8.0] |
| Pressure support (cmH20) | 10.0 [8.0, 10.0] | 10.0 [8.0, 10.0] | 10.0 [8.0, 10.0] |
| FiO2 | 1.0 [0.5, 1.0] | 1.0 [0.8, 1.0] | 1.0 [0.6, 1.0] |
| Mean airway pressure (cmH2O) | 14.8 [11.0, 20.0] | 14.0 [11.0, 20.0] | 14.3 [11.0, 20.0] |
| Frequency (Hz) | 8.0 [6.0, 8.0] | 7.0 [6.0, 8.0] | 7.0 [6.0, 8.0] |
| Amplitude (cmH2O) | 41.0 [36.0, 50.0] | 39.0 [35.0, 46.0] | 40.0 [35.0, 47.0] |
ECMO = extracorporeal membrane oxygenation; eCPR = extracorporeal cardiopulmonary resuscitation; CPB = cardiopulmonary bypass; VA = venoarterial; VV = veno-venous; IQR = interquartile range; PIP = peak inspiratory pressure; PEEP = positive end-expiratory pressure.
Vasoactive inotropic score: none/low = 0 to <20, high = ≥20.
Table 2:
Pre-extracorporeal membrane oxygenation laboratory findings by vital status
| Vital status at hospital discharge | |||
|---|---|---|---|
| Laboratory test, median [IQR] | Dead (N = 232) | Alive (N = 282) | Overall (N = 514) |
| Lactate (mmol/L) | 5.1 [1.8, 9.9] | 3.0 [1.6, 6.9] | 4.0 [1.7, 8.0] |
| pH | 7.3 [7.1, 7.4] | 7.3 [7.2, 7.4] | 7.3 [7.1, 7.4] |
| PaO2 (mmHg) | 48.5 [34.2, 95.1] | 53.0 [37.0, 89.7] | 52.0 [35.2, 93.7] |
| PaCO2 (mmHg) | 50.0 [38.4, 66.9] | 50.0 [39.0, 66.0] | 50.0 [38.8, 66.8] |
| PaO2/FiO2 ratio | 73.8 [37.4, 175.3] | 61.2 [37.4, 142.5] | 64.0 [37.4, 153.3] |
| Oxygenation index | 24.4 [8.3, 49.2] | 30.2 [12.5, 50.0] | 28.3 [10.9, 50.0] |
| Prothrombin time (seconds) | 18.0 [14.7, 21.7] | 16.0 [13.3, 18.5] | 16.8 [13.8, 19.7] |
| Partial thromboplastin time (seconds) | 46.9 [34.8, 65.9] | 41.5 [32.7, 53.0] | 43.7 [33.4, 58.4] |
| International normalized ratio | 1.6 [1.3, 2.0] | 1.4 [1.2, 1.7] | 1.5 [1.2, 1.8] |
| Fibrinogen (mg/dL) | 210.5 [136.5, 284.5] | 238.0 [177.0, 320.0] | 217.0 [156.0, 306.0] |
| Leukocytes (103/μL) | 13.7 [8.9, 18.5] | 13.8 [8.8, 19.1] | 13.8 [8.8, 19.0] |
| Hemoglobin (g/dL) | 13.2 [11.1, 15.0] | 13.3 [11.4, 15.4] | 13.2 [11.2, 15.3] |
| Platelets (103/μL) | 180.5 [109.0, 253.0] | 172.0 [120.0, 237.0] | 174.0 [114.0, 248.0] |
| Sodium (mmol/L) | 140.0 [136.0, 145.0] | 140.0 [136.0, 145.0] | 140.0 [136.0, 145.0] |
| Potassium (mmol/L) | 4.0 [3.5, 4.6] | 3.7 [3.3, 4.3] | 3.8 [3.4, 4.4] |
| Chloride (mmol/L) | 104.0 [99.0, 109.0] | 104.0 [99.0, 109.0] | 104.0 [99.0, 109.0] |
| Blood urea nitrogen (mg/dL) | 15.0 [8.0, 26.0] | 16.0 [10.0, 23.0] | 16.0 [9.0, 24.0] |
| Creatinine (mg/dL) | 0.6 [0.4, 0.9] | 0.6 [0.4, 0.8] | 0.6 [0.4, 0.9] |
| Glucose (mg/dL) | 127.0 [94.0, 202.0] | 138.0 [101.0, 183.0] | 135.5 [98.0, 188.0] |
| Albumin (g/dL) | 2.7 [2.1, 3.2] | 2.6 [2.2, 3.2] | 2.6 [2.2, 3.2] |
| Alkaline phosphatase (IU/L) | 118.0 [83.0, 154.5] | 119.5 [76.0, 199.0] | 118.5 [80.0, 174.0] |
| Alanine aminotransferase (IU/L) | 36.0 [18.0, 99.0] | 30.5 [21.0, 54.0] | 32.0 [20.0, 68.0] |
| Aspartate aminotransferase (IU/L) | 92.5 [44.5, 320.5] | 66.0 [35.0, 130.0] | 71.5 [39.0, 180.0] |
| Total bilirubin (mg/dL) | 2.3 [1.0, 4.3] | 2.0 [0.6, 5.2] | 2.2 [0.7, 4.6] |
IQR = interquartile range
Univariable analysis and missingness of candidate variables included in the PEP model prior to and after imputation are found in Table 3, with a complete list of candidate variables found in Supplemental Table 4. Variables excluded because they were missing after imputation, i.e. not applicable in >10% of the cohort were: vital signs (mean, systolic, and diastolic arterial blood pressure and heart rate) as these were not applicable in eCPR subjects (14%); baseline ventilator settings were not applicable in non-intubated subjects (13%). Indication for ECMO of the non-intubated patients were respiratory 1.4%, cardiac 8.4%, and eCPR 3.5%. Similarly, indices using PaO2 were excluded as they were not relevant in patients with congenital heart disease with the potential of intracardiac shunting (56%). Of these excluded variables only mean arerial pressure was associated with mortality by univariate analysis (p = 0.024) but was not significant by multivariate analysis using full case selection. An analysis performed excluding subjects who received eCPR also found that vital signs were not independantly predictive of mortality.
Table 3:
Univariable analyses of pre-extracorporeal membrane oxygenation candidate predictors of in-hospital mortality included in the multivariable model
| In-hospital mortality | ||||
|---|---|---|---|---|
| Variable | Odds ratio (95% CI) |
P-value | % Missing pre-imputation |
% Missing post-imputation |
| Primary ECMO indication | <0.001 | 0% | 0% | |
| Respiratory | Reference | |||
| Cardiac | 1.91 (1.30, 2.80) | |||
| eCPR | 3.14 (1.81, 5.46) | |||
| Age | 0.001 | 0% | 0% | |
| Pre-term neonate | 3.71 (1.93, 7.15) | |||
| Full-term neonate | Reference | |||
| Infant | 1.75 (1.11, 2.74) | |||
| Child | 1.43 (0.85, 2.40) | |||
| Adolescent | 1.67 (0.88, 3.19) | |||
| Acute diagnoses | ||||
| Congenital diaphragmatic hernia | 2.03 (1.15, 3.58) | 0.014 | 0% | 0% |
| Meconium aspiration syndrome | 0.10 (0.03, 0.28) | <0.001 | 0% | 0% |
| Infection | ||||
| Level of evidence | ||||
| Documented | 1.34 (0.83, 2.16) | 0.233 | 0% | 0% |
| Pathogen | ||||
| Bacterial | 1.82 (0.99, 3.37) | 0.055 | 0% | 0% |
| Viral | 0.70 (0.35, 1.43) | 0.327 | 0% | 0% |
| System | ||||
| Blood | 3.71 (1.54, 8.93) | 0.003 | 0% | 0% |
| Respiratory | 0.80 (0.46, 1.41) | 0.442 | 0% | 0% |
| Other | 2.44 (0.22, 27.12) | 0.467 | 0% | 0% |
| Labs | ||||
| pH | 0.32 (0.10, 1.02) | 0.054 | 18% | 0% |
| Partial thromboplastin time (seconds) | 1.13 (1.04, 1.22) | 0.003 | 36% | 0% |
| International normalized ratio | 1.99 (1.36, 2.90) | <0.001 | 37% | 0% |
514 records were analyzed in models for in-hospital mortality. Analysis based on non-imputed data.
ECMO = extracorporeal membrane oxygenation; eCPR = extracorporeal cardiopulmonary resuscitation
Table 4 displays the PEP model. The variables retained after final multiple logistic regression analysis included age (pre-term neonate, full-term neonate, infant, child, and adolescent), indication for ECMO, meconium aspiration syndrome (MAS), congenital diaphragmatic hernia CDH, D-BSI, pH in arterial blood, partial thromboplastin time (aPTT), and INR. The highest risk of mortality was associated with D-BSI (OR 5.26; CI 1.90–14.57) followed by eCPR (OR = 4.36; CI 2.23–8.51). In contrast, a diagnosis of MAS was protective (OR = 0.18 CI 0.05–0.63).
Table 4:
Multivariable model of in-hospital mortality
| Variable | Odds ratio (95% CI) | P-value |
|---|---|---|
| Indication for ECMO | <0.001 | |
| eCPR | 4.36 (2.23, 8.51) | |
| Cardiac | 2.42 (1.46, 4.02) | |
| Respiratory | Reference | |
| Age | 0.031 | |
| Pre-term neonate | 3.10 (1.52, 6.33) | |
| Full-term neonate | Reference | |
| Infant | 1.47 (0.87, 2.48) | |
| Child | 1.11 (0.60, 2.07) | |
| Adolescent | 1.44 (0.67, 3.07) | |
| Congenital diaphragmatic hernia | 3.11 (1.49, 6.49) | 0.002 |
| Meconium aspiration syndrome | 0.18 (0.05, 0.63) | 0.007 |
| Baseline pH in arterial blood | 0.22 (0.06, 0.80) | 0.022 |
| Partial thromboplastin time (increase of 10 seconds) | 1.07 (1.00, 1.14) | 0.048 |
| International normalized ratio | 1.45 (0.95, 2.23) | 0.085 |
| Documented blood infection prior to ECMO | 5.26 (1.90, 14.57) | 0.001 |
Estimated probability of mortality is p = ex/(1+ex), where x = 9.081 + 0.887[Indication = cardiac] + 1.468[Indication = eCPR] + 1.132[Age = pre-term neonate] + 0.378[Age = infant] + 0.109[Age = child] + 0.360[Age = adolescent] + 1.138[Congenital diaphragmatic hernia] - 1.710[Meconium aspiration syndrome] - 1.534[pH in arterial blood] + 0.0064[Partial thromboplastin time (seconds)] + 0.357[International normalized ratio] + 1.659[Documented blood infection] is the log odds of mortality from the multivariable model. Using leave-one-out cross validation to prevent small-sample bias, the area under the ROC curve is 0.75 (0.70, 0.80).
A total of 27 subjects had D-BSI prior to ECMO. Mortality by infection type was 14/20 bacterial, 4/5 viral, and 2/2 fungal. Of the seven survivors, four utilized VA ECMO. The most common organisms isolated in the blood stream were Staphylococcus aureus (n=4) and Streptococcus pneumonia (n=4) followed by adenovirus and Escherichia coli (n=2 each). Both patients with adenovirus had chronic immune dysfunction. Only one subject grew Staphylococcal epidermidis. Fevers occurred in equal proportions (8%) of those with and without a D-BSI prior to ECMO. The median time from blood culture last drawn to cannulation for those with D-BSI was 22.3 hours (see supplemental Table 2).
The c-statistic was 0.75 (95% CI, 0.70–0.80) and the Hosmer-Lemeshow p-value was 0.94, indicating modest model discrimination. When evaluating subjects without imputed data (complete case analysis; n = 298), the c-statistic improved only to 0.78. The prediction equation for probability of mortality is: p = ex/(1+ex), where x = 9.081 + 0.887[Indication = cardiac] + 1.468[Indication = eCPR] + 1.132[Age = pre-term neonate] + 0.378[Age = infant] + 0.109[Age = child] + 0.360[Age = adolescent] + 1.138[CDH] – 1.710[MAS] – 1.534[pH in arterial blood] + 0.0064[aPTT] + 0.357[INR] + 1.659[D-BSI] is the log odds of mortality from the multivariable model. An online calculator is available at https://www.cpccrn.org/calculators/ecmoprediction/.
Figure 1 displays the distribution of patients across the estimated risk of mortality and the calibration plot for observed to expected mortality demonstrating good calibration across all risk categories, notably including the lowest and highest.
Figure 1.
Distribution of estimated probability of mortality (Panel A) and the calibration plot for observed to expected mortality by risk group (Panel B). An even distribution and good calibration is demonstrated across all risk groups.
DISCUSSION
The PEP prognostic model provides an in-hospital mortality prediction model for all respiratory, cardiac, or eCPR ECMO candidates <19 years old. Of all currently existing pediatric ECMO risk adjustment models, only Neo-RESCUERS, which is limited to neonates with respiratory failure, has a higher (c-statistic (0.75 vs 0.78).
The PEP model is unique from previously published models because of the omnibus approach of incorporating all pediatric subjects who received ECMO for any indication, including eCPR subjects. Our model allows for centers to benchmark performance using all patients who receive ECMO Additionally, we identified three unique covariates associated with mortality: INR, aPTT, and D-BSI. Commensurate with prior publications, we found patient age, ECMO mode, neonatal diagnoses of MAS or CDH, and severity of acidosis pre-ECMO to be associated with mortality (2, 5, 14–16).
The odds of death conferred from a D-BSI in the PEP model exceeded all other variables. The antibiotic status of those with a D-BSI prior to ECMO is unknown, suggesting that perhaps source control prior to ECMO could be improved. Only one patient grew a coagulase negative staphylococcus organism suggesting that the infections were not merely contaminants. Usual screening methods such as fevers and leukocytosis may be insufficient given that fevers were not a senstive marker for D-BSI, and white blood cell count was not independently associated with mortality. Additionally studies evaluating the physiologic benefits and ECMO practices in the setting of blood stream infections including the variation related to antibiotic use and rapid screening is warranted.
The use of ECMO to reverse cardiac arrest is mechanistically plausible as ECMO restores both perfusion and oxygenation. The benefit of ECMO in the setting of infection is less clear given that the mainstay of treatment for infection is source control which may be more difficult to achieve and maintain on ECMO.
In prior studies, hepatic failure has been associated with greater mortality (2, 5). It remains unknown whether attempts to normalize these prolonged clotting tests prior to ECMO initiation would alter the increased mortality risk; however, these data suggest that early and deliberate screening for pre-ECMO liver injury using INR and aPTT may improve pre-ECMO risk stratification.
Due to missingness >10% after imputation the PEP model notably excludes some variables that are clinically appealing or have been predictive in prior models including measures of oxygenation and vital signs. It is important to remember that oxygenation is largely determined by the clinician’s preference for escalating support and is not driven by protocol. Moreover, vital signs in the setting of sedation and interventions such as dexmetetomidine and cooling do not accurately reflect the patient’s state or degree of illness. Nonetheless, we did perform a separate multivariate analysis using only full case selection to allow for inclusion of vital signs which also demonstrated no significant association with mortality.
Implications
The value of the PEP model is that it can be applied to all pediatric ECMO patients without excluding for age or ECMO indication. Thus, it may afford improved discrimination into the center specific processes that influence survival across cohorts that are currently not evaluated with existing models as well as anticipate and benchmark mortality for similar ECMO cohorts (10, 11, 37).
The PEP model performance decreases from 0.75 to 0.73 (95% CI, 0.69–0.77; p <0.05) when using only variables available in the ELSO registry at time of model development (age, indication, CDH, MAS, D-BSI, and baseline arterial pH) suggesting that the collection of pre-ECMO aPTT and INR could improve calibration of risk among ELSO centers. External validation using the ELSO registry and prospective validation is warranted.
Limitations
The results must be interpreted within the limitations of the study design. First, because there is no comparison database of non-ECMO patients with similar illnesses, the findings only apply to pediatric cohorts for whom ECMO is chosen as a therapy and is not intended for individual patient predictions or selection for ECMO. Second, the model is limited to covariates available in the BATE database. Therefore, it is possible that other non-measured exposures such as antibiotic use might serve as better predictors or were covariants with variables included in our model. For example, center infrastructure covariates were not analyzed (20). Third, misclassification may have been introduced because many of the diagnostic categories used lack explicit definitions. Fourth, a larger sample size may detect a mortality difference across centers (37–39). Fifth, the data collection concluded in 2014 and calibration over time will be required given that pre-ECMO care and patient factors continue to evolve (2, 21).
Finally, missingness prior to imputation was not proportional across all variables, and therefore, may not have been missing at random. Vital signs were not included in the final model because they were not recorded during eCPR which represented >10% of subjects, however none of them were independently predictive by multivariate analuysis using full case selection or after excluding patients who received eCPR. The finding that the c-statistic using full case selection was relatively similar to the findings using the imputed data (0.75 vs. 0.78) further mitigates the concerns related to the missingness of values and the use of imputation.
CONCLUSIONS
The PEP model represents a model to prognosticate in-hospital mortality for all patients <19 years of age who receive ECMO for any indication. Because it is not limited to any narrow group of patients (e.g neonates with respiratory failure) it can be applied to all patients at a single instution or across institutions which is a current limitation of all existing models. It expands upon current models by including commonly monitored variables not previously available in the ELSO registry such as INR and aPTT. Consequently, it is anticipated that the model will be most useful for risk stratifying patient specific features that influence ECMO mortatlity in order to improve benchmarking of ECMO performance.
Supplementary Material
Acknowledgements
We thank Drs. Robert Tamburro and Tammara Jenkins for their valuable input.
Grants and Support:
This work was supported by the following cooperative agreements from the Eunice Kennedy Shriver National Institute of Child Health and Human Development, National Institutes of Health, Department of Health and Human Services: U10HD050096, U10HD049981, U10HD049983, U10HD050012, U10HD063108, U10HD063114, and U01HD049934.
Abbreviations:
- D-BSI
documented blood stream infection
- ECMO
extracorporeal membrane oxygenation
- ELSO
extracorporeal life support organization
- BATE
bleeding and thrombosis on ECMO
- INR
international normalized ratio
- IQR
Interquartile range
- aPTT
activated partial thromboplastin time
- CDH
congenital diaphragmatic hernia
- MAS
meconium aspiration syndrome
Footnotes
Copyright form disclosure: Drs. Bailly, Reeder, Pollack, Moler, Meert, Berg, Carcillo, Zuppa, Newth, Berger, Bell, Dean, Garcia-Filion, Wessel, Heidemann, Harrison, and Dalton received support for article research from the National Institutes of Health (NIH). Drs. Reeder, Pollack, Meert, Berger, Wessel, Heidemann, Doctor, Harrison, and Dalton’s institutions received funding from the NIH. Dr. Barbaro disclosed that he is the Extracorporeal Life Support Organization (ELSO) Registry Chair. Drs. Moler, Berg, Carcillo, Zuppa, Newth, Bell, and Dean’s institutions received funding from the National Institute of Child Health and Human Development. Dr. Moler’s institution also received funding from the National Heart, Lung, and Blood Institute. Dr. Carcillo’s institution also received funding from the National Institute of General Medical Sciences. Dr. Newth received funding from Philips Research North America. Dr. Berger’s institution also received funding from the Association of Pediatric Pulmonary Hypertension and Actelion. Dr. Nicholson disclosed government work. Dr. Doctor’s institution also received funding from the Department of Defense and Kalocyte. Dr. Dalton received funding from Innovative ECMO Concepts (consultant). Ms. Winder has disclosed that she does not have any potential conflicts of interest
REFERENCES
- 1.Annich GM, Lynch WR, MacLaren G, et al. : Extracorporeal Cardiopulmonary Support in Critical Care. Fifth. Ann Arbor, Extracorporeal Life Support Organization; 2017 [Google Scholar]
- 2.Bailly DK, Reeder RW, Zabrocki LA, et al. : Development and Validation of a Score to Predict Mortality in Children Undergoing Extracorporeal Membrane Oxygenation for Respiratory Failure: Pediatric Pulmonary Rescue With Extracorporeal Membrane Oxygenation Prediction Score. Crit Care Med 2016; 1–9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Nance ML, Nadkarni VM, Hedrick HL, et al. : Effect of preextracorporeal membrane oxygenation ventilation days and age on extracorporeal membrane oxygenation survival in critically ill children. J Pediatr Surg 2009; 44:1606–1610 [DOI] [PubMed] [Google Scholar]
- 4.Minneci PC, Kilbaugh TJ, Chandler HK, et al. : Factors associated with mortality in pediatric patients requiring extracorporeal life support for severe pneumonia. Pediatr Crit Care Med 2013; 14:e26–33 [DOI] [PubMed] [Google Scholar]
- 5.Zabrocki LA, Brogan T V, Statler KD, et al. : Extracorporeal membrane oxygenation for pediatric respiratory failure: Survival and predictors of mortality. Crit Care Med 2011; 39:364–70 [DOI] [PubMed] [Google Scholar]
- 6.Jacobs JP, O’Brien SM, Pasquali SK, et al. : The Society of Thoracic Surgeons Congenital Heart Surgery Database Mortality Risk Model: Part 2-Clinical Application. Ann Thorac Surg 2015; 100:1063–1070 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Thabut G, Christie JD, Kremers WK, et al. : Survival differences following lung transplantation among US transplant centers. J Am Med Assoc 2010; 304:53–60 [DOI] [PubMed] [Google Scholar]
- 8.Richardson DK, Gray JE, McCormick MC, et al. : Score for Neonatal Acute Physiology: a physiologic severity index for neonatal intensive care. Pediatrics 1993; 91:617–623 [PubMed] [Google Scholar]
- 9.Cohen ME, Ko CY, Bilimoria KY, et al. : Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus. J Am Coll Surg 2013; 217:336–46.e1 [DOI] [PubMed] [Google Scholar]
- 10.Pollack MM, Holubkov R, Funai T, et al. : Simultaneous prediction of new morbidity, mortality, and survival without new morbidity from pediatric intensive care: A new paradigm for outcomes assessment. Crit Care Med 2015; 43:1699–1709 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Pollack MM, Ruttimann UE, Getson PR: Accurate prediction of the outcome of pediatric intensive care. A new quantitative method. N Engl J Med 1987; 316:134–139 [DOI] [PubMed] [Google Scholar]
- 12.Krumholz HM, Wang Y, Mattera JA, et al. : An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with heart failure. Circulation 2006; 113:1693–1701 [DOI] [PubMed] [Google Scholar]
- 13.Krumholz HM: Standards for statistical models used for public reporting of health outcomes: an American Heart Association scientific statement from the quality of care and outcomes research interdisciplinary writing group. Circulation 2006; 113:456–462 [DOI] [PubMed] [Google Scholar]
- 14.Barbaro RP, Boonstra PS, Paden ML, et al. : Development and validation of the pediatric risk estimate score for children using extracorporeal respiratory support ( Ped - RESCUERS ). Intensive Care Med 2016; 42:879–888 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Barbaro RP, Bartlett RH, Chapman RL, et al. : Development and validation of the neonatal risk estimate score for children using extracorporeal respiratory support. J Pediatr 2016; 173:56–61.e3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Maul TM, Kuch BA, Wearden PD: Development of Risk Indices for Neonatal Respiratory Extracorporeal Membrane Oxygenation. ASAIO J 2016; 62:584–590 [DOI] [PubMed] [Google Scholar]
- 17.Schmidt M, Bailey M, Sheldrake J, et al. : Predicting survival after extracorporeal membrane oxygenation for severe acute respiratory failure. The Respiratory Extracorporeal Membrane Oxygenation Survival Prediction (RESP) score. Am J Respir Crit Care Med 2014; 189:1374–1382 [DOI] [PubMed] [Google Scholar]
- 18.Dalton HJ, Garcia-Filion P, Holubkov R, et al. : Association of Bleeding and Thrombosis With Outcome in Extracorporeal Life Support. Pediatr Crit Care Med 2015; 16:167–174 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Collins GS, Reitsma JB, Altman DG, et al. : Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): The TRIPOD Statement. Eur Urol 2015; 67:1142–1151 [DOI] [PubMed] [Google Scholar]
- 20.Pathan N, Ridout DA, Smith E, et al. : Predictors of outcome for children requiring respiratory extra-corporeal life support: implications for inclusion and exclusion criteria. Intensive Care Med 2008; 34:2256–2263 [DOI] [PubMed] [Google Scholar]
- 21.Zabrocki LA, Brogan T V, Statler KD, et al. : Extracorporeal membrane oxygenation for pediatric respiratory failure: Survival and predictors of mortality. Crit Care Med 2011; 39:364–370 [DOI] [PubMed] [Google Scholar]
- 22.Gow KW, Heiss KF, Wulkan ML, et al. : Extracorporeal life support for support of children with malignancy and respiratory or cardiac failure: The extracorporeal life support experience. Crit Care Med 2009; 37:1308–1316 [DOI] [PubMed] [Google Scholar]
- 23.Feudtner C, Feinstein JA, Zhong W, et al. : Pediatric complex chronic conditions classification system version 2: updated for ICD-10 and complex medical technology dependence and transplantation. BMC Pediatr 2014; 14:199. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Centers for Disease Control and Prevention: Division of Nutrition, Physical Activity, and Obesity: Growth Chart Training. 2017. Available at: http://www.cdc.gov/nccdphp/dnpao/growthcharts/resources/. Accessed October 25, 2017.
- 25.Finney H: Reference ranges for plasma cystatin C and creatinine measurements in premature infants, neonates, and older children. Arch Dis Child 2000; 82:71–75 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Zappitelli M, Parikh CR, Akcan-Arikan A, et al. : Ascertainment and epidemiology of acute kidney injury varies with definition interpretation. Clin J Am Soc Nephrol 2008; 3:948–54 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.It F, Graded N, Graded N, et al. : Section 2: AKI Definition. Kidney Int Suppl 2012; 2:19–36 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Fleming GM, Sahay R, Zappitelli M, et al. : The Incidence of Acute Kidney Injury and Its Effect on Neonatal and Pediatric Extracorporeal Membrane Oxygenation Outcomes. Pediatr Crit Care Med 2016; 17:1157–1169 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Gaies MG, Gurney JG, Yen AH, et al. : Vasoactive-inotropic score as a predictor of morbidity and mortality in infants after cardiopulmonary bypass. Pediatr Crit Care Med 2010; 11:234–8 [DOI] [PubMed] [Google Scholar]
- 30.Gaies Michael G., Jeffries Howard E., Niebler Robert A., Pasquali Sara K., Donohue Janet E., Yu Sunkyung, Christine Gall TBR: Vasoactive-Inotropic Score (VIS) is Associated with Outcome After Infant Cardiac Surgery: An Analysis from the Pediatric Cardiac Critical Care Consortium (PC4 ) and Virtual PICU System Registries. Ped Crit Care Med 2015; 15:529–537 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Bonafide CP, Brady PW, Keren R, et al. : Development of heart and respiratory rate percentile curves for hospitalized children. Pediatrics 2013; 131:e1150–7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Pollack MM, Patel KM, Ruttimann UE: PRISM III: An updated pediatric risk of mortality score. Crit Care Med 1996; 24:743–752 [DOI] [PubMed] [Google Scholar]
- 33.Leteurtre S, Duhamel A, Salleron J, et al. : PELOD-2: an update of the PEdiatric logistic organ dysfunction score. Crit Care Med 2013; 41:1761–1773 [DOI] [PubMed] [Google Scholar]
- 34.Kleinman ME, Chameides L, Schexnayder SM, et al. : Pediatric advanced life support: 2010 American Heart Association Guidelines for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care. Am Acad Pediatr 2006; 117:e1361–e1399 [DOI] [PubMed] [Google Scholar]
- 35.Wood AM, White IR, Royston P: How should variable selection be performed with multiply imputed data? Stat Med 2008; 27:3227–3246 [DOI] [PubMed] [Google Scholar]
- 36.Rubin D: Multiple Imputation for Nonresponse in Surveys. Hoboken, NJ: John Wiley & Sons, Inc.; 1987. [Google Scholar]
- 37.Barbaro RP, Odetola FO, Kidwell KM, et al. : Association of hospital-level volume of extracorporeal membrane oxygenation cases and mortality. Analysis of the extracorporeal life support organization registry. Am J Respir Crit Care Med 2015; 191:894–901 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Barrett CS, Chan TT, Wilkes J, et al. : Association of Pediatric Cardiac Surgical Volume and Mortality After Cardiac ECMO. ASAIO J 2017; 63:802–809 [DOI] [PubMed] [Google Scholar]
- 39.Bratton SL, Chan T, Barrett CS, et al. : Metrics to Assess Extracorporeal Membrane Oxygenation Utilization in Pediatric Cardiac Surgery Programs. Pediatr Crit Care Med 2017; 18:779–786 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.