Development and validation of a model to predict mortality risk among extremely preterm infants during the early postnatal period: a multicentre prospective cohort study

Abstract

Background

Recently, with the rapid development of the perinatal medical system and related life-saving techniques, both the short-term and long-term prognoses of extremely preterm infants (EPIs) have improved significantly. In rapidly industrialising countries like China, the survival rates of EPIs have notably increased due to the swift socioeconomic development. However, there is still a reasonably lower positive response towards the treatment of EPIs than we expected, and the current situation of withdrawing care is an urgent task for perinatal medical practitioners.

Objective

To develop and validate a model that is practicable for EPIs as soon as possible after birth by regression analysis, to assess the risk of mortality and chance of survival.

Methods

This multicentre prospective cohort study used datasets from the Sino-Northern Neonatal Network, including 46 neonatal intensive care units (NICUs). Risk factors including maternal and neonatal variables were collected within 1 hour post-childbirth. The training set consisted of data from 41 NICUs located within the Shandong Province of China, while the validation set included data from 5 NICUs outside Shandong Province. A total of 1363 neonates were included in the study.

Results

Gestational age, birth weight, pH and lactic acid in blood gas analysis within the first hour of birth, moderate-to-severe hypothermia on admission and adequate antenatal corticosteroids were influencing factors for EPIs’ mortality with important predictive ability. The area under the curve values for internal validation of our prediction model and Clinical Risk Index for Babies-II scores were 0.81 and 0.76, and for external validation, 0.80 and 0.51, respectively. Moreover, the Hosmer-Lemeshow test showed that our model has a constant degree of calibration.

Conclusions

There was good predictive accuracy for mortality of EPIs based on influencing factors prenatally and within 1 hour after delivery. Predicting the risk of mortality of EPIs as soon as possible after birth can effectively guide parents to be proactive in treating more EPIs with life-saving value.

Trial registration number

ChiCTR1900025234.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• Development of a mortality prediction model based on a multicentre prospective cohort.

• The population included in the training and validation sets covers a wide geographical area and is well represented.

• Published equation and online nomogram for reproducibility and clinical translation of their final model.

• As attitudes towards active care of extremely preterm infants shift, the risk of death will also change, and this model may need to be recalibrated frequently to remain accurate.

• The applicability to other situations requires more data for external validation.

Introduction

The United Nations’ sustainable development plan proposes a goal to end preventable deaths of newborns by 2030.¹ Premature infants, especially those born at a gestational age (GA) of less than 28 weeks, still face a high risk of neonatal death worldwide.² Developed countries advocate for active treatment for extremely preterm infants (EPIs) who are above the limit of viability.^3–5 However, for low-income and middle-income countries like China, the survival of EPIs is not only restricted by the level of treatment, but also depends on the parents’ voluntary choice of avoidable death—withdrawal of care.⁶

Our provincial perinatal centre’s real-world study from 2010 to 2019 showed that 74.1% of EPIs with stable vital signs died due to withdrawing care in the delivery room (DR) and neonatal intensive care unit (NICU).⁷ Additionally, our findings indicate that the primary factor leading to care withdrawal was parental lack of confidence, rather than the perceived loss of survival potential for the EPIs.⁸ As a rapidly industrialising country, China’s life-saving capability has been enhanced with the rapid development of its economy recently. Nevertheless the lack of a clear definition for the survival limit in the country has resulted in a less proactive approach towards rescuing EPIs. It is urgent for us to develop a model to scientifically predict the risk of death in early post partum, so as to reduce withdrawal of care due to poor communication between doctors and patients and to keep EPIs who are likely to survive from death due to treatment being withdrawn.

Developed countries have sequentially implemented mortality prediction models to estimate the early survival probability of premature infants and formulate more proactive guidelines.^9–11 Among them, Clinical Risk Index for Babies (CRIB)-II is commonly used. In both the Korean Neonatal Network (area under the curve (AUC) >0.8) and the Australian population (AUC=0.913), CRIB-II demonstrated superior performance in predicting mortality compared with GA or birth weight (BW) alone.^{12 13} But among neonates born at 32 weeks’ gestation or earlier in the UK and the Gambia, it fared a bit worse (AUC <0.8).¹⁰ Therefore, the applicability of the model to populations in our region deserves further exploration.

Based on the clinical research database—Sino-Northern Neonatal Network (SNN), this study prospectively collected the case data of EPIs in 46 NICUs, and established a CARE-Preterm (Chinese Adverse Prognosis of Very Preterm infants cohort study) prospective cohort. The data from 41 units located in Shandong Province, which has consistently had the highest fertility rate in China, are used as the training set. Data from an additional five units, situated in provinces with significant populations, were employed for external validation. We collected objective and accurate variables on demographic variables, antenatal management and within 1 hour of birth to develop predictive models for assessing the risk of mortality in EPIs, so as to establish a foundation for enhancing parental confidence in doctor–patient communication and facilitating life-saving treatments.

Methods

Data sources, participants and outcome

We carried out a multicentre cohort study based on a prospective cohort from the SNN. The cohort study collected EPIs with a GA less than 28 weeks who were admitted between 1 January 2018 and 30 June 2022 from 46 NICUs. Cases with incomplete information, congenital malformations and withdrawal of care due to socioeconomic considerations were excluded. Among all the units, 41 are located in Shandong Province of China and are used as training sets for model establishment. The five NICUs outside Shandong Province are considered as the validation set for external validation of the model (online supplemental figure 1). Case data mainly include maternal characteristics, perinatal information, neonatal characteristics, treatment information, prognosis and death classification.

The outcome variable was death. Outcome assessments were completed at 40 weeks of corrected GA or before discharge. Potential risk factors (variables) for the outcome were selected from the database based on the existing literature and relevance to the outcomes.

Death category

All of these infants received intensive care initially. We have strict criteria for categorising infants whose deaths occur after admission to the NICU: (1) redirection of care (ROC): infants who have developed a series of very serious complications or have reached the end of their lives during active treatment; (2) socioeconomic considerations (SEC): the infants’ guardians lacked the financial support or worried about possible severe sequelae, even when infants did not suffer severe neurological injury or in terminal status; (3) maximal intensive care (MIC): life-sustaining therapies such as ventilatory and cardiovascular support and resuscitation efforts were pursued until death was pronounced. ROC and MIC were categorised as active treatment, while SEC was classified as withdrawal of care.⁷ After the exclusion of 240 deaths due to withdrawal of care, all deaths in this paper occurred after active treatment.

Important diagnostic definitions

GA was assessed by early pregnancy ultrasound, obstetric examination and obstetric history. If there was a 2-week difference between obstetric and paediatric assessments, paediatric assessments were used.¹⁴ According to the definition of the WHO, the normal temperature is 36.5~37.5℃, and the admission temperature <36℃ is moderate-to-severe hypothermia.¹⁵ The pH and lactate values from the first blood gas analysis within 1 hour of birth (if umbilical artery blood was spared at delivery, the results of umbilical artery blood gas analysis was preferred) were used. Hypertensive disorders of pregnancy was defined as systolic blood pressure ≥140 mm Hg and/or diastolic blood pressure ≥90 mm Hg measured at least twice in the same arm.¹⁶ Gestational diabetes mellitus is diabetes mellitus that is metabolically normal or potentially hypoglycaemic prior to pregnancy, but is only present or diagnosed during pregnancy.¹⁷ Chorioamnionitis is a term encompassing a broad spectrum of disease during pregnancy that is characterised by inflammation and/or infection of intrauterine structures such as the placenta, chorion and amnion.¹⁸ Delayed umbilical cord clamping is when the cord is ligated 30 s after the fetus is delivered.¹⁹ Abnormal amniotic fluid includes bloody amniotic fluid and contaminated amniotic fluid. A course of antenatal corticosteroids is dexamethasone sodium phosphate 24 mg intramuscularly in two divided doses of 12 mg at 24-hour intervals or in four divided doses of 6 mg at 12-hour intervals. Adequate antenatal corticosteroid treatment consists of one or more courses.²⁰

Statistical analysis methods

In this study, after identifying the important significant variables by reviewing the literature, we chose the most classically used stepwise selection for further screening of variables and modelling of the risk of in-hospital mortality. The stepwise selection method can take into account the correlation between variables and reduce the complexity of the model.^{21 22} Then, we combine theoretical knowledge to ensure the reasonableness of the selected predictor variables. Discrimination—that is, the ability of the model to correctly predict life or death—was assessed by calculating receiver operating characteristic curves and their associated AUC. An AUC value of 0.5 indicates no ability to discriminate, and larger values indicate increasing ability. A value of 0.8 is considered good. The Hosmer-Lemeshow (HL) test was used to measure the goodness of fit of the models. This test compares the difference between observed and expected mortality. A non-significant p value of the HL statistic indicates a model with a constant discriminatory ability across strata. Internal validation was performed using 10-fold cross-validation on the training set data. The validation set data are then substituted into the model for external validation. Finally, the model is displayed in the form of a nomogram. R Studio V.4.1.2 (R Project for Statistical Computing) statistical software package was used for all data analyses.

Informed consent and trial registration

Informed parental consent was obtained for all newborns at the time of admission. SNN is a clinical research database in China and has strict data entry and quality control standards. Trained data abstractors prospectively collected infant information at each NICU. The research protocol has been registered in the Chinese Clinical Trial Registration Center (ID: ChiCTR1900025234).

Patient and public involvement

Patients and/or the general public were not involved in the design, execution or drafting of this secondary analysis.

Results

In this study, the included 46 hospitals reported EPI deliveries ranging from 11 to 154. The mortality rates associated with withdrawing care ranged from 5.1% to 37.5%, while the mortality rates after active care ranged from 5.6% to 38.9%. During the study, a total of 1393 EPIs were recorded, excluding 6 cases with incomplete data and 1 case due to congenital malformations. Additionally, there were 240 cases (240 of 1386, 17.3%) where care was withdrawn; thus, 1146 infants were included for analysis from 1 January 2018 to 30 June 2022. Among them, 1015 cases were in the training set while 131 were in the validation set. The mortality rates in the training and validation sets were 23.0% (233 of 1015) and 21.4% (28 of 131), respectively. The mortality rate was 23.0% (233 of 1015) in the training set and 21.4% (28 of 131) in the validation set. Figure 1 presents a flow diagram illustrating the inclusion of infants in the study.

Figure 1.

Participant flow diagram. EPIs, extremely preterm infants.

The univariate analysis of the training dataset is displayed in table 1. The following variables were associated with NICU mortality: GA, BW, lactic acid, pH, multiple births, sex, moderate-to-severe hypothermia, adequate antenatal corticosteroids and premature rupture of fetal membranes ≥24 hours.

Table 1.

Univariate analysis from the training set (n=1015)

Variables	Survival (n=782）	Death (n=233）	OR	95% CI	P value
Continuous variables, median (IQR)
Gestational age, weeks	27.1 (26.4–27.6）	26.3 (25.4–27.1）	0.450	0.383 to 0.526	<0.001
Birth weight, g	980 (850–1100）	850 (700–970）	0.996	0.995 to 0.997	<0.001
Lactic acid, mmol/L	2.4 (1.6–4.5）	4.3 (2.3–6.5）	1.203	1.142 to 1.268	<0.001
Lowest base deficit	−4.6 (–7.0, –2.6)	−6.1 (–9.7, –3.4)	0.926	0.893 to 0.959	<0.001
Categorical variables, no (%)
pH <7	35 (4.4）	41 (17.5）	4.558	2.828 to 7.323	<0.001
Multiple births	216 (27.6)	91 (39.0)	1.679	1.234 to 2.279	0.001
Sex (male)	453 (57.94)	132 (56.6)	0.949	0.707 to 1.277	0.046
Moderate-to-severe hypothermia	301 (38.4)	161 (69.0)	3.573	2.623 to 4.908	<0.001
Caesarean section	330 (42.1)	91 (39.0)	0.878	0.650 to 1.182	0.393
Abnormal amniotic fluid	95 (12.1)	34 (14.5)	1.236	0.801 to 1.868	0.326
Adequate antenatal corticosteroids	341 (43.6)	42 (18.0)	0.284	0.196 to 0.405	<0.001
Antenatal magnesium	345 (44.1)	106 (45.4)	1.057	0.787 to 1.418	0.711
Gestational diabetes	106 (13.5)	29 (12.4)	0.907	0.575 to 1.390	0.662
Maternal hypertension	110 (14.0)	28 (12.0)	0.834	0.527 to 1.283	0.424
Premature rupture of fetal membranes ≥24 hours	137 (17.5)	49 (21.0)	0.656	0.447 to 0.946	0.027
Chorioamnionitis	67 (8.5)	19 (8.1)	0.947	0.543 to 1.582	0.842
Antibiotics within 24 hours before delivery	316 (40.4)	107 (45.9)	1.252	0.932 to 1.681	0.134
Delayed umbilical cord clamping	208 (26.5)	63 (27.0)	1.023	0.732 to 1.416	0.894

From the stepwise logistic regression multivariate analysis, a six-variable logistic regression model was generated (table 2). GA, BW, lactic acid, pH, moderate-to-severe hypothermia on admission and adequate antenatal corticosteroids were retained in the final model. Among them, GA, BW, lactic acid, pH and moderate-to-severe hypothermia are independent risk factors for death. Additionally, adequate antenatal corticosteroids were found to effectively reduce the risk of mortality. The equation used to generate the probability of death from the model is as follows: logit=14.107–0.519 GA−0.002 BW+0.077 lactic acid+0.894 PH+0.940 moderate-to-severe hypothermia−1.130 adequate antenatal corticosteroids. The probability of death (Y) is given by the equation Y=exp(logit)/[1+exp(logit)].

Table 2.

Multivariate analysis from the training set

Variables	β	SE	P value	OR	95% CI
Gestational age	−0.519	0.107	<0.001	0.595	0.481 to 0.732
Birth weight	−0.002	0.001	0.002	0.998	0.997 to 0.999
Lactic acid	0.077	0.037	0.041	1.080	1.003 to 1.162
pH	0.894	0.343	0.009	2.445	1.244 to 4.797
Moderate-to-severe hypothermia	0.940	0.177	<0.001	2.561	1.816 to 3.634
Adequate antenatal corticosteroids	−1.130	0.207	<0.001	0.323	0.213 to 0.480

Our logistic model for neonatal mortality risk scoring had an AUC value on derivation data equal to 0.81 (95%CI 0.777 to 0.838) for the NICUs in Shandong Province of China. Validation of the CRIB-II score using the same derived data resulted in an AUC of 0.75 (95% CI 0.720 to 0.793). As shown by the receiver operating characteristic curve of the training set consisting of data from NICUs outside Shandong Province, comparison of areas under the receiver operating characteristic curves for our model (0.80 (95% CI 0.777 to 0.838)) and CRIB-II (0.51 (95% CI 0.471 to 0.553)) indicated that discriminatory performance of our model was superior to that of CRIB-II (figure 2).

Figure 2.

Receiver operating characteristic curves for derivation and validation datasets of our model with the CRIB-II scores. (A) Receiver operating characteristic curves for derivation data from our model; (B) receiver operating characteristic curves for derivation data from CRIB-II scores; (C) receiver operating characteristic curves for validation data from our model; (D) receiver operating characteristic curves for validation data from CRIB-II scores. AUC, area under the curve; CRIB-II, Clinical Risk Index for Babies-II.

Calibration of the model was visually accurate since observed and predicted mortality risks were similar, as shown in figure 3. P value of the HL is 0.428 (>0.05). A non-significant p value of the HL statistic indicates a model with a constant discriminatory ability across strata.

Figure 3.

Calibration plots of validation datasets.

An example nomogram is shown in figure 4. Each of the six predictors is assigned a score based on its weight, and these scores are then summed to calculate the total points. The total score is directly correlated with the risk of mortality. In simpler terms, the higher the overall score for this infant, the more likely it is to die than others.

Figure 4.

A simplified nomogram to predict mortality among extremely preterm infants. Our model’s online forecasts are available at: https://xieyt2000.github.io/death_prob/ (usage instructions: (1) for continuous variables, enter true gestational age and birth weight; (2) for categorical variables, enter 1 for adequate antenatal corticosteroids, 1 for moderate-to-severe hypothermia and 1 for pH <7; the reverse is entered as 0).

Online supplemental table 1 describes the survival outcomes according to GA stratification. The survival rate is extremely low for EPIs who are near the limit of survival, such as those born at 23 or 24 weeks of gestation. With advancing GA, there is a gradual improvement in the success rate of treatment. Nonetheless, for more advanced GAs, such as 26 or 27 weeks, a significant proportion of deaths can be attributed to the withdrawal of care.

Further, we compared a number of prenatal complications and infant characteristics in online supplemental table 2. Statistical analysis reveals no significant differences in the characteristics of infants admitted to the NICU who had care withdrawn compared with those who received active treatment.

Discussion

The topic of withholding or withdrawal of medical treatment from EPIs who are on the edge of survival has been a subject of debate for several decades.^{23 24} Due to advancements in perinatal medicine and technology, the level of care for preterm infants in China has significantly improved, resulting in a success for treating older preterm infants that is comparable with that of developed countries. In developed countries like Japan and Germany with relatively positive attitudes towards treatment, survival rates for preterm infants under 24 weeks have reached high levels.²⁵ In China, there is ambiguity in defining survival limit for preterm. Previous textbooks still define preterm births as occurring 28–37 weeks, while those born before 28 weeks are categorised as late abortions.^{26 27} Our provincial perinatal centre’s real-world study found that 74.1% of all EPIs born alive died as a result of withdrawing care in the DR and NICU.⁷ With the rapid development of industrialisation, the main reason for withdrawing care is no longer poverty, rather the guiding conversation of medical staff in the process of communication between doctors and patients makes parents lose confidence in treatment.^25–30 We need to find a way to predict the risk of death so that parents can be as positive as possible in choosing life-saving treatments. The CRIB-II score is a widely used tool in neonatal wards worldwide.³¹ Nevertheless, our analysis of the study data revealed its limited efficiency, thereby posing challenges in meeting our clinical requirements. Consequently, we incorporated objective and accurate variables into our prediction model for assessing the risk of death. This approach aims to provide guidance to neonatologists, enabling them to make more informed medical decisions and ultimately reduce the occurrence of preventable neonatal deaths.

Consistent with the current study, the survival rate of EPIs is closely related to the demographic characteristics, including GA and BW, with GA being influential.^32–35 Countries such as the USA, Japan, Sweden and the UK are beginning to look at pushing down the survival threshold to 22 weeks for providing active medical intervention for preterm infants.³⁶ In developed European countries, a majority (82–96%) of practitioners will actively resuscitate EPIs at 24 weeks, and a higher proportion (85.4–100%) choose to actively treat EPIs at 25 weeks.³⁷ As the level of survival improved, our treatment success rates from 23 weeks to 28 weeks were 10.0%, 34.1%, 62.5%, 72.6% and 86.6%, respectively. But in this study, the proportion of EPIs at older GAs such as 26 and 27 weeks, where withdrawal of care resulted in death, was still up to 19.1% and 13.0%. By maintaining a positive attitude towards neonatal resuscitation, we can increase the chances of survival for more newborns.

Besides demographic characteristics, pH lactic acid levels indicate the fundamental vital signs and internal conditions during delivery, particularly in cases of asphyxia resuscitation. In our univariate analysis, the ORs for pH and lactate were approximately 4.6 and 1.2, which were strongly associated with mortality. Previous evidence-based medical research on acidosis has indicated a close relationship between umbilical artery blood gas pH values and neonatal mortality as well as severe nervous system disorders.^{38 39} The level of blood lactic acid can serve as an objective indicator for assessing the degree of neonatal asphyxia, as it reflects the extent of hypoxia, which is closely associated with the disease’s severity and prognosis.^{40 41} Animal experiments also confirmed that lactic acid was produced earlier during the process of asphyxia, which could be used as an early warning signal.⁴² It follows that postnatal umbilical artery blood gases are both the gold standard for asphyxia and a reliable predictor of death.

Moderate and severe hypothermia at admission was also an independent risk factor for death of EPIs in this cohort. Several clinical studies have consistently identified hypothermia in premature infants as a primary risk factor for neonatal mortality and severe complications.^43–45 Our previous retrospective survey of 24 NICUs, conducted based on the SNN, revealed that the hospital admission rate of hypothermia among very low BW infants reached an alarming 89.3%. Moreover, infants in the moderate-to-severe hypothermia group faced a fourfold higher risk of mortality compared with those in the normothermia group.⁴⁶ During the resuscitation and transfer of preterm infants, our team paid more attention to the warmth aspect, which reduced the incidence of hypothermia to 88.2%.^{47 48} After further strengthening of leadership execution, the incidence of hypothermia dropped to 77.9%.¹⁵ After quality improvement for hypothermia, there is still a gap in the incidence of hypothermia in our country compared with foreign countries. In fact, in developing countries like China, where a significant number of hospitals have not implemented hypothermia quality improvement programmes, the prevalence of hypothermia is even more pronounced. This highlights the significance of hypothermia as both a contributing factor to mortality and a crucial variable for predicting future fatalities.

In recent years, adequate antenatal corticosteroids have been one of the most concerning contents of the perinatal management of premature infants, especially for the EPIs of 22–25 weeks.^49–51 The data of this study showed that the utilisation of antenatal corticosteroids in the surviving infants was 43.6%, while 18% in the treatment failed. The issue of very low antenatal steroid exposure may be related to our attitudes towards EPIs at younger GAs. Owing to diferences in medical condition, economic foundation and cultural background, the minimum GA recommendation for active treatment in developing countries, such as China, is ambiguous. As a result, there is a lack of confidence in the successful rescue of EPIs, leading to a relatively lower level of proactive antenatal management. This factor is also a significant contributor to the region’s ineligibility for CRIB-II. While our model may not be universally applicable to centres with high or improving rates of antenatal steroid administration, it could still prove valuable in developing countries that exhibit comparable rates of steroid usage. Although the utilisation rate is lower compared with that of developed countries, it still plays a crucial role in preventing mortality. Numerous studies on glucocorticoids have demonstrated the substantial benefits of antenatal corticosteroids in enhancing the survival rates of EPIs with low GA and reducing the incidence of severe complications. Even if due to special pathological factors such as emergency labour or caesarean section, if the pregnant mother fails to complete the use of adequate antenatal corticosteroids, giving one dose of this hormone can still reduce the mortality by 26% 3 hours before delivery.^52–54 Therefore, antenatal corticosteroids should be included as important factors in mortality prediction models.

We carried out a multicentre cohort study based on a prospective cohort, and the data were divided into a training set and a validation set to further incorporate objective and accurate variables for model building. Then, we also created an online website to facilitate timely evaluation (https://xieyt2000.github.io/death_prob/). As shown by the receiver operating characteristic of the training set consisting of data from NICUs, comparison of areas under the receiver operating characteristic curves for our model and CRIB-II indicated that discriminatory performance of our model was superior to that of CRIB-II in both internal and external validations. The stepwise progression employed in this study for variable selection is a method that has the potential to result in overfitting of the model. Moreover, the stepwise progression is an automated variable screening process that has the potential to exclude significant variables or result in a poorly interpretable model. Therefore, the variables selected for the model must be evaluated within the clinical context. We conducted an external validation and incorporated our understanding of clinical theory to ensure the justification of variable selection. As attitudes towards active care of EPIs both prenatally and postnatally shift, so as the risk of mortality, and our model will need frequent recalibration to remain accurate. CRIB-II performance will likely improve as our model becomes less applicable.

In summary, the models developed by incorporating demographic variables such as GA and BW, prenatal management variables such as antenatal corticosteroids, objectively accessible variables within 1 hour of birth such as lactic acid, pH and moderate-to-severe hypothermia on admission were able to perform well for the estimation of mortality risk. In contrast to CRIB-II, our model is tailored to meet the specific requirements of the local population. By using our model, more infants with life-saving potential can receive appropriate treatment, leading to a significantly improved prognosis for these vulnerable individuals. Furthermore, our model holds the potential for broader applicability across different geographical regions and populations, especially in low-income and middle-income countries.

Conclusions

It is feasible to predict the risk of death using demographic variables, prenatal management factors and factors affecting vital signs. The model helps to guide parents in choosing more aggressive treatment for infants who can be saved, thereby avoiding preventable deaths and making reasonable medical decisions.

Data availability statement

Data are available upon reasonable request. The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

Ethics statements

Patient consent for publication

Not required.

Ethics approval

This study involves human participants and was approved by the Ethics Committee of the Provincial Hospital Affiliated to Shandong First Medical University (LCYJ no. 2019-132). Informed consent to participate in the study was obtained from participants’ parents.