Development and validation of a predictive model for feeding intolerance in intensive care unit patients with sepsis

Abstract

Background:

Feeding intolerance in patients with sepsis is associated with a lower enteral nutrition (EN) intake and worse clinical outcomes. The aim of this study was to develop and validate a predictive model for enteral feeding intolerance in the intensive care unit patients with sepsis.

Methods:

In this dual-center, retrospective, case-control study, a total of 195 intensive care unit patients with sepsis were enrolled from June 2018 to June 2020. Data of 124 patients for 27 clinical indicators from one hospital were used to train the model, and data from 71 patients from another hospital were used to assess the external predictive performance. The predictive models included logistic regression, naive Bayesian, random forest, gradient boosting tree, and deep learning (multilayer artificial neural network) models.

Results:

Eighty-six (44.1%) patients were diagnosed with enteral feeding intolerance. The deep learning model achieved the best performance, with areas under the receiver operating characteristic curve of 0.82 (95% confidence interval = 0.74–0.90) and 0.79 (95% confidence interval = 0.68–0.89) in the training and external sets, respectively. The deep learning model showed good calibration; based on the decision curve analysis, the model's clinical benefit was considered useful. Lower respiratory tract infection was the most important contributing factor, followed by peptide EN and shock.

Conclusions:

The new prediction model based on deep learning can effectively predict enteral feeding intolerance in intensive care unit patients with sepsis. Simple clinical information such as infection site, nutrient type, and septic shock can be useful in stratifying a septic patient's risk of EN intolerance.

INTRODUCTION

Sepsis is a life-threatening organ dysfunction caused by a dysregulated host response to infection,[1] and remains an important global health concern. Sepsis is associated with the breakdown of carbohydrate, lipid, and protein stores. Despite the increased nutritional requirements of patients with sepsis, their nutrition intake decreases, leading to a substantial energy shortage.[2] Enteral nutrition (EN) is recommended in critically ill patients by the European Society of Intensive Care Medicine (ESICM).[3] Unfortunately, intensive care unit (ICU) patients diagnosed with sepsis are likely to develop enteral feeding intolerance (FI).[4]

FI indicates intolerance to enteral feeding for clinical reasons such as vomiting, diarrhea, high-gastric residuals, or the presence of entero-cutaneous fistulas.[5] Compared to the patients without FI, patients with FI show 10% less EN intake. Moreover, patients with enteral FI have more ventilator days and longer ICU length of stay than those without enteral FI. The daily mortality hazard rate could increase significantly once enteral FI is noted.[4]

Early FI prevention in patients with sepsis has clinical significance. Previous studies have mostly focused on the analysis of susceptibility factors for FI in patients with sepsis, and there are no prediction models for FI in patients with sepsis. Thus, we aimed to develop and validate a predictive model for FI in patients with sepsis, providing a basis for clinical decision-making during the period for early prevention of EN intolerance.

PATIENTS AND METHODS

Study design and data collection

We conducted a retrospective, dual center clinical study. The study was approved by our hospital ethics committee, and informed consent was not required. A total of 235 consecutive adult patients with sepsis (age >18 years) who were admitted to the ICU of the People's Hospital of Guangxi Zhuang Autonomous Region were enrolled between June 2018 and June 2020. Furthermore, 136 adult patients with sepsis admitted to Yongwu Hospital of Guangxi Zhuang Autonomous Region were enrolled in the same period. The diagnostic criteria for sepsis were the Sepsis 3.0 Consensus criteria.[6] All enrolled patients had no contraindications to EN, whereas patients with severe septic shock, gastrointestinal tumor, chronic diarrhea, gastrointestinal bleeding, gastrointestinal surgery, or length of ICU stay less than 7 days were excluded. The enrolled patients were allocated to the feeding tolerance (FT) group and the FI group, according to whether they had FI during their ICU stay. The patients were fed through a nasoduodenal tube or nasogastric tube and intermittent (q4h, four times a day) or bolus feeding. The nutrition formula was a short-peptide or intact-protein formula. Vomiting, distension, high-gastric residuals (gastric residual volume ≥500 mL/24 h), diarrhea, and high intra-abdominal pressure (intra-abdominal pressure >12 mmHg) were considered to be the outcomes of enteral FI.[5]

Based on previous studies,[4,5,7,8,9,10,11] the collected exposure data included information on demographics (age and sex), medical history (hypertension, diabetes, coronary heart disease, kidney failure, chronic obstructive pulmonary disease, cerebrovascular accident), infection site, clinical presentation, laboratory results on admission (albumin, creatinine, urea nitrogen), disease severity (Acute Physiology and Chronic Health Evaluation II score, Sequential Organ Failure Assessment score), clinical treatment (sedation, analgesia, antibiotics, continuous feeding), EN type (short peptides, intact protein), and patient outcomes. All exposure data were collected from the database of electronic medical records and laboratory test reports.

Statistical analyses

A total of 195 patients were included in the study. Moreover, 124 patients from the People's Hospital of Guangxi Zhuang Autonomous Region were assigned to the training dataset, and 71 patients from Yongwu Hospital of Guangxi Zhuang Autonomous Region were assigned to the testing dataset. The continuous variables are expressed as mean with standard deviation, and categorical variables are presented as frequencies with percentages.

We constructed different models based on the training set through the grid search and five-fold cross-validation. These models included logistic regression, naive Bayesian, random forest, gradient boosting tree, and deep learning (multilayer feedforward artificial neural network [ANN] algorithm). We evaluated the performance of different models on the testing dataset as external validation. The area under the receiver operating characteristic curve (AUC) was used as the evaluation metrics. We selected the model that had the best performance and constructed an online predictive tool. Besides, we also established calibration curves and performed the decision curve analysis. All analyses were performed using R software (version 4.0.1).

RESULTS

Demographics and clinical characteristics

A total of 195 patients were included in the study, with 124 and 71 patients in the training set and the testing set, respectively. Among all the patients, 86 (44.1%) were diagnosed with FI, 49 (39.5%) with FI from the training set, and 37 (52.1%) with FI in the testing set. The majority of the patients in the training set were older men (average age 66.7 ± 17.2 years; male patients 70.9%). The demographic information of the testing set was generally similar to that of the training set (average age 71.5 ± 13.8 years; male patients 70.4%). The most common site of infection was the lower respiratory tract (94.3% patients in the training set and 78.9% patients in the testing set), while the most common underlying disease was hypertension (64.5% patients in the training set and 38.0% patients in the testing set). The patient characteristics in the primary cohorts and validation cohorts are presented in Table 1.

Table 1.

Characteristics of patients in the training and validation cohorts

Characteristics n (%/mean±SD)	Training cohort (n=124)			Validation cohort (n=71)
	Feeding intolerance (n=49)	Feeding tolerance (n=75)	P	Feeding intolerance (n=37)	Feeding tolerance (n=34)	P
Age, years	65.9 (20.2)	67.2 (15.0)	0.088	72.8 (13.3)	70.1 (14.4)	0.463
Sex (male)	39 (79.5%)	49 (65.3%)	0.089	29 (78.4%)	20 (60.6%)	0.128
Comorbidity
Diabetes	16 (32.6%)	31 (41.3%)	0.330	6 (16.2%)	7 (21.2%)	0.866
Hypertension	32 (65.3%)	48 (64.0%)	0.882	16 (43.2%)	11 (33.3%)	0.484
Coronary heart disease	6 (12.2%)	13 (17.3%)	0.442	7 (18.9%)	10 (30.3%)	0.449
Chronic pulmonary disease	6 (12.2%)	8 (10.6%)	0.786	3 (8.1%)	5 (15.2%)	0.615
Chronic kidney failure	11 (22.4%)	14 (18.6%)	0.776	2 (5.4%)	5 (15.2%)	0.360
Cerebrovascular accident	18 (36.7%)	25 (33.3%)	0.845	16 (43.2%)	10 (30.3%)	0.336
Ventilation	43 (87.8%)	63 (84.0%)	0.749	24 (64.9%)	18 (54.5%)	0.436
Infection
Pneumonia	47 (95.9%)	70 (93.3%)	0.832	34 (91.9%)	22 (66.7%)	0.012
Urinary tract infection	6 (12.2%)	7 (9.3%)	0.828	1 (2.7%)	4 (12.1%)	0.305
Intracranial infection	1 (2.0%)	3 (4.0%)	0.933	0 (0%)	2 (6.1%)	0.436
Blood infection	5 (10.2%)	8 (10.6%)	1.000	2 (5.4%)	2 (6.1%)	1.000
Abdominal infection	1 (2.0%)	1 (1.3%)	1.000	2 (5.4%)	0 (0%)	0.511
Skin infection	3 (6.1%)	3 (4.0%)	0.912	3 (8.1%)	6 (18.2%)	0.395
APACHE II score	23.2 (6.7)	22.6 (7.6)	0.730	22.2 (7.4)	23.6 (4.7)	0.430
Feeding method
Intermittent	6 (12.25%)	17 (22.67%)	0.221	21 (56.76%)	29 (85.2%)	0.018
Bolus	43 (87.75%)	58 (77.33%)	0.221	16 (43.24%)	5 (14.71%)	0.018
Feeding route
Gastric tube	40 (81.63%)	71 (94.67%)	0.044	35 (94.59%)	34 (100%)	0.511
Jejunal tube	9 (18.37%)	4 (5.33%)	0.044	2 (5.41%)	0 (0%)	0.511
Total EN (mL)	580 (200-1000)	630 (200-1500)	0.688	680 (200-1000)	700 (200-1000)	0.783
Gastric motility drugs	17 (34.69%)	7 (9.33%)	0.001	7 (18.91%)	3 (8.82%)	0.379
Sedation	45 (91.84%)	64 (85.33%)	0.421	34 (91.89%)	30 (88.23%)	0.906
28-day mortality	2 (4.1%)	4 (5.3%)	1.000	4 (10.8%)	2 (6.1%)	0.750

EN, enteral nutrition; SD, standard deviation; APACHE II, Acute Physiology and Chronic Health Evaluation II

Development and choice of a prediction model

We constructed five different models, with an AUC ranging between 0.70 and 0.94 [Figure 1]. Subsequently, we evaluated the predictive performance of the five models with external validation. The AUC of these models for the testing set ranged between 0.60 and 0.79 [Figure 2]. The deep learning model, which had the highest AUC (0.82, 95% confidence interval = 0.74–0.90) in the testing set achieved the best performance. Twenty-seven variables were selected for modeling since the model incorporating these variables had the best predictive power. The relative importance of the top 15 variables calculated by the deep learning model is shown in [Figure 3]. The lower respiratory tract infection (infection site of the lung) was the most important contributing factor, followed by peptide EN, and shock.

Figure 1.

ROC curve of the training cohort. AUC of each model is 0.70 (95% confidence interval [CI]; 0.60–0.80) in Bayes Net model, 0.73 (95% CI 0.64–0.82) in Logistic regression model, 0.82 (95% CI 0.74–0.90) in Deep Learning model 0.92 (95% CI 0.87–0.97) in Random Forest model, and 0.94 (95% CI 0.90–0.99) in Gradient Boosting model.

Figure 2.

ROC curve of the validation cohort. AUC of each model is 0.73 (95% CI 0.60–0.85) in Bayes Net model, 0.69 (95% CI 0.56–0.81) in Logistic regression model, 0.79 (95% CI 0.68–0.89) in Deep Learning model 0.63 (95% CI 0.50–0.76) in Random Forest model, and 0.60 (95% CI 0.47–0.74) in Gradient Boosting model.

Figure 3.

The ranking of top 15 predictive variables in the deep learning model. COPD: chronic obstructive pulmonary disease

Discrimination and calibration

The calibration curve for the probability of FI in the overall datasets demonstrated good agreement between prediction and observation [Figure 4]. There was an overprediction of the incidence of FI when the risk was lower than 80%. Additionally, there was an underprediction of the incidence of FI when the risk was higher than 80%.

Figure 4.

Calibration curve of deep learning model in overall datasets. The red line represents a perfect prediction by an ideal model. The black line represents the performance of the deep learning model. The calibration curve in overall cohorts shows the agreement between predicted (x-axis) and observed (y-axis) risk of feeding intolerance in ICU patients with sepsis.

Clinical use

The decision curve analysis for the deep learning model is presented in Figure 5. The decision curve showed that if the threshold probability of a patient was >15%, the net benefit of the deep learning model was better than the treat-all-patients scheme to predict FI. To enhance clinical usage, we published our deep learning model on https://xdeng3.shinyapps.io/NIPM/.

Figure 5.

Decision Curve analysis for the deep learning model. The y-axis measures the net benefit. The red line represents the deep learning model. The gray line represents the assumption that all the patients have feeding intolerance. The black line represents the assumption that no patients have feeding intolerance.

Figure 6 shows the screenshots of the web calculator with a clinical example from the website.

Figure 6.

The deep learning-based model showed a risk of 19% for a patient with sepsis aged 69 years with a history of hypertension and diabetes, who had the following features: infection, pneumonia; APACHE II score, 22; albumin, 30.8 g/L; creatinine, 87 μmol/L; urea nitrogen, 8.75 mmol/L; lactic acid, 1.2 mmol/L; sedation and analgesia treatment; treatment with two types of antibiotics; intermittent feeding with short peptides; and intra-abdominal pressure, <12 mmHg.

DISCUSSION

The prevalence of FI in the ICU patients ranges from 2% to 75%,[12] and the prevalence of FI in patients with sepsis is approximately 35%.[13] The incidence of FI in this study was consistent with those reported in the previous studies. FI occurs frequently and is associated with higher mortality and longer ICU stay than FT.[12] There was no difference in the 28-day mortality between the FI and FT groups in this study, as the hospital mortality rate due to sepsis has decreased significantly, which has been attributed to innovations in ICU care.[14] However, numerous patients with sepsis cannot return to a meaningful quality of life owing to ICU-acquired weakness (ICU-AW).[15] Optimal nutrition is an essential intervention to address ICU-AW.[16] To seek optimal nutrition, early prediction of FI is of particular importance among patients with sepsis.

We constructed and validated a deep learning (ANN) model (https://xdeng3.shinyapps.io/NIPM/) for the individualized prediction of FI in ICU patients with sepsis in this study. This model effectively predicts FI risk within 48 h from ICU admission, which can help ICU physicians take timely measures to prevent the occurrence of FI in patients with sepsis. Moreover, FI in septic patients is affected by various factors. This study demonstrated several clinical variables that can significantly predict FI in patients with sepsis, including infection site, nutrition type, shock, continuous feeding, coronary heart disease, antibiotic category, chronic obstructive pulmonary disease, ventilation, hypertension, cerebrovascular accident, intra-abdominal pressure, and analgesia. All these variables are clinically available.

The strongest predictor identified in our study by deep learning was the lower respiratory tract infection, which has never been reported previously. The lower respiratory tract was the leading site of infection in this study (94.35% of the patients in the training set, 78.87% of the patients in the testing set). In most cases, sepsis-associated with FI was secondary to pneumonia [Table 1]. Besides, most patients with abdominal sepsis in our department had abdominal compartment syndrome (ACS), which had contraindications for EN. Thus, the lower respiratory tract was the infection site in most patients enrolled in this study. A previous epidemiological survey has shown that sepsis with pneumonia was associated with a poor outcome.[8] We hypothesize that patients with sepsis secondary to pneumonia have a more intense inflammatory response, which may play a role in FI. Furthermore, most patients with pneumonia received mechanical ventilation, which is an independent risk factor for FI.[9,17] Accordingly, we recommend early intervention for preventing FI in septic patients with pneumonia.

Additionally, we found that nutrition type is the second crucial predictive variable of FI. Short-peptide EN is predisposed to FI (38 [45.8%] in short-peptide EN, 48 [42.8%] in intact-protein formula) for patients with sepsis, especially diarrhea (28 [33.7%] in short-peptide EN, 16 [14.3%] in intact-protein formula) in our study. The SPIRIT trial by Jakob et al.[18] demonstrated that no differences in the number of diarrhea events were observed between the hydrolyzed protein EN and whole protein feeding. van Zanten et al.[19] combined the meta-analysis data[10] with the SPIRIT trial results (211 patients) and found no benefits with respect to the diarrhea incidence during the ICU stay, favoring peptide-based EN.[19] The high osmotic pressure of short-peptide EN may contribute to diarrhea. More studies are required to investigate whether short-peptide EN is beneficial for patients with sepsis. Moreover, we recommend an intact-protein formula for patients with sepsis in the ICU.

Shock was the third significant predictor, consistent with the previous studies. Mao et al.[11] showed that elevated serum lactate levels were more likely to be associated with FI development in the elderly patients with sepsis treated with vasopressors. The patients with septic shock who receive adequate fluid resuscitation and receive norepinephrine doses of <0.14 μg/kg/min may tolerate early EN.[20] The NUTRIREA-2 trial, the largest randomized controlled trial evaluating EN in septic shock, indicated that the EN group had a higher FI incidence than the parenteral nutrition (PN) group.[21] Societal guidelines offer varying recommendations for the use of EN in patients with septic shock. The recent European Society of Parenteral and Enteral Nutrition (ESPEN) guideline suggests early and progressive EN should be used in sepsis patients after hemodynamic stabilization.[22] The optimal dose and timing of initiation of feeding in patients with septic shock remain unclear.[23] More studies are required to determine the timing and dose of EN in patients with septic shock. Presently, enteral feeding needs to be evaluated according to FT on a case-by-case basis.

To the best of our knowledge, this is the first attempt to use the multilayer feedforward ANN model, a type of deep learning model, as a prediction tool in FI of patients with sepsis. Deep learning refers to the class of machine learning (ML) methods that make use of more abstract representations of the input data to perform a specific task.[24] ANN is an information management model that is similar to the biological nervous system function of the human brain.[25] Our ANN model was constructed by training with stochastic gradient descent using backpropagation. ANN models are flexible and of high predictive accuracy since we can choose different activation functions. In addition, ANN models have other features, including adaptive learning rate, rate annealing, dropout, and L1 or L2 regularization. They can also address the collinearity problem comparing with traditional regression models. We divided the dataset and trained different models using grid search in our study. Therefore, the ANN model outperformed the classical approach (logistic regression model) and other ML algorithms, including random forest, naive Bayesian, and gradient boosting tree. Thus, we selected ANNs as our final predictive model.

Our study has a few limitations. First, since this study was a retrospective, observational study, this study is subject to the disadvantages of a retrospective study. Second, as this was a dual center study, the results may not be generalizable to other populations within China or other countries. We suggest that in the future, multicenter studies should be conducted to validate this model further. Third, the sample size was small because of the strict inclusion and exclusion criteria. Thus, large sample sizes are needed in the future. The performance of a prediction model will be continuously improved when there are large sample sizes, benefiting from the strong self-learning ability of ANNs.

In conclusion, a new prediction model was developed and validated using deep learning, which can effectively predict the occurrence of EN intolerance in ICU patients with sepsis (https://xdeng3.shinyapps.io/NIPM/). Through this predictive model, timely preventive measures can be undertaken by clinicians to reduce the occurrence of FI in patients with sepsis. The lower respiratory tract as the infection site is the strongest predictor for FI. Simple clinical information such as nutrient and septic shock type can be useful in stratifying a septic patient's risk of EN intolerance.

Financial support and sponsorship

This study was supported by grants from the Science and Technology Project Foundation of Nanning, no. 2020032 and Foundation of the Key Laboratory of Guangxi Zhuang Autonomous Region Health Commission, no. ZZH 2020013.

Conflicts of interest

There are no conflicts of interest.