ABSTRACT
Background
Acute kidney injury (AKI) is a significant complication in patients with trauma. The early identification of AKI in these patients poses challenges. This study aimed to predict AKI in trauma patients 24 or 48 hours in advance using an extreme gradient boosting (XGBoost) model.
Methods
We analyzed 17 859 trauma patients admitted to a regional trauma center between January 2015 and July 2023. Demographic, clinical, and laboratory parameters were collected. The model was developed using data until July 2021 and validated using data from August 2021. We developed models to predict AKI stages 1–3 and AKI stages 2 and 3 occurring 48 and 24 hours later and measured predictive performance in the validation group. The models' performance was evaluated using the area under the receiver operating characteristic curve (AUROC), and feature importance was assessed through SHapley Additive exPlanations values.
Results
The study population exhibited an incidence of AKI of 6.6% in the development group and 5.4% in the validation group. The models demonstrated predictive performance with AUROCs of 0.864 and 0.886 for 48-hour predictions of AKI stages 1–3 and stages 2 and 3, and 0.904 and 0.903 for 24-hour predictions of AKI stages 1–3 and stages 2 and 3, respectively. Key features influencing model predictions included baseline and in-hospital serum creatinine values, injury severity score, age, lactate dehydrogenase, D-dimer, platelets, albumin, and C-reactive protein levels.
Conclusions
The XGBoost models effectively predicted AKI in trauma patients up to 48 hours in advance using clinical data.
Keywords: acute kidney injury, gradient boosting, machine learning, ROC curve, trauma
KEY LEARNING POINTS.
What was known:
Acute kidney injury (AKI) is a common complication in trauma patients associated with increased mortality and longer hospital stays.
Trauma patients may have different risk factors for AKI, including hemorrhage, direct kidney trauma, and rhabdomyolysis, distinct from the general hospital population.
Early identification of AKI risk in trauma patients is challenging, and predictive models specifically for this population were lacking.
This study adds:
An extreme gradient boosting machine learning model was developed and validated to predict AKI in trauma patients 24 and 48 hours in advance with high accuracy (area under ROC curve 0.864–0.904).
The predictive model achieved a sensitivity of 65%–74% with a specificity of 90%–91% using an optimal cut-off value.
The model identified key predictive factors for AKI in trauma patients, including serum creatinine, age, injury severity score (ISS), and biomarkers like LDH, CRP, and D-dimer.
Potential impact:
The study demonstrates the feasibility of using readily available electronic health record data to create an early predicting system for AKI in trauma settings.
By identifying key risk factors, this study may inform targeted research into AKI prevention and treatment strategies specific to trauma patients.
Implementation of this predictive model could enable earlier intervention and prevention strategies for AKI in trauma patients, potentially reducing morbidity and mortality.
INTRODUCTION
Acute kidney injury (AKI) is a common complication among trauma patients admitted to hospital. A systematic review has revealed that a pooled rate of 24% of trauma patients admitted to intensive care units develop AKI [1]. The reported incidence rate of AKI in trauma patients is as high as the AKI incidence from data in general hospitalized patients [2]. Similar to AKI in general patients, AKI in trauma patients is also associated with significant increases in in-hospital mortality and longer hospital stays [3–6]. Trauma patients have distinct characteristics compared with general hospitalized patients. For example, these individuals are often younger and typically present with a lower burden of comorbid health conditions [7]. Furthermore, the pathogenesis of AKI in trauma patients is multifactorial, with unique contributors such as hemorrhage, direct traumatic injury to the kidneys, and rhabdomyolysis, each of which can lead to kidney damage [8]. In addition to the well-known risk factors for AKI, such as the presence of diabetes, advanced age, baseline kidney function impairment, and use of nephrotoxins, studies of trauma patients have suggested potential risk factors for AKI in trauma patients, such as type of injury, history of blood transfusions, rhabdomyolysis, and hyperlactatemia [9–11].
The diagnosis of AKI is straightforward, guided by well-established criteria regarding serum creatinine levels and urine output as outlined in the Kidney Disease: Improving Global Outcomes (KDIGO) guideline [12]. However, clinical recognition and management of AKI is often suboptimal, resulting from inadequate assessment of its risk factors [13]. Hence, the initial approach in managing AKI encompasses the early detection of risk factors predisposing to AKI and kidney function decline, followed by targeted interventions that address the underlying pathophysiological etiologies [14].
Prediction models using clinical features have been developed for the early diagnosis of AKI [15]. With the recent advances in machine learning models, AKI prediction models have shown improvements in accuracy [16, 17]. The extreme gradient boosting (XGBoost) machine learning model is a tree-based model based on decision trees that classify data based on conditions, and it uses the Boosting algorithm, one of the ensemble learning methods that combines the results of multiple models to improve prediction performance, to improve the performance of the model. The Boosting algorithm works sequentially by weighting the errors that resulted from the failure to predict in the previous decision tree and taking them into account in the next tree to make predictions [18].
In this study we aimed to develop and validate a predictive algorithm based on machine learning techniques for the early identification of AKI in trauma patients. The timely prediction of AKI using a machine learning model has the potential to improve prognostic outcomes in trauma patients.
MATERIALS AND METHODS
This retrospective, single-center study using electronic health records (EHRs) was conducted under a protocol reviewed and approved by the Institutional Review Board of Dankook University Hospital (IRB number 2023–12-006). The institutional review board waived the requirement for informed consent for this study.
Data description and inclusion criteria
We included a trauma patient cohort in the study institution, the trauma center of Dankook University Hospital, from January 2015 to July 2023. The study institution is one of the major regional trauma centers in South Korea, specifically serving the Chungnam province, which has a population of ∼2.06 million individuals. The clinical data included laboratory results, diagnoses as International Statistical Classification of Diseases and Related Health Problems (ICD-10) codes, injury severity score (ISS) at admission, medication, and demographic data. Briefly, the data were collected from the EHRs of patients visiting the trauma center within the study period. Out of a total of 18 900 screened patients, a total of 17 859 were included in the study, excluding those who were not admitted to the hospital and discharged the same day. The study’s inclusion criteria were as follows: (i) age between 18 and 90 years; (ii) presence of baseline serum creatinine value within the first 24 hours of hospitalization; (iii) a baseline serum creatinine value below 4.0 mg/dL or receiving renal replacement therapy within 24 hours of admission; and (iv) subsequent serum creatinine measurement conducted at least 24 hours following the initial serum creatinine test. We followed the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis using machine learning methods (TRIPOD + AI) reporting guideline [19].
Outcome definition
The study had two primary outcomes: the composite outcome of AKI stages 1–3 and, separately, AKI stages 2 and 3, defined based on serum creatinine levels. At each 24-hour interval, the stages of AKI were assessed based on the KDIGO guideline for AKI definition based on serum creatinine. The baseline serum creatinine was obtained within 24 hours of admission and excluded from the analysis if there was no serum creatinine value within the timeframe. Urine output data were excluded from our study due to concerns about the accuracy of these measurements as documented in EHRs.
Feature selection and data preparation
In this study, feature selection was conducted based on indicators that were both available in our database and recognized as potential risk factors for AKI in previous studies [9–11]. The predictive model incorporates features such as patient demographics (age and gender), type of injury categorized according to ICD-10 diagnostic codes, the ISS index, and baseline serum creatinine value recorded within the first 24 hours of admission. Time-series features included laboratory measurements and medications administered within the initial 7-day period following hospital admission. Given the established risk factors for AKI identified in prior research and the regularity of clinical testing, we included a comprehensive range of laboratory features. These encompassed hemoglobin, platelet count, white blood cell (WBC) count, serum albumin, chloride, D-dimer, total bilirubin, creatine kinase (CK), bicarbonate, lactate, C-reactive protein (CRP), procalcitonin, lactate dehydrogenase (LDH), arterial blood gas pH, and the urinary analysis parameters specific gravity, WBCs, red blood cell (RBC) count, protein, renal epithelial cells, and squamous cells. Medications were categorized as binary variables, assessing the prescription and administration of vancomycin, aminoglycosides, non-steroidal anti-inflammatory drugs (NSAIDs), iodine-based contrast agents, mannitol, inotropic drugs, diuretics, colistin, penicillins, cephalosporins, renin–angiotensin system (RAS) inhibitors, lipid-lowering agents, and colloid fluids. Time-series features missing within 24-hour intervals were treated as previously tested values and imputed Not a Number (NaN) values when baseline values were unavailable.
Predictive model development and validation
To develop and validate prediction models for AKI, we organized patients' time-series clinical data in the order of test records to enable gradient tree models to learn from the data. Based on the inclusion and exclusion criteria, patients who did not have a baseline creatinine value within 24 hours of hospital admission were excluded from the study, and the missing values were not imputed with the reference creatinine value. For laboratory baseline data other than serum creatinine, missing values were handled as follows: if actual measurements were available, these were designated as baseline values. When there was a gap between consecutive laboratory tests, the result of the earlier test was assumed to continue until just prior to the subsequent test. The preprocessed data were split into development and validation sets based on the time of admission. Patients admitted between January 2015 and July 2021 were assigned to the model development group. Subsequently, individuals hospitalized from August 2021 to July 2023 were allocated to the validation group. The XGBoost model was trained on the training set using a gradient boosting framework. For the hyperparameter settings of the gradient decision tree models, the number of trees was set to 100, the maximum depth of each tree was limited to 6, and the prediction results of the previous tree were reflected in the next boosted decision tree with a ratio of 0.3. The model's performance was evaluated using the metric of the area under the receiver operating characteristic curve (AUROC) of the models. To present clinically applicable diagnostic criteria, sensitivity and specificity were measured according to cut-off values. We assessed the relative importance of features contributing to the predictions of the XGBoost model by analyzing SHAP (SHapley Additive exPlanations) values [20]. SHAP scores provide a game-theoretic approach to explain the output of machine learning models by assigning each feature an importance value for a particular prediction.
Statistical analysis
Patient demographics, laboratory values, medication, and outcomes were compared among the development and validation cohorts. We displayed the statistical results as numbers and percentages for categorical or discrete variables and as median and interquartile ranges for continuous variables. For these comparisons, based on the distributions of the variables, we used t-tests, analysis of variance, and χ2 tests as appropriate. All analyses including model development and validation were performed using the pandas 1.3.5, scikit-learn 1.0.2, Matplotlib 3.5.3, shap 0.42.1, Seaborn 0.12.2, and XGBoost 1.6.2 libraries within a Python 3.7.16 (Python Software Foundation) environment. The level of significance was set at a two-sided P < .05.
RESULTS
Baseline characteristics
The flowchart of the study is shown in Fig. 1. A total of 11 687 patients (9063 in the development set and 2624 in the validation set) were included in the analysis. Of the patients, the overall incidence rates of AKI stages 1–3 was 6.6% in the development group and 5.4% in the validation group. The incidence rates of AKI stages 2 and 3, which can be considered severe forms of AKI, were 2.2% and 2.5%, respectively. In-hospital mortality was 4.9% in the development group and 4.3% in the validation group. The cohort assignment, based on specific time intervals rather than random selection, resulted in statistically significant variances in most baseline characteristics between the development and validation groups.
Figure 1:
Flowchart for patient selection. A total of 18 900 patients were screened using their medical records from January 2015 to July 2023. A total of 6169 patients were excluded due to exclusion criteria based on medical chart review. The development cohort comprised individuals admitted until July 2021. Subsequently, the validation cohort consisted of patients admitted after August 2021. ER, emergency room; RRT, renal replacement therapy.
The baseline characteristics of the participants are shown in Table 1. Statistically significant differences were observed between the development and validation groups in terms of sex ratio, injury site distribution, diabetes prevalence, mean age, ISS, and baseline renal function as assessed by the Chronic Kidney Disease Epidemiology Collaboration with eGFR (CKD-EPI eGFR). During hospitalization, a high frequency of use of analgesics such as NSAIDs and acetaminophen was observed in both the development and validation groups. Additionally, several medications that may contribute to the development of AKI were identified, including RBC transfusions, loop diuretics, RAS inhibitors, and intravenous mannitol. The baseline characteristics stratified by AKI stage are presented in Supplementary Table S2.
Table 1:
Baseline characteristics of development and validation groups.
| Characteristic (total n = 11 687) | Development group (n = 9063) | Validation group (n = 2624) | P-value |
|---|---|---|---|
| Sex, female, n (%) | 2790 (30.8) | 898 (34.2) | <.01 |
| Age, years (IQR) | 58.0 (44.0, 71.0) | 60.0 (46.0, 73.0) | <.01 |
| Injury severity score, points (IQR) | 9.0 (4.0, 17.0) | 10.0 (5.0, 17.0) | <.01 |
| AKI, n (%) | |||
| AKI stage 1–3 | 596 (6.6) | 142 (5.4) | .18 |
| AKI stage 2–3 | 201 (2.2) | 66 (2.5) | .45 |
| Mortality, n (%) | 442 (4.9) | 114( 4.3) | .28 |
| Injury site, n (%) | |||
| Abdominopelvis | 1121 (12.4) | 330 (12.6) | <.01 |
| Thorax | 782 (8.6) | 277 (10.6) | |
| Head | 2432 (26.8) | 722 (27.5) | |
| Others | 4503 (49.7) | 1278 (48.7) | |
| Diabetes mellitus | 1274 (14.1) | 460 (17.5) | <.01 |
| Clinical parameters | |||
| Serum creatinine, mg/dL (IQR) | 0.8 (0.7, 1.0) | 0.8 (0.7, 1.0) | .63 |
| EPI-CKD eGFR, ml/min/1.73 m2 (IQR) | 99.9 (87.0, 111.1) | 98.4 (85.8, 109.3) | .01 |
| Hemoglobin, g/dL (IQR) | 12.6 (11.0, 14.0) | 12.6 (11.1, 14.0) | .85 |
| Platelets, 1000/μL (IQR) | 210.0 (172.0, 250.0) | 218.0 (179.0, 261.0) | <.01 |
| White blood cells, 1000/μL (IQR) | 10.1 (7.7, 13.2) | 10.3 (7.8, 13.4) | .01 |
| Serum bilirubin, mg/dL (IQR) | 0.5 (0.4, 0.8) | 0.5 (0.4, 0.7) | .42 |
| Serum albumin, g/L (IQR) | 4.2 (3.8, 4.5) | 4.1 (3.8, 4.4) | .79 |
| Serum LDH, U/L (IQR) | 271.0 (220.0, 364.0) | 275.0 (223.0, 363.8) | .19 |
| Serum lactate, mmol/L (IQR) | 1.4 (0.9, 2.3) | 1.2 (0.8, 2.2) | <.01 |
| CK, U/L (IQR) | 211.0 (118.2, 427.0) | 199.0 (113.0, 374.0) | .11 |
| D-dimer, ng/mL (IQR) | 3286.0 (856.5, 10 764.5) | 3051.0 (829.8, 11 698.5) | .09 |
| CRP, mg/dL (IQR) | 0.2 (0.1, 1.2) | 0.2 (0.1, 0.9) | .10 |
| Arterial pH, (IQR) | 7.4 (7.4, 7.5) | 7.4 (7.4, 7.4) | <.01 |
| Arterial total CO2, mmol/L (IQR) | 24.7 (21.8, 27.1) | 21.7 (19.6, 23.5) | <.01 |
| Arterial bicarbonate, mmol/L (IQR) | 25.4 (23.3, 27.2) | 25.2 (23.5, 26.8) | .94 |
| Urine specific gravity, (IQR) | 1.020 (1.014, 1.030) | 1.022 (1.015, 1.036) | <.01 |
| Presence of urine RBC, n (%) | 3497 (38.6) | 1210 (46.1) | <.01 |
| Presence of urine WBC, n (%) | 3240 (35.7) | 1374 (52.4) | <.01 |
| Presence of urine squamous cells, n (%) | 435 (4.8) | 205 (7.8) | .13 |
| Presence of urine renal epithelial cells, n (%) | 28 (0.3) | 5 (0.2) | .08 |
| Medication use | |||
| NSAIDs, n (%) | 3277 (36.2) | 1269 (48.4) | <.01 |
| Acetaminophen, n (%) | 6946 (76.6) | 2206 (84.1) | <.01 |
| IV vancomycin, n (%) | 570 (6.3) | 201 (7.7) | .02 |
| Aminoglycosides, n (%) | 1203 (11.3) | 269 (10.3) | .10 |
| IV acyclovir, n (%) | 15 (0.2) | 4 (0.2) | 1.00 |
| Colistin, n (%) | 101 (1.1) | 57 (2.2) | <.01 |
| Norepinephrine, n (%) | 298 (3.3) | 271 (10.3) | <.01 |
| Mannitol, n (%) | 942 (10.4) | 430 (16.4) | <.01 |
| Loop diuretics, n (%) | 2841 (31.3) | 961 (36.6) | <.01 |
| Spironolactone, n (%) | 203 (2.2) | 101 (3.8) | <.01 |
| Thiazide, n (%) | 362 (4.0) | 111 (4.2) | .72 |
| RAS inhibitors, n (%) | 1528 (16.9) | 572 (21.8) | <.01 |
| RBC transfusion, n (%) | 3670 (40.5) | 1095 (41.7) | .53 |
IQR, interquartile range; IV, intravenous.
Performance of the predictive model
We performed an internal and temporal validation of the predictive performance of an XGBoost model trained using clinical data from the development set. The performance of the predictive model was quantified by calculating the AUROC curve. In the validation group, the 48-hour predictive model for AKI stages 1–3 demonstrated an AUROC of 0.864, while for the model for AKI stages 2 and 3 exhibited an AUROC of 0.886. Additionally, we utilized the XGBoost model to predict the occurrence of AKI within 24 hours, employing the same development–validation dataset previously identified. The 24-hour predictive model for AKI stages 1–3 showed an AUROC of 0.904, and the model for AKI stages 2 and 3 achieved an AUROC of 0.903 (Fig. 2). For longer-term prediction, the 72-hour predictive model showed an AUROC of 0.809 for AKI stages 1–3 and 0.783 for stages 2 and 3 (Supplementary Fig. S1). Compared with other machine learning algorithms, including random forest, support vector machine, ridge regression, and logistic regression, our XGBoost model demonstrated superior performance across most AKI prediction scenarios (Supplementary Table S1).
Figure 2:
ROCs for XGBoost models. The plots present the ROC curves for the XGBoost models, which plot model sensitivity against false-positive rate across a range of all possible risk thresholds for predicting the binary AKI outcome. The diagonal line represents the performance of the random classifier, which serves as a baseline for comparison. The ROC curves demonstrate the prediction performance for AKI stages 1–3 (a) and AKI stages 2 and 3 (b) 48 hours ahead of time, as well as the prediction performance for AKI stages 1–3 (c) and AKI stages 2 and 3 (d) 24 hours ahead of time. The AUROC curve values are indicated in the legend of each plot.
To assess the clinical diagnostic usefulness of the predictive model, we evaluated the diagnostic sensitivity and specificity using various cut-off probabilities. For the 48-hour prediction of AKI stages 1–3, the model demonstrated a sensitivity of 68.0% and a specificity of 91.2% using a cut-off probability value of 0.008. The 48-hour prediction model for AKI stages 2 and 3 exhibited a sensitivity of 65.1% and a specificity of 90.5% at a cut-off value of 0.0007. Additionally, the 24-hour prediction model for AKI stages 1–3 showed a sensitivity of 73.9% and a specificity of 90.4% with a cut-off value of 0.008, while the 24-hour prediction model for AKI stages 2 and 3 achieved a sensitivity of 74.2% and a specificity of 90.4% at a cut-off value of 0.0004 (Fig. 3). The corresponding accuracy and F1 scores for different cut-off values are provided in Supplementary Table S3.
Figure 3:
Sensitivity and specificity of XGBoost models by cut-off probabilities. Each panel depicts bar plots comparing the sensitivity and specificity across a range of probability cut-off values for the prediction model. Sensitivity (represented by orange bars) measures the model's ability to correctly identify true AKI cases, whereas specificity (represented by blue bars) measures the model's ability to correctly identify true negative cases. The bar plots illustrate the trade-off between sensitivity and specificity across different cut-off values for the prediction of AKI stages 1–3 (a) and AKI stages 2 and 3 (b) 48 hours ahead of time, as well as for the prediction of AKI stages 1–3 (c) and AKI stages 2 and 3 (d) 24 hours ahead of time.
Feature importance
We analyzed feature importance by calculating SHAP scores and illustrate summary plots for each predictive model in Fig. 4. Across all predictive models, in-hospital serum creatinine value was the most influential feature driving the model predictions. Baseline serum creatinine at admission, age, ISS, LDH, CRP, and D-dimer levels were found to be significant factors in the models, although their importance varied across the different prediction models. Meanwhile, administered medications such as intravenous vancomycin, NSAIDs, colistin, and anti-hypertensive agents were found to contribute relatively less to the prediction models compared with the aforementioned clinical and laboratory parameters.
Figure 4:
Contributions of the top 15 features on SHAP summary plots of prediction models. The higher the SHAP value of a feature, the greater its contribution to the prediction of AKI occurrence by the model. Each dot represents a single patient's feature attribution value for the corresponding feature in the model. The vertical position of a dot indicates the impact of that feature on the model output for that particular patient, with dots accumulating vertically to depict the density of patients at each impact level. The color of the dots represents the actual value of the feature for the patient, with red indicating higher feature values and blue indicating lower feature values. The SHAP summary plots show the feature contributions for AKI stages 1–3 (a) and AKI stages 2 and 3 (b) 48 hours ahead of time, as well as for AKI stages 1–3 (c) and AKI stages 2 and 3 (d) 24 hours ahead of time.
DISCUSSION
In this study we developed and validated a series of AKI prediction models using the XGBoost machine learning algorithm to forecast AKI events 24 and 48 hours in advance among trauma patients. To the best of our knowledge, this is the first study to report the prediction of AKI in trauma patients using a gradient boosting model. We investigated the ability to predict AKI in this patient population using XGBoost, a gradient boosting model that has demonstrated superior performance compared with other types of models, such as logistic regression, random forest, support vector machine, and artificial neural network models, in predicting AKI based on time-series clinical data [21–24].
The predictive models in our study demonstrated effective performance in forecasting in-hospital AKI among trauma patients in a temporal validation cohort, as evidenced by the AUROC values. Furthermore, to interpret the clinical features contributing to the model's predictions and further elucidate the factors driving the model's decision-making process, we employed SHAP values, a method of machine learning model explanation [25]. The SHAP summary plot (Fig. 4) represents variable importance, where the Y-axis lists the variables in order of their impact on the model's predictions. The X-axis represents the summarized SHAP value, which indicates the degree of change in log odds. This approach allows a comprehensive understanding of how each feature influences the model's output, providing insights into the relative importance of various clinical parameters in predicting AKI. The SHAP analysis revealed that ISS and in-hospital laboratory values before the AKI event, such as serum creatinine, LDH, CRP, CK, and D-dimer, played a significant role in the model's predictions. These findings suggest that monitoring and considering temporal changes in these specific laboratory parameters can aid in assessing a patient's risk of developing AKI during their hospitalization.
AKI is a common condition among hospitalized patients, with a reported incidence of more than 10%–20% and an associated increased risk of in-hospital mortality [2]. In contrast, trauma patients tend to be younger and have fewer comorbidities compared with the general hospitalized population. Hence, the incidence of AKI in trauma patients is generally lower, although reported rates vary across different studies [8, 9]. In this study the mean age of hospitalized patients was relatively low at 57 years, and the mean value of baseline EPI-CKD eGFR was 99 mL/min/1.73 m², which is similar to that of the general population [26].
In addition to the distinctions between trauma patients and the general hospitalized population, there are specific factors inherent to trauma patients that are known to be associated with the development of AKI. These factors include the site of injury (e.g. abdominal injury), mechanism of injury (e.g. burns), higher ISS, and elevated CK levels resulting from rhabdomyolysis [10, 27, 28]. The presence of these unique risk factors and patient characteristics provide a compelling rationale for the development of dedicated predictive models tailored specifically for forecasting in-hospital AKI among patients admitted to trauma centers. By incorporating these trauma-specific variables into the predictive models, in conjunction with other pertinent clinical data, the accuracy and performance of the models in identifying patients at high risk of AKI can be enhanced. Consequently, this approach can facilitate the implementation of early intervention strategies and optimize patient management in the trauma setting, ultimately leading to improved patient outcomes. Recently, some positive results have been reported for systems that utilize EHR-based real-time AKI prediction models to alert clinicians to AKI events or provide clinical decision support systems to prevent mortality [29–32]. In order to prevent the occurrence of AKI, it is crucial to develop a comprehensive clinical supporting system that can forecast AKI in advance. This system should accurately predict AKI with minimized false negatives and provide clear explanations for its predictions. In this study, we present a predictive model that utilizes time-series clinical indicators to forecast all stages of AKI, including severe cases, with a specificity of 90%–95%. Furthermore, our model demonstrates an accuracy of over 70% in predicting AKI 24 hours in advance and 60% accuracy in predicting AKI 48 hours in advance in trauma patients.
We acknowledge several limitations of our study. First, the retrospective and observational nature of our EHR-based design of the research could have introduced inevitable selection bias, as the study relied on pre-existing data and did not involve random assignment of participants to groups. Second, our study utilized only laboratory results and medications administered during hospitalization as features for the time-series clinical data, which can be considered both a strength and a weakness. On one hand, focusing on robust data that can be easily collected from EHRs reduces the data requirements for a real-world clinical setting, minimizing errors and the need for secondary data processing. This approach enhances the feasibility and reproducibility of the study. On the other hand, the lack of utilization of additional features, such as documented clinical history, time-averaged urine output, systolic and diastolic blood pressure, and pulse rate, may limit the predictive power of the model. Moreover, the inability to include urine output-based AKI diagnosis due to the inaccuracy of urine output recording and the difficulty of applying the KDIGO AKI urine output criteria is a significant drawback. To overcome these limitations, an infrastructure capable of supporting a comprehensive real-time EHR database is needed, along with a method to accurately quantify vital sign values and hourly urine output, which can be subject to measurement errors. Third, given the risk of overfitting a predictive model trained using inpatient data, external validation using a separate database with a similar EHR structure is a more robust method for assessing the model's predictive performance. In this study, we employed a temporal validation technique to ensure heterogeneity between the development group and the validation group. This approach helps to mitigate the risk of overfitting and provides a more reliable, albeit imperfect, assessment of the model's generalizability. However, as a focus of future research, developing an AKI risk surveillance model with real-time feature value updates, integrating it into EHRs, and implementing it across multiple trauma centers could help to address these limitations and improve prediction performance. Such a live prediction model would allow the continuous monitoring of patients' risk factors and enable timely interventions to prevent or mitigate AKI. Furthermore, the increased diversity of data from multiple centers would enhance the model's robustness and external validity.
In conclusion, the predictive performance of our XGBoost model for AKI prediction in trauma patients was not inferior to that of conventional machine learning models. This finding demonstrates that it is possible to predict AKI 24 or 48 hours in advance using parameters readily available in the EHR database. Based on these results, the development and implementation of a clinical decision support tool that integrates our predictive model could provide an opportunity for early intervention in trauma patients at risk of developing AKI.
Supplementary Material
Contributor Information
Yongjin Yi, Department of Internal Medicine, College of Medicine, Dankook University, Dongnam-gu, Chungcheongnam-do, Cheonan-si, Republic of Korea.
Minwoo Tae, Department of Artificial Intelligence-based Convergence, Dankook University, Suji-gu, Gyeonggi-do, Yongin-si, Republic of Korea.
Sujong Shin, Department of Artificial Intelligence-based Convergence, Dankook University, Suji-gu, Gyeonggi-do, Yongin-si, Republic of Korea.
Sang-Il Choi, Department of Computer Engineering, Dankook University, Gyeonggi-do, Yongin-si, Republic of Korea.
FUNDING
This research was supported by the Bio&Medical Technology Development Program of the National Research Foundation, funded by the Korean government (MSIT) (RS-2023-00220408) and the ICT Challenge and Advanced Network of Human Resource Development (ICAN) support program, supervised by the Institute for Information & Communications Technology Planning & Evaluation (IITP-2023-00259867).
DATA AVAILABILITY STATEMENT
Individual participant data will not be available. The study protocol, statistical analysis plan, and analytic code are to be used for approved proposal aims only. Send data access requests to yongjin.neph@gmail.com and sign a data access agreement.
CONFLICT OF INTEREST STATEMENT
All authors have nothing to disclose.
REFERENCES
- 1. Søvik S, Isachsen MS, Nordhuus KM et al. Acute kidney injury in trauma patients admitted to the ICU: a systematic review and meta-analysis. Intensive Care Med 2019;45:407–19. 10.1007/s00134-019-05535-y [DOI] [PubMed] [Google Scholar]
- 2. Susantitaphong P, Cruz DN, Cerda J et al. World incidence of AKI: a meta-analysis. Clin J Am Soc Nephrol 2013;8:1482–93. 10.2215/CJN.00710113 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Bagshaw SM, George C, Gibney RTN et al. A multi-center evaluation of early acute kidney injury in critically ill trauma patients. Ren Fail 2008;30:581–9. 10.1080/08860220802134649 [DOI] [PubMed] [Google Scholar]
- 4. Skinner DL, Hardcastle TC, Rodseth RN et al. The incidence and outcomes of acute kidney injury amongst patients admitted to a level I trauma unit. Injury 2014;45:259–64. 10.1016/j.injury.2013.07.013 [DOI] [PubMed] [Google Scholar]
- 5. Stewart IJ, Sosnov JA, Howard JT et al. Acute kidney injury in critically injured combat veterans: a retrospective cohort study. Am J Kidney Dis 2016;68:564–70. 10.1053/j.ajkd.2016.03.419 [DOI] [PubMed] [Google Scholar]
- 6. Haines RW, Lin SP, Hewson R et al. Acute kidney injury in trauma patients admitted to critical care: development and validation of a diagnostic prediction model. Sci Rep 2018;8:3665. 10.1038/s41598-018-21929-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Eriksson M, Brattström O, Mårtensson J et al. Acute kidney injury following severe trauma: risk factors and long-term outcome. J Trauma Acute Care Surg 2015;79:407–12. 10.1097/TA.0000000000000727 [DOI] [PubMed] [Google Scholar]
- 8. Harrois A, Libert N, Duranteau J. Acute kidney injury in trauma patients. Curr Opin Crit Care 2017;23:447–56. 10.1097/MCC.0000000000000463 [DOI] [PubMed] [Google Scholar]
- 9. Bihorac A, Delano MJ, Schold JD et al. Incidence, clinical predictors, genomics, and outcome of acute kidney injury among trauma patients. Ann Surg 2010;252:158–65. 10.1097/SLA.0b013e3181deb6bc [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Nasu T, Ueda K, Kawashima S et al. Prehospital blood pressure and lactate are early predictors of acute kidney injury after trauma. J Surg Res 2021;265:180–6. 10.1016/j.jss.2021.03.037 [DOI] [PubMed] [Google Scholar]
- 11. Shashaty MGS, Meyer NJ, Localio AR et al. African American race, obesity, and blood product transfusion are risk factors for acute kidney injury in critically ill trauma patients. J Crit Care 2012;27:496–504. 10.1016/j.jcrc.2012.02.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Khwaja A. KDIGO clinical practice guidelines for acute kidney injury. Nephron Clin Pract 2012;120:c179–84. 10.1159/000339789 [DOI] [PubMed] [Google Scholar]
- 13. Joslin J, Wilson H, Zubli D et al. Recognition and management of acute kidney injury in hospitalised patients can be partially improved with the use of a care bundle. Clin. Med 2015;15:431–6. 10.7861/clinmedicine.15-5-431 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Kellum JA, Romagnani P, Ashuntantang G et al. Acute kidney injury. Nat Rev Dis Primers 2021;7:1–17. 10.1038/s41572-021-00284-z [DOI] [PubMed] [Google Scholar]
- 15. Park S, Lee H. Acute kidney injury prediction models: current concepts and future strategies. Curr Opin Nephrol Hypertens 2019;28:552–9. 10.1097/MNH.0000000000000536 [DOI] [PubMed] [Google Scholar]
- 16. Song X, Liu X, Liu F et al. Comparison of machine learning and logistic regression models in predicting acute kidney injury: a systematic review and meta-analysis. Int J Med Informatics 2021;151:104484. 10.1016/j.ijmedinf.2021.104484 [DOI] [PubMed] [Google Scholar]
- 17. Vagliano I, Chesnaye NC, Leopold JH et al. Machine learning models for predicting acute kidney injury: a systematic review and critical appraisal. Clin Kidney J 2022;15:2266–80. 10.1093/ckj/sfac181 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Chen T, Guestrin C. XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM; 2016:785–94. 10.1145/2939672.2939785 [DOI] [Google Scholar]
- 19. Collins GS, Moons KGM, Dhiman P et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ 2024;385:e078378. 10.1136/bmj-2023-078378 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Lundberg SM, Lee SI. A unified approach to interpreting model predictions. Advances in neural information processing systems 2017;30:4765–74. 10.48550/arXiv.1705.07874 [DOI] [Google Scholar]
- 21. Tomašev N, Glorot X, Rae JW et al. A clinically applicable approach to continuous prediction of future acute kidney injury. Nature 2019;572:116–9. 10.1038/s41586-019-1390-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Goodwin TR, Demner-Fushman D. A customizable deep learning model for nosocomial risk prediction from critical care notes with indirect supervision. J Am Med Inform Assoc 2020;27:567–76. 10.1093/jamia/ocaa004 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Tseng PY, Chen YT, Wang CH et al. Prediction of the development of acute kidney injury following cardiac surgery by machine learning. Crit Care 2020;24:478. 10.1186/s13054-020-03179-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Martinez DA, Levin SR, Klein EY et al. Early prediction of acute kidney injury in the emergency department with machine-learning methods applied to electronic health record data. Ann Emerg Med 2020;76:501–14. 10.1016/j.annemergmed.2020.05.026 [DOI] [PubMed] [Google Scholar]
- 25. Lundberg SM, Erion G, Chen H et al. From local explanations to global understanding with explainable AI for trees. Nat Mach Intell 2020;2:56–67. 10.1038/s42256-019-0138-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Lee S, Park S, Kim Y et al. Impact of variability in estimated glomerular filtration rate on major clinical outcomes: a nationwide population-based study. PLoS One 2020;15:e0244156. 10.1371/journal.pone.0244156 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Harrois A, Soyer B, Gauss T et al. Prevalence and risk factors for acute kidney injury among trauma patients: a multicenter cohort study. Crit Care 2018;22:344. 10.1186/s13054-018-2265-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Legrand M, Clark AT, Neyra JA et al. Acute kidney injury in patients with burns. Nat Rev Nephrol 2024;20:188–200. 10.1038/s41581-023-00769-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Al-Jaghbeer M, Dealmeida D, Bilderback A et al. Clinical decision support for in-hospital AKI. JASN 2018;29:654–60. 10.1681/ASN.2017070765 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Wilson FP, Greenberg JH. Acute kidney injury in real time: prediction, alerts, and clinical decision support. Nephron 2018;140:116–9. 10.1159/000492064 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Park S, Baek SH, Ahn S et al. Impact of electronic acute kidney injury (AKI) alerts with automated nephrologist consultation on detection and severity of AKI: a quality improvement study. Am J Kidney Dis 2018;71:9–19. 10.1053/j.ajkd.2017.06.008 [DOI] [PubMed] [Google Scholar]
- 32. Tome ACN, Ramalho RJ, Dos Santos KF et al. Impact of an electronic alert in combination with a care bundle on the outcomes of acute kidney injury. Diagnostics 2022;12:3121. 10.3390/diagnostics12123121 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
Individual participant data will not be available. The study protocol, statistical analysis plan, and analytic code are to be used for approved proposal aims only. Send data access requests to yongjin.neph@gmail.com and sign a data access agreement.