ABSTRACT
Background
Models developed to predict hospital-acquired acute kidney injury (HA-AKI) in non-critically ill patients have a low sensitivity, do not include dynamic changes of risk factors and do not allow the establishment of a time relationship between exposure to risk factors and AKI. We developed and externally validated a predictive model of HA-AKI integrating electronic health databases and recording the exposure to risk factors prior to the detection of AKI.
Methods
The study set was 36 852 non-critically ill hospitalized patients admitted from January to December 2017. Using stepwise logistic analyses, including demography, chronic comorbidities and exposure to risk factors prior to AKI detection, we developed a multivariate model to predict HA-AKI. This model was then externally validated in 21 545 non-critical patients admitted to the validation centre in the period from June 2017 to December 2018.
Results
The incidence of AKI in the study set was 3.9%. Among chronic comorbidities, the highest odds ratios (ORs) were conferred by chronic kidney disease, urologic disease and liver disease. Among acute complications, the highest ORs were associated with acute respiratory failure, anaemia, systemic inflammatory response syndrome, circulatory shock and major surgery. The model showed an area under the curve (AUC) of 0.907 [95% confidence interval (CI) 0.902–0.908), a sensitivity of 82.7 (95% CI 80.7–84.6) and a specificity of 84.2 (95% CI 83.9–84.6) to predict HA-AKI, with an adequate goodness-of-fit for all risk categories (χ2 = 6.02, P = 0.64). In the validation set, the prevalence of AKI was 3.2%. The model showed an AUC of 0.905 (95% CI 0.904–0.910), a sensitivity of 81.2 (95% CI 79.2–83.1) and a specificity of 82.5 (95% CI 82.2–83) to predict HA-AKI and had an adequate goodness-of-fit for all risk categories (χ2 = 4.2, P = 0.83). An online tool (predaki.amalfianalytics.com) is available to calculate the risk of AKI in other hospital environments.
Conclusions
By using electronic health data records, our study provides a model that can be used in clinical practice to obtain an accurate dynamic and updated assessment of the individual risk of HA-AKI during the hospital admission period in non-critically ill patients.
Keywords: acute kidney injury, electronic health data records, hospital-acquired, prediction, risk score
INTRODUCTION
Acute kidney injury (AKI) is a frequent and serious complication in hospitalized patients [1–3]. In addition, AKI has been associated with long-term morbidity and mortality after hospital discharge [4, 5].
Most cases of AKI in hospitalized patients are caused by ischaemia or nephrotoxicity [6–8]. The risk of developing AKI depends on the characteristics of the patient in terms of age, presence of previous kidney disease and number and types of comorbidities [9]. Since a large part of the AKI episodes are due to potentially avoidable causes, knowing as accurately as possible the individual risk at any time of the hospital stay could help decision making and implementation of preventive measures to reduce the incidence of hospital AKI [10]. The Kidney Disease: Improving Global Outcomes (KDIGO) guidelines recommend that patients be stratified for risk of AKI at admission and managed according to their susceptibilities and exposures to reduce the risk of AKI [11]. The incidence and risk factors associated with AKI in patients admitted to intensive care units (ICUs) have been extensively analysed [12, 13]. However, these models are difficult to extrapolate to non-critically ill patients since they have been developed for patients that are under the influence of a cluster of risk factors related to haemodynamic instability, use of vasoactive drugs, low tissue oxygenation, inflammatory response and invasive procedures such as mechanical ventilation that are unique to this environment [14]. The few studies analysing the epidemiology and risk factors associated with AKI in non-critically ill patients have two main limitations to identify accurately the risk factors associated with AKI. First, all of them are based on demographic characteristics and comorbidities that have been registered retrospectively from the discharge administrative codes and therefore are subject to a potential bias in the collection of coded information [15–20]. Second, they do not allow us to know whether the exposure to risk factors preceded the detection of the AKI episode [21].
The aim of our study was to develop and validate a predictive model of hospital-acquired AKI (HA-AKI) in non-critically ill patients in which risk factors are automatically obtained by integrating electronic health databases, it is ensured that the exposure to risk factors precedes the detection of the AKI episode and AKI episodes are automatically detected through electronic systems based on the calculation of differences in creatinine levels.
MATERIALS AND METHODS
This prospective study was performed at two different hospital centres. The first centre developed the predictive model (study set) and the second centre performed the external validation of the predictive model (validation set).
Study set
The study set included patients admitted to the Vall d’Hebron University Hospital from January to December 2017. Vall d’Hebron is a tertiary, high-complexity hospital that provides assistance to a population of 500 000 habitants in Barcelona, Spain and provides all kinds of medical and surgical procedures, including neurosurgery, cardiac surgery, endovascular catheter-guided procedures and lung, liver, kidney and bone marrow transplantation programmes. We included all patients >18 years of age who were admitted to the hospital during this period and did not meet any of the following exclusion criteria: admission for community-acquired AKI; hospital stay <24 h; admission for elective heart surgery; direct admission from the emergency room to the ICU; admission as a recipient of a renal, lung, liver or bone marrow transplant; absence of serum creatinine measurements done at least 12 months after hospital admission; chronic haemodialysis treatment and denial of written consent to participate in the study. Community-acquired AKI was diagnosed whenever patients met the AKI criteria within the first 24 h of hospital admission. Patients initially admitted to conventional hospitalization wards who afterwards required admission to the ICU were only included if the AKI episode was detected while they were admitted in non-critically ill wards prior to their admission to the ICU.
Baseline kidney function
Our patient care system integrates the laboratory databases of the hospital and primary care registers, thus allowing historical data to be obtained for all patients who are hospitalized, provided that these data appear in those registers. Baseline kidney function was obtained from the electronic laboratory data records of primary healthcare and defined as the most recent glomerular filtration rate, estimated by the Chronic Kidney Disease Epidemiology Collaboration equation, within the 12 months prior to hospital admission.
Definition of AKI
AKI was defined and classified in severity stages according to the KDIGO criteria [11]. HA-AKI was defined as an increase in serum creatinine ≥0.3 mg/dL or >50% over the baseline occurring from the first 24 h to any time within the hospital admission.
AKI detection
The software integrated into the electronic laboratory database was used to perform repeated comparisons among all serum creatinine levels available for each patient during the hospital stay and generated an identification code, assigning 1 when the AKI criteria were met and 0 when not. It also assigned a level of AKI severity according to the maximum differences in serum creatinine detected. The date of AKI detection was also recorded. The number of the admission episode was used as a filter so that patients with more than one AKI episode during the hospital stay were entered into the database only once, corresponding with the more severe episode of AKI.
Clinical evaluation at hospital admission and during hospital stay
At hospital admission, a team of 10 trained nurses and 4 nephrologists examined the medical data and interviewed all patients to record age, gender, ethnic group and the presence of the following chronic comorbidities: diabetes mellitus, hypertension, ischaemic heart disease (IHD), ischaemic cerebrovascular disease (ICD), ischaemic peripheral vascular disease (PVD), chronic digestive disease, chronic liver disease, chronic congestive heart failure (CCHF), malnutrition (MN), chronic obstructive pulmonary disease (COPD), malignancy, dementia, rheumatologic disease, acquired immunodeficiency syndrome (AIDS)/human immunodeficiency virus (HIV), urologic disease or chronic kidney disease (CKD). All these variables were recorded in the general study database according to the criteria detailed in the Supplementary methods operational definitions. Nutritional status was assessed using theNutritional Risk Screening 2002 test [22]. The allocation of comorbidity codes to each patient was carried out by consensus among clinical researchers. All patients were followed up until hospital discharge. During the hospital stay, the data of six electronic health databases, i.e. vital signs, laboratory, pharmacy prescription, interventional radiology, interventional cardiology and surgery, were integrated together using the number of the admission episode, which is unique for each patient and common to all these databases. Overall, the information extracted from these six databases included haemoglobin levels, leucocyte count, oxygen saturation, body temperature, blood pressure, heart rate and respiratory rate, as well as a complete list of nephrotoxic drugs (detailed in Supplementary data, Table S1) and exposure to contrast dyes or major surgery. Every 24 h, updated information from all these data was dumped into the general study database, which also contained the comorbidity data and all available values of serum creatinine for each patient. From these data, software generated classification codes for anaemia, hypoxaemic acute respiratory failure, systemic inflammatory response syndrome (SIRS), shock, exposure to nephrotoxic drugs, contrast dyes and major surgery. Using these codes, the exposure to all these risk factors was classified as positive (= 1), when the system detected at least one exposure during the hospital stay, or negative (= 0), when no exposure was detected. In all cases, the system recorded the data of exposure to each of these variables as well as the number of exposures to them. In patients with a code of AKI = 1, the exposure to these risk factors only was classified as equal to 1 when it occurred within a maximum period of time prior to AKI detection (48 h for anaemia, SIRS and shock, 72 h for contrast dyes and surgery and 7 days for nephrotoxic drugs). Figure 1 shows a schematic view of the interrelation process among the different electronic databases carried out to obtain the information on the clinical variables during the hospital stay.
FIGURE 1:
Schematic representation of the interrelation between electronic databases performed to obtain updated clinical information during the hospital stay.
At hospital admission (A), chronic comorbidities are checked by the research team according to explicit criteria and recorded in the general database. During the hospital stay (B), the data of five different electronic health databases are integrated using the admission episode number and all of them dump the requested information into the general study database. The laboratory database performs repeated comparisons among all serum creatinine levels and generates an identification code, assigning a 1 when the AKI criteria are met and a 0 when not. It also assigns a level of AKI severity according to the maximum differences in serum creatinine detected. The date of AKI detection is also recorded. The admission episode number is used as a filter so that patients with more than one AKI episode during the hospital stay are entered into the system only once, corresponding with the more severe episode of AKI. The follow-up of haemoglobin levels is used to generate a classification code of anaemia. The level of oxygen saturation is used to generate a code of hypoxaemic acute failure. Information on blood leucocyte levels, together with temperature, heart and respiratory rate, are integrated to generate a code for SIRS and information on blood pressure, together with the prescription of vasoactive drugs, is used to generate a code for shock. A complete list of direct nephrotoxic drugs is introduced in the pharmacy prescription database, which generates a code of exposure every time the prescription list contains any of these drugs. The databases of radiology, angioradiology and interventional cardiology provide information about the exposure to contrast dyes and the database of surgery provides information about major surgery and anaesthesia. In all cases, the system records the data for exposure to each one of these factors. In patients with a code of AKI = 1, the exposure to risk factors is classified as equal to 1 only when it occurs within a maximum period of time prior to AKI detection (48 h for anaemia, SIRS and shock, 72 h for contrast dyes and surgery and 7 days for nephrotoxic drugs). In patients with a code of AKI = 0, the exposure to risk factors is classified as positive (= 1), when the system detects at least one exposure during the hospital stay, or negative (= 0), when none is detected. In both cases (AKI and no AKI), the number of exposures to each risk factor is also recorded.
Unlike the haemoglobin level, arterial oxygen saturation, heart rate, respiratory rate and blood pressure level, being numerical variables that can be directly transferred into the general database, both circulatory shock and SIRS are complex variables that, to be automatically detected using a software-guided detection code, require the integration of data from various electronic records and the definition of classification algorithms. In both cases, before using them in statistical analyses, we analysed the accuracy of the automatic detection systems in a sample of 3426 patients. To do this, using data blindly obtained by two independent clinical investigators, we performed a concordance analysis between the identification of cases using electronic detection systems and the diagnosis made by the investigators using clinical criteria, as well as an analysis of interobserver agreement for both clinical diagnoses. The results of these analyses are summarized in Supplementary data, Table S2.
Validation set
The predictive model obtained at the Vall d’Hebron Hospital was externally validated in patients admitted to the Arnau de Vilanova Hospital of Lleida between June 2017 and December 2018. Arnau de Vilanova Hospital is a high-complexity teaching centre that provides assistance to 490 000 habitants. This centre provides similar activities as the Hospital Vall d’Hebron with the exceptions of transplant programmes and cardiac surgery. The selection of patients and the study procedures were done according to the same criteria stated for the study set. The external validation study was performed by an independent research team that did not participate in the development of the predictive model and it was tested in the hospital electronic health record only.
The ethics committee of the Arnau de Vilanova Hospital was consulted and they decided that informed consent was not necessary for the validation of the model, given that no type of intervention was carried out on the patients.
Statistics
The incidence and prevalence calculations included the total number of admissions. For patients who developed more than one AKI episode during the hospital admission, only the most severe episode was included in the study. Patients were considered to be at risk each time they were admitted to the hospital and therefore patients who were admitted two or more times during the study period were included in the calculations on each admission, except when readmission occurred within 30 days after hospital discharge. Results are given as the mean ± standard deviation (SD) or median and [interquartile range (IQR), 25th percentile–75th percentile]. Differences in risk factors between groups were calculated by the Student’s unpaired t-test oranalysis of variance test. Qualitative variables were compared using the chi-squared test. Concordance analyses between qualitative variables were done by the kappa coefficient. P-values ˂0.05 were considered statistically significant. To determine which variables were independently associated with AKI, we carried out a univariate analysis comparing patients with and without AKI. All the variables with a P-value ˂0.1 in the univariate analysis were entered into stepwise multiple logistic regression analysis with a forward selection method based on changes in the likelihood ratio (LR). Odds ratios (ORs) were calculated from the regression coefficients as an approximation of the relative risk. The predictive value of the logistic model was evaluated using the C statistic, Cox and Snell R2 and Nagelgerke R2. Model overfitting was prevented using the Akaike information criterion (AIC) [23, 24]. The Hosmer–Lemeshow test [25] was used as well to calculate the discrimination power and goodness-of-fit of the logistic model. Results are presented according to theTransparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis guidelines for risk prediction models [26, 27]. Once obtained in the study set, the predictive logistic model was blindly tested on the external validation set by an independent group of researchers who did not participate in the development of the predictive model. Statistical analyses were performed with the Statistical Package for the Social Sciences for Windows version 20.0 (IBM, Armonk, NY, USA).
RESULTS
Study set
During the study period there were 42 449 hospital discharges. Figure 2 shows the flow chart for patient selection. The final study group comprised 36 852 patients. Of this cohort, 1453 (3.9%) developed AKI, with an incidence of 39 AKI episodes/1000 hospital admissions. Distribution by AKI stages was Stage 1, n = 1069 (73.5%); Stage 2, n = 258 (17.8%) and Stage 3, n = 126 (8.7%).
FIGURE 2:
Flow chart for patient selection.
Table 1 summarizes the demographic characteristics, comorbidities, clinical events and procedures during the hospital stay in the study group, classified according to the presence of AKI. AKI patients were older and more frequently male than non-AKI patients. Comorbidities including IHD, ICD, ischaemic PVD, chronic liver disease, CCHF, MN, COPD, malignancy, urologic disease and CKD stages were also more frequent in AKI patients. The AKI risk increased linearly as glomerular filtration rate decreased. Patients with AKI also showed significantly higher rates of urgent admission, anaemia, acute respiratory failure, SIRS, shock, major surgery and exposure to contrast dyes and nephrotoxic drugs.
Table 1.
Demographic characteristics, chronic comorbidities, clinical events and procedures during the hospital admission and univariate logistic analysis of variables associated with HA-AKI in the study group
| Variables | Total | AKI | Non-AKI | OR (95 % CI) |
|---|---|---|---|---|
| Patients, n (%) | 36 852 (100) | 1453 (3.9) | 35 399 (96) | |
| Gender (male), n (%) | 16 782 (45.5) | 879 (60.5) | 15 903 (44.9) | 1.76 (1.59–1.96) |
| Age (years), mean (SD) | 54.9 (20.6) | 73 (15.0) | 54 (20.5) | 1.065 (1.061–1.070) |
| Chronic comorbidities, n (%) | ||||
| Diabetes | 6837 (18.6) | 574 (39.5) | 6263 (17.7) | 3.04 (2.73–3.39) |
| Hypertension | 14 507 (39.4) | 990 (68.1) | 13 517 (38.2) | 3.46 (3.09–3.87) |
| IHD | 2728 (7.4) | 194 (13.4) | 2534 (7.2) | 2.00 (1.71–2.34) |
| ICD | 2560 (6.9) | 181 (12.5) | 2379 (6.7) | 1.98 (1.68–2.32) |
| Ischaemic PVD | 1924 (5.2) | 138 (9.5) | 1786 (5.0) | 1.98(1.65–2.37) |
| Chronic digestive disease | 2132 (5.8) | 70 (4.8) | 2062 (5.8) | 0.82 (0.64–1.05) |
| Chronic liver disease | 1277 (3.5) | 123 (8.5) | 1154 (3.3) | 2.74 (2.26–3.33) |
| CCHF | 2988 (8.1) | 225 (15.5) | 2763 (7.8) | 2.16 (1.87–2.51) |
| MN | 8524 (23.1) | 766 (52.7) | 7758 (21.9) | 3.97 (3.57–4.42) |
| COPD | 5383 (14.6) | 537 (37.0) | 4846 (13.7) | 3.7 (3.10–4.30) |
| Malignancy | 5278 (14.3) | 496 (34.1) | 4782 (13.5) | 3.32 (2.97–3.71) |
| Dementia | 332 (0.9) | 14 (1.0) | 318 (0.9) | 1.07 (0.63–1.84) |
| Rheumatologic disease | 1543 (4.2) | 58 (4.0) | 1486 (4.2) | 0.95 (0.73–1.24) |
| AIDS/HIV | 293 (0.8) | 28 (1.9) | 265 (0.7) | 2.61 (0.76–3.86) |
| Urologic disease | 2731 (7.4) | 172 (11.8) | 2559 (7.2) | 1.72 (1.46–2.07) |
| CKD stages, n (%) | ||||
| 0 + 1 | 30 260 (82.1) | 879 (60.5) | 29 381 (83.0) | Reference |
| 2 | 3654 (9.9) | 192 (13.2) | 3462 (9.8) | 1.85 (1.58–2.18) |
| 3 | 2171(5.9) | 231 (15.9) | 1940 (5.5) | 3.98 (3.42–4.63) |
| 4 | 767 (2.1) | 151 (10.4) | 616 (1.7) | 8.19 (6.77–9.91) |
| Clinical variables during hospital admission, n (%) | ||||
| Urgent admission | 24 441 (66.3) | 1282 (88.2) | 23 159 (65.4) | 3.96 (3.37–4.66) |
| Anaemia | 5417 (14.7) | 528 (36.3) | 4889 (13.8) | 3.56 (3.19–3.98) |
| Acute respiratory failure | 1827 (5.0) | 286 (19.7) | 1541 (4.4) | 5.39 (4.69–6.19) |
| SIRS | 658 (1.8) | 271 (18.7) | 387 (1.1) | 20.74 (17.58–24.48) |
| Circulatory shock | 650 (1.8) | 300 (20.6) | 350 (1.0) | 26 (22.09–30.73) |
| Major surgery | 12 127 (32.9) | 594 (40.9) | 11 533 (32.6) | 1.43 (1.29–1.59) |
| Exposure to contrast media | 3353 (9.1) | 303 (20.9) | 3050 (8.6) | 2.80 (2.45–3.19) |
| Exposure to nephrotoxic drugs | 19 145 (52.0) | 1011 (69.6) | 18 134 (51.2) | 2.18 (1.94–2.44) |
The results of the logistic model to predict AKI are summarized in Table 2. The highest ORs were associated with advanced stages of CKD, shock, acute respiratory failure and SIRS. The model showed an AUC of 0.907 (95% CI 0.902–0.908), with a sensitivity of 82.7 (95% CI 80.7–84.6) and a specificity of 84.2 (95% CI 83.9–84.6) to predict HA-AKI and showed an adequate goodness-of-fit for all risk categories (Table 3; χ2 = 6.02, P = 0.64).
Table 2.
Final multivariate model selected by forward stepwise logistic regression to predict HA-AKI
| Variables | β | Standard error | Wald | OR (95% CI) | P-value |
|---|---|---|---|---|---|
| Gender (male), n (%) | 0.21 | 0.069 | 9.63 | 1.29 (1.08–1.42) | 0.002 |
| Age (years), mean (SD) | 0.05 | 0.003 | 218.25 | 1.05 (1.04–1.05) | <0.001 |
| Chronic comorbidities | |||||
| Diabetes | 0.48 | 0.085 | 31.67 | 1.61 (1.36–1.90) | <0.001 |
| Hypertension | 0.17 | 0.073 | 5.76 | 1.19 (1.03–1.37) | 0.016 |
| Ischaemic heart disease | 0.32 | 0.101 | 10.02 | 1.38 (1.13–1.67) | 0.002 |
| Ischaemic peripheral vascular disease | 0.41 | 0.123 | 11.31 | 1.51 (1.19–1.93) | 0.001 |
| Chronic liver disease | 1.04 | 0.126 | 68.95 | 2.84 (2.22–3.63) | <0.001 |
| CCHF | 0.48 | 0.079 | 37.05 | 1.61 (1.38–1.88) | <0.001 |
| MN | 0.25 | 0.078 | 10.23 | 1.29 (1.10–1.50) | 0.001 |
| COPD | 0.32 | 0.085 | 14.03 | 1.37 (1.16–1.62) | <0.001 |
| Malignancy | 0.59 | 0.089 | 40.27 | 1.76 (1.48–2.10) | <0.001 |
| Chronic urologic disease | 0.96 | 0.117 | 67.02 | 2.60 (2.07–3.27) | <0.001 |
| CKD stages | |||||
| 0 + 1 | Reference | ||||
| 2 | 0.89 | 0.096 | 84.42 | 2.42 (2.01–2.93) | <0.001 |
| 3 | 1.38 | 0.098 | 198.25 | 3.98 (3.28–4.82) | <0.001 |
| 4 | 2.04 | 0.125 | 265.63 | 7.67 (5.99–9.78) | <0.001 |
| Clinical variables during hospital admission | |||||
| Urgent admission | 0.79 | 0.097 | 66.42 | 2.21 (1.83–2.67) | <0.001 |
| Anaemia | 0.78 | 0.069 | 125.61 | 2.18 (1.90–2.49) | <0.001 |
| Acute respiratory failure | 1.26 | 0.097 | 169.11 | 3.53 (2.92–4.27) | <0.001 |
| Acute heart failure | 0.69 | 0.095 | 53.24 | 2.00 (1.66–2.41) | <0.001 |
| SIRS | 1.25 | 0.129 | 94.88 | 3.50 (2.72–4.5) | <0.001 |
| Circulatory shock | 1.82 | 0.127 | 205.48 | 6.16 (4.80–7.89) | <0.001 |
| Major surgery | 0.99 | 0.076 | 169.43 | 2.70 (2.32–3.13) | <0.001 |
| Exposure to contrast media | 0.52 | 0.087 | 36.52 | 1.69 (1.43–2.00) | <0.001 |
| Exposure to nephrotoxic drugs | 0.57 | 0.070 | 67.04 | 1.77 (1.54–2.03) | <0.001 |
Table 3.
Hosmer–Lemeshow’s goodness-of-fit of the logistic predictive model in the study group
| Risk deciles | AKI = 0 | AKI = 1 | Total | ||
|---|---|---|---|---|---|
| Observed | Expected | Observed | Expected | ||
| <0.0008504 | 3684 | 3683.9 | 2 | 2.1 | 3686 |
| 0.0085041–0.0016950 | 3678 | 3680.7 | 8 | 5.3 | 3686 |
| 0.0016951–0.0031772 | 3675 | 3675.8 | 11 | 10.1 | 3686 |
| 0.0031773–0.0053733 | 3663 | 3660.6 | 15 | 17.3 | 3678 |
| 0.0053733–0.0087714 | 3655 | 3656.7 | 30 | 28.2 | 3685 |
| 0.0087715–0.0140660 | 3644 | 3641.7 | 42 | 44.2 | 3686 |
| 0.0140661–0.0228354 | 3628 | 3615.7 | 56 | 68.2 | 3684 |
| 0.0228355–0.0394850 | 3586 | 3575.0 | 100 | 110.9 | 3686 |
| 0.0394851–0.0850955 | 3471 | 3477.2 | 214 | 207.7 | 3685 |
| >0.0850955 | 2715 | 2731.0 | 974 | 957.9 | 3689 |
χ2 = 6.01. P = 0.645.
Supplementary data, Table S3 summarizes the results of the stepwise forward procedures done to develop the final logistic model, including changes in the LRs, Cox and Snell R2, Nagelkerke R2 and AIC.
Validation set
The demographic characteristics, comorbidities and clinical parameters of the study and external validation cohorts are summarized in Table 4.
Table 4.
Comparison of demographic characteristics, comorbidities and clinical variables between the study set and the external validation set
| Variables | Study set | Validation set | P-value |
|---|---|---|---|
| Patients, n | 36 852 | 21 545 | – |
| Gender (male), n (%) | 16 782 (45.5) | 9932 (46) | 0.19 |
| Age (years), mean (SD) | 549 (20.6) | 60.1 ± 19.7 | 0.38 |
| Chronic comorbidities, n (%) | |||
| Diabetes | 6837 (18.6) | 3942 (18.3) | 0.44 |
| Hypertension | 14 507 (39.4) | 8389 (38.9) | 0.27 |
| IHD | 2728 (7.4) | 1573 (7.3) | 0.66 |
| ICD | 2560 (6.9) | 1486 (6.9) | 0.83 |
| Ischaemic PVD | 1924 (5.2) | 1163 (5.4) | 0.36 |
| Chronic digestive disease | 2132 (5.8) | 1228 (5.7) | 0.68 |
| Chronic liver disease | 1277 (3.5) | 775 (3.6) | 0.41 |
| CCHF | 2988 (8.1) | 1659 (7.7) | 0.08 |
| MN | 8524 (23.1) | 4869 (22.6) | 0.14 |
| COPD | 5383 (14.6) | 3102 (14.4) | 0.49 |
| Malignancy | 5278 (14.3) | 3038 (14.1) | 0.47 |
| Dementia | 332 (0.9) | 172 (0.8) | 0.1 |
| Rheumatologic disease | 1543 (4.2) | 851 (4) | 0.56 |
| AIDS/HIV | 293 (0.8) | 86 (0.4) | <0.0001 |
| Urologic disease | 2731 (7.4) | 1573 (7.3) | 0.63 |
| CKD stages, n (%) | |||
| 0 + 1 | 30 260 (82.1) | 17 731 (82.3) | 0.015 |
| 2 | 3654 (9.9) | 2198 (10.2) | – |
| 3 | 2171 (5.9) | 1142 (5.3) | – |
| 4 | 767 (2.1) | 474 (2.2) | – |
| Clinical variables during the hospital admission, n (%) | |||
| Urgent admission | 24 441 (66.3) | 14 422 (66.9) | 0.13 |
| Anaemia | 5417 (14.7) | 3189 (14.8) | 0.74 |
| Acute respiratory failure | 1827 (5) | 1120 (0.5) | 0.19 |
| SIRS | 658 (1.8) | 383 (1.8) | 0.97 |
| Circulatory shock | 650 (1.8) | 370 (1.72) | 0.68 |
| Major surgery | 12 127 (32.9) | 6753 (31.3) | <0.0001 |
| Exposure to contrast media | 3353 (9.1) | 2068 (9.6) | 0.6 |
| Exposure to nephrotoxic drugs | 19 145 (52) | 11 144 (51.7) | 0.59 |
When compared with the study set, patients of the validation set showed significantly lower prevalences of major surgery and patients with AIDS/HIV. There was as well a significant difference in the distribution of CKD stages between the two centres. In the validation set, 807/21 545 (3.7%) developed HA-AKI, with an incidence of 37.4 AKI episodes/1000 hospital admissions. Distribution by AKI stages was Stage 1, n = 605 (75%); Stage 2, n = 129 (16%) and Stage 3, n = 736 (9%), with no significant differences between the study set and validation set. When the predictive model was tested in the validation set, it showed an AUC of 0.905 (95% CI 0.904–0.910), with a sensitivity of 81.2 (95% CI 79.2–83.1) and a specificity of 82.5 (95% CI 82.2–83) to predict HA-AKI and an adequate goodness-of-fit for all risk categories (χ2 = 4.2,P = 0.83; Table 5).
Table 5.
Hosmer–Lemeshow’s goodness-of-fit of the logistic predictive model in the validation group
| Deciles of risk | AKI = 0 | AKI = 1 | Total | ||
|---|---|---|---|---|---|
| Observed | Expected | Observed | Expected | ||
| <0.0009026 | 2154 | 2153.6 | 1 | 1.4 | 2155 |
| 0.0009027–0.0018699 | 2151 | 2149.9 | 2 | 3.1 | 2153 |
| 0.0018700–0.0032362 | 2150 | 2149.5 | 5 | 5.4 | 2155 |
| 0.0032363–0.0054175 | 2143 | 2145.2 | 11 | 8.7 | 2154 |
| 0.0054176–0.0085382 | 2144 | 2140.2 | 10 | 13.8 | 2154 |
| 0.0085383–0.0127563 | 2134 | 2131.9 | 20 | 22.1 | 2154 |
| 0.0127564–0.0214063 | 2121 | 2118.3 | 33 | 35.6 | 2154 |
| 0.0214064–0.0391575 | 2093 | 2092.6 | 61 | 61.3 | 2154 |
| 0.0391575–0.0874214 | 2034 | 2035.0 | 120 | 118.9 | 2154 |
| >0.0874214 | 1612 | 1623.3 | 546 | 533.6 | 2158 |
χ2 = 4.2. P = 0.836.
There were no significant differences between the AUC obtained in the study set and that obtained in the validation set (Figures 3 and 4).
FIGURE 3:
Flow chart for patient selection.
FIGURE 4:
Comparison of AUCs obtained in the study set and in the validation set.
DISCUSSION
In this study we integrated the information of six electronic health databases commonly used in clinical practice to develop and externally validate a predictive dynamic model that allows one to accurately estimate the individual likelihood of suffering AKI at any time during a hospital stay in non-critically ill patients. In the study group, the final logistic model identified two sets of risk factors. The first set included the demographic data and the patient’s chronic comorbidities. The second included a set of risk factors related to the patient’s clinical status and to the exposure to major surgery, contrast media or nephrotoxic drugs during the hospital stay. This model showed a high sensitivity and specificity to predict hospital AKI and showed an adequate calibration for all risk categories, both in the study group and in the validation group. When compared with previously published risk models, our model differs at various points. First, unlike previous studies, our study provides a model that allows estimating the risk of HA-AKI tailored to patients admitted to non-critical hospital wards. Moreover, in order to obtain a predictive model that could be exportable to hospitals with different characteristics and complexities, patients who were admitted for programmes and/or procedures such as cardiac surgery or solid organ or bone marrow transplantation that are not commonly available at all hospital centres were deliberately excluded. This potential scalability to less complex centres could be demonstrated as the model had the same performance in the validation set as in the study set. Second, in our study, comorbidities were not obtained through the administrative discharge codes but were checked case by case. Moreover, the classification of comorbidities was performed using explicit and objective definition criteria. In this way, biases related to discrepancies in assigning administrative codes to clinical conditions or to the lack of coding of certain comorbidities were minimized as much as possible. As proof of this, the prevalences of certain comorbidities observed in our cohort of patients are higher than those reported from administrative discharge codes in previous studies [28, 29]. Additionally, in our cohort, the prevalence of comorbidities such as MN, which are barely recorded in the diagnostic codes of discharge, showed similar figures than those described in studies specifically designed to analyse its prevalence [30]. Overall, these differences are in agreement with previously published data that demonstrate the variability and limitations of administrative data to define co-morbidities and clinical conditions [31]. Third, comorbidities were considered separately, which allowed assigning a risk to each of them and identifying independent predictors of AKI risk such as MN, which are not described in previous models. In addition, our model allowed stratifying the risk associated with renal function in greater detail than that provided by the dichotomous classification, depending on the presence or absence of chronic renal failure. Fourth, the main novelty of our study is the prospective monitoring of the evolution of the clinical data of the patients through integration and cross-talk between the different electronic databases containing all these data. This procedure allowed us to analyse the dynamic exposure to risk factors related to the clinical status of patients during the hospital stay, such as hypoxaemia, haemoglobin level, blood pressure changes, contrast dyes or nephrotoxic drugs, prior to the detection of the AKI episode. This integration allowed as well to perform an accurate transformation of single variables such as blood pressure, heart rate, arterial oxygen saturation, prescription of vasoactive drugs or blood leucocyte counts into more complex variables defining specific syndromes such as SIRS and circulatory shock. Electronic records also allowed us to record the exposure to the same variables and risk factors in patients who did not develop AKI during the hospital admission. This approach made it possible to estimate the individual risk, based on the actual exposure to each risk factor. Lastly, since our predictive model was developed from the values of risk factors assessed prior to AKI detection, our model allows one perform dynamic monitoring of risk and even to predict the changes in the individual risk that are expected to happen every time the values of different predictive risk factors change.
Our group recently performed external validation of one of the most recent predictive models of acute renal failure, the Madrid Acute Kidney Injury Prediction Score (MAKIPS) [32]. This model can be calculated automatically from electronic medical records and could be easily implemented in clinical practice. With our validation, we conclude that the MAKIPS can be a useful tool, easily obtainable from electronic records data, to predict AKI in hospitals of different complexity. However, this model, as well as many others described, has the main limitation that it does not include dynamic factors. The inclusion of dynamic changes of possible acute precipitants in the models is technically complex and constitutes a challenge for future research. It could lead to a significant improvement in the discrimination of predictive models and could also generate dynamic predictive models capable of detecting changes in the risk profile of patients throughout the hospital stay.
Our model has some limitations that affect neither its predictive capacity nor its calibration but must be highlighted. First, the record of clinical variables such as blood pressure, heart rate, respiratory rate or oxygen saturation were automatically dumped into the study database; however, these values are not without potential error related to the variability in the manual introduction of these variables into their corresponding databases. Second, although the model allows AKI to be accurately predicted, it does not predict its severity stage. Third, our data indicate that integrating data from different electronic databases make it possible to obtain a reliable prediction of the risk of AKI. However, the model obtained in our study is not the only one that can be obtained with the combination of these data. As exposure to each of the acute complications or nephrotoxic agents can occur at different times after hospital admission, in order to relate the exposure to them with the development of AKI it was necessary to define a maximum period of time between exposure and detection of AKI. In our study, the duration of this period of time was defined by consensus of the research group using pathophysiological criteria. The definition of other periods of time, based on alternative criteria, would modify the prevalence of exposure to these risk factors and consequently the magnitude of the associations found between these variables and AKI.
In conclusion, our study provides an externally validated model based on demographic data, specific comorbidities, acute clinical conditions and procedures that can be used in clinical practice to obtain an accurate dynamic assessment of the individual risk of suffering AKI during the entire hospital stay period in patients admitted into non-critical hospitalization wards. This model is highly versatile and allows for performing repeated manual risk estimation, using the prediction algorithm, to provide an automatic risk measure updated in real time in those centres where it is possible to carry out a complete integration of the healthcare databases containing the necessary information.
SUPPLEMENTARY DATA
Supplementary data are available at ckj online.
FUNDING
This study was funded by grants from Amgen S.A. and Menarini S.A, but these companies did not participate in the study design, data collection, analyses or writing of the study.
CONFLICT OF INTEREST STATEMENT
None declared.
Supplementary Material
Contributor Information
Alfons Segarra, Department of Nephrology, Arnau de Vilanova University Hospital, Lleida, Spain; Department of Nephrology, Vall d'Hebron University Hospital, Barcelona, Spain.
Jacqueline Del Carpio, Department of Nephrology, Arnau de Vilanova University Hospital, Lleida, Spain; Department of Medicine, Autonomous University of Barcelona, Barcelona, Spain.
Maria Paz Marco, Department of Nephrology, Arnau de Vilanova University Hospital, Lleida, Spain.
Elias Jatem, Department of Nephrology, Arnau de Vilanova University Hospital, Lleida, Spain.
Jorge Gonzalez, Department of Nephrology, Arnau de Vilanova University Hospital, Lleida, Spain.
Pamela Chang, Department of Nephrology, Arnau de Vilanova University Hospital, Lleida, Spain.
Natalia Ramos, Department of Nephrology, Vall d'Hebron University Hospital, Barcelona, Spain.
Judith de la Torre, Department of Nephrology, Vall d'Hebron University Hospital, Barcelona, Spain; Department of Nephrology, Althaia Foundation, Manresa, Spain.
Joana Prat, Department of Development, Parc Salut Hospital, Barcelona, Spain; Department of Informatics, Vall d'Hebron University Hospital, Barcelona, Spain.
Maria J Torres, Department of Informatics, Vall d'Hebron University Hospital, Barcelona, Spain; Department of Information, Southern Metropolitan Territorial Management, Barcelona, Spain.
Bruno Montoro, Department of Hospital Pharmacy, Vall d'Hebron University Hospital, Barcelona, Spain.
Mercedes Ibarz, Laboratory Department, Arnau de Vilanova University Hospital, Lleida, Spain.
Silvia Pico, Laboratory Department, Arnau de Vilanova University Hospital, Lleida, Spain.
Gloria Falcon, Technical Secretary and Territorial Management of Lleida-Pirineus, Lleida, Spain.
Marina Canales, Technical Secretary and Territorial Management of Lleida-Pirineus, Lleida, Spain.
Elisard Huertas, Informatic Unit of the Catalonian Institute of Health–Territorial Management, Lleida, Spain.
Iñaki Romero, Territorial Management Information Systems, Catalonian Institute of Health, Lleida, Spain.
Nacho Nieto, Department of Informatics, Vall d'Hebron University Hospital, Barcelona, Spain; Department of Information, Southern Metropolitan Territorial Management, Barcelona, Spain.
REFERENCES
- 1. Susantitaphong P, Cruz DN, Cerda J et al. World incidence of AKI: a meta-analysis. Clin J Am Soc Nephrol 2013; 8: 1482–1493 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Chertow GM, Burdick E, Honour M et al. Acute kidney injury, mortality, length of stay, and costs in hospitalized patients. J Am Soc Nephrol 2005; 16: 3365–3370 [DOI] [PubMed] [Google Scholar]
- 3. Mehta RL, Burdmann EA, Cerda J et al. Recognition and management of acute kidney injury in the International Society of Nephrology 0by25 global snapshot: a multinational cross-sectional study. Lancet 2016; 387: 2017–2025 [DOI] [PubMed] [Google Scholar]
- 4. Forni LG, Darmon M, Ostermann M et al. Renal recovery after acute kidney injury. Intensive Care Med 2017; 43: 855–866 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Bucaloiu ID, Kirchner HL, Norfolk ER et al. Increased risk of death and de novo chronic kidney disease following reversible acute kidney injury. Kidney Int 2012; 81: 477–485 [DOI] [PubMed] [Google Scholar]
- 6. Xu X, Nie S, Liu Z et al. Epidemiology and clinical correlates of AKI in Chinese hospitalized adults. Clin J Am Soc Nephrol 2015; 10: 1510–1518 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Wang HE, Muntner P, Chertow GM et al. Acute kidney injury and mortality in hospitalized patients. Am J Nephrol 2012; 35: 349–355 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Nie S, Feng Z, Tang L et al. Risk factor analysis for AKI including laboratory indicators: a nationwide multicenter study of hospitalized patients. Kidney Blood Press Res 2017; 42: 761–773 [DOI] [PubMed] [Google Scholar]
- 9. Kashani K, Macedo E, Burdmann EA et al. Acute kidney injury risk assessment: differences and similarities between resource-limited and resource-rich countries. Kidney Int Rep 2017; 2: 519–529 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Harty J. Prevention and management of acute kidney injury. Ulster Med J 2014; 83: 149–157 [PMC free article] [PubMed] [Google Scholar]
- 11. Kidney Disease: Improving Global Outcomes Acute Kidney Injury Work Group. KDIGO clinical practice guideline for acute kidney injury. Kidney Int Suppl 2012; 2: 1–138 [Google Scholar]
- 12. Mas-Font S, Ros-Martinez J, Pérez-Calvo C et al. Prevention of acute kidney injury in intensive care units. Med Intensiva 2017; 41: 116–126 [DOI] [PubMed] [Google Scholar]
- 13. Seller-Pérez G, Más-Font S, Pérez-Calvo C et al. Acute kidney injury: renal disease in the ICU. Med Intensiva 2016; 40: 374–382 [DOI] [PubMed] [Google Scholar]
- 14. Hoste EA, Bagshaw SM, Bellomo R et al. Epidemiology of acute kidney injury in critically ill patients: the multinational AKI-EPI study. Intensive Care Med 2015; 41: 1411–1423 [DOI] [PubMed] [Google Scholar]
- 15. Barrantes F, Feng Y, Ivanov O et al. Acute kidney injury predicts outcomes of non-critically iII patients. Mayo Clin Proc 2009; 84: 401–416 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Bedford M, Stevens P, Coulton S et al. Development of risk models for the prediction of new or worsening acute kidney injury on or during hospital admission: a cohort and nested study. Health Services and Delivery Research No. 4.6. Southampton, UK: NIHR Journals Library, 2016 [PubMed] [Google Scholar]
- 17. James M, Hobson CH, Darmon M et al. Applications for detection of acute kidney injury using electronic medical records and clinical information systems: workgroup statements from the 15th ADQI Consensus Conference. Can J Kidney Health Dis 2016; 3: 9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Hodgson L, Dimitrov B, Roderick P et al. Predicting AKI in emergency admissions: an external validation study of the acute kidney injury prediction score (APS). BMJ Open 2017; 7: e013511. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Koyner J, Adhikari R, Edelson D et al. Development of multicenter ward-based AKI prediction model. Clin J Am Soc Nephrol 2016; 11: 1935–1946 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Malley K, Cook K, Price M et al. Measuring diagnoses: ICD code accuracy. Health Serv Res 2005; 40: 1620–1639 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Hodgson LE, Sarnowski A, Roderick PJ et al. Systematic review of prognostic prediction models for acute kidney injury (AKI) in general hospital populations. BMJ Open 2017; 7: e016591. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Kondrup J, Rasmussen HH, Hamberg O et al. Nutritional risk screening (NRS 2002): a new method based on an analysis of controlled clinical trials. Clin Nutr 2003; 22: 321–336 [DOI] [PubMed] [Google Scholar]
- 23. Akaike H. Likelihood of a model and information criteria. J Econometr 1981; 16: 3–14 [Google Scholar]
- 24. Cavanaugh JE. Unifying the derivations of the Akaike and corrected Akaike information criteria. Stat Probabil Lett 1997; 33: 201–208 [Google Scholar]
- 25. Hosmer DW, Lemeshow S. Confidence interval estimates of an index of quality performance based on logistic regression models. Stat Med 1995; 106: 565–570 [DOI] [PubMed] [Google Scholar]
- 26. Moons KG, Altman DG, Reitsma JB et al. Transparent reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med 2015; 162: W1–W73 [DOI] [PubMed] [Google Scholar]
- 27. Pencina MJ, D’Agostino RB, D’Agostino RB et al. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 2008; 27: 157–172 [DOI] [PubMed] [Google Scholar]
- 28. Martin C, Molinero LM, Ortiz A et al. Development and internal validation of a prediction model for hospital-acquired acute kidney injury. Clin Kidney J 2019; 14: 1–8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Tomašev N, Glorot X, Rae J et al. A clinically applicable approach to continuous prediction of future acute kidney injury. Nature 2019; 572: 116–119 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Fiaccadori E, Lombardi M, Leonardi S et al. Prevalencce and clinical outcome associated with preexisting malnutrition in acute renal failure: a prospective cohort study. J Am Soc Nephrol 1999; 10: 581–593 [DOI] [PubMed] [Google Scholar]
- 31. Ohnuma T, Uchino S. Prediction models and their external validation studies for mortality of patients with acute kidney injury: a systematic review. PLoS One 2017; 12: e0169341. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Martin-Cleary C, Molinero-Casares LM, Ortiz A et al. Development and internal validation of a prediction model for hospital-acquired acute kidney injury. Clin Kidney J 2021; 14: 309–316 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.