Validation of a Model To Predict the Risk of Nephrotoxicity in Patients Receiving Colistin

Abstract

Despite concerns about its nephrotoxicity, colistin often remains the only effective agent for treating multidrug-resistant Gram-negative infections. Published studies have reported a wide range of nephrotoxicity risk factors. To assess the clinical utility of various models, we compared their performances for predicting the risk of nephrotoxicity. We identified a model demonstrating reasonable overall risk assessment, with an observed/expected ratio of 1.29 (95% confidence interval [CI], 0.68 to 1.90) and a positive predictive value of 87.5% for identifying patients at high risk of developing nephrotoxicity.

TEXT

Multidrug-resistant (MDR) infections caused by Gram-negative bacteria are increasing worldwide and are associated with significant morbidity and mortality (1, 2). With a lack of new antibiotics in development, there has been a resurgence in the use of colistin, as it still demonstrates activity against many of these MDR organisms. However, colistin is associated with significant nephrotoxicity, which limits its widespread use.

Numerous studies have evaluated the prevalence of colistin-associated nephrotoxicity, but the rates vary widely in the literature, ranging anywhere from 10% to 48% (3–6). The reasons for such wide variability are multifactorial. One possible explanation is the different definitions used for nephrotoxicity. Earlier studies did not utilize a standardized definition and generally reported lower rates of nephrotoxicity (4, 7). In more recent years, studies have used the risk, injury, failure, loss, and end-stage renal disease (RIFLE) criteria as a standardized definition for acute kidney injury (8). The rates of colistin-associated nephrotoxicity in these studies are more consistent, with a range between 31% and 48% (5, 6, 9–12). However, the reported risk factors associated with nephrotoxicity vary considerably in these studies. Thus, it is unclear which risk factors are most predictive of colistin-associated nephrotoxicity.

With the high rate of nephrotoxicity associated with colistin use, identifying patients at high risk for developing nephrotoxicity is important for optimizing colistin therapy (i.e., weighing the risk versus benefits of initiating or continuing therapy). A direct comparison of the different mathematical models used to identify patients at high risk for colistin-associated nephrotoxicity has not been undertaken. To assess the clinical utility of various mathematical prediction models, the objectives of this study were to compare their performances in (i) predicting the overall risk of nephrotoxicity in a population of patients receiving colistin and (ii) identifying patients at high risk for developing colistin-associated nephrotoxicity.

A literature search was conducted using PubMed to identify prediction models from published studies that evaluated independent risk factors for colistin-associated nephrotoxicity. Studies were included if they were published in English in the last 10 years and they defined nephrotoxicity according to the RIFLE criteria. The full logistic equation for each prediction model was obtained by contacting the respective corresponding author.

Patients admitted from January 2012 through January 2014 to Baylor St. Luke's Medical Center, an 850-bed tertiary care center in Houston, TX, were retrospectively screened as a validation cohort. Patients aged ≥18 years who received ≥72 h of intravenous colistin for suspected or documented infections were included. Patients were excluded if they had severe renal insufficiency (on any form of renal replacement therapy or baseline serum creatinine of >1.5 mg/dl) or fluctuating renal function (increase or decrease in serum creatinine of >50% in the 72 h immediately prior to the initiation of colistin). The patients were monitored for up to 30 days (or until hospital discharge, whichever occurred first), and nephrotoxicity was defined according to the RIFLE criteria. Institutional review board approval was obtained prior to the initiation of this study, and the need for informed consent was waived due to the retrospective nature of this study.

To predict the risk of nephrotoxicity for a patient, various prediction models were conditioned using specific risk factors from individual patients in the validation cohort (see Appendix). The performances of the models for predicting the overall risk of nephrotoxicity were assessed by comparing the percentage of actual observed nephrotoxicity to the average predicted rate of nephrotoxicity in the patient population, using the observed-to-expected (O/E) ratio. Classification and regression tree (CART) analysis was used to identify the most significant breakpoint for stratifying patients into high- and low-risk categories. The positive predictive value (PPV), negative predictive value (NPV), sensitivity, and specificity were calculated for each prediction model. The patients with a calculated risk of nephrotoxicity above or below the breakpoint were considered high and low risk, respectively. High-risk patients who developed nephrotoxicity and did not develop nephrotoxicity were considered true positives and false positives, respectively. In contrast, low-risk patients who developed nephrotoxicity and did not develop nephrotoxicity were considered false negatives and true negatives, respectively. Positive predictive value was calculated by comparing the number of true positives to the total number of true and false positives. Negative predictive value was calculated by comparing the number of true negatives to the total number of true and false negatives.

A total of 32 patients were included in the validation cohort, and their characteristics are summarized in Table 1. The overall observed rate of nephrotoxicity was 53.1% (risk, 21.9%; injury, 15.6%; failure, 15.6%). The prediction models from 3 published studies were evaluated (9, 10, 12); the reasons for excluding models are shown in Table A1. The corresponding O/E ratios for nephrotoxicity were 1.29 (95% confidence interval [CI], 0.68 to 1.90), 6.11 (95% CI, 3.20 to 9.01), and 1.10 (95% CI, 0.58 to 1.63) for the three studies (Table 2). Because the Phe and Collins models performed reasonably well in predicting the overall risk of nephrotoxicity, they were selected for further evaluation. Using CART analysis, the most significant breakpoints for stratifying patients into high- and low-risk groups were 55.7% in the Collins model and 58.3% in the Phe model. After stratification, the PPV and NPV in the Phe model were found to be 87.5% and 58.3%, respectively. In contrast, the Collins model demonstrated a PPV and NPV of 40.0% and 35.3%, respectively. The sensitivity and specificity in the Phe model were 41.2% and 93.3%, respectively. The Collins model demonstrated a sensitivity and specificity of 35.3% and 40.0%, respectively. Apparently, the Pogue model did not perform as well; a possible explanation is that concomitant rifampin was identified as an independent risk factor for nephrotoxicity. The basis for this risk factor is unclear, and no patients in the validation cohort received concurrent rifampin therapy.

TABLE 1.

Pertinent patient characteristics^a

Characteristic	Data
No. (%) of males	17 (53.1)
No. (%) Caucasians	16 (50.0)
Age (mean ± SD) (yr)	54.7 ± 17.6
Daily dose (mean ± SD) (mg/kg of IBW/day)	4.2 ± 1.3
Duration of therapy (mean ± SD) (days)	9.2 ± 6.2
No. of concurrent nephrotoxins (mean ± SD)	1.8 ± 1.4
No. (%) with concurrent rifampin	0 (0.0)
No. (%) with cystic fibrosis	2 (6.3)
No. (%) with observed nephrotoxicity	17 (53.1)

^an = 32.

TABLE A1.

Reasons for exclusion of study

Reference	Reason(s) for exclusion
5	No response from corresponding author
6	Evaluated overweight and obese patients only
11	No reported odds ratio
	Risk factors not adjusted for confounders in multivariate analysis
13	Risk factors evaluated based on a time-to-event analysis
14	Plasma colistin concn monitoring not widely available
15	Risk factors evaluated based on a time-to-event analysis
16	Did not report risk factors

TABLE 2.

Reported risk factors for colistin-associated nephrotoxicity and overall risk assessment

Reference	No. of subjects examined	No. (%) with nephrotoxicity	Risk factors identified	O/E ratio	95% CI
9	121	41 (34)	Age, duration of therapy, daily dose by IBW, cystic fibrosis (protective)	1.29	0.68 to 1.90
10	126	54 (43)	Colistin daily dose ≥5.0 mg/kg of IBW, receipt of concomitant rifampin, receipt of ≥3 concomitant nephrotoxins	6.11	3.20 to 9.01
12	174	84 (48)	Age, receipt of concomitant nephrotoxins	1.10	0.58 to 1.63

With the increasing rate of infections caused by MDR Gram-negative organisms, colistin is often used as the drug of last resort. However, nephrotoxicity associated with colistin remains a major concern. Thus, risk stratification strategies to identify those at the highest risk are of high clinical relevance. A wide range of risk factors for nephrotoxicity has been reported. To our knowledge, this is the first study to compare and validate the performances of various models for predicting the risk of colistin-associated nephrotoxicity. In our study, the independent risk factors for nephrotoxicity included age, duration of therapy, and daily dose by ideal body weight (IBW), which are consistent with other models (6, 9–12). Interestingly, we also found cystic fibrosis to be protective against the development of nephrotoxicity. Concomitant nephrotoxins were not found to be a risk factor in our model.

The Phe model performed reasonably well in predicting the overall risk of nephrotoxicity in a population of patients. After stratifying patients into high- and low-risk categories based on the most significant breakpoint for nephrotoxicity, the model had a high PPV and specificity for identifying patients at a high risk for developing nephrotoxicity. However, there were some limitations to our study. First, we could only evaluate models for which we had access to the full logistic equations. Second, our validation cohort was limited to a small sample size of patients with normal baseline renal function in a single center; therefore, the generalizability to other institutions with different patient populations and prescribing patterns is unknown. Third, it was difficult to quantitatively assess the influence of concomitant nephrotoxins in view of the method of administration (e.g., intravenous [i.v.] contrast given as a one-time dose, nonsteroidal anti-inflammatory drugs [NSAIDs] administered on an as-needed basis, and vasopressin titrated to clinical response). This makes it difficult to accurately assess how much the patient actually received. Future directions include validating the model in a larger cohort at multiple centers.

In conclusion, we identified a model demonstrating a reasonable overall risk assessment and high PPV for identifying patients at high risk of developing colistin-associated nephrotoxicity. Mathematical prediction models could be used to identify patients at a high risk for nephrotoxicity. Strategies to minimize nephrotoxicity may be selectively implemented in these patients (e.g., closer monitoring of renal function using more sensitive biomarkers of renal injury, limiting the duration of therapy when possible, and ensuring the colistin dose is appropriate).

APPENDIX

Computation of the predicted risk of nephrotoxicity for an individual patient. The reasons for excluding models are shown in Table A1.

A 46-year-old (“46” in the Phe and Collins models below) patient without cystic fibrosis (“0” in the Phe model) was given colistin 4.1 mg/kg of IBW/day (“4.1” in the Phe model and the second “0” in the Pogue model) for 14 days (“14” in the Phe model). The patient was also on 2 concomitant nephrotoxins (first “0” in the Pogue model and “1” in the Collins model) but received no rifampin (last “0” in the Pogue model).

Phe model:

$$\begin{array}{l}\text{logit}=-4.419+0.036(46)+0.337(4.1)+0.076(14)-3.554(0)\hfill \\ e^{\text{logit}}=0.73\hfill \\ P_{\text{nephrotoxicity}}\hspace{0.17em}=42.2\%\hfill \end{array}$$

Pogue model:

$$\begin{array}{l}\text{logit}=-4.2354+1.9164(0)+3.1532(0)+1.3362(0)\hfill \\ e^{\text{logit}}=0.01\hfill \\ P_{\text{nephrotoxicity}}=1.4\%\hfill \end{array}$$

Collins model:

$$\begin{array}{l}\text{logit}=-2.9016+0.03053(46)+1.38827(1)\hfill \\ e^{\text{logit}}=0.9\hfill \\ P_{\text{nephrotoxicity}}\hspace{0.17em}=47.3\%\hfill \end{array}$$

Note:

$$\begin{array}{l}{P}_{\text{nephrotoxicity}}\hspace{0.17em}=e^{\text{logit}}/\left(e^{\text{logit}}+1\right)\hfill \\ P_{\text{nephrotoxicity}}:\text{predicted risk of nephrotoxicity}\hfill \end{array}$$