Risk prediction models for contrast induced nephropathy: systematic review

Abstract

Objectives To look at the available literature on validated prediction models for contrast induced nephropathy and describe their characteristics.

Design Systematic review.

Data sources Medline, Embase, and CINAHL (cumulative index to nursing and allied health literature) databases.

Review methods Databases searched from inception to 2015, and the retrieved reference lists hand searched. Dual reviews were conducted to identify studies published in the English language of prediction models tested with patients that included derivation and validation cohorts. Data were extracted on baseline patient characteristics, procedural characteristics, modelling methods, metrics of model performance, risk of bias, and clinical usefulness. Eligible studies evaluated characteristics of predictive models that identified patients at risk of contrast induced nephropathy among adults undergoing a diagnostic or interventional procedure using conventional radiocontrast media (media used for computed tomography or angiography, and not gadolinium based contrast).

Results 16 studies were identified, describing 12 prediction models. Substantial interstudy heterogeneity was identified, as a result of different clinical settings, cointerventions, and the timing of creatinine measurement to define contrast induced nephropathy. Ten models were validated internally and six were validated externally. Discrimination varied in studies that were validated internally (C statistic 0.61-0.95) and externally (0.57-0.86). Only one study presented reclassification indices. The majority of higher performing models included measures of pre-existing chronic kidney disease, age, diabetes, heart failure or impaired ejection fraction, and hypotension or shock. No prediction model evaluated its effect on clinical decision making or patient outcomes.

Conclusions Most predictive models for contrast induced nephropathy in clinical use have modest ability, and are only relevant to patients receiving contrast for coronary angiography. Further research is needed to develop models that can better inform patient centred decision making, as well as improve the use of prevention strategies for contrast induced nephropathy.

Introduction

Every year, over 80 million iodinated contrast studies are performed worldwide, ordered by a wide range of medical specialties.¹ With the increasing trend towards minimally invasive diagnostic and interventional procedures that often need intravenous or intra-arterial contrast, there has been a concomitant rise in the incidence of acute kidney injury after exposure to radiocontrast, often termed contrast induced nephropathy.² In fact, contrast induced nephropathy could be as high as the third most common cause of acute kidney injury in patients admitted to hospital, after ischaemic and drug induced injury.³

Studies have shown a strong association between contrast induced nephropathy and adverse clinical outcomes, including cardiovascular complications, provision of dialysis, and death.^{4 5 6} Therefore, the use of prediction models for contrast induced nephropathy could have several benefits. Firstly, they may help identify patients at high risk for the disorder, who might benefit from peri-procedural strategies that protect the kidney.^{7 8} Secondly, patients identified as high risk would also be an ideal population to study novel therapies for the prevention and treatment of the disorder. Finally, prediction models for contrast induced nephropathy could improve preintervention counselling to facilitate informed patient centred decision making.

Despite these potential benefits, clinical prediction rules have several weaknesses that limit their application in daily practice. These include differences in derivation, inconsistent external validation, and complexity.⁹ Most importantly, the downstream effects of adopting clinical prediction rules to guide decision making and improve patient outcomes are often not evaluated.¹⁰ These factors make it challenging for a clinician to select the ideal model to use in practice. To address this knowledge gap, we did a systematic review to look at the available literature on validated prediction models for contrast induced nephropathy, and to describe their performance and clinical usefulness.

Methods

We performed this systematic review in accordance with the preferred reporting items for systematic reviews and meta-analyses guidelines.¹¹ Our objective was to systematically review prediction models for contrast induced nephropathy.

Data sources and searches

We used a strategy developed with a health informatics specialist to search Ovid Medline (1946 to 9 March 2015), Embase (1947 to week 10 in 2015), and CINAHL (cumulative index to nursing and allied health literature; 1993 to March 2015). We reviewed the bibliographies of identified articles to locate further eligible studies. The web appendix shows the search strategies performed.

Study selection

Studies published in the English language were eligible for inclusion if they evaluated the characteristics of a predictive model for identifying patients at risk of contrast induced nephropathy among adults undergoing a diagnostic or interventional procedure that used conventional, iodinated radiocontrast (media used for computed tomography (CT) or angiography, and not gadolinium based contrast). Because a set of predictive factors derived in only one population could lack validity and applicability, we only included studies in which both development and validation of the prediction model was conducted. We did not prespecify the method of validation, nor did we exclude studies where the derivation and validation cohorts were drawn from the same population. We excluded unpublished conference abstracts.

Owing to the anticipated heterogeneity in the criteria for contrast induced nephropathy between studies and the well described association between even mild elevations of serum creatinine levels and adverse outcomes, we accepted each study’s definition of the disorder.¹² These included relative or absolute increases in serum creatinine after contrast exposure.

Two authors (ZH and SAS) scanned titles and abstracts for initial selection. Selected articles were reviewed in full and independently assessed for eligibility by the same two reviewers. Discrepancies were resolved by consensus.

Data extraction

From each study, we abstracted data on baseline patient characteristics, procedural characteristics, criteria to define contrast induced nephropathy, the number of events, predictor variables included in the risk model, internal and external validation, measures of discrimination, measures of calibration, and methodological features indicative of study quality. To facilitate a comparison of predictor variables, we grouped final model variables into six categories: demographic data, anthropometric data, medical history, physical examination and clinical presentation, procedural characteristics, and laboratory values.

Model performance

We evaluated the internal validity of each model by examining model discrimination, calibration, and reclassification. The concordance (the C statistic) of the prediction tool was used as a measure of discrimination; however, other performance statistics such as sensitivity and specificity were included if the C statistic was not reported.¹⁰ The C statistic is equivalent to the area under the curve, and represents the model’s ability to distinguish patients who will develop contrast induced nephropathy from those who will not. C statistic values range from 0.5 (no discrimination, no better than chance) to 1.0 (perfect discrimination). A C statistic of 0.7-0.8 indicates modest discriminative ability, while a C statistic greater than 0.8 indicates good discriminative ability.¹³

Model calibration was measured by the Hosmer-Lemeshow statistic, which refers to the concordance between observed and predicted risks. A Hosmer-Lemeshow statistic with a small P value indicates poor calibration.¹³ If the Hosmer-Lemeshow statistic was not reported, we reported the range of observed rates of contrast induced nephropathy from the lowest to highest predicted risk groupings. Reclassification was evaluated by net reclassification improvement.¹⁴ Net reclassification improvement refers to the proportion of individuals who, after incorporating the prediction tool, are reclassified to a risk stratum that is a better reflection of their actual outcome. The net reclassification improvement indicates the frequency with which appropriate reclassification occurs compared to inappropriate reclassification with use of the new model. For this test, a value of P<0.05 suggests that a significantly greater number of patients are being reclassified appropriately than are being reclassified inappropriately.¹⁴

Quality assessment, clinical usefulness, and external validation

We assessed study quality using a modification of the criteria recommended by Hayden and colleagues.¹⁵ The criteria involve assessment of seven categories related to study participation (sampling bias), study attrition (attrition bias), prognostic factor selection, prognostic factor measurement, outcome measurement (ascertainment bias), statistical analysis, and model performance (discrimination, calibration). These criteria are explained further in the web appendix.

Similar to a previous systematic review, we also assessed the clinical usefulness of each study, which was defined as the combination of clinical utility and usability.¹⁶ For clinical utility (the effect on a clinical decision linked to a risk category or threshold), we assessed whether authors linked their models to specific risk categories and discussed how the risk categories would aid diagnostic evaluations. For usability (the availability of a clinical decision aid), we noted whether authors included a calculator or risk score that would facilitate knowledge translation and use at the bedside. These criteria are explained further in the web appendix. We also evaluated the generalisability of each prediction model by determining whether it had been externally validated in an independent patient population, either in the original or a subsequent publication.

Data synthesis

We qualitatively synthesised results focusing on the populations in which the risk score had been tested, the types of variables contained within the prediction models, model discrimination, external validation and practical aspects of model implementation. We did not perform meta-analyses because the included studies were too heterogeneous.

Patient involvement

No patients were involved in setting the research question or the outcome measures, nor were they involved in the design and implementation of the study. There are no plans to involve patients in dissemination.

Results

Our search strategy yielded 3567 unique citations (fig). We excluded 3509 citations based on screening of title and abstract mainly due to non-relevant outcomes, animal studies and review articles, leaving 58 articles for full text review. We subsequently excluded 42 studies that did not fulfil our inclusion criteria because they consisted of unpublished abstracts (n=17), models that did not have an associated risk score for contrast induced nephropathy (n=13), letters to the editor and narrative reviews (n=6), duplicate publications (n=3), studies with outcomes not related to contrast induced nephropathy (n=2), and non-validated risk scores (n=1). This yielded 16 studies comprising 12 unique risk prediction models.^{6 17 18 19 20 21 22 23 24 25 26 27}

Flow diagram of included studies. CIN=contrast induced nephropathy

Risk prediction scores and variables

The studies included a total of 72 214 patients (range 218-48 001) and 3062 (4.0%) events of contrast induced nephropathy in the respective derivation cohorts (table 1). Only one model was derived in a population encompassing more than one hospital (Gurm and colleagues).²⁰ All the included studies comprised patients undergoing coronary angiography or percutaneous coronary intervention. No studies included patients undergoing intravenous contrast (for example, with computed tomography) or endovascular procedures involving the aorta or peripheral arteries.

Table 1.

 Characteristics of predictive models for contrast induced nephropathy

Study (year), country	Population and setting	Definition of contrast induced nephropathy	Cointervention	Patients (No)		Events (No (%))	Predictors (No)
				Derivation cohort	Validation cohort
Bartholemew et al (2004), USA	Coronary interventional procedures at one hospital	≥1.0 mg/dL increase in serum creatinine from baseline within 48 h of PCI	IV NS (pre)	10 481	9997	DC: 293 (2.8), VC: 120 (1.2)	8
Chen et al (2014), China	PCI at one hospital	>0.5 mg/dL (44.2 µmol/L) or 25% increase in serum creatinine from baseline within 5 days of PCI	IV NS for CKD (pre/post procedure) with NAC	1500	1000	DC: 246 (16.4), VC: 172 (17.2)	9
Fu et al (2012), China	PCI at one hospital	>0.5 mg/dL (44.2 umol/L) or 25% increase in serum creatinine from baseline within 48-72 h of PCI	IV NS (pre/post)	668	277	DC: 105 (15.7), VC: 46 (16.1)	9
Ghani et al (2009), Kuwait	PCI at one hospital	>0.5 mg/dL increase in serum creatinine from baseline within 48 h of PCI	IV NaHCO3 with or without NAC (pre/post procedure)	247	100	DC: 13 (5.52), VC: 5 (5)	5
Gao et al (2014), China	Coronary angiography or PCI at one hospital	>0.5 mg/dL or 25% increase in serum creatinine from baseline within 72 h of PCI	None	2764	1181	DC: 127 (4.6), VC: 50 (4.2)	7
Gurm et al (2013), USA	PCI in multiple non-federal hospitals	>0.5 mg/dL increase in creatinine from baseline within 7 days of PCI	NR	48001	20572	DC: 1243 (2.59), VC: 505 (2.45)	15
Liu et al (2015), China	PCI at one hospital	>0.5 mg/dL (44.2 umol/L) increase in serum creatinine from baseline within 48-72 h of PCI	IV NS (pre/post procedure)	495	233	DC: 15 (3.0), VC: NR	3
Maioli et al (2010), Italy	Coronary angiography or PCI at one hospital	>0.5mg/dL increase in creatinine from baseline within 5 days of PCI	IV NS hydration with NAC	1281	502	DC: 114 (9.4), VC: 54 (10.8)	7
Marenzi et al (2004), Italy	PCI for STEMI in one hospital	>0.5 mg/dL increase in creatinine from baseline within 48 h of PCI	IV NS (post contrast)	218	891*	DC: 40 (19), VC: 126 (14.4)	5
Mehran et al (2004), USA	PCI at one hospital	>0.5 mg/dL or 25% increase in serum creatinine from baseline within 48 h of PCI	IV hNS (pre/post procedure)	5571	2786	DC: 729 (13.1), VC: 386 (13.9)	8
Tziakas et al (2013), Greece	Elective or emergency PCI at one hospital	>0.5 mg/dL or 25% increase in serum creatinine from baseline within 48 h of PCI	IV NS hydration (pre/post procedure) with or without NAC	488	200	DC: 50 (10.2), VC: 25 (14)	5
Victor et al (2014), India	PCI at one hospital	>0.5 mg/dL or 25% increase in serum creatinine from baseline within 48 h of PCI	IV hNS with NAC	900	300	DC: 87 (9.7), VC: 26 (8.7)	7

1 mg/dL=88.4 µmol/L. PCI=percutaneous coronary intervention; CIN=contrast induced nephropathy; STEMI=ST elevated myocardial infarction; DC= derivation cohort; VC=validation cohort; NS=normal saline; hNS=half normal strength saline; CKD=chronic kidney disease; NAC=N-acetyl cysteine; NR=not recorded; IV=intravenous.

*External validation in paper by Sgura and colleagues (cohort of patients with STEMI).²⁸

The outcome of interest, contrast induced nephropathy, was defined in 11 studies as an increase of at least 0.5 mg/dL (1 mg/dL=88.4 μmol/L) in serum creatinine from baseline; one study defined the increase as at least 1.0 mg/dL in serum creatinine from baseline. However, the required time frame in which the increase in creatinine was measured after the administration of contrast varied among the studies from two to seven days.

Ten studies reported on interventions administered to mitigate the risk of contrast induced nephropathy, both before and after contrast exposure. In all 10 studies, intravenous fluid was administered (0.9% normal saline in seven, 0.45% half normal strength saline in two, and isotonic sodium bicarbonate in one), and in five studies, N-acetylcysteine was given. One study did not administer any form of prophylaxis (Gao and colleagues),²⁷ and another study did not report whether any prophylaxis was administered (Gurm and colleagues).²⁰

The number of predictors in each risk model varied from three to 15 (table 2). Of 12 models, 11 included measures of pre-existing chronic kidney disease via estimated glomerular filtration rate, creatinine clearance, or serum creatinine cut-off levels. One study did not include any measure of pre-existing chronic kidney disease (Marenzi and colleagues).²³ Other common measures included in the majority of risk models were age (eight models), diabetes (eight models), heart failure or impaired ejection fraction (eight models), and contrast volume (seven models). Very few models included patient sex, drug treatments, or laboratory values other than serum creatinine. Only one model included proteinuria as a risk factor (Victor and colleagues).²⁶ The web appendix shows the complete list of variables contained within each model.

Table 2.

 Variables included in predictive models for contrast induced nephropathy

Variable	Study (sample size)
	Bartholemew (n=10 481)	Chen (n=1500)	Fu (n=668)	Ghani (n=247)	Gao (n=2764)	Gurm (n=48 001)	Liu (n=495)	Maioli (n=1281)	Marenzi (n=218)	Mehran (n=5571)	Tziakas (n=488)	Victor (n=900)
Demographic
Age		X	X		X	X	X	X	X	X
Female sex				X
Anthropometric data
Height						X
Weight						X
Medical history
Renal insufficiency*	X	X	X	X	X	X	X	X		X	X	X
Anaemia*		X	X							X
Diabetes mellitus	X	X	X	X		X		X		X		X
Hypertension	X				X
Heart failure	X				X	X				X
Impaired LVEF*		X	X				X	X
Previous MI		X	X		X
Recent cardiac procedure/PCI								X			X
Peripheral vascular disease	X										X	X
Metformin use											X
Physical examination/clinical presentation
Hypotension*		X	X							X		X
Shock*				X		X
CAD presentation						X
Use of IABP	X				X				X	X
Anterior MI									X
Time to reperfusion									X
Procedure
Urgent/emergent	X	X	X			X
PCI indication						X
Contrast volume	X		X		X				X	X	X	X
Multivessel PCI				X
One procedure in past 72 h								X
Laboratory values at presentation
Albuminuria												X
Pre-procedure Cr>baseline Cr								X
HDL<1		X				X
CK-MB						X
Haemoglobin						X						X
Troponin I						X
Troponin T						X

X=variable included in each model; LVEF=left ventricular ejection fraction, MI=myocardial infarction; PCI=percutaneous coronary intervention; CAD=coronary artery disease; IABP=intra-aortic balloon pump; Cr=creatinine; HDL=high density lipoprotein; CK-MB=creatine kinase isoenzyme-MB.

*Definition of variable varied by study (web appendix).

Model performance and validation

Table 3 summarises model performance (discrimination and calibration) and validation. Ten models were validated internally: seven by split samples, one by bootstrapping, one by random forest plots, and one by using the same population. The other two models were validated externally. All models reported the C statistic for the validation cohort, which ranged from 0.61 to 0.95, indicating a degree of discriminative performance that varied from poor to excellent. Six models were validated externally; some of which had more than one external validation study. These six models demonstrated a wide range of discriminative ability (C statistic 0.57-0.86). Calibration was reported in five studies for the derivation cohort using the Hosmer-Lemeshow test (table 3). Net reclassification index was reported in one study (Gurm and colleagues).²⁰

Table 3.

 Metrics of model performances to predict risk of contrast induced nephropathy

Derivation cohort			Validation cohort (internal)			Validation cohort (external)
Study	Population	Sample size, discrimination, calibration	Study	Type of internal validation	Sample size, discrimination, calibration	Study	Population	Sample size, discrimination, calibration
Bartholemew et al (2004)	Coronary interventional procedures at one hospital	N=10 481; 0.89; P=0.10	Same	Split sample	N=9998; NR; risk score range associated with CIN risk (%): 0-4 (0.2), 5-6 (2.8), 7-8 (10), 9-11 (28)	—	—	—
			—	—	—	Tziakas et al (2014)	Patients admitted for elective/urgent PCI in six tertiary care centres across four countries	N=2689; 0.59; NR
			—	—	—	Tziakas et al (2013)	Patients undergoing elective or emergency PCI at one hospital	N=488; 0.58; NR
Chen et al (2014)	PCI at one hospital	N=1500; 0.82; P=0.89	Same	Split sample	N=1000; 0.82; risk score range associated with CIN risk (%): low (5.3), moderate (19.9), high (32.5), very high (59.5)	—	—	—
Fu et al (2012)	PCI at one hospital	N=668; NR; NR	Same	None	—	Same	Cohort of elderly patients at same institution	N=277; 0.79; P>0.05
Ghani et al (2009)	PCI at one hospital	N=247; NR; NR	Same	Split sample	N=100; 0.61; risk score range associated with CIN risk (%): <4 (9.2), 5-8 (32.1), 9-12 (54.2), >12 (84)	—	—	—
Gao et al (2014)	Coronary angiography or PCI at one hospital	N=2764; 0.76; P>0.05	Same	Split sample	N=1181; 0.71; P=0.54	—	—	—
Gurm et al (2013)	PCI in multiple hospitals	N=48 001; NR; NR	Same	Random forest	N=20 572; 0.84; risk score range associated with CIN rate (%): low (0.51), medium (2.8), high (12.99)	—	—	—
Liu et al (2015)	PCI at one hospital	N=495; 0.79; NR	Same	Split sample	N=233; 0.86; risk score range associated with CIN rate (%): low (0), medium (5.1), high (19.44)	—	—	—
Maioli et al (2010)	Coronary angiography or PCI at one hospital	N=1281; NR; NR	Same	None	—	Same	Independent cohort of patients with CrC <60	N=502; 0.82; risk score range associated with CIN risk (%): 0-1 (0), 2-3 (1), 4 (2), 5 (6), 6 (12), 7 (19), 8 (24), 9 (36), 10 (50), >11 (57)
Marenzi et al (2004)	PCI for STEMI in one hospital	N=218; NR; NR	Same	Same population	NR; NR; risk score range associated with CIN rate (%): 0 (4), 1 (8), 2 (24), 3 (39), 4-5 (100)	Sgura et al (2010)	Patients admitted to CCU for STEMI treated with primary PCI	N=891; 0.57; NR
Mehran et al (2004)	PCI at one hospital	N=5571; 0.69 (Cr), 0.70 (eGFR); P=0.43 (Cr), P=0.42 (eGFR)	Same	Split sample	N=2786 (model combines Cr and eGFR); 0.67; risk score range associated with CIN rate (%): low (8.4), medium (12.8), high (29.9), very high (55.9)	Gao et al (2014)	Adult patients undergoing cardiac catheterisation or PCI at one institution	N=3945; 0.57; NR
			—	—	—	Tziakas et al (2013)	Patients undergoing elective or emergency PCI at one hospital	N=5571; 0.59; NR
			—	—	—	Tziakas et al (2014)	Patients admitted for elective/urgent PCI in six tertiary care centres across four countries	N=2689; 0.565; NR
			—	—	—	Liu et al (2014)	Patients admitted for PCI at one hospital	N=728; 0.84; NR
			—	—	—	Sgura et al (2010)	Patients admitted to CCU for STEMI treated with primary PCI	N=891; 0.57; risk score range associated with CIN risk (%): low (12.10), medium (14.75), high (19.28), very high (34.48)
Tziakas et al (2013)	Elective or emergency PCI at one hospital	N=488; 0.76; P>0.05	Same	Bootstrap	N=1000; 0.75; NR	Same	PCI patients at same institution	N=200; 0.86; NR
			—	—	—	Tziakas et al (2014)	Patients admitted for elective/urgent PCI in six tertiary care centres across four countries	N=2689; 0.7; P=0.184
Victor et al (2014)	PCI at one hospital	N=900; 0.93 (sensitivity 93.9%, specificity 89.5%); NR	Same	Split sample	N=300; 0.95 (sensitivity 92.3%, specificity 82.1%); NR	—	—	—

NR=not reported; CIN=contrast induced nephropathy; CCU=coronary care unit; STEMI=ST elevated myocardial infarction; PCI=percutaneous coronary intervention; Cr=creatinine; eGFR=estimated glomerular filtration rate; CrC=creatinine clearance.

Of 12 models, five had good discriminative ability on validation (C statistic >0.80), with the number of predictors in each risk model ranging from three to 15. These higher performing models included the following risk factors: pre-existing kidney disease (all models), age (four models), diabetes (four models), heart failure or impaired ejection fraction (four models), and hypotension or shock (three models). Only one of these models included contrast volume (Victor and colleagues).²⁶ Of these five models with good discriminative ability, only the study by Maioli and colleagues²² was validated externally (C statistic 0.82), although the external cohort was from the same hospital as the derivation cohort.

Three models had moderate discriminative ability (C statistic 0.70-0.80). These models included the following risk factors: pre-existing kidney disease (all models), age (two models), diabetes (one model), heart failure or impaired ejection fraction (two models), and hypotension or shock (one model). All these models included contrast volume as a risk factor.

Quality assessment and clinical usefulness

There were important differences in the risk of bias among the studies, with no single study satisfying all seven variables (table 4). Although all studies adequately described the selection of the study sample and the ascertainment of the study outcome, study attrition was described in only one study. In addition, of the 12 models, most studies provided little information on prognostic factor selection (nine models) and measurement (seven models). As described above, model calibration and reclassification was absent from most studies. Four studies excluded patients with missing data, whereas the remaining studies did not report the degree of missing data. No study reported the use of imputation techniques.

Table 4.

 Risk of bias and clinical usefulness*

Study	Bias†							Usefulness†
	Study participation	Study attrition	Prognostic factor selection	Prognostic factor measurement	Outcome measurement	Analysis	Reporting of model performance	Clinical utility	Clinical usability
Bartholemew et al (2004)	Low	?	?	?	Low	Low	High	Yes	No
Chen et al (2014)	Low	?	?	?	Low	Low	High	Yes	No
Fu et al (2012)	Low	?	High	Low	Low	Low	High	No	Yes
Ghani et al (2009)	Low	?	?	?	Low	Low	High	No	No
Gao et al (2014)	Low	?	?	?	Low	Low	Low	Yes	No
Gurm et al (2013)	Low	?	Low	Low	Low	Low	High	Yes	Yes
Liu et al (2015)	Low	?	High	High	Low	Low	High	Yes	No
Maioli et al (2010)	Low	?	?	?	Low	Low	High	Yes	No
Marenzi et al (2004)	Low	Low	High	High	Low	Low	High	Yes	No
Mehran et al (2004)	Low	?	Low	Low	Low	Low	Low	Yes	Yes
Tziakas et al (2013)	Low	?	High	Low	Low	Low	Low	Yes	No
Victor et al (2014)	Low	?	Low	Low	Low	Low	High	No	Yes

*Adapted from references 15 and 16 (web appendix provides more details).

†Bias evaluated as low risk, high risk, or unknown (?) risk; clinical usefulness evaluated as yes or no. Two authors (ZH and SAS) evaluated the studies on these criteria independently. Discrepancies were resolved by consensus.

Most models (nine of 12) stratified their cohorts into risk categories (low, moderate, high risk). However, none of the models explained how assignment to a risk category would affect diagnostic or therapeutic decisions. Simple risk calculators or nomograms were provided by four studies. Three additional studies included electronic calculators to facilitate knowledge translation. These studies were by Gurm and colleagues (online calculator, https://bmc2.org/calculators/cin),²⁰ Mehran and colleagues (smartphone application, www.qxmd.com/calculate-online/nephrology/contrast-nephropathy-post-pci),²¹ and Victor and colleagues (Excel spreadsheet in their manuscript’s web appendix).²⁶ No study evaluated the implications of the prediction model on clinical decision making or patient outcomes.

Discussion

Principal findings

In our systematic review, we found 16 studies that described 12 prediction models for contrast induced nephropathy in patients administered contrast for interventional cardiac procedures. The discriminative ability for the prediction of contrast induced nephropathy in these models varied from poor to excellent. Models with good discriminative ability included measures of chronic kidney disease, age, diabetes mellitus, heart failure or impaired ejection fraction, and hypotension or shock. None of the models had been evaluated in clinical practice. Our study is the first to synthesise the available literature on predictive models of contrast induced nephropathy, and highlights the need for further development and refinement of these models.

Eight of the models had moderate to good discrimination, based on the C statistic. However, this is only the first step in the development of a clinical prediction rule.²⁹ The next steps involve testing the rule in a separate population (external validation) and measuring the effect of its application on clinical outcomes (impact analysis). From these eight models, two were externally validated in the same hospital as the derivation cohort (in the studies by Fu and colleagues¹⁸ and Maioli and colleagues),²² which limits their generalisability. The model in Tziakas and colleagues’ study was externally validated in two different settings: once in the same hospital as the derivation cohort²⁴ and once as part of a multicentre study.³⁰ The C statistic is a function of the sensitivity and specificity of a given risk score, and both sensitivity and specificity are influenced by case mix, disease severity, or risk factors for disease.³¹ Therefore, variations in the C statistic were expected for the same model among different studies because of differences in population characteristics in each of the externally validated studies.

Some model characteristics make their clinical application challenging. Both studies by Fu and colleagues¹⁸ and Tziakas and colleagues²⁴ included contrast volume in their models; however, the volume of contrast needed is usually not known until the procedure has been performed. Since decisions on diagnostic testing and prophylactic therapies are usually made before the procedure, inclusion of contrast volume as a model variable limits model usefulness. This criticism applies to other promising models, such as those in the studies by Gao and colleagues²⁷ and Victor and colleagues.²⁶ Maioli and colleagues did not include contrast volume as a variable in their model, and so this model may be a reasonable starting point for further refinement.²²

Higher performing models usually included measures of pre-existing chronic kidney disease, age, diabetes, heart failure or impaired ejection fraction, and hypotension or shock. These variables should all be considered in future model development. However, prognostic factor selection and measurement was poorly described across most studies.

It is likely that other factors that were not included in the risk models also contribute to the risk of contrast induced nephropathy.^{6 32 33} For example, drug treatments such as inhibitors of the renin-angiotensin aldosterone system and diuretic agents could increase susceptibility to contrast induced nephropathy through alterations in kidney haemodynamics.³⁴ The contribution of these risk factors could be especially important in emergency settings when these treatments are not discontinued before administration of contrast. In addition, the use of prophylactic drugs and intravenous fluid was not included in the risk scores, despite being administered to patients in 10 of 12 studies. This may have led to differences in the rate of contrast induced nephropathy among the studies, and contributed to interstudy heterogeneity. Heterogeneity also precluded the performance of meta-analyses. Contributors to heterogeneity included different clinical settings, cointerventions, the type of contrast administered, the timing of creatinine measurement to ascertain contrast induced nephropathy, and the method used to define baseline creatinine.

Use of prediction models in clinical practice

Predictive models for contrast induced nephropathy have been available for clinical use for almost 10 years. However, uptake by cardiologists and radiologists has been low, judging by their omission from recent clinical practice guidelines^{35 36} and survey studies.^{37 38} Even though prediction models for contrast induced nephropathy perform similarly to other popular prediction models (Framingham risk score,³⁹ QRISK2⁴⁰) on the basis of the C statistic, several important differences have limited their use in clinical practice.

Firstly, these aforementioned models have been externally validated in multicenter studies. Secondly, many predictive models, including the Framingham risk score and QRISK 2, clearly outline how assignment to a risk category affects diagnostic or therapeutic decisions.^{39 41} Finally, electronic risk calculators exist for both the Framingham risk score and QRISK2. None of the predictive models for contrast induced nephropathy satisfies all three of these elements, and only the model in the study by Mehran and colleagues satisfies two of three elements, but with a median C statistic of 0.57 on external validation.²¹

Another reason for the low clinical uptake of predictive models for contrast induced nephropathy is that they have focused exclusively on populations receiving intra-arterial contrast for coronary angiographic procedures. These procedures represent a small proportion of all contrast procedures, with contrast enhanced computer tomography (CT) scans being much more common. Indeed, the risk of contrast induced nephropathy associated with intravenous, contrast enhanced CT procedures is not rare, occurring in 11% of a low risk population.⁴²

In addition, the pathophysiological mechanism of contrast induced nephropathy related to contrast enhanced CT procedures could differ from that associated with coronary angiography procedures.⁴³ For example, in intravenous procedures involving contrast enhanced CT, a large volume of intravenous contrast is often injected within 10 to 20 seconds compared with small intra-arterial injections of contrast occurring over minutes in coronary procedures. As such, predictive models for contrast induced nephropathy derived from patients undergoing coronary angiography might not be generalisable to individuals undergoing intravenous contrast enhanced CT procedures or CT angiography.

Study limitations

Our study had several limitations. Firstly, our included studies were heterogeneous in terms of their populations, use of prophylactic therapies, and definitions of contrast induced nephropathy, which could have led to a differential risk for contrast induced nephropathy. Secondly, our review focused only on studies in which contrast was administered for a coronary procedure; therefore, the risk models reviewed might not be generalisable to other procedures, such as intravenous contrast enhanced CT, CT angiography, and non-coronary angiography. However, no studies have derived a predictive score for contrast induced nephropathy in a population receiving intravenous contrast.

Thirdly, only one of the 12 models in our review included an analysis of reclassification using clinically meaningful risk categories. Because net reclassification is a relatively new construct, this may explain why it was not included during the validation of the models included in our review. Fourthly, the criteria for risk of bias and clinical usefulness were adapted from the existing literature on clinical risk prediction and have not been extensively studied or prospectively validated.^{15 16} However, this exercise is necessary because the inclusion of methodologically weak studies can threaten the internal validity of a systematic review.¹⁵ Finally, no internally or externally validated models were prospectively evaluated in clinical practice to determine their effect on clinical decision making and patient outcomes.

Clinical applications and future directions

Currently, there are no definitively effective strategies for prophylaxis or treatment of contrast induced nephropathy.^{44 45} The provision of intravenous fluids (containing saline or bicarbonate) and N-acetyl cysteine have been extensively studied for prophylaxis; however, no conclusion on efficacy has been reached despite multiple prospective trials and several meta-analyses.⁴⁶ As a result, the Prevention of Serious Adverse Events following Angiography (PRESERVE) trial is underway, with the plan to enrol 8680 patients to compare the effectiveness of isotonic sodium bicarbonate versus isotonic saline and oral N-acetyl cysteine versus oral placebo (NCT01467466; GMC is on the steering committee of the PRESERVE trial).

Another effective prevention strategy for contrast induced nephropathy could be system based quality improvement efforts. Brown and colleagues reduced the rate of contrast induced nephropathy by 20% in consecutive patients with percutaneous coronary intervention at multiple centres through a multifaceted intervention. These improvements included standardised fluid orders, loosening nil per mouth restrictions before a procedure, cessation of nephrotoxic medications, self-expansion of the extracellular fluid volume, mandatory procedure delays to ensure adequate volume status, and team training.⁴⁷

Predictive risk scores for contrast induced nephropathy were not used in either the PRESERVE trial or the quality improvement study, representing a potential missed opportunity. For the PRESERVE trial, a robust risk score might help target ideal candidates for enrolment to enrich event rates of contrast induced nephropathy and ensure that a beneficial effect is not missed. For quality improvement, a risk score would allow the clinical team to concentrate their system changes on the patients at highest risk. This targeting would help protect against improvement fatigue given the multiple quality initiatives that now exist. In fact, patients with an estimated glomerular filtration rate of less than 60 mL/min/1.73 m² seemed to benefit more from the intervention (28% reduction), suggesting that integration of a risk score to target high risk patients could achieve an even greater reduction in the rate of contrast induced nephropathy.⁴⁷

Further research is needed to develop a prediction model for contrast induced nephropathy that is of comparable quality to risk scores in other disease states. Many of the existing models have been adequately derived, but newly derived models should ensure that standardised definitions are used to select and measure prognostic factors in order to reduce heterogeneity and misclassification bias. Subsequent steps would involve external validation in multicentre cohorts and integrating risk assessment with diagnostic or therapeutic decisions. This second step will require experts to suggest clinically relevant risk thresholds for contrast induced nephropathy in the clinical practice guidelines for acute kidney injury, which are currently lacking.⁴⁸ Lastly, there is a need for model derivation that includes patients who undergo intravenous CT procedures, CT angiography, and non-coronary angiography. The risks associated with these procedures are likely to be different from the risk associated with arterial contrast for coronary angiography.

Clinicians should consider using one of the higher performing scores that do not include contrast volume to estimate a patient’s risk of contrast induced nephropathy (as shown in the studies by Maioli and colleagues,²² Chen and colleagues,¹⁷ Gurm and colleagues,²⁰ and Liu and colleagues).²⁵ There are many potential applications of predictive models in general. Firstly, predictive models can inform decision making centred towards patients, whereby those at high risk can choose alternative imaging methods or opt out of further investigation. Secondly, they allow for the selective use of preprocedural manoeuvres that could mitigate the risk of contrast induced nephropathy (for example, holding diuretics to prevent intravascular volume depletion, or fluid hydration^{44 45}). Thirdly, they allow clinical trials and quality improvement interventions to target patients most likely to benefit from these complex efforts.

Conclusion

Our systematic review demonstrates that risk prediction for contrast induced nephropathy is still in its early stages. Although higher performing models usually include pre-existing chronic kidney disease, age, diabetes, heart failure or impaired ejection fraction, and hypotension or shock, most have limited predictive ability when validated externally and are not relevant to individuals receiving intravenous contrast or non-coronary angiography. Given the increasing incidence of contrast induced nephropathy and the many clinical applications of risk prediction, it is necessary to build on current models to develop a clinically useful and generalisable prediction model for the disorder that can improve clinical decision making and patient outcomes.

What is already known about this topic

• Acute kidney injury following exposure to radiocontrast, often referred to as contrast induced nephropathy, has been associated with significant morbidity and mortality

• Several models exist to predict contrast induced nephropathy, but their impact on decision making and patient outcomes is unclear

What this paper adds

• Most predictive models for contrast induced nephropathy have modest ability, and are only relevant to patients receiving contrast for coronary angiography

• Further research should involve the external validation of models, as well as the integration of risk assessment with diagnostic or therapeutic decisions

Cite this as: BMJ 2015;351:h4395

Web Extra. Extra material supplied by the author

Web appendix: Supplemental material