Derivation and External Validation of Prediction Models for Advanced Chronic Kidney Disease Following Acute Kidney Injury

Matthew T James; Neesh Pannu; Brenda R Hemmelgarn; Peter C Austin; Zhi Tan; Eric McArthur; Braden J Manns; Marcello Tonelli; Ron Wald; Robert R Quinn; Pietro Ravani; Amit X Garg

doi:10.1001/jama.2017.16326

. 2017 Nov 14;318(18):1787–1797. doi: 10.1001/jama.2017.16326

Derivation and External Validation of Prediction Models for Advanced Chronic Kidney Disease Following Acute Kidney Injury

Matthew T James ^1,^2,^3,^4,^✉, Neesh Pannu ⁵, Brenda R Hemmelgarn ^1,^2,^3,⁴, Peter C Austin ^6,⁷, Zhi Tan ¹, Eric McArthur ⁶, Braden J Manns ^1,^2,^3,⁴, Marcello Tonelli ^1,^2,^3,⁴, Ron Wald ^6,⁸, Robert R Quinn ^1,^2,^3,⁴, Pietro Ravani ^1,^2,^3,⁴, Amit X Garg ^6,⁹

¹Department of Medicine, Cumming School of Medicine, University of Calgary, Alberta, Canada

²Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Alberta, Canada

³O’Brien Institute for Public Health, Cumming School of Medicine, University of Calgary, Alberta, Canada

⁴Libin Cardiovascular Institute of Alberta, Cumming School of Medicine, University of Calgary, Alberta, Canada

⁵Department of Medicine, University of Alberta, Edmonton, Canada

⁶Institute for Clinical Evaluative Sciences, Ontario, Canada

⁷Institute of Health Policy, Management and Evaluation, University of Toronto, Ontario, Canada

⁸Department of Medicine, University of Toronto, Ontario, Canada

⁹Department of Medicine, Western University, London, Ontario, Canada

^✉

Corresponding Author: Matthew T. James, MD, PhD, Foothills Medical Center, 3280 Hospital Dr NW, Calgary, AB T2N 4Z6, Canada (mjames@ucalgary.ca).

Accepted for Publication: October 4, 2017.

Author Contributions: Dr James had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: James, Pannu, Hemmelgarn, Tan, Manns, Wald, Quinn, Ravani.

Acquisition, analysis, or interpretation of data: James, Pannu, Hemmelgarn, Austin, Tan, McArthur, Tonelli, Wald, Quinn, Ravani, Garg.

Drafting of the manuscript: James, Tan, Ravani.

Critical revision of the manuscript for important intellectual content: Pannu, Hemmelgarn, Austin, Tan, McArthur, Manns, Tonelli, Wald, Quinn, Ravani, Garg.

Statistical analysis: James, Pannu, Austin, Tan, McArthur, Ravani.

Obtained funding: James, Hemmelgarn, Wald.

Administrative, technical, or material support: James, Hemmelgarn, Manns, Tonelli, Garg.

Supervision: James, Hemmelgarn, Manns, Quinn.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr James reported receiving support from a Canadian Institutes of Health Research (CIHR) New Investigator and CIHR Foundation award and grant support from Amgen Canada. Dr Hemmelgarn reported receiving support from the Roy and Vi Bay chair in Kidney Research at the University of Calgary. Dr Austin reported receiving support from a Career Investigator award from the Heart and Stroke Foundation of Canada. Dr Manns reported receiving support from the Svare Professorship in Health Economics at the University of Calgary. Dr Tonelli reported receiving support from a CIHR Foundation award. Dr Quinn reported having a patent for a dialysis measurement analysis reporting system. Dr Garg was supported by the Dr Adam Linton Chair in Kidney Health Analytics at Western University and a CIHR Clinician Investigator Award. No other disclosures are reported.

Funding/Support: The study was funded by an Open Operating grant from the Canadian Institutes of Health Research (CIHR). This study was supported by the Institute for Clinical Evaluative Sciences (ICES) Western site. ICES is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). Core funding for ICES Western is provided by the Academic Medical Organization of Southwestern Ontario, the Schulich School of Medicine and Dentistry, Western University, the Lawson Health Research Institute, and 14 clinical departments.

Role of the Funder/Sponsor: None of sponsors had a role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: This study is based in part on data provided by Alberta Health and Alberta Health Services. The interpretation and conclusions contained herein are those of the researchers and do not necessarily represent the views of the government of Alberta or Alberta Health Services. Neither the government of Alberta nor the Alberta Health or Alberta Health Services express any opinion in relation to this study. The opinions, results, and conclusions reported herein are those of the authors and are independent from these funding sources. No endorsement by ICES or the Ontario MOHLTC is intended or should be inferred. Parts of this material are based on data and information compiled and provided by the Canadian Institute for Health Information (CIHI). However, the analyses, conclusions, opinions, and statements expressed herein are those of the authors, and not necessarily those of CIHI. Some of this work was conducted with members of the provincial ICES Kidney, Dialysis, and Transplantation Research Program, which receives programmatic grant funding from CIHR.

^✉

Corresponding author.

PMCID: PMC5820711 PMID: 29136443

Key Points

Question

Can the risk of progression to advanced chronic kidney disease be accurately predicted when patients who had acute kidney injury during their hospital stay are discharged from the hospital?

Findings

A multivariable model was developed with 9973 participants and was externally validated with 2761 participants. In the external validation cohort, a model that included age, sex, acute kidney injury stage, prehospitalization serum creatinine level, albuminuria, and discharge serum creatinine achieved a C statistic of 0.81 for predicting advanced chronic kidney disease after hospital discharge.

Meaning

This model was able to predict advanced chronic kidney disease following hospitalization with acute kidney injury but requires evaluation of its utility in a clinical setting.

Abstract

Importance

Some patients will develop chronic kidney disease after a hospitalization with acute kidney injury; however, no risk-prediction tools have been developed to identify high-risk patients requiring follow-up.

Objective

To derive and validate predictive models for progression of acute kidney injury to advanced chronic kidney disease.

Design, Setting, and Participants

Data from 2 population-based cohorts of patients with a prehospitalization estimated glomerular filtration rate (eGFR) of more than 45 mL/min/1.73 m² and who had survived hospitalization with acute kidney injury (defined by a serum creatinine increase during hospitalization > 0.3 mg/dL or > 50% of their prehospitalization baseline), were used to derive and validate multivariable prediction models. The risk models were derived from 9973 patients hospitalized in Alberta, Canada (April 2004-March 2014, with follow-up to March 2015). The risk models were externally validated with data from a cohort of 2761 patients hospitalized in Ontario, Canada (June 2004-March 2012, with follow-up to March 2013).

Exposures

Demographic, laboratory, and comorbidity variables measured prior to discharge.

Main Outcomes and Measures

Advanced chronic kidney disease was defined by a sustained reduction in eGFR less than 30 mL/min/1.73 m² for at least 3 months during the year after discharge. All participants were followed up for up to 1 year.

Results

The participants (mean [SD] age, 66 [15] years in the derivation and internal validation cohorts and 69 [11] years in the external validation cohort; 40%-43% women per cohort) had a mean (SD) baseline serum creatinine level of 1.0 (0.2) mg/dL and more than 20% had stage 2 or 3 acute kidney injury. Advanced chronic kidney disease developed in 408 (2.7%) of 9973 patients in the derivation cohort and 62 (2.2%) of 2761 patients in the external validation cohort. In the derivation cohort, 6 variables were independently associated with the outcome: older age, female sex, higher baseline serum creatinine value, albuminuria, greater severity of acute kidney injury, and higher serum creatinine value at discharge. In the external validation cohort, a multivariable model including these 6 variables had a C statistic of 0.81 (95% CI, 0.75-0.86) and improved discrimination and reclassification compared with reduced models that included age, sex, and discharge serum creatinine value alone (integrated discrimination improvement, 2.6%; 95% CI, 1.1%-4.0%; categorical net reclassification index, 13.5%; 95% CI, 1.9%-25.1%) or included age, sex, and acute kidney injury stage alone (integrated discrimination improvement, 8.0%; 95% CI, 5.1%-11.0%; categorical net reclassification index, 79.9%; 95% CI, 60.9%-98.9%).

Conclusions and Relevance

A multivariable model using routine laboratory data was able to predict advanced chronic kidney disease following hospitalization with acute kidney injury. The utility of this model in clinical care requires further research.

This study uses data from Canadian administrative and clinical ambulatory databases to derive and validate risk prediction models for advanced chronic kidney disease in patients hosptialized with acute kidney injury.

Introduction

Acute kidney injury is common among hospitalized patients and associated with poor long-term outcomes. Several studies between 2008 to 2017 have demonstrated that although acute kidney injury is usually reversible, some patients may experience incomplete recovery of kidney function, while others subsequently develop accelerated loss of kidney function, resulting in an increased risk of chronic kidney disease. Current clinical practice guidelines recommend that patients be followed-up within 3 months to assess whether they have developed chronic kidney disease and when identified undergo treatment according to current guidelines.

Despite these recommendations, many patients with acute kidney injury do not receive a follow-up assessment, nor do they receive appropriate care when kidney function has not recovered, resulting in lost opportunities to intervene and potentially improve long-term outcomes. However, because not all patients who survive acute kidney injury progress to chronic kidney disease, follow-up of all patients hospitalized with acute kidney injury could lead to unnecessary use of clinical resources. Identifying and screening patients at high risk of developing advanced chronic kidney disease could improve outcomes for patients following acute kidney injury.

We used population-based laboratory and administrative data from 2 provinces in Canada to derive and validate risk prediction models for advanced chronic kidney disease following hospitalization with acute kidney injury. Our objective was to develop a practical risk stratification approach that could be used to identify patients at high risk of chronic kidney disease after they are discharged.

Methods

We followed the TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis) statement for reporting multivariable prediction model development and validation. The health research ethics boards of the Universities of Calgary, Alberta, and Sunnybrook Health Sciences Centre, Ontario, Canada, approved the study and granted waiver of participant consent.

Derivation and Validation Cohorts

We formed a derivation cohort using the Alberta Kidney Disease Network (AKDN) database, a population-based repository that includes serum creatinine measurements from more than 1.8 million adults obtained from acute care and outpatient settings for all 3.5 million residents of the province, from April 1, 2004, through March 31, 2014. The database includes patient-level data from the provincial administrative health sources of Alberta Health. Patient comorbidities and procedures are characterized using the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis and the Canadian Classification of Procedures (CCP) and the ICD-10 diagnosis and the Canadian Classification of Heath Intervention (CCI) procedure codes. We randomly selected two-thirds of the AKDN cohort to serve as a model derivation cohort and used the remaining third as an internal validation cohort.

We formed the external validation cohort from Southwestern Ontario using data from the Institute of Clinical Evaluative Sciences Kidney, Dialysis and Transplantation (ICES KDT) Program. These data include laboratory information from more than 74 000 patients from 2 regional laboratory providers (Cerner and Gamma-Dynacare) in Southwestern Ontario (catchment population 2.5 million) from June 1, 2004, through March 31, 2012. The laboratory data sets were linked with comprehensive provincial administrative data using unique encoded identifiers and analyzed at ICES. The ICD-9-CM-CPP and ICD-10-CCI coding approaches were used to characterize patient comorbidities and procedures.

Study Population

We identified all adult residents 18 years of age or older who were hospitalized during the study enrollment period (Figure 1). Eligible participants required at least 1 outpatient estimated glomerular filtration rate (eGFR) value between 7 and 365 days prior to hospitalization to establish baseline kidney function (outpatient serum creatinine tests in this period generally reflect stable values). We restricted the cohorts to those hospitalized with acute kidney injury (based on an increase in serum creatinine during hospitalization >0.3 mg/dL or >50% of their prehospitalization baseline), a baseline eGFR higher than 45 mL/min/1.73 m², and at least 2 outpatient serum creatinine measurements separated by a minimum of 3 months with the first between 30 days and 1 year after the hospital discharge date. This required that patients survived more than a month after hospital charge and had sufficient follow-up information to ascertain development of advanced chronic kidney disease, and reflected a pragmatic approach that recognizes that organizing chronic kidney disease care is not needed when patients do not survive long enough to develop advanced chronic kidney disease. We excluded people treated with chronic dialysis or who had had a kidney transplant prior to or during the index hospitalization, identified by a validated approach using physician billing claims for a continuous period of 90 days of dialysis in provincial administrative databases or registration for chronic dialysis or kidney transplant with the Canadian Organ Replacement Registry.

Study Outcome

The outcome of the study was progression to advanced chronic kidney disease, defined as a sustained outpatient eGFR value of less than 30 mL/min/1.73 m² (estimated based on age, sex, and serum creatinine levels using the Chronic Kidney Disease Epidemiology Collaboration equation) identified between 30 days to 1 year after hospital discharge or initiation of chronic dialysis up to 1 year after hospital discharge. This stage of chronic kidney disease is associated with a high risk of adverse outcomes and is accompanied by guideline recommendations and performance indicators for referral to a nephrologist. We required a minimum of 2 outpatient measurements of eGFR lower than 30 mL/min/1.73 m² separated by at least 3 months, including an average of all outpatient eGFR measurements within this period of less than 30 mL/min/1.73 m² to satisfy the definition of this outcome to align with current consensus criteria for chronic kidney disease.

Candidate Predictor Variables

We identified candidate predictor variables for potential inclusion in our models from a review of the literature and identified potential predictor variables available prior to and during the index hospitalization with acute kidney injury. Baseline kidney function was determined from the most recent outpatient serum creatinine measurement made 7 to 365 days prior to the index hospitalization. Albuminuria was characterized by urine albumin:creatinine ratio (ACR) or dipstick using random spot urine measurements during the index admission or for up to 6 months prior to admission. We used urine ACR measurements preferentially, with urine dipstick measurements added for patients without ACR measurements in the derivation cohort. We defined albuminuria categories as normal (ACR, <30 mg/g or urine-dipstick negative), mild (ACR, 30-300 mg/g or urine dipstick trace or 1+), or heavy (ACR, >300 mg/g or urine dipstick positive ≥2+) and characterized patients with multiple tests using the median of multiple measurements, recognizing that albuminuria can be transient. Absence of a result for albuminuria was included as a separate category. We categorized the severity of acute kidney injury according to 3 categories of the Kidney Disease, Improving Global Outcomes (KDIGO) staging system based on the highest serum creatinine value identified during hospitalization or requirement for dialysis during the index hospitalization. We identified the extent of kidney function recovery at the time of hospital discharge based on the last inpatient serum creatinine value measured before hospital discharge. Comorbidities and procedures were identified from hospital discharge records and physician claims using validated ICD-9-CM and ICD-10 coding algorithms (eTable 1 in Supplement 1).

Model Derivation

Candidate variables from the derivation cohort were included as potential covariates in multivariable logistic regression models. Stepwise backward variable selection with a significance level of .05 for variable retention was used in 1000 bootstrapped samples of the same size as the derivation sample, to develop a parsimonious predictor model while minimizing overfitting. Variables selected in at least 70% of samples were included in the full multivariable model. We then fit a series of reduced models by sequentially removing variables and compared the full multivariable model with the simplified models. Reduced models were structured to include variables with greater ease of ascertainment at hospital discharge and to include removing variables requiring prehospitalization laboratory data. To further support clinical use of the model, we created a simple integer risk index using the method described by Sullivan et al. The performance of the risk index was determined based on modeling the logit of the total risk index for each participant.

Prediction Model Performance

The regression coefficients from the logistic regression models in the derivation sample were fixed. The fitted models were applied to both the internal and external validation cohorts. The overall goodness-of-fit of the models was compared based on Akaike Information Criteria. Model discrimination was compared based on the C statistic and integrated discrimination improvement (IDI). Model calibration was assessed by the calibration intercept, by the calibration slope, and graphically by locally weighted scatterplot smoothing (LOESS) plots of observed vs predicted probabilities of the outcome. The ability of models to reclassify patients into high- or low-risk categories was compared using the net reclassification index (NRI). Risk thresholds for the outcome were defined as less than 1.0%, 1.0% to less than 5.0%, 5.0% to less than 10.0%, 10.0% to less than 20.0%, and 20.0% or higher. Reclassification was also compared using the continuous NRI, which assesses reclassification across a continuous range of risk thresholds. A P value of .05 was considered statistically significant using 2-sided testing. Statistical analyses were performed using SAS version 9.4 (SAS Institute Inc) and R (https://r-project.org).

Results

Characteristics of the Cohorts

A total of 14 958 participants were included in the derivation and internal validation cohorts (9973 in the derivation cohort and 4985 in the internal validation cohort) and 2761 in the external validation cohort (Figure 1). The mean age of participants ranged from 66 years through 69 years, the mean baseline serum creatinine value was 1.0 mg/dL, obtained a median of 66 days (interquartile range [IQR], 24-161 days) prior to admission in the derivation and internal validation cohorts and 44 days (IQR, 17-124 days) in the external validation cohort, more than 20% had stage 2 or 3 acute kidney injury and the serum creatinine value was 1.3 mg/dL or higher at the time of hospital discharge in one-third of patients (Table 1; to convert creatinine from mg/dL to μmol/L, multiply by 88.4). Albuminuria status was available for 57% of the derivation and internal validation cohorts and 36% of the external validation cohort. Compared with the derivation and internal validation cohorts, patients in the external validation cohort were slightly older and were more likely to have received cardiovascular procedures or mechanical ventilation. The median number of outpatient serum creatinine measurements per patient during follow-up was 11 (IQR, 6-22) in the derivation and internal validation cohorts and 12 (IQR, 7-21) in the external validation cohort. There were 29 patients (0.2%) in the derivation and internal validation cohorts with only 1 eGFR less than 30 mL/min/1.73 m² and did not have a subsequent measurement during follow-up. They were not considered to have developed advanced chronic kidney disease.

Table 1. Baseline Characteristics of the Derivation, Internal Validation, and External Validation Cohorts.

	Cohort, No. (%) of Patients^a
	Derivation (n = 9973)	Internal Validation (n = 4985)	External Validation (n = 2761)
Demographics
Age, mean (SD), y	65.7 (14.9)	65.9 (15)	69.2 (11.4)
Women	4258 (42.7)	2091 (41.95)	1107 (40.1)
Laboratory data
Baseline measures, mean (SD)
Serum creatinine, mg/dL	1.0 (0.2)	1.0 (0.2)	1.0 (0.2)
eGFR, mL/min/1.73 m²	76 (20)	76 (20)	73 (18)
Albuminuria measurement^b
Normal	3775 (37.8)	1881 (37.7)	798 (28.9)
Mild	1373 (13.8)	662 (13.3)	137 (5.0)
Heavy	494 (4.9)	243 (4.9)	56 (2.0)
Unmeasured	4331 (43.4)	2199 (44.1)	1770 (64.1)
Acute kidney injury stage^c
1	7686 (77.1)	3806 (76.4)	2165 (78.4)
2	1357 (13.6)	699 (14.0)	356 (12.9)
3	930 (9.3)	480 (9.6)	240 (8.7)
Discharge serum creatinine, mg/dL
<1.0	3394 (34.0)	1721 (34.5)	1057 (38.3)
1.0-<1.3	3207 (32.2)	1542 (30.9)	912 (33.0)
1.3-<1.6	1963 (19.7)	1012 (20.3)	535 (19.3)
1.6-<1.9	799 (8.0)	383 (7.7)	153 (5.5)
≥1.9	610 (6.1)	327 (6.6)	107 (3.9)
Discharge eGFR, mean (SD), mL/min/1.73 m²	62 (23)	62 (23.0)	59 (22)
Comorbidities^d
Diabetes	920 (9.2)	498 (10.0)	1077 (39.0)
Cancer	2638 (26.4)	1329 (26.7)	1149 (41.6)
Congestive heart failure	2422 (24.3)	1197 (24.0)	543 (19.7)
Metastatic solid tumor	451 (4.5)	221 (4.4)	96 (3.5)
Myocardial Infarction	1796 (18.0)	902 (18.1)	306 (11.1)
Mild liver disease	302 (4.7)	150 (4.7)	45 (1.6)
Moderate or severe liver disease	189 (1.9)	99 (2.0)	33 (1.2)
Peripheral vascular disease	1079 (10.8)	505 (10.1)	222 (8.0)
Rheumatologic disease	404 (4.0)	215 (4.3)	54 (2.0)
Hypertension	1734 (17.4)	861 (17.3)	677 (24.5)
Procedures in hospital
Cardiac catheterization	629 (6.3)	316 (6.3)	634 (23.0)
Cardiac surgery	754 (7.6)	301 (6.0)	522 (18.9)
Abdominal aortic aneurysm repair	139 (1.4)	53 (1.1)	56 (2.0)
Mechanic ventilation	1752 (17.6)	818 (16.4)	745 (27.0)
Dialysis	175 (1.8)	83 (1.7)	47 (1.7)
Length of hospital stay, median (IQR), d	9 (5-18)	9 (5-19)	8 (5-14)

Open in a new tab

Abbreviations: eGFR, estimated glomerular filtration rate; IQR, interquartile range.

SI conversion factor: To convert serum creatinine from mg/dL to μmol/L, multiply by 88.4.

^{^a}

Percentages may not sum to 100 due to rounding.

^{^b}

Normal albuminuria is defined by an albumin:creatinine ratio (ACR) of less than 30 mg/g or dipstick urinalysis protein negative; mild, ACR from 30 through 300 mg/g or dipstick urinalysis protein trace of 1+; and heavy, ACR of more than 300 mg/g or dipstick urinalysis protein of 2+ or higher. To convert urine ACR to mg/mmol, multiply by 0.113.

^{^c}

Acute kidney injury stage 1 is defined by a serum creatinine value increase of 0.3 mg/dL or more or 1.5 to 1.9 times the baseline measure within the index hospitalization; stage 2, serum creatinine value increase of 2.0 to 2.9 times the baseline measure within index hospitalization; and stage 3, serum creatinine value increase 3.0 times or higher than the baseline or an increase to 4.0 mg/dL or higher, or acute dialysis within index hospitalization.

^{^d}

Patients could have more than 1 comorbidity.

Four hundred eight patients (2.7%) in the derivation and internal validation cohorts developed the outcome of having a sustained eGFR value of less than 30 mL/min/1.73m² after hospital discharge. Seven hundred seventy-five patients (5.2%) in the derivation and internal validation cohorts had an eGFR value of less than 30 mL/min/1.73 m² at discharge, of whom 172 (22.2%) later developed advanced chronic kidney disease, which comprised 42.2% of the total number of advanced chronic kidney disease events. A similar proportion of patients experienced the outcome in the external validation cohort (62 events, 2.2%).

Prediction of Advanced Chronic Kidney Disease in the Derivation Cohort

Six variables were associated with a higher risk of progression to advanced chronic kidney disease in bootstrapped samples of the derivation cohort: older age, female sex, higher baseline serum creatinine values, albuminuria, greater acute kidney injury severity, and higher discharge serum creatinine values (Table 2; eTable 2 in Supplement 1). The 6-variable model (model 1) had the highest C statistic (0.86; 95% CI, 0.84-0.89) and lowest Akaike Information Criterion (1967.0). Based on the predicted risk using model 1, 5228 patients (52.4%) in the derivation cohort had less than 1%; 3512 (35.2%), 1% to less than 5%; 626 (6.2%), 5% to less than 10%; 360 (3.6%), 10% to less than 20%; and 247 (2.5%), 20% or more risk of developing chronic kidney disease (eTable 3 in Supplement 1). Discrimination (based on the C statistic and integrated discrimination improvement [IDI]) was lower in reduced models without inclusion of albuminuria, discharge serum creatinine values, and baseline serum creatinine values. The lowest C statistic was observed for model 5, which included age, sex, and acute kidney injury stage only (C statistic, 0.71; 95% CI, 0.67-0.74).

Table 2. Advanced Chronic Kidney Disease Predictors and Performance of Models in the Derivation Cohort^a.

	Models, Odds Ratio (95% CI)
	1. Age, Sex, Acute Kidney Injury Stage, Albuminuria, and Baseline and Discharge Serum Creatinine Values	2. Age, Sex, Acute Kidney Injury Stage, and Baseline and Discharge Serum Creatinine Values	3. Age, Sex, Acute Kidney Injury Stage, Baseline Serum Creatinine Values	4. Age, Sex, Discharge Serum Creatinine Values	5. Age, Sex, Acute Kidney Injury Stage
Predictors
Age, per year increase	1.02 (1.01-1.03)	1.01 (1.01-1.02)	1.01 (1-1.02)	1.01 (1.01-1.02)	1.02 (1.01-1.03)
Sex
Men	1 [Reference]	1 [Reference]	1 [Reference]	1 [Reference]	1 [Reference]
Women	2.93 (2.16-3.97)	3.03 (2.24-4.11)	2.78 (2.07-3.73)	2.24 (1.72-2.90)	1.15 (0.9-1.47)
Acute kidney injury stage^b
1	1 [Reference]	1 [Reference]	1 [Reference]		1 [Reference]
2	1.28 (0.88-1.87)	1.30 (0.89-1.89)	2.2 (1.56-3.12)		1.9 (1.35-2.67)
3	2.47 (1.73-3.52)	2.52 (1.77-3.59)	6.84 (5.15-9.10)		6.34 (4.81-8.37)
Serum creatinine, mg/dL
Baseline per 0.1 mg/dL increase	1.18 (1.10-1.18)	1.20 (1.12-1.29)	1.45 (1.37-1.54)
Discharge
<1.0	1 [Reference]	1 [Reference]		1 [Reference]
1.0-<1.3	2.93 (1.55-5.56)	2.88 (1.52-5.46)		3.39 (1.8-6.41)
1.3-<1.6	7.78 (4.19-14.44)	7.67 (4.14-14.22)		10.73 (5.86-19.67)
1.6-<1.9	11.35 (5.86-21.97)	11.45 (5.93-22.12)		19.05 (10.06-36.06)
≥1.9	37.01 (19.46-70.37)	36.81 (19.43-69.77)		84.44 (46.75-152.5)
Albuminuria^b
Normal	1 [Reference]
Mild	1.25 (0.81-1.92)
Heavy	3.13 (2.00-4.91)
Unmeasured	1.67 (1.21-2.29)
Model Performance Measures
Akaike Information Criterion	1967.0	1987.3	2205.0	2022.3	2352.9
C statistic	0.86 (0.84-0.89)	0.85 (0.83-0.88)	0.80 (0.78-0.83)	0.84 (0.78-0.87)	0.71 (0.67-0.74)
Difference	1 [Reference]^c	0.01 (0.002-0.012)	0.06 (0.04-0.08)	0.02 (0.01-0.03)	0.15 (0.12-0.19)
P value		.005	<.001	<.001	<.001
Integrated discrimination improvement, %,		0.6 (0.1-1.2)	5.8 (4.8-6.8)	1.6 (0.7-2.4)	7.8 (6.6-9.0)
P value		.014	<.001	<.001	<.001

Open in a new tab

SI conversion factor: To convert serum creatinine to μmol/L, multiply by 88.4.

^{^a}

Of the 9973 patients in the derivation cohort, 272 had advanced chronic kidney disease events.

^{^b}

See Table 1 footnote for acute kidney injury stage and albuminuria severity definitions.

^{^c}

Models 2 through 5 were each compared with model 1. For model comparisons, integrated discrimination improvement values and differences in a C statistic greater than 0 indicate better performance for the full model 1 than for the reduced models.

Prediction Model Performance in the Internal Validation Cohort

In the internal validation cohort, the 6-variable model remained well calibrated (eFigure 1 in Supplement 1) and also had a higher C statistic (0.87; 95% CI, 0.84-0.90) than did the models with fewer variables (Table 3; eTable 4 in Supplement 1). The IDI, continuous NRI, and categorical NRI favored the 6-variable model compared with the reduced-variable models. For example, significantly improved classification into risk categories was observed with the 6-variable model 1 compared with model 4—age, sex, and discharge serum creatinine values (categorical NRI, 16.6%; 95% CI, 7.7%-25.6%; P value <.001) and model 5—age, sex, and acute kidney injury stage (categorical NRI, 99.3%; 95% CI, 87.6%-110.9%; P value <.001). Model calibration and discrimination were similar for patients with baseline serum creatinine measurements taken within 3 months or taken more than 3 months prior to admission and with baseline eGFR greater than vs less than 60 mL/min/1.73 m² (eTable 5 in Supplement 1), as well as for patients excluded from the derivation and validation cohorts due to absence of outpatient serum creatinine measurements prior to index hospitalization (eTable 6 in Supplement 1).

Table 3. Predictive Performance of Models for Advanced Chronic Kidney Disease in Internal and External Validation Cohorts^a.

	Models, Measures of Predictive Performance
	1. Age, Sex, Acute Kidney Injury Stage, Albuminuria, and Baseline and Discharge Serum Creatinine Values		2. Age, Sex, Acute Kidney Injury Stage, and Baseline and Discharge Serum Creatinine Values		3. Age, Sex, Acute Kidney Injury Stage, Baseline Serum Creatinine Values		4. Age, Sex, Discharge Serum Creatinine Values		5. Age, Sex, Acute Kidney Injury Stage
	Internal Validation Cohort	External Validation Cohort	Internal Validation Cohort	External Validation Cohort	Internal Validation Cohort	External Validation Cohort	Internal Validation Cohort	External Validation Cohort	Internal Validation Cohort	External Validation Cohort
Calibration intercept	0.14	−0.43	0.12	−0.37	0.70	−0.25	−0.05	−0.16	0.22	-0.24
P value	.46	.15	.55	.23	.01	.52	.82	.63	.53	.65
Calibration slope	1.07	0.91	1.05	0.90	1.23	1.00	0.99	0.94	1.07	1.00
P value	.32	.32	.44	.28	.01	.99	.93	.54	.50	.97
C statistic (95%CI)	0.87 (0.84 to 0.90)	0.81 (0.75 to 0.86)	0.86 (0.83 to 0.89)	0.80 (0.74 to 0.86)	0.84 (0.81 to 0.87)	0.78 (0.73 to 0.83)	0.84 (0.80 to 0.88)	0.79 (0.74 to 0.85)	0.72 (0.67 to 0.76)	0.70 (0.64 to 0.77)
Difference in C statistics (95% CI)	1 [Reference]^b		0.01 (0.00 to 0.02)	0.01 (−0.01 to 0.04)	0.03 (0.00 to 0.06)	0.03 (−0.02 to 0.08)	0.03 (0.02 to 0.04)	0.02 (0.00 to 0.05)	0.15 (0.10 to 0.20)	0.11 (0.05 to 0.18)
P value			.04	.34	.03	.26	<.001	.09	<.001	<.001
IDI (95% CI), %			1.1 (0.3 to 1.8)	0.6 (−0.3 to 1.5)	6.8 (5.1 to 8.5)	5.3 (2.9 to 7.7)	2.9 (1.9 to 4.0)	2.6 (1.1 to 4.0)	9.8 (7.9 to 11.6)	8.0 (5.1 to 11.0)
P value			.01	.18	<.001	<.001	<.001	<.001	<.001	<.001
NRI (95% CI), %
Continuous			40.0 (24.1 to 55.8)	27.5 (6.6 to 48.5)	80.7 (65.2 to 96.2)	73.7 (49.4 to 98.0)	73.5 (58.2 to 88.9)	47.2 (26.1 to 68.4)	106.1 (92.5 to 119.7)	87.7 (65.2 to 110.1)
P value			<.001	.03	<.001	<.001	<.001	<.001	<.001	<.001
Categorical^c			6.4 (−1.5 to 14.2)	2.3 (−3.9 to 8.5)	51.9 (36.5 to 64.3)	39.2 (21.5 to 57.0)	16.6 (7.7 to 25.6)	13.5 (1.9 to 25.1)	99.3 (87.6 to 110.9)	79.9 (60.9 to 98.9)
P value			.12	.49	<.001	<.001	<.001	.044	<.001	<.001
Events, No. (%)^d			8 (5.9)	≤5 (≤8.1)	47 (34.6)	11 (17.7)	24 (17.6)	15 (24.2)	74 (54.4)	20 (32.3)
Nonevents, No. (%)			23 (0.5)	23 (0.5)	840 (17.3)	580 (21.5)	−50 (−1.0)	−289 (−10.7)	2175 (44.8)	1304 (48.3)
Overall, No. (%)			31 (6.4)	NR	887 (51.9)	591 (39.2)	26 (16.6)	−274 (13.5)	2249 (99.3)	1324 (79.9)

Open in a new tab

Abbreviations: IDI, integrated discrimination improvement; NRI, net reclassification improvement; NR, not reportable (cells associated with 5 or fewer events are empty because of Institute for Clinical Evaluative Sciences policy).

^{^a}

Of the 4985 patients in the internal validation cohort, 136 had advanced chronic kidney disease events, and of the 2761 patients in the external cohort, 62 patients had chronic kidney disease events.

^{^b}

Models 2 through 5 were each compared with model 1. For model comparisons differences in the C statistic, IDI, and NRI, values greater than 0 indicate better performance for model 1 than the reduced models.

^{^c}

Risk categories include patients with less than 1%, 1% to less than 5%, 5% to less than 10%, 10% to less than 20%, and 20% or higher risk of developing chronic kidney disease.

^{^d}

Net reclassification improvement events refer to development of advanced chronic kidney disease, and NRI nonevents refer to no development of advanced chronic kidney disease.

The integer-based risk index developed from the 6-variable model (Figure 2) showed similar discrimination in the internal validation cohort (C statistic, 0.87; 95% CI, 0.83-0.91) to that of the logistic regression model and also remained well calibrated (calibration intercept, −0.08; P value = .72; calibration slope, 1.02; P value = .98). Observed vs predicted risks from the integer-based risk index across the range of values of the risk index are shown in Figure 3A.

Figure 3. — Internal validation cohort (n = 4956) and external validation cohort (n = 2761). Error bars indicate 95% confidence intervals. The risk scores observed in both cohorts ranged from a minimum value of 0 to maximum value of 24.

Prediction Model Performance in the External Validation Cohort

The 6-variable model remained well calibrated in the external validation cohort (eFigure 2 in Supplement 1) and had a C statistic of 0.81 (95% CI, 0.75-0.86). Discrimination and reclassification remained improved using the 6-variable model compared with simpler models—model 4, age, sex, and discharge serum creatinine or model 5, age, sex, and acute kidney injury stage (Table 3). The observed vs predicted risks applying the integer-based risk index in the external validation cohort is shown in Figure 3B.

Clinical Utility of the Prediction Model

The potential clinical utility of applying the 6-variable risk index to plan follow-up care, based on a range of predicted risk thresholds for advanced chronic kidney disease, is shown in eTable 7 in Supplement 1. The percentage of patients above the risk threshold for follow-up ranged from 48% to 2.6%, while positive predictive values ranged from 5.2% to 30.5% for corresponding risk thresholds of 1% or higher and 20% or higher, respectively. Negative predictive values were 98% or higher for each of the risk thresholds.

Discussion

A 6-variable model and corresponding risk index (including age, sex, baseline serum creatinine value, albuminuria, acute kidney injury severity, and discharge serum creatinine value) showed the best performance for predicting whether patients who had acute kidney injury during their hospital stay would later develop advanced chronic kidney disease in internal and external validation cohorts, with better performance than reduced models based on age, sex, and discharge serum creatinine alone or based on age, sex, or acute kidney injury severity alone. The predictor variables used in this model and risk index may be readily ascertained at the time of hospital discharge making it feasible for implementation within discharge planning processes to identify patients at high risk of developing advanced chronic kidney disease following hospitalization with acute kidney injury. Such risk stratification could help guide prognostic assessment and follow-up during transition to outpatient medical care, and inform resource allocation since advanced chronic kidney disease carries a high risk of complications, is accompanied by recommendations for assessment by nephrology specialists, and leads to high resource requirements to provide care. The predictor variables contained in these risk models have been previously associated with chronic kidney disease or end-stage kidney disease following acute kidney injury. This work has integrated these variables into multivariable risk models and a practical risk index, so that these variables can be readily used to obtain individualized estimates of advanced chronic kidney disease for patients with acute kidney injury encountered in clinical practice.

Model validation is an important aspect of developing clinical prediction models. There was no deterioration in model calibration upon evaluation in the internal validation cohort, indicating that the models were likely not overfit. An even more important aspect is external validation in a different population of patients before widespread model adoption and use. These models performed well in the geographically distinct Ontario cohort, despite some expected loss of discriminative performance (model 1 C statistic, 0.81; 95% CI, 0.75-0.86). Models are typically considered useful for clinical decision making when the C statistic is higher than 0.70 and strong when the C statistic exceeds 0.80, suggesting this model could support clinical decision making. Furthermore, the net reclassification improvement of the full model compared with simpler models demonstrates that using this model could improve accuracy of decision making for follow-up compared with that based on fewer variables alone.

Strengths of this study include that the model was developed in a large, population-representative cohort of adults hospitalized with acute kidney injury. The predictor variables in these risk models consist of readily available patient demographics and laboratory tests, which could allow them to be implemented into clinical practice using automated approaches within electronic medical record systems (an electronic risk calculator is available in Supplement 2). The models were validated in a distinct, external cohort.

Limitations

This study has several limitations. First, candidate variables were identified from secondary analysis of data, which does not include all potential risk factors for chronic kidney disease such as blood pressure, urine sediment, cause of acute kidney injury, and details of dialysis treatments. However, similar to this study, another risk prediction model for chronic kidney disease also found that clinical variables provided little improvement in model performance beyond kidney-related laboratory variables. Second, if low-risk patients were systematically excluded from these cohorts due to lack of follow-up creatinine testing, then estimates from the resulting models could overestimate risk of advanced chronic kidney disease. However, the predicted risks of excluded patients were similar to those of included patients, suggesting that such bias is unlikely. Third, some participants did not have albuminuria measured, which may influence the performance of the models. However, models without albuminuria also performed well, suggesting that risk prediction for chronic kidney disease following acute kidney injury could still be achieved without albuminuria measurements. Fourth, models were derived and validated in cohorts from Canada, and generalizability to patients from other regions was not examined.

This study may have implications for patients, clinicians, and policy makers. Studies from both Canada and the United States have demonstrated low rates of follow-up for chronic kidney disease care and poor clinical outcomes in individuals with chronic kidney disease after acute kidney injury. This may be because clinicians in the community are not aware of acute kidney injury episodes that occur during hospitalization, lack awareness of the prognostic implications of these events, or encounter barriers to timely access to chronic kidney disease care in the community. This highlights the common clinical challenge of providing continuity of care between the hospital and community. These risk prediction models provide an accurate but simple strategy that could be used to stratify patients into clinically meaningful risk groups at the time of hospital discharge and guide further management in the community.

Conclusions

Supplement 1.

eTable 1. Diagnostic and procedure codes used to identify comorbidities and procedures in the derivation, internal validation (AKDN) and external validation (ICES) cohorts

eTable 2. Predictors of advanced CKD and their frequency in the derivation, internal validation (AKDN) and external validation (ICES) cohorts

eTable 3. Frequency, and observed risk of risk categories for 5 prediction models for advanced CKD in the derivation (AKDN) cohort

eTable 4. Distribution of model predicted risk of advanced CKD (%) estimated in the derivation, internal validation (AKDN), and external validation (ICES) cohorts

eTable 5. Discrimination and calibration of a six-variable model for advanced CKD in the internal validation (AKDN) cohort, stratified by timing of baseline Scr measurement and level of baseline eGFR

eTable 6. Predictive performance of models for advanced CKD in the patients excluded from original (AKDN) cohorts due to lack of pre-hospitalization Scr measurements between 7-365 days prior to hospital admission

eTable 7. Varying thresholds of predicted risk based on the six-variable risk-index, proportion of patients who would be risk stratified for community CKD follow-up, and corresponding sensitivity, specificity, positive and negative predictive values for progression to advanced CKD during follow-up after a hospitalization with AKI

eFigure 1. Calibration of the six-variable (Model 1) and reduced models (Models 2-5) in the internal validation (AKDN) cohort using locally weighted least squares regression smoother plots

eFigure 2. Calibration of the six-variable (Model 1) and reduced models (Models 2-5) in the external validation (ICES) cohort using locally weighted least squares regression smoother plots

Click here for additional data file.^{(1.5MB, pdf)}

Supplement 2.

Risk calculator for advanced chronic kidney disease following acute kidney injury

Click here for additional data file.^{(26.5KB, xlsx)}

References

1.Coca SG, Singanamala S, Parikh CR. Chronic kidney disease after acute kidney injury: a systematic review and meta-analysis. Kidney Int. 2012;81(5):442-448. [DOI] [PMC free article] [PubMed] [Google Scholar]
2.Sawhney S, Marks A, Fluck N, et al. . Post-discharge kidney function is associated with subsequent ten-year renal progression risk among survivors of acute kidney injury. Kidney Int. 2017;92(2):440-452. [DOI] [PMC free article] [PubMed] [Google Scholar]
3.Ishani A, Nelson D, Clothier B, et al. . The magnitude of acute serum creatinine increase after cardiac surgery and the risk of chronic kidney disease, progression of kidney disease, and death. Arch Intern Med. 2011;171(3):226-233. [DOI] [PubMed] [Google Scholar]
4.Kidney International Supplements KDIGO clinical practice guideline for acute kidney injury. http://www.kisupplements.org/issue/S2157-1716(12)X7200-9. Published March 2012. Accessed October 5, 2017.
5.Siew ED, Peterson JF, Eden SK, et al. . Outpatient nephrology referral rates after acute kidney injury. J Am Soc Nephrol. 2012;23(2):305-312. [DOI] [PMC free article] [PubMed] [Google Scholar]
6.Harel Z, Wald R, Bargman JM, et al. . Nephrologist follow-up improves all-cause mortality of severe acute kidney injury survivors. Kidney Int. 2013;83(5):901-908. [DOI] [PubMed] [Google Scholar]
7.Goldstein SL, Jaber BL, Faubel S, Chawla LS; Acute Kidney Injury Advisory Group of American Society of Nephrology . AKI transition of care: a potential opportunity to detect and prevent CKD. Clin J Am Soc Nephrol. 2013;8(3):476-483. [DOI] [PubMed] [Google Scholar]
8.James M, Bouchard J, Ho J, et al. . Canadian Society of Nephrology commentary on the 2012 KDIGO clinical practice guideline for acute kidney injury. Am J Kidney Dis. 2013;61(5):673-685. [DOI] [PubMed] [Google Scholar]
9.Coleman EA, Berenson RA. Lost in transition: challenges and opportunities for improving the quality of transitional care. Ann Intern Med. 2004;141(7):533-536. [DOI] [PubMed] [Google Scholar]
10.Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD). Ann Intern Med. 2015;162(10):735-736. [DOI] [PubMed] [Google Scholar]
11.Hemmelgarn BR, Clement F, Manns BJ, et al. . Overview of the Alberta Kidney Disease Network. BMC Nephrol. 2009;10:30. [DOI] [PMC free article] [PubMed] [Google Scholar]
12.Quan H, Sundararajan V, Halfon P, et al. . Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43(11):1130-1139. [DOI] [PubMed] [Google Scholar]
13.Quan H, Khan N, Hemmelgarn BR, et al. ; Hypertension Outcome and Surveillance Team of the Canadian Hypertension Education Programs . Validation of a case definition to define hypertension using administrative data. Hypertension. 2009;54(6):1423-1428. [DOI] [PubMed] [Google Scholar]
14.Hux JE, Ivis F, Flintoft V, Bica A. Diabetes in Ontario: determination of prevalence and incidence using a validated administrative data algorithm. Diabetes Care. 2002;25(3):512-516. [DOI] [PubMed] [Google Scholar]
15.Jain AK, Cuerden MS, McLeod I, et al. . Reporting of the estimated glomerular filtration rate was associated with increased use of angiotensin-converting enzyme inhibitors and angiotensin-II receptor blockers in CKD. Kidney Int. 2012;81(12):1248-1253. [DOI] [PubMed] [Google Scholar]
16.Juurlink D, Preyra C, Croxford R, et al. . Canadian Institute for Health Information Discharge Abstract Database: A Validation Study. Toronto, ON: Institute for Clinical Evaluative Sciences; 2006. [Google Scholar]
17.Siew ED, Ikizler TA, Matheny ME, et al. . Estimating baseline kidney function in hospitalized patients with impaired kidney function. Clin J Am Soc Nephrol. 2012;7(5):712-719. [DOI] [PMC free article] [PubMed] [Google Scholar]
18.Clement FM, James MT, Chin R, et al. ; Alberta Kidney Disease Network . Validation of a case definition to define chronic dialysis using outpatient administrative data. BMC Med Res Methodol. 2011;11:25. [DOI] [PMC free article] [PubMed] [Google Scholar]
19.Levey AS, Stevens LA, Schmid CH, et al. ; CKD-EPI (Chronic Kidney Disease Epidemiology Collaboration) . A new equation to estimate glomerular filtration rate. Ann Intern Med. 2009;150(9):604-612. [DOI] [PMC free article] [PubMed] [Google Scholar]
20.Go AS, Chertow GM, Fan D, McCulloch CE, Hsu CY. Chronic kidney disease and the risks of death, cardiovascular events, and hospitalization. N Engl J Med. 2004;351(13):1296-1305. [DOI] [PubMed] [Google Scholar]
21.Hemmelgarn BR, Manns BJ, Lloyd A, et al. ; Alberta Kidney Disease Network . Relation between kidney function, proteinuria, and adverse outcomes. JAMA. 2010;303(5):423-429. [DOI] [PubMed] [Google Scholar]
22.Andrassy KM. Comments on “KDIGO 2012 clinical practice guideline for the evaluation and management of chronic kidney disease.” Kidney Int. 2013;84(3):622-623. [DOI] [PubMed] [Google Scholar]
23.Levin A, Hemmelgarn B, Culleton B, et al. ; Canadian Society of Nephrology . Guidelines for the management of chronic kidney disease. CMAJ. 2008;179(11):1154-1162. [DOI] [PMC free article] [PubMed] [Google Scholar]
24.National Institute for Health and Care Excellence, National Clinical Guideline Centre Chronic kidney disease in adults in primary and secondary care: clinical guideline 182. Methods, evidence and recommendations. https://www.nice.org.uk/guidance/cg182. July 2014. Accessed April 17, 2017.
25.Stevens LA, Greene T, Levey AS. Surrogate end points for clinical trials of kidney disease progression. Clin J Am Soc Nephrol. 2006;1(4):874-884. [DOI] [PubMed] [Google Scholar]
26.Lamb EJ, MacKenzie F, Stevens PE. How should proteinuria be detected and measured? Ann Clin Biochem. 2009;46(pt 3):205-217. [DOI] [PubMed] [Google Scholar]
27.Waikar SS, Wald R, Chertow GM, et al. . Validity of International Classification of Diseases, Ninth Revision, clinical modification codes for acute renal failure. J Am Soc Nephrol. 2006;17(6):1688-1694. [DOI] [PubMed] [Google Scholar]
28.Kellum JA. How can we define recovery after acute kidney injury? considerations from epidemiology and clinical trial design. Nephron Clin Pract. 2014;127(1-4):81-88. [DOI] [PubMed] [Google Scholar]
29.Wald R, Quinn RR, Luo J, et al. ; University of Toronto Acute Kidney Injury Research Group . Chronic dialysis and death among survivors of acute kidney injury requiring dialysis. JAMA. 2009;302(11):1179-1185. [DOI] [PubMed] [Google Scholar]
30.Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15(4):361-387. [DOI] [PubMed] [Google Scholar]
31.Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York, NY: Springer; 2009. [Google Scholar]
32.Sullivan LM, Massaro JM, D’Agostino RB Sr. Presentation of multivariate data for clinical use: the Framingham Study risk score functions. Stat Med. 2004;23(10):1631-1660. [DOI] [PubMed] [Google Scholar]
33.Steyerberg EW, Vickers AJ, Cook NR, et al. . Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128-138. [DOI] [PMC free article] [PubMed] [Google Scholar]
34.Austin PC, Steyerberg EW. Graphical assessment of internal and external calibration of logistic regression models by using loess smoothers. Stat Med. 2014;33(3):517-535. [DOI] [PMC free article] [PubMed] [Google Scholar]
35.Pencina MJ, D’Agostino RB Sr, D’Agostino RB Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157-172. [DOI] [PubMed] [Google Scholar]
36.Cook NR, Ridker PM. Advances in measuring the effect of individual predictors of cardiovascular risk: the role of reclassification measures. Ann Intern Med. 2009;150(11):795-802. [DOI] [PMC free article] [PubMed] [Google Scholar]
37.Hosmer DW, Lemeshow S. Applied Logistic Regression. 2nd ed New York, NY: John Wiley & Sons; 2000. [Google Scholar]
38.Tangri N, Stevens LA, Griffith J, et al. . A predictive model for progression of chronic kidney disease to kidney failure. JAMA. 2011;305(15):1553-1559. [DOI] [PubMed] [Google Scholar]
39.Haggerty JL, Reid RJ, Freeman GK, Starfield BH, Adair CE, McKendry R. Continuity of care: a multidisciplinary review. BMJ. 2003;327(7425):1219-1221. [DOI] [PMC free article] [PubMed] [Google Scholar]
40.Rennke S, Nguyen OK, Shoeb MH, Magan Y, Wachter RM, Ranji SR. Hospital-initiated transitional care interventions as a patient safety strategy: a systematic review. Ann Intern Med. 2013;158(5 pt 2):433-440. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplement 1.

eTable 1. Diagnostic and procedure codes used to identify comorbidities and procedures in the derivation, internal validation (AKDN) and external validation (ICES) cohorts

eTable 2. Predictors of advanced CKD and their frequency in the derivation, internal validation (AKDN) and external validation (ICES) cohorts

eTable 3. Frequency, and observed risk of risk categories for 5 prediction models for advanced CKD in the derivation (AKDN) cohort

eTable 4. Distribution of model predicted risk of advanced CKD (%) estimated in the derivation, internal validation (AKDN), and external validation (ICES) cohorts

eFigure 1. Calibration of the six-variable (Model 1) and reduced models (Models 2-5) in the internal validation (AKDN) cohort using locally weighted least squares regression smoother plots

eFigure 2. Calibration of the six-variable (Model 1) and reduced models (Models 2-5) in the external validation (ICES) cohort using locally weighted least squares regression smoother plots

Click here for additional data file.^{(1.5MB, pdf)}

Supplement 2.

Risk calculator for advanced chronic kidney disease following acute kidney injury

Click here for additional data file.^{(26.5KB, xlsx)}

[joi170125r1] 1.Coca SG, Singanamala S, Parikh CR. Chronic kidney disease after acute kidney injury: a systematic review and meta-analysis. Kidney Int. 2012;81(5):442-448. [DOI] [PMC free article] [PubMed] [Google Scholar]

[joi170125r2] 2.Sawhney S, Marks A, Fluck N, et al. . Post-discharge kidney function is associated with subsequent ten-year renal progression risk among survivors of acute kidney injury. Kidney Int. 2017;92(2):440-452. [DOI] [PMC free article] [PubMed] [Google Scholar]

[joi170125r3] 3.Ishani A, Nelson D, Clothier B, et al. . The magnitude of acute serum creatinine increase after cardiac surgery and the risk of chronic kidney disease, progression of kidney disease, and death. Arch Intern Med. 2011;171(3):226-233. [DOI] [PubMed] [Google Scholar]

[joi170125r4] 4.Kidney International Supplements KDIGO clinical practice guideline for acute kidney injury. http://www.kisupplements.org/issue/S2157-1716(12)X7200-9. Published March 2012. Accessed October 5, 2017.

[joi170125r5] 5.Siew ED, Peterson JF, Eden SK, et al. . Outpatient nephrology referral rates after acute kidney injury. J Am Soc Nephrol. 2012;23(2):305-312. [DOI] [PMC free article] [PubMed] [Google Scholar]

[joi170125r6] 6.Harel Z, Wald R, Bargman JM, et al. . Nephrologist follow-up improves all-cause mortality of severe acute kidney injury survivors. Kidney Int. 2013;83(5):901-908. [DOI] [PubMed] [Google Scholar]

[joi170125r7] 7.Goldstein SL, Jaber BL, Faubel S, Chawla LS; Acute Kidney Injury Advisory Group of American Society of Nephrology . AKI transition of care: a potential opportunity to detect and prevent CKD. Clin J Am Soc Nephrol. 2013;8(3):476-483. [DOI] [PubMed] [Google Scholar]

[joi170125r8] 8.James M, Bouchard J, Ho J, et al. . Canadian Society of Nephrology commentary on the 2012 KDIGO clinical practice guideline for acute kidney injury. Am J Kidney Dis. 2013;61(5):673-685. [DOI] [PubMed] [Google Scholar]

[joi170125r9] 9.Coleman EA, Berenson RA. Lost in transition: challenges and opportunities for improving the quality of transitional care. Ann Intern Med. 2004;141(7):533-536. [DOI] [PubMed] [Google Scholar]

[joi170125r10] 10.Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD). Ann Intern Med. 2015;162(10):735-736. [DOI] [PubMed] [Google Scholar]

[joi170125r11] 11.Hemmelgarn BR, Clement F, Manns BJ, et al. . Overview of the Alberta Kidney Disease Network. BMC Nephrol. 2009;10:30. [DOI] [PMC free article] [PubMed] [Google Scholar]

[joi170125r12] 12.Quan H, Sundararajan V, Halfon P, et al. . Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43(11):1130-1139. [DOI] [PubMed] [Google Scholar]

[joi170125r13] 13.Quan H, Khan N, Hemmelgarn BR, et al. ; Hypertension Outcome and Surveillance Team of the Canadian Hypertension Education Programs . Validation of a case definition to define hypertension using administrative data. Hypertension. 2009;54(6):1423-1428. [DOI] [PubMed] [Google Scholar]

[joi170125r14] 14.Hux JE, Ivis F, Flintoft V, Bica A. Diabetes in Ontario: determination of prevalence and incidence using a validated administrative data algorithm. Diabetes Care. 2002;25(3):512-516. [DOI] [PubMed] [Google Scholar]

[joi170125r15] 15.Jain AK, Cuerden MS, McLeod I, et al. . Reporting of the estimated glomerular filtration rate was associated with increased use of angiotensin-converting enzyme inhibitors and angiotensin-II receptor blockers in CKD. Kidney Int. 2012;81(12):1248-1253. [DOI] [PubMed] [Google Scholar]

[joi170125r16] 16.Juurlink D, Preyra C, Croxford R, et al. . Canadian Institute for Health Information Discharge Abstract Database: A Validation Study. Toronto, ON: Institute for Clinical Evaluative Sciences; 2006. [Google Scholar]

[joi170125r17] 17.Siew ED, Ikizler TA, Matheny ME, et al. . Estimating baseline kidney function in hospitalized patients with impaired kidney function. Clin J Am Soc Nephrol. 2012;7(5):712-719. [DOI] [PMC free article] [PubMed] [Google Scholar]

[joi170125r18] 18.Clement FM, James MT, Chin R, et al. ; Alberta Kidney Disease Network . Validation of a case definition to define chronic dialysis using outpatient administrative data. BMC Med Res Methodol. 2011;11:25. [DOI] [PMC free article] [PubMed] [Google Scholar]

[joi170125r19] 19.Levey AS, Stevens LA, Schmid CH, et al. ; CKD-EPI (Chronic Kidney Disease Epidemiology Collaboration) . A new equation to estimate glomerular filtration rate. Ann Intern Med. 2009;150(9):604-612. [DOI] [PMC free article] [PubMed] [Google Scholar]

[joi170125r20] 20.Go AS, Chertow GM, Fan D, McCulloch CE, Hsu CY. Chronic kidney disease and the risks of death, cardiovascular events, and hospitalization. N Engl J Med. 2004;351(13):1296-1305. [DOI] [PubMed] [Google Scholar]

[joi170125r21] 21.Hemmelgarn BR, Manns BJ, Lloyd A, et al. ; Alberta Kidney Disease Network . Relation between kidney function, proteinuria, and adverse outcomes. JAMA. 2010;303(5):423-429. [DOI] [PubMed] [Google Scholar]

[joi170125r22] 22.Andrassy KM. Comments on “KDIGO 2012 clinical practice guideline for the evaluation and management of chronic kidney disease.” Kidney Int. 2013;84(3):622-623. [DOI] [PubMed] [Google Scholar]

[joi170125r23] 23.Levin A, Hemmelgarn B, Culleton B, et al. ; Canadian Society of Nephrology . Guidelines for the management of chronic kidney disease. CMAJ. 2008;179(11):1154-1162. [DOI] [PMC free article] [PubMed] [Google Scholar]

[joi170125r24] 24.National Institute for Health and Care Excellence, National Clinical Guideline Centre Chronic kidney disease in adults in primary and secondary care: clinical guideline 182. Methods, evidence and recommendations. https://www.nice.org.uk/guidance/cg182. July 2014. Accessed April 17, 2017.

[joi170125r25] 25.Stevens LA, Greene T, Levey AS. Surrogate end points for clinical trials of kidney disease progression. Clin J Am Soc Nephrol. 2006;1(4):874-884. [DOI] [PubMed] [Google Scholar]

[joi170125r26] 26.Lamb EJ, MacKenzie F, Stevens PE. How should proteinuria be detected and measured? Ann Clin Biochem. 2009;46(pt 3):205-217. [DOI] [PubMed] [Google Scholar]

[joi170125r27] 27.Waikar SS, Wald R, Chertow GM, et al. . Validity of International Classification of Diseases, Ninth Revision, clinical modification codes for acute renal failure. J Am Soc Nephrol. 2006;17(6):1688-1694. [DOI] [PubMed] [Google Scholar]

[joi170125r28] 28.Kellum JA. How can we define recovery after acute kidney injury? considerations from epidemiology and clinical trial design. Nephron Clin Pract. 2014;127(1-4):81-88. [DOI] [PubMed] [Google Scholar]

[joi170125r29] 29.Wald R, Quinn RR, Luo J, et al. ; University of Toronto Acute Kidney Injury Research Group . Chronic dialysis and death among survivors of acute kidney injury requiring dialysis. JAMA. 2009;302(11):1179-1185. [DOI] [PubMed] [Google Scholar]

[joi170125r30] 30.Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15(4):361-387. [DOI] [PubMed] [Google Scholar]

[joi170125r31] 31.Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York, NY: Springer; 2009. [Google Scholar]

[joi170125r32] 32.Sullivan LM, Massaro JM, D’Agostino RB Sr. Presentation of multivariate data for clinical use: the Framingham Study risk score functions. Stat Med. 2004;23(10):1631-1660. [DOI] [PubMed] [Google Scholar]

[joi170125r33] 33.Steyerberg EW, Vickers AJ, Cook NR, et al. . Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128-138. [DOI] [PMC free article] [PubMed] [Google Scholar]

[joi170125r34] 34.Austin PC, Steyerberg EW. Graphical assessment of internal and external calibration of logistic regression models by using loess smoothers. Stat Med. 2014;33(3):517-535. [DOI] [PMC free article] [PubMed] [Google Scholar]

[joi170125r35] 35.Pencina MJ, D’Agostino RB Sr, D’Agostino RB Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157-172. [DOI] [PubMed] [Google Scholar]

[joi170125r36] 36.Cook NR, Ridker PM. Advances in measuring the effect of individual predictors of cardiovascular risk: the role of reclassification measures. Ann Intern Med. 2009;150(11):795-802. [DOI] [PMC free article] [PubMed] [Google Scholar]

[joi170125r37] 37.Hosmer DW, Lemeshow S. Applied Logistic Regression. 2nd ed New York, NY: John Wiley & Sons; 2000. [Google Scholar]

[joi170125r38] 38.Tangri N, Stevens LA, Griffith J, et al. . A predictive model for progression of chronic kidney disease to kidney failure. JAMA. 2011;305(15):1553-1559. [DOI] [PubMed] [Google Scholar]

[joi170125r39] 39.Haggerty JL, Reid RJ, Freeman GK, Starfield BH, Adair CE, McKendry R. Continuity of care: a multidisciplinary review. BMJ. 2003;327(7425):1219-1221. [DOI] [PMC free article] [PubMed] [Google Scholar]

[joi170125r40] 40.Rennke S, Nguyen OK, Shoeb MH, Magan Y, Wachter RM, Ranji SR. Hospital-initiated transitional care interventions as a patient safety strategy: a systematic review. Ann Intern Med. 2013;158(5 pt 2):433-440. [DOI] [PubMed] [Google Scholar]

PERMALINK

Derivation and External Validation of Prediction Models for Advanced Chronic Kidney Disease Following Acute Kidney Injury

Matthew T James, MD, PhD

Neesh Pannu, MD, SM

Brenda R Hemmelgarn, MD, PhD

Peter C Austin, PhD

Zhi Tan, MSc

Eric McArthur, MSc

Braden J Manns, MD, MSc

Marcello Tonelli, MD, SM

Ron Wald, MD

Robert R Quinn, MD, PhD

Pietro Ravani, MD, PhD

Amit X Garg, MD, PhD

Key Points

Question

Findings

Meaning

Abstract

Importance

Objective

Design, Setting, and Participants

Exposures

Main Outcomes and Measures

Results

Conclusions and Relevance

Introduction

Methods

Derivation and Validation Cohorts

Study Population

Figure 1. Formation of the Derivation, Internal Validation, and External Validation Cohorts Hospitalized With Acute Kidney Injury.

Study Outcome

Candidate Predictor Variables

Model Derivation

Prediction Model Performance

Results

Characteristics of the Cohorts

Table 1. Baseline Characteristics of the Derivation, Internal Validation, and External Validation Cohorts.

Prediction of Advanced Chronic Kidney Disease in the Derivation Cohort

Table 2. Advanced Chronic Kidney Disease Predictors and Performance of Models in the Derivation Cohorta.

Prediction Model Performance in the Internal Validation Cohort

Table 3. Predictive Performance of Models for Advanced Chronic Kidney Disease in Internal and External Validation Cohortsa.

Figure 2. Six-Variable Risk Index for Advanced Chronic Kidney Disease Following Hospitalization With Acute Kidney Injury.

Figure 3. Predicted vs Observed Probability of Advanced Chronic Kidney Disease by the 6-Variable Risk index in the Internal Validation and External Validation Cohorts.

Prediction Model Performance in the External Validation Cohort

Clinical Utility of the Prediction Model

Discussion

Limitations

Conclusions

References

Associated Data

Supplementary Materials

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases

Table 2. Advanced Chronic Kidney Disease Predictors and Performance of Models in the Derivation Cohort^a.

Table 3. Predictive Performance of Models for Advanced Chronic Kidney Disease in Internal and External Validation Cohorts^a.