Abstract
Objective
To evaluate the validity of the International Classification of Diseases, Tenth Revision (ICD-10) code N17x for acute kidney injury (AKI) in elderly patients in two settings: at presentation to the emergency department and at hospital admission.
Design
A population-based retrospective validation study.
Setting
Southwestern Ontario, Canada, from 2003 to 2010.
Participants
Elderly patients with serum creatinine measurements at presentation to the emergency department (n=36 049) or hospital admission (n=38 566). The baseline serum creatinine measurement was a median of 102 and 39 days prior to presentation to the emergency department and hospital admission, respectively.
Main outcome measures
Sensitivity, specificity and positive and negative predictive values of ICD-10 diagnostic coding algorithms for AKI using a reference standard based on changes in serum creatinine from the baseline value. Median changes in serum creatinine of patients who were code positive and code negative for AKI.
Results
The sensitivity of the best-performing coding algorithm for AKI (defined as a ≥2-fold increase in serum creatinine concentration) was 37.4% (95% CI 32.1% to 43.1%) at presentation to the emergency department and 61.6% (95% CI 57.5% to 65.5%) at hospital admission. The specificity was greater than 95% in both settings. In patients who were code positive for AKI, the median (IQR) increase in serum creatinine from the baseline was 133 (62 to 288) µmol/l at presentation to the emergency department and 98 (43 to 200) µmol/l at hospital admission. In those who were code negative, the increase in serum creatinine was 2 (−8 to 14) and 6 (−4 to 20) µmol/l, respectively.
Conclusions
The presence or absence of ICD-10 code N17× differentiates two groups of patients with distinct changes in serum creatinine at the time of a hospital encounter. However, the code underestimates the true incidence of AKI due to a limited sensitivity.
Keywords: Epidemiology
Article summary.
Article focus
Validation of administrative database codes is a prerequisite to their optimal use in research.
The aim of this study was to describe the validity of the International Classification of Diseases, Tenth Revision (ICD-10) code N17× for acute kidney injury (AKI) compared with a reference standard based on changes in serum creatinine.
Key messages
The ICD-10 code N17× for AKI has a moderate sensitivity and high specificity.
The sensitivity of the N17× code improves for more severe forms of AKI.
The code was successful in identifying a group of patients admitted to hospital with a median increase in serum creatinine of 98 µmol/l.
Strengths and limitations of this study
This is the first study to provide information on the diagnostic performance of ICD-10 code N17× for AKI using laboratory values as the reference standard.
It was a large population-based validation study that included serum creatinine measurements from 12 hospitals.
Future validation studies in younger patients are required.
Background
Healthcare administrative databases can provide researchers and policy makers with information on a large number of patients in an efficient manner. When using these data resources for clinical or health services research, the validity of the research depends upon the accuracy of the diagnostic and procedural codes that have been recorded.1 However, the accuracy of coding is not guaranteed because administrative databases are not primarily intended for research.2 Consequently, understanding the validity of administrative codes is a prerequisite to their optimal use in the assessment of patient outcomes.
Clinically, acute kidney injury (AKI) is characterised by an abrupt decline in the renal function that may result in disordered fluid, acid–base and electrolyte homeostasis and retention of waste products from nitrogen metabolism, such as creatinine and urea and/or a decreased urine output.3–5 Two systems for defining and quantifying the severity of AKI are widely used: the Acute Kidney Injury Network (AKIN) classification6 and the Risk-Injury-Failure-Loss-ESRD (RIFLE) criteria.7 These staging systems define AKI severity according to absolute and relative (percentage) increases in serum creatinine, a blood test universally used for indicating kidney function. While the incidence of AKI is dependent on the definition used, it is recognised that this condition is common, affecting 2–9% of patients at hospital admission.8–11 Moreover, patients who develop AKI have both poor short-term and long-term outcomes and their care is expensive.8 9 12–20
The purpose of the present study was to evaluate the accuracy of the International Classification of Diseases, Tenth Revision (ICD-10) code N17× for AKI for applications in clinical and health services research, particularly in pharmacoepidemiological studies. We compared this code against changes in serum creatinine concentration in two settings: (1) at presentation to the emergency department and (2) at hospital admission. In addition, we investigated the effect of baseline chronic kidney disease (CKD) status on the diagnostic performance of the code in the two settings. Based on the findings of a previous validation study on ICD-9 codes, we anticipated the sensitivity for ICD-10 code N17× would be low, improving with more severe definitions of AKI.8 10 21 22 Moreover, we expected higher sensitivity in patients with CKD than those without, as the former typically have larger absolute increases in serum creatinine for a given amount of AKI.
Methods
Study design and setting
We conducted a population-based retrospective validation study using Ontario's linked healthcare administrative databases and laboratory data from Southwestern Ontario. All residents receive universal access to hospital and physician services under a single provincial payer system, providing a comprehensive set of health administrative data.
Using a diagnostic test assessment framework, we obtained diagnostic performance characteristics (sensitivity, specificity, positive-predictive value and negative-predictive value) of various diagnostic coding algorithms for ICD-10 code N17×, which is defined as ‘acute renal failure’. We used changes in serum creatinine from the baseline value as the reference standard (see online supplementary table S1 for a sample two-by-two table). Moreover, we compared the change in serum creatinine between patients who were N17× code positive with those who were N17× code negative.
Our protocol was approved by the institutional review board at Sunnybrook Health Sciences Centre (Toronto, Ontario). The relevant datasets are held at the Institute for Clinical Evaluative Sciences. The reporting of this study follows guidelines set out for studies assessing diagnostic accuracy (see online supplementary table S2).23
Data sources
Patient records from the seven databases were linked using encrypted unique identifiers. We identified laboratory measurements, including serum creatinine, using a system that keeps patients’ electronic medical records (Cerner, Kansas City, Missouri, USA).24 This system contains inpatient, outpatient and the emergency department laboratory measurements for 12 Southwestern Ontario hospitals. For a subpopulation, we also obtained previous laboratory measurements from Gamma Dynacare, a provider of outpatient laboratory services to residents in Southwestern Ontario. We obtained inpatient and emergency department patient diagnostic information from the Canadian Institute of Health Information (CIHI) Discharge Abstract Database (DAD) and National Ambulatory Care Reporting System database (NACRS), respectively. We obtained information on inpatient and outpatient physician services from the Ontario Health Insurance Plan database (OHIP). We ascertained patients’ demographic information from the Ontario's Registered Persons Database (RPDB) and prescribed drug use for patients 65 years of age and older from the Ontario Drug Benefit database (ODB). These databases have been used extensively to research health outcomes and health services.25–28
Accrual of elderly patients in Two settings: at presentation to the emergency department and at hospital admission
We developed two separate cohorts. The first cohort consisted of patients with a serum creatinine measurement at presentation to the emergency department (Emergency Department Cohort). The second cohort consisted of patients with a serum creatinine measurement at hospital admission (Hospitalised Cohort). All patients had a baseline serum creatinine measurement (described below). The period of accrual was from 1 June 2003 to 30 September 2010.
We restricted cohort entry to patients 66 years of age and older to ensure at least 1 year of baseline prescription records for all patients. We excluded the following patients from both cohorts: (1) those who had end-stage renal disease (defined by the receipt of dialysis in the 120 days prior to the hospital encounter), (2) those who received a kidney or liver transplant in the 5 years prior to the hospital encounter, (3) those whose date of serum creatinine measurement (from Cerner) and the hospital encounter (from CIHI-DAD) did not align (less than 1.4% patients excluded for this reason; see online supplementary figure S1 for explanation) and (4) those without at least one serum creatinine measurement in the 7–365 days prior to the hospital encounter to serve as the baseline value. In cases where a patient had multiple baseline measurements we selected the most recent one. Additional selection criteria for each cohort are described below and illustrated in online supplementary figure S1.
We excluded patients who did not have a serum creatinine measurement in the emergency department from entry to the Emergency Department Cohort. We excluded the following patients from entry to the Hospitalised Cohort: (1) those with a hospital admission that resulted in a stay greater than 90 days (to ensure we had data for the full hospital admission to the time of discharge, that is, particularly for patients accrued in the second half of the year 2010) and (2) those without a serum creatinine measurement either in the emergency department prior to hospital admission or during the first 2 days of hospital admission. When multiple eligible hospital presentations were identified for a given patient over the study period, we randomly selected one hospital encounter for each cohort.
Within the two cohorts, we identified patients diagnosed with CKD (assessed by the presence of ICD-10 code N18x defined as ‘chronic kidney disease’) in the 5 years prior to their hospital encounter to evaluate potential differential classification of AKI in patients with baseline CKD compared with those without.
The reference standard: serum creatinine-based definitions of AKI
Measuring changes in serum creatinine is the most common method of identifying AKI in clinical practice. We adapted the reference standard used in this study from four widely used serum creatinine-based definitions of AKI: (1) AKIN Stage 1 or greater: ≥27 µmol/l (0.3 mg/dl) or 50% increase in serum creatinine concentration from the baseline, (2) RIFLE Risk: ≥1.5-fold increase in serum creatinine concentration from the baseline, (3) RIFLE Injury: ≥2-fold increase in serum creatinine concentration from the baseline and (4) RIFLE Failure: ≥3-fold increase in serum creatinine concentration from the baseline or a baseline serum creatinine concentration ≥354 µmol/l (4.0 mg/dl) with ≥44 µmol/l (0.5 mg/dl) increase from the baseline.6 7 A Roche Modular Ion Selective Electrode system (Basel, Switzerland) was used to measure serum creatinine.
For the Emergency Department Cohort, we categorised the difference between patients’ peak (highest) serum creatinine concentration at presentation to the emergency department and the baseline serum creatinine concentration into the serum creatinine-based definitions. To be classified as having AKI in the Hospitalised Cohort, patients had to have a ≥27 µmol/l (0.3 mg/dl) or 50% increase in serum creatinine concentration from their baseline value at the emergency department or during the first 2 days of hospital admission. This was done to ensure all patients classified as having AKI by the serum creatinine-based definitions manifested AKI at hospital admission rather than developing the condition de novo during the hospital stay. However, there may be a delay from the time of injury to when the peak serum creatinine concentration is realised. Thus, in those with AKI, we categorised the severity as defined by the serum creatinine-based definitions, using the peak serum creatinine concentration in either the emergency department or in the first 5 days of hospital admission.
ICD-10 coding administrative database algorithms for AKI
Following a discharge from the hospital, trained coders review all charts to record appropriate diagnosis codes and their associated attributes. The coders follow the Canadian Coding Standards developed by CIHI.29 According to CIHI's guidelines, the coders are not permitted to interpret laboratory measurements, but can record a condition based on laboratory findings if the physician documents the condition as diagnosed in the patient's chart. For ambulatory care records (included in CIHI-NACRS), coders are allowed to include up to 10 diagnoses per visit. The first diagnosis listed is the main problem for the patient's visit that required evaluation and/or treatment or management as determined by the physician at the end of the visit. For hospitalisation records (included in CIHI-DAD), coders may record up to 25 conditions using ICD-10 diagnostic codes. Additionally, they must indicate diagnosis type ‘M’ for the condition that was most responsible for the greatest portion of the length of stay or used the greatest amount of resources. They may also indicate diagnosis type ‘1’ for any condition that existed prior to the admission and was treated during the hospital stay.29
In this study, we tested two unique algorithms to identify patients with AKI at presentation to the emergency department and three unique algorithms to identify patients at hospital admission. Each of these algorithms used the ICD-10 code N17×, but varied the possible diagnosis types. In the Emergency Department Cohort, we examined the code: (1) as the main problem (referred to as ‘main diagnosis’) or (2) in any of the 10 potential diagnostic fields with any diagnosis type (referred to as ‘all diagnoses’). In the Hospitalised Cohort, we examined the code: (1) as the diagnosis type of ‘M’ (most responsible; referred to as the ‘most responsible diagnosis’), (2) as the diagnosis type of ‘1’ (pre-admit comorbidity; referred to as ‘admission diagnosis’) or (3) in any one of the 25 potential diagnosis fields with any diagnosis type (referred to as ‘all diagnoses’).
Statistical analysis
We used descriptive statistics to summarise demographic characteristics, comorbidities, prescription drug claim information and baseline laboratory measurements for patients in both settings. We calculated sensitivity, specificity, positive-predictive value and negative-predictive value for each diagnostic coding algorithm (formulas presented in online supplementary table S1). We calculated 95% CI for single proportions using the Wilson Score method.30 We performed these calculations against the four reference standard definitions. We expressed the changes in patient serum creatinine concentration from the baseline as medians with IQR and we compared the means using the Mann-Whitney test. We conducted all analyses using the SAS (Statistical Analysis Software) V.9.2 (SAS Institute Incorporated, Cary, North Carolina, USA, 2008).
Results
We identified a total of 36 049 patients for the Emergency Department Cohort and 38 566 patients for the Hospitalised Cohort. Baseline characteristics of the two cohorts are presented in table 1, including the proportion of patients who satisfied different definitions of AKI. For example, 294 (0.8%) and 567 (1.5%) patients satisfied the RIFLE Injury definition of AKI (defined by ≥2-fold increase in serum creatinine concentration), in each cohort, respectively.
Table 1.
Baseline characteristics for patients in the Emergency Department and Hospitalised Cohorts
| Emergency Department Cohort (n=36049) | Hospitalised Cohort (n=38566) | |
|---|---|---|
| Demographics | ||
| Median age (IQR), years | 77 (72–83) | 76 (71–82) |
| Women (n (%)) | 19262 (53.4) | 19070 (49.4) |
| Income Quintile(n (%)) | ||
| One (lowest) | 7678 (21.3) | 8027 (20.8) |
| Two | 7306 (20.3) | 7765 (20.1) |
| Three (middle) | 7062 (19.6) | 7654 (19.8) |
| Four | 6110 (16.9) | 6797 (17.6) |
| Five (highest) | 7301 (20.3) | 7816 (20.3) |
| Year of Cohort Entry(n (%)) | ||
| 2003–2004 | 3648 (10.1) | 6733 (17.5) |
| 2005–2006 | 8348 (23.2) | 9256 (24.0) |
| 2007–2008 | 11954 (33.2) | 11380 (29.5) |
| 2009–2010 | 12099 (33.6) | 11197 (29.0) |
| Rural location(n (%)) | 5397 (15.0) | 7165 (18.6) |
| Resident in a long-term care facility(n (%)) | 1454 (4.0) | 1298 (3.4) |
| Comorbidities* (n (%)) | ||
| Chronic kidney disease† | 1526 (4.2) | 1632 (4.2) |
| Diabetes mellitus‡ | 8497 (23.6) | 8650 (22.4) |
| Peripheral vascular disease | 1137 (3.2) | 2077 (5.4) |
| Coronary artery disease§ | 16847 (46.7) | 18844 (48.9) |
| Congestive heart failure | 8860 (24.6) | 9224 (23.9) |
| Stroke/transient ischaemic attack | 1434 (4.0) | 1467 (3.8) |
| Chronic liver disease | 837 (2.3) | 1074 (2.8) |
| Medication Use* (n (%)) | ||
| Angiotensin-converting enzyme inhibitor | 13781 (38.2) | 14859 (38.5) |
| Angiotensin-receptor blocker | 6540 (18.1) | 6514 (16.9) |
| Potassium sparing diuretic | 3643 (10.1) | 3949 (10.2) |
| Non-potassium sparing diuretic | 16308 (45.2) | 17145 (44.5) |
| Calcium channel blocker | 11785 (32.7) | 12553 (32.5) |
| β-Adrenergic antagonist | 13646 (37.9) | 14662 (38.0) |
| Statins | 15706 (43.6) | 16602 (43.0) |
| NSAIDs (excluding aspirin) | 6520 (18.1) | 7761 (20.1) |
| Anticonvulsants | 2297 (6.4) | 2244 (5.8) |
| Antidepressants | 9187 (25.5) | 8938 (23.2) |
| Antipsychotics | 1883 (5.2) | 1692 (4.4) |
| Benzodiazepines | 9035 (25.1) | 9414 (24.4) |
| Antineoplastics | 2217 (6.1) | 2377 (6.2) |
| Thyroid hormone | 6172 (17.1) | 6150 (15.9) |
| Baseline laboratory measurements¶ | ||
| Serum creatinine concentration, µmol/L, median (IQR) | 91 (75–113) | 90 (75–114) |
| eGFR ml/min/1.73 m2,** median (IQR) | 61 (46–75) | 62 (47–77) |
| eGFR category(n (%)) | ||
| ≥60 ml/min/1.73 m2 | 18382 (51.0) | 20716 (53.7) |
| 45–59 ml/min/1.73 m2 | 9043 (25.1) | 9011 (23.4) |
| 30–44 ml/min/1.73m2 | 5622 (15.6) | 5633 (14.6) |
| 15–29 ml/min/1.73m2 | 2415 (6.7) | 2537 (6.6) |
| <15 ml/min/1.73m2 | 587 (1.6) | 669 (1.7) |
| Urine dipstick protein(n (%)) | ||
| negative | 4186 (84.0) | 3252 (81.4) |
| 0.3g/l | 415 (8.3) | 409 (10.2) |
| 1.0g/l | 296 (5.9) | 257 (6.4) |
| ≥3.0g/l | 87 (1.7) | 79 (2.0) |
| Serum sodium concentration, mmol/l, median (IQR) | 139 (137–142) | 139 (137–141) |
| Serum potassium concentration, mmol/l, median (IQR) | 4.0 (4.0–5.0) | 4.0 (4.0–5.0) |
| AKI definitions for all patients(n (%)) | ||
| AKIN Stage 1 or greater | 5312 (14.7) | 6879 (17.8) |
| RIFLE Risk | 473 (1.3) | 884 (2.3) |
| RIFLE Injury | 294 (0.8) | 567 (1.5) |
| RIFLE Failure | 527 (1.5) | 920 (2.4) |
| AKI definitions for patients with CKD† (n (%)) | ||
| AKIN Stage 1 or greater | 524 (34.3) | 644 (39.5) |
| RIFLE Risk | 25 (1.6) | 65 (4.0) |
| RIFLE Injury | 12 (0.8) | 41 (2.5) |
| RIFLE Failure | 154 (10.1) | 246 (15.1) |
*Comorbidities and medication usage in the 5 and 6 months preceding the hospital encounter were considered, respectively.
†CKD was assessed by the ICD-10 code N18×, defined as ‘chronic kidney disease’.
‡Diabetes mellitus was assessed by the diabetic medication use in the previous 6 months.
§Coronary artery disease includes the receipt of coronary artery bypass graft surgery, percutaneous coronary intervention and diagnoses of angina.
¶The baseline measurements for serum creatinine were taken at a median (IQR) of 102 (41–204) and 39 (16–128) days prior to the hospital encounter for the Emergency Department Cohort and the Hospitalised Cohort, respectively. Baseline urine protein and serum sodium and potassium were available for a subset of patients. Emergency Department cohort: A total of 4984, 29 746 and 30 040 patients had a baseline urine protein and serum sodium and potassium measurement available in the 7 to 365 days prior to the index date, respectively. Hospitalised cohort: A total of 3997, 34 407 and 34 538 patients had a baseline urine protein and serum sodium and potassium measurements available in the 7–365 days prior to the index date, respectively
**eGFR was calculated using the CKD-Epi equation.
CKD-Epi equation:141×min((serum creatinine in μmol/L/88.4)/κ, 1)α×max((serum creatinine in μmol/L/88.4)/κ, 1)−1.209×0.993Age×1.018 (if female)×1.159 (if an African–American) κ=0.7 for females and 0.9 for males, α=−0.329 for females and −0.411 for males, min=the minimum of Scr/κ or 1, max=the maximum of Scr/κ or 1. Racial information was not available in our data sources and all patients were assumed not to be of a non African–Canadian race. This was a reasonable assumption; as of 2006, African–Canadians represented less than 7% of the Ontario population. Source: http://www12.statcan.ca/census-recensement/2006/dp-pd/hlt/97-562/index.cfm?Lang=E
eGFR, estimated glomerular filtration rate.
The diagnostic performance of the various coding algorithms is presented in table 2. For both types of hospital encounters, ‘all diagnoses’ was the best performing ICD-10 N17× coding algorithm. At presentation to the emergency department, the sensitivity of the ICD-10 code for the RIFLE Injury definition of AKI (≥2-fold increase in serum creatinine concentration from the baseline) was 37.4% (95% CI 32.1% to 43.1%).
Table 2.
Diagnostic performance characteristics of three different algorithms for ICD-10 code N17× using four different serum creatinine-based definitions of AKI as the reference standard
| Diagnostic performance characteristics (95% CI) |
|||
|---|---|---|---|
| Diagnostic coding algorithm | Definition | Emergency Department Cohort | Hospitalised Cohort |
| All diagnoses | AKIN Stage 1 | Sn=7.2 (6.6 to 8.0) | Sn=21.8 (20.9 to 22.8) |
| Sp=99.9 (99.8 to 99.9) | Sp=98.4 (98.2 to 98.5) | ||
| PPV=90.4 (87.2 to 92.8) | PPV=74.2 (72.3 to 76.1) | ||
| NPV=86.2 (85.8 to 86.5) | NPV=85.3 (84.9 to 85.6) | ||
| RIFLE Risk | Sn=30.4 (26.5 to 34.7) | Sn=56.4 (53.2 to 59.7) | |
| Sp=99.2 (99.1 to 99.3) | Sp=96.0 (95.8 to 96.2) | ||
| PPV=33.8 (29.5 to 38.4) | PPV=24.7 (22.8 to 26.6) | ||
| NPV=99.1 (99.0 to 99.2) | NPV=98.9 (98.8 to 99.1) | ||
| RIFLE Injury | Sn=37.4 (32.1 to 43.1) | Sn=61.6 (57.5 to 65.5) | |
| Sp=99.1 (99.0 to 99.2) | Sp=95.6 (95.4 to 95.8) | ||
| PPV=25.8 (21.9 to 30.2) | PPV=17.3 (15.7 to 19.0) | ||
| NPV=99.5 (99.4 to 99.6) | NPV=99.4 (99.3 to 99.5) | ||
| RIFLE Failure | Sn=30.2 (26.4 to 34.2) | Sn=59.1 (55.9 to 62.3) | |
| Sp=99.2 (99.2 to 99.3) | Sp=96.1 (95.9 to 96.3) | ||
| PPV=37.3 (32.9 to 42.0) | PPV=26.9 (25.0 to 28.9) | ||
| NPV=99.0 (98.9 to 99.1) | NPV=99.0 (98.9 to 99.1) | ||
| Main diagnosis/most responsible diagnosis | AKIN Stage 1 | Sn=4.1 (3.6 to 4.6) | Sn=5.1 (4.6 to 5.7) |
| Sp=100.0 (99.9 to 100.0) | Sp=99.9 (99.8 to 99.9) | ||
| PPV=94.7 (91.0 to 97.0) | PPV=90.7 (87.4 to 93.2) | ||
| NPV=85.8 (85.4 to 86.1) | NPV=82.9 (82.5 to 83.3) | ||
| RIFLE Risk | Sn=21.4 (17.9 to 25.3) | Sn=18.7 (16.2 to 21.4) | |
| Sp=99.6 (99.6 to 99.7) | Sp=99.4 (99.3 to 99.5) | ||
| PPV=44.5 (38.2 to 51.0) | PPV=42.5 (33.7 to 47.5) | ||
| NPV=99.0 (98.9 to 99.1) | NPV=98.1 (98.0 to 98.3) | ||
| RIFLE Injury | Sn=27.9 (23.1 to 33.3) | Sn=22.8 (19.5 to 26.4) | |
| Sp=99.6 (99.6 to 99.7) | Sp=99.3 (99.2 to 99.4) | ||
| PPV=36.1 (30.2 to 42.6) | PPV=33.3 (28.7 to 38.1) | ||
| NPV=99.4 (99.3 to 99.5) | NPV=98.9 (98.7 to 99.0) | ||
| RIFLE Failure | Sn=22.0 (18.7 to 25.7) | Sn=22.9 (20.3 to 25.8) | |
| Sp=99.6 (99.5 to 99.7) | Sp=99.5 (99.5 to 99.6) | ||
| PPV=51.1 (44.6 to 57.5) | PPV=54.4 (49.4 to 59.3) | ||
| NPV=98.9 (98.7 to 99.0) | NPV=98.1 (98.0 to 98.3) | ||
| Admission diagnosis | AKIN Stage 1 | n/a | Sn=15.8 (15.0 to 16.7) |
| Sp=99.2 (99.1 to 99.3) | |||
| PPV=81.6 (79.4 to 83.6) | |||
| NPV=84.5 (84.1 to 84.8) | |||
| RIFLE Risk | Sn=43.1 (39.9 to 46.4) | ||
| Sp=97.5 (97.3 to 97.6) | |||
| PPV=28.5 (26.2 to 31.0) | |||
| NPV=98.6 (98.5 to 98.8) | |||
| RIFLE Injury | Sn=48.3 (44.2 to 52.4) | ||
| Sp=97.2 (97.0 to 97.4) | |||
| PPV=20.5 (18.4 to 22.8) | |||
| NPV=99.2 (99.1 to 99.3) | |||
| RIFLE Failure | Sn=47.4 (44.2 to 50.6) | ||
| Sp=97.6 (97.5 to 97.8) | |||
| PPV=32.6 (30.2 to 35.2) | |||
| NPV=98.7 (98.6 to 98.8) | |||
All values are presented as percentages (%).
To convert serum creatinine from μmol/l to mg/dl divide by 88.4.
AKI, acute kidney injury; ICD-10, International Classification of Diseases, tenth revision; NPV, negative predictive value; PPV, positive predictive value; Sn, sensitivity; Sp, specificity; n/a, not applicable.
Sensitivities were higher at hospital admission than at presentation to the emergency department for all four serum creatinine-based definitions of AKI. For example, at hospital admission, the sensitivity of the code for the RIFLE Injury definition was 61.6% (95% CI 57.5% to 65.5%). The sensitivity of the code improved for more severe definitions of AKI, peaking at the RIFLE Injury definition. There was no substantial difference in specificity for both types of hospital encounters, with values greater than 95% in both settings. The positive-predictive value of the code decreased for more severe definitions of AKI, with a nadir at the RIFLE Injury definition in both settings.
The absolute change in serum creatinine (the peak value—the baseline value) and relative change in serum creatinine ((the peak value—the baseline value)/the baseline value)) for patients with hospital encounters who were positive and negative for the ICD-10 N17× code are presented in table 3. When considering the ‘all diagnoses’ algorithm, 1.2% of patients at presentation to the emergency department and 5.2% of patients at hospital admission were code positive for AKI.
Table 3.
Change in serum creatinine concentration from the baseline in all patients with and without the ICD-10 N17× code for AKI (referred to as code positive and code negative)
| Emergency department cohort |
Hospitalised cohort |
||||||
|---|---|---|---|---|---|---|---|
| Absolute change (µmol/l) |
Relative change (%)* |
Absolute change (µmol/l) |
Relative change (%)* |
||||
| Diagnostic coding algorithm | Code | N | Median (IQR) | N | Median (IQR) | ||
| All diagnoses | + | 426 | 133 (62 to 288) | 87 (43 to 204) | 2023 | 98 (43 to 200) | 69 (28 to 153) |
| − | 35623 | 2 (−8 to 14) | 2 (−9 to 15) | 36 543 | 6 (−4 to 20) | 7 (−5 to 22) | |
| Main diagnosis/most responsible diagnosis | + | 227 | 187 (89 to 383) | 128 (62 to 295) | 388 | 196 (93 to 396) | 121 (49 to 275) |
| − | 35822 | 2 (−8 to 14) | 2 (−9 to 16) | 38 178 | 7 (−4 to 22) | 7 (−4 to 24) | |
| Admission diagnosis | + | n/a | 1366 | 114 (39 to 187) | 75 (30 to 169) | ||
| − | 37 230 | 6 (−4 to 21) | 7 (−4 to 23) | ||||
Both absolute and relative changes in serum creatinine were significantly different between code positive and code negative patients in both types of hospital encounters (all p values <0.001) (means presented in box plot; online supplementary figures S2 and 3)
*((peak serum creatinine—baseline serum creatinine)/baseline serum creatinine)).
To convert serum creatinine from μmol/l to mg/dl divide by 88.4.
AKI, acute kidney injury; CKD, chronic kidney disease; ICD-10, International Classification of Diseases, tenth revision; N, number; +, code positive; −, code negative; n/a, not applicable.
The median (IQR) absolute change in serum creatinine concentration for code positive patients was 133 (62 to 288) μmol/l and 98 (43 to 200) μmol/l in each setting, respectively. The change for code negative patients was 2 (−8 to 14) μmol/l and 6 (−4 to 20) μmol/l in each setting, respectively.
When expressed in relative terms, the median (IQR) change for code positive patients was 87 (43 to 204)% and 69 (28 to 153)% in each setting, respectively. The relative change for code negative patients was 2 (−9 to 15)% and 7 (−5 to 22)% in each setting, respectively. In both settings, the difference in the mean absolute and relative change in serum creatinine between code-positive and code-negative patients was highly statistically significant (p<0.001).
The diagnostic performance of the ‘all diagnoses’ algorithm at hospital admission in patients with and without CKD is presented in table 4. The sensitivity of the ICD-10 code for AKIN Stage 1 or greater, RIFLE Risk and RIFLE Injury definitions were higher in patients with CKD than those without CKD. For example, the sensitivity of the code for the RIFLE Injury definition was 75.6% (95% CI 60.7% to 86.2%) in patients with CKD and 60.5% (95% CI 56.2% to 64.5%) in patients without CKD.
Table 4.
Diagnostic performance characteristics of the ICD-10 N17x code in hospitalised patients with and without CKD using serum creatinine-based definitions of AKI as the reference standard*
| Diagnostic performance characteristics (95% CI) |
||
|---|---|---|
| Definition | Patients with CKD | Patients without CKD |
| AKIN Stage 1 | Sn=35.6 (32.0 to 39.3) | Sn=20.4 (19.4 to 21.4) |
| Sp=92.0 (90.2 to 93.5) | Sp=98.6 (98.4 to 98.7) | |
| PPV=74.4 (69.2 to 78.9) | PPV=74.2 (72.1 to 76.2) | |
| NPV=68.7 (66.1 to 71.1) | NPV=85.9 (85.5 to 86.3) | |
| RIFLE Risk | Sn=76.9 (65.4 to 85.5) | Sn=54.8 (51.4 to 58.2) |
| Sp=83.5 (81.6 to 85.3) | Sp=96.5 (96.3 to 96.7) | |
| PPV=16.2 (12.5 to 20.8) | PPV=26.2 (24.2 to 28.3) | |
| NPV=98.9 (98.1 to 99.3) | NPV=98.9 (98.8 to 99.1) | |
| RIFLE Injury | Sn=75.6 (60.7 to 86.2) | Sn=60.5 (56.2 to 64.5) |
| Sp=82.6 (80.7 to 84.4) | Sp=96.2 (96.0 to 96.4) | |
| PPV=10.1 (7.2 to 13.9) | PPV=18.5 (16.8 to 20.5) | |
| NPV=99.2 (98.6 to 99.6) | NPV=99.4 (99.3 to 99.5) | |
| RIFLE Failure | Sn=48.4 (42.2 to 54.6) | Sn=63.1 (59.4 to 66.6) |
| Sp=86.4 (84.5 to 88.1) | Sp=96.4 (96.3 to 96.6) | |
| PPV=38.6 (33.4 to 44.2) | PPV=24.8 (22.8 to 26.9) | |
| NPV=90.4 (88.7 to 91.9) | NPV=99.3 (99.2 to 99.4) | |
All values are presented as percentages (%).
*The ICD-10 N17× coding algorithm considered is all diagnoses.
AKI, acute kidney injury; CKD, Chronic kidney disease; ICD-10, International Classification of Diseases, tenth revision; NPV, negative predictive value; PPV, positive predictive value; Sn, sensitivity; Sp, specificity.
The code demonstrated the highest sensitivity for the RIFLE Risk definition in patients with CKD and the RIFLE Failure definition in patients without CKD. The specificities of the code were lower in patients with CKD than those without CKD for all four definitions. For example, the specificity of the code for the RIFLE Injury definition was 82.6% (95% CI 80.7% to 84.4%) in patients with CKD and 96.2% (95% CI 96.0% to 96.4%) in patients without CKD.
The absolute and relative changes in serum creatinine at hospital admission in patients with and without CKD who were code positive and code negative are presented in table 5. When considering the ‘all diagnoses’ algorithm, a total of 18.9% of patients with CKD and 4.6% of patients without CKD were code positive for AKI.
Table 5.
Change in serum creatinine concentration from the baseline in hospitalised patients with and without CKD where ICD-10 code N17× did and did not indicate AKI (referred to as code positive and code negative)*
| Patients with CKD |
Patients without CKD |
|||||
|---|---|---|---|---|---|---|
| Absolute change (µmol/l) |
Relative change (%)† |
Absolute change (µmol/l) |
Relative change (%)† |
|||
| Code | N | Median (IQR) | N | Median(IQR) | ||
| + | 308 | 108 (48 to 215) | 53 (20 to 104) | 1715 | 95 (43 to 197) | 72 (29 to 161) |
| − | 1324 | 16 (−8 to 51) | 9 (−4 to 26) | 35 219 | 6 (−4 to 19) | 6 (−5 to 22) |
Code positive and code negative patients were significantly different (all p-values <0.001).
Relative changes in patients with and without CKD were statistically different (p<0.0001).
*The ICD-10 N17× coding algorithm considered is all diagnoses.
†((peak serum creatinine—baseline serum creatinine)/baseline serum creatinine)).
To convert serum creatinine from μmol/l to mg/dl divide by 88.4.
CKD, chronic kidney disease; ICD-10, International Classification of Diseases, tenth revision; N, number; +, code positive; −, code negative; n/a, not applicable.
The median (IQR) absolute change in serum creatinine concentration in patients with CKD who were code positive was 108 (48 to 215) μmol/l and in patients without CKD who were code positive was 95 (43 to 197) μmol/l. The difference in the absolute change in serum creatinine between patients with and without CKD who were AKI code positive was not significantly different (p=0.910). The median (IQR) absolute change in patients with CKD who were code negative was 16 (−8 to 51) μmol/l and in patients without CKD who were code negative was 6 (−4 to 19) μmol/l.
When expressed in relative terms, the median (IQR) change in serum creatinine in patients with CKD who were code positive was significantly lower than in patients without CKD who were code positive (53 (20 to 104)% vs 72 (29 to 161)%; p<0.0001). The median (IQR) relative change in patients with CKD who were code negative was 9 (−4 to 26)% and in patients without CKD who were code negative was 6 (−5 to 22)%. For both patients with and without CKD, the difference in the mean absolute and relative changes in serum creatinine between code positive and negative patients was highly statistically significant (p<0.001).
Discussion
In this population-based retrospective validation study, we evaluated the diagnostic performance of ICD-10 code N17× for AKI. We discovered that the best performing coding algorithm both at presentation to the emergency department and at hospital admission was when the code was expressed as ‘all diagnoses’ (ie, when the code was indicated by any diagnosis type).
At hospital admission, the poorest performing coding algorithm was the ‘most responsible diagnosis’. A likely explanation for this is that AKI frequently presents as a complication of other conditions requiring hospital admission, such as acute myocardial infarction or sepsis.5 AKI is rarely the primary reason for hospital admission so would less likely be coded as the ‘most responsible diagnosis’.
The ICD-10 code demonstrated modest sensitivity for the serum creatinine-based definitions of AKI, indicating its inability to identify a portion of patients who experienced a clinically significant increase in serum creatinine. Of the patients who satisfied the RIFLE Injury definition of AKI from their increase in serum creatinine, 61.6 (95% CI 57.5 to 65.5)% of them were code positive AKI when the ICD-10 code was expressed as ‘all diagnoses’. However, the code demonstrated high specificity for all four serum creatinine-based definitions. Of the patients who did not satisfy the RIFLE Injury definition of AKI, 95.6 (95% CI 95.4 to 95.8)% of them were code negative for AKI. These characteristics of the ICD-10 AKI code are similar to those described in ICD-9 code validation studies that used serum creatinine-based definitions as the reference standard.8 10 21 22 The use of a serum creatinine-based definition of AKI as the reference standard is preferred over a chart review, recognising the latter have appeared in a number of previous validation studies.31–35
The ICD-10 code showed improved sensitivity for more severe definitions of AKI and was the highest for the RIFLE Injury definition. This characteristic of the ICD-10 code is similar to that of ICD-9 codes for AKI.21 A greater increase in serum creatinine reflects more severe AKI, is more likely to be documented in the medical chart and thus detected by coders.
The positive predictive value of the ICD-10 code decreased for more severe definitions of AKI and was the lowest for the RIFLE Injury definition. This fluctuation in the positive predictive values can in part be attributed to the varying prevalence of the four serum creatinine-based definitions.
At both types of hospital encounters, patients who were code positive had a significantly higher increase in serum creatinine than those who were code negative (p<0.001). In other words, the code does successfully differentiate between two groups of patients with and without distinct changes in serum creatinine.
We also assessed the diagnostic performance of the ICD-10 code in subgroups of patients with and without CKD prior to the hospital encounter. For all definitions, with the exception of RIFLE Failure, the code demonstrated higher sensitivity in patients with CKD than those without CKD. However, the specificity of the code was lower in patients with CKD than those without CKD (for all definitions of AKI). The latter finding suggests a portion of patients with stable CKD are misclassified as having AKI at their hospital encounter. For example, clinicians may not have access to patients’ baseline serum creatinine measurements, or may make an AKI diagnosis without investigating the baseline measurements. In such cases, an elevated serum creatinine concentration at hospital presentation that is no different from the baseline value may still be misdiagnosed as AKI.
Among patients with CKD, presence or absence of the ICD-10 AKI code was also able to differentiate between two groups of patients with distinct changes in their serum creatinine from the baseline value. Although the difference in mean absolute change in serum creatinine was no different in patients with and without CKD who had a positive AKI code, the relative change was lower in patients with CKD than those without CKD. This is consistent with physicians defining AKI more commonly in absolute rather than in relative terms. With a given absolute increase in serum creatinine, the relative increase in serum creatinine is lower in patients with CKD than those without CKD.
Our study has several strengths. To our knowledge, it is the first study to provide information on the diagnostic performance of the ICD-10 code for AKI using serum creatinine-based definitions of AKI as the reference standard.8 10 21 22 We assessed different diagnostic coding algorithms of the ICD-10 code at both presentation to the emergency department and at hospital admission. Moreover, we evaluated the diagnostic performance of the code in subgroups of patients with and without CKD prior to the hospital encounter. We studied patients from 12 hospitals across Southwestern Ontario with representation from both academic and community care centres. This helped minimise a selection bias. The large number of patients resulted in good precision for the estimates provided in the study.
Our study does have some limitations. We evaluated the validity of the ICD-10 code for AKI in patients 66 years of age and older. These findings should generalise well to elderly patients, a segment of the population at a high risk of AKI.11 36 The findings are also useful for pharmacoepidemiological studies using Ontario's healthcare administrative databases, where prescription information on Ontario residents aged 65 and older is available from the universal drug benefit plan. However, future validation studies in younger patients are needed. Moreover, we did not know the degree to which patients with AKI were symptomatic from diminished kidney function or the indication that prompted presentation to the emergency department or hospital admission.
It is important to acknowledge that for the definitions of AKI used in this study, we adapted the serum creatinine-based component of the AKIN and RIFLE classification systems. The AKIN and RIFLE classification systems recommend using both serum creatinine and urine output measurements in determining the presence and severity of AKI.6 7 In addition, it is recommended that the AKIN classification is applied only after an optimal state of hydration is achieved.6 However, urine output measurement and hydration status were not available in the data sources used in the study. In reality, the accuracy of bedside urine output measurement is notoriously poor outside of intensive care settings with an indwelling catheter. Nonetheless, the change in serum creatinine is a widely used measure of kidney function in clinical settings. Moreover, the serum creatinine-based component has been solely used to identify patients with AKI using the AKIN and RIFLE classification systems in previous studies.37 38
The median (IQR) period between the baseline serum creatinine measurements and the hospital encounter was 102 (41–204) days for patients who presented to the emergency department and 39 (16–128) days for patients admitted to the hospital. While these are reasonable baseline measurements, the AKIN and RIFLE classification systems require the change in serum creatinine to occur within 48 h and within 7 days, respectively.6 7 Although it is likely that serum creatinine changes occurred just prior to the hospital encounter, we cannot say this with complete certainty given the absence of available measurements during this period.
Finally, we could not examine the validity of outpatient claims for AKI in this study. However, the diagnostic performance of outpatient claims in our jurisdiction is notoriously poor. Nonetheless, emergency department and hospital inpatient records hold information on more severe forms of AKI, which are of particular interest to clinicians, researchers and policymakers. Moreover, we recognise that we did not capture those patients who may have had severe forms of AKI, but did not present to the emergency department or hospital, or those who presented, but did not have their serum creatinine measured. However, the latter situation is unlikely given that serum creatinine measurements are a standard laboratory test for most patients who present to a hospital encounter for acute medical care.
Conclusion
Although the use of healthcare administrative databases for clinical or healthcare services research has several merits, there are inherent limitations, including the accuracy with which certain conditions (ie, AKI) can be identified. The sensitivity of ICD-10 code N17× for AKI was limited, particularly for less-severe definitions. This results in an underestimation of the true incidence of the condition. Nonetheless, the presence or absence of the ICD-10 code successfully differentiates two groups of elderly patients with and without distinct increases in serum creatinine from baseline values at the time of a hospital encounter. The results from this study guide's judicious use of ICD-10 code N17× in future research using large healthcare administrative databases.
Supplementary Material
Acknowledgments
We thank Barbara Jones, Jeff Lamond and the late Milton Haines for their help in providing access to the Gamma-Dynacare laboratory data. We thank the team at London Health Sciences Centre, St. Joseph's Health Care and the Thames Valley Hospitals for providing access to the Cerner laboratory data.
Footnotes
Contributors: YJH participated in the study coordination and study design and analysis, provided interpretation of study results and draughted the manuscript. SZS participated in the study design, acquisition of data and performed the analysis. SG contributed to the study design and interpretation of study results. RW, EC and JLF contributed to the study design and provided feedback on the manuscript. AXG conceived of the study, participated in its design and interpretation and provided a feedback on the manuscript. All authors read and approved the final manuscript.
Funding: This study was supported by the Canadian Institutes of Health Research and the Institute for Clinical Evaluative Sciences (ICES), the latter which is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). Gandhi was supported by a Canadian Institutes of Health Research Drug Safety and Effectiveness Cross-Disciplinary Training Program Award. Dr Garg was supported by a Clinician Scientist Award from the Canadian Institutes of Health Research.
Competing interests: None.
Data sharing statement: We cannot share due to the protection of patients’ privacy.
References
- 1.Iezzoni LI. Assessing quality using administrative data. Ann Intern Med 1997;127:666–74 [DOI] [PubMed] [Google Scholar]
- 2.Mohammed MA, Stevens A. The value of administrative databases. BMJ 2007;334:1014–15 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Brady HR, Singer GG. Acute renal failure. Lancet 1995;346:1533–40 [DOI] [PubMed] [Google Scholar]
- 4.Lameire N, Van Biesen W, Vanholder R. Acute renal failure. Lancet 2005;365:417–30 [DOI] [PubMed] [Google Scholar]
- 5.Bellomo R, Kellum JA, Ronco C. Acute kidney injury. Lancet 2012;380:756–66 [DOI] [PubMed] [Google Scholar]
- 6.Mehta RL, Kellum JA, Shah SV, et al. Acute kidney injury network: report of an initiative to improve outcomes in acute kidney injury. Crit Care 2007;11:R31. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Bellomo R, Ronco C, Kellum JA, et al. Acute dialysis quality initiative w. Acute renal failure—definition, outcome measures, animal models, fluid therapy and information technology needs: the second international consensus conference of the acute dialysis quality initiative (ADQI) Group. Crit Care 2004;8:R204–12 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Liangos O, Wald R, O'Bell JW, et al. Epidemiology and outcomes of acute renal failure in hospitalized patients: a national survey. Clin J Am Soc Nephrol 2006;1:43–51 [DOI] [PubMed] [Google Scholar]
- 9.Uchino S, Bellomo R, Goldsmith D, et al. An assessment of the RIFLE criteria for acute renal failure in hospitalized patients. Crit Care Med 2006;34:1913–17 [DOI] [PubMed] [Google Scholar]
- 10.Waikar SS, Curhan GC, Wald R, et al. Declining mortality in patients with acute renal failure, 1988 to 2002. J Am Soc Nephrol 2006;17:1143–50 [DOI] [PubMed] [Google Scholar]
- 11.Xue JL, Daniels F, Star RA, et al. Incidence and mortality of acute renal failure in Medicare beneficiaries, 1992 to 2001. J Am Soc Nephrol 2006;17:1135–42 [DOI] [PubMed] [Google Scholar]
- 12.Bell M, Granath F, Schön S, et al. Continuous renal replacement therapy is associated with less chronic renal failure than intermittent haemodialysis after acute renal failure. Intensive Care Med 2007;33:773–80 [DOI] [PubMed] [Google Scholar]
- 13.Chertow GM, Burdick E, Honour M, et al. Acute kidney injury, mortality, length of stay, and costs in hospitalized patients. J Am Soc Nephrol 2005;16:3365–70 [DOI] [PubMed] [Google Scholar]
- 14.Hoste EA, Clermont G, Kersten A, et al. RIFLE criteria for acute kidney injury are associated with hospital mortality in critically ill patients: a cohort analysis. Crit Care 2006;10:R73. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Hoste EA, Lameire NH, Vanholder RC, et al. Acute renal failure in patients with sepsis in a surgical ICU: predictive factors, incidence, comorbidity, and outcome. J Am Soc Nephrol 2003;14:1022–30 [DOI] [PubMed] [Google Scholar]
- 16.Metnitz PG, Krenn CG, Steltzer H, et al. Effect of acute renal failure requiring renal replacement therapy on outcome in critically ill patients. Crit Care Med 2002;30:2051–8 [DOI] [PubMed] [Google Scholar]
- 17.Ostermann M, Chang RW. Acute kidney injury in the intensive care unit according to RIFLE. Crit Care Med 2007;35:1837–43 [DOI] [PubMed] [Google Scholar]
- 18.Ricci Z, Cruz D, Ronco C. The RIFLE criteria and mortality in acute kidney injury: a systematic review. Kidney Int 2008;73:538–46 [DOI] [PubMed] [Google Scholar]
- 19.Uchino S, Kellum JA, Bellomo R, et al. Acute renal failure in critically ill patients: a multinational, multicenter study. JAMA 2005;294:813–18 [DOI] [PubMed] [Google Scholar]
- 20.Bagshaw SM, Laupland KB, Doig CJ, et al. Prognosis for long-term survival and renal recovery in critically ill patients with severe acute renal failure: a population-based study. Crit Care 2005;9:R700–9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Wald R, Liangos O, Waikar SS, et al. The identification of patients with acute renal failure (ARF) using administrative data: a validation study. J Am Soc Nephrol 2005;16:F-PO1031 [Google Scholar]
- 22.Waikar SS, Wald R, Chertow GM, et al. Validity of International Classification of Diseases, ninth revision, clinical modification codes for acute renal failure. J Am Soc Nephrol 2006;17:1688–94 [DOI] [PubMed] [Google Scholar]
- 23.Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. BMJ 2003;326:41–4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Cerner. Laboratory. http://www.cerner.com/solutions/Hospitals_and_Health_Systems/Laboratory/ (accessed 12 May 2012)
- 25.Zhao YY, Weir MA, Manno M, et al. New fibrate use and acute renal outcomes in elderly adults: a population-based study. Ann Intern Med 2012;156:560–9 [DOI] [PubMed] [Google Scholar]
- 26.Jain AK, Cuerden MS, McLeod I, et al. Reporting of the estimated glomerular filtration rate was associated with increased use of angiotensin-converting enzyme inhibitors and angiotensin-II receptor blockers in CKD. Kidney Int 2012;81:1248–53 [DOI] [PubMed] [Google Scholar]
- 27.Molnar AO, Coca SG, Devereaux PJ, et al. Statin use associates with a lower incidence of acute kidney injury after major elective surgery. J Am Soc Nephrol 2011;22:939–46 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Weir MA, Gomes T, Mamdani M, et al. Impaired renal function modifies the risk of severe hypoglycaemia among users of insulin but not glyburide: a population-based nested case-control study. Nephrol Dial Transplant 2011;26:1888–94 [DOI] [PubMed] [Google Scholar]
- 29.Canadian Institute for Health Information Canadian coding standards for Version 2012 ICD-10-CA and CCI. http://secure.cihi.ca/free_products/canadian_coding_standards_2012_e.pdf (accessed 12 May 2012)
- 30.Newcombe RG. Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat Med 1998;17:857–72 [DOI] [PubMed] [Google Scholar]
- 31.Geraci JM, Ashton CM, Kuykendall DH, et al. International Classification of Diseases, 9th revision, clinical modification codes in discharge abstracts are poor measures of complication occurrence in medical inpatients. Med Care 1997;35:589–602 [DOI] [PubMed] [Google Scholar]
- 32.Parker JP, Li Z, Damberg CL, et al. Administrative versus clinical data for coronary artery bypass graft surgery report cards: the view from California. Med Care 2006;44:687–95 [DOI] [PubMed] [Google Scholar]
- 33.Quan H, Parsons GA, Ghali WA. Assessing accuracy of diagnosis-type indicators for flagging complications in administrative data. J Clin Epidemiol 2004;57:366–72 [DOI] [PubMed] [Google Scholar]
- 34.So L, Evans D, Quan H. ICD-10 coding algorithms for defining comorbidities of acute myocardial infarction. BMC Health Serv Res 2006;6:161. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Juurlink D, Preyra C, Croxford R, et al. Canadian Institute for Health Information Discharge Abstract Database: A Validation Study. Toronto, Canada: Institute for Clinical Evaluative Sciences, 2006 [Google Scholar]
- 36.Hoste EA, Schurgers M. Epidemiology of acute kidney injury: how big is the problem? Crit Care Med 2008;36:S146–51 [DOI] [PubMed] [Google Scholar]
- 37.Bagshaw SM, Uchino S, Cruz D, et al. A comparison of observed versus estimated baseline creatinine for determination of RIFLE class in patients with acute kidney injury. Nephrol Dial Transplant 2009;24:2739–44 [DOI] [PubMed] [Google Scholar]
- 38.Thakar CV, Christianson A, Freyberg R, et al. Incidence and outcomes of acute kidney injury in intensive care units: a Veterans Administration study. Crit Care Med 2009;37:2552–8 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.