Performance and Limitations of Administrative Data in the Identification of AKI

Abstract

Background and objectives

Billing codes are frequently used to identify AKI events in epidemiologic research. The goals of this study were to validate billing code–identified AKI against the current AKI consensus definition and to ascertain whether sensitivity and specificity vary by patient characteristic or over time.

Design, setting, participants, & measurements

The study population included 10,056 Atherosclerosis Risk in Communities study participants hospitalized between 1996 and 2008. Billing code–identified AKI was compared with the 2012 Kidney Disease Improving Global Outcomes (KDIGO) creatinine-based criteria (AKI_cr) and an approximation of the 2012 KDIGO creatinine- and urine output–based criteria (AKI_{cr_uop}) in a subset with available outpatient data. Sensitivity and specificity of billing code–identified AKI were evaluated over time and according to patient age, race, sex, diabetes status, and CKD status in 546 charts selected for review, with estimates adjusted for sampling technique.

Results

A total of 34,179 hospitalizations were identified; 1353 had a billing code for AKI. The sensitivity of billing code–identified AKI was 17.2% (95% confidence interval [95% CI], 13.2% to 21.2%) compared with AKI_cr (n=1970 hospitalizations) and 11.7% (95% CI, 8.8% to 14.5%) compared with AKI_{cr_uop} (n=1839 hospitalizations). Specificity was >98% in both cases. Sensitivity was significantly higher in the more recent time period (2002–2008) and among participants aged 65 years and older. Billing code–identified AKI captured a more severe spectrum of disease than did AKI_cr and AKI_{cr_uop}, with a larger proportion of patients with stage 3 AKI (34.9%, 19.7%, and 11.5%, respectively) and higher in-hospital mortality (41.2%, 18.7%, and 12.8%, respectively).

Conclusions

The use of billing codes to identify AKI has low sensitivity compared with the current KDIGO consensus definition, especially when the urine output criterion is included, and results in the identification of a more severe phenotype. Epidemiologic studies using billing codes may benefit from a high specificity, but the variation in sensitivity may result in bias, particularly when trends over time are the outcome of interest.

Introduction

AKI is now recognized as a major public health concern; it is common, costly, and associated with exceedingly poor outcomes (1–4). Concomitant with the increase in physician awareness, the consensus definition of AKI has expanded, and milder cases (e.g., a 0.3-mg/dl increase in serum creatinine) are now considered clinically significant (5,6). AKI is often identified by hospital billing codes in epidemiologic studies (7–11), but the extent to which billing code practice has followed the consensus definition is unknown. Inconsistencies in billing codes may lead to biased estimates in clinical studies, particularly when accuracy varies with variables of interest, such as era.

Unfortunately, there is no gold standard method by which to evaluate AKI identification. No standardized procedure for event adjudication exists. Some epidemiologic studies use chart adjudication, some use automated abstraction of electronic medical records, and most do not adjudicate. The current Kidney Disease Improving Global Outcomes (KDIGO) AKI definition remains controversial, particularly with respect to the inclusion of KDIGO stage 1 (an increase in serum creatinine of 0.3 mg/dl over 48 hours, or a 50% increase in creatinine over 7 days) and the urine output criterion (urine output<0.5 ml/min per kg for >6 hours) (5,6,12). Few studies have collected urine output data; hence, the evidence supporting its incorporation in the AKI definition is substantially weaker than that for serum creatinine (6,12).

Billing code–identified AKI has poor sensitivity compared with older creatinine-based definitions, but this has not been updated with the KDIGO definition of AKI (13). Acknowledging the uncertainty regarding urine output and small changes in serum creatinine, we analyzed the performance of AKI billing codes compared with two KDIGO-based AKI definitions: AKI_cr, based on the KDIGO creatinine criteria only, AKI_{cr_uop}, based on both the KDIGO creatinine criteria and an approximation of the urine output–based criteria, and by AKI stage. In addition, we evaluated time period, patient age, race, sex, and comorbid conditions as potential modifiers of billing code sensitivity and specificity. Finally, we evaluated the case mix captured by AKI billing code, AKI_cr, and AKI_{cr_uop}, and their associations with subsequent in-hospital and 30-day mortality.

Materials and Methods

Study Population

The Atherosclerosis Risk in Communities (ARIC) study is an ongoing prospective cohort designed to study risk factors for cardiovascular disease and atherosclerosis (14). Original enrollment (1987–1989) included 15,792 individuals aged 45–64 years and recruited from four United States communities: Washington County, Maryland; Minneapolis, Minnesota; Jackson, Mississippi; and Forsyth County, North Carolina. Because of the difficulty in obtaining hospital charts from more historical periods, only participants with a hospitalization between 1996 and 2008 were included (n=10,056).

Data Source: Hospitalization Billing Codes

Vital status and intervening hospitalizations for ARIC participants are determined annually by telephone follow-up (response rate, 92% in year 20) as well as by active surveillance of community hospital discharge lists, local newspaper obituaries, state death lists, and death certificates from the Department of Vital Statistics. For each hospitalization, trained abstractors record all (or, if more than 26 codes were used, at least 26) discrete International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) discharge billing codes, including both diagnostic and procedure codes; for deaths, abstractors additionally record all ICD-10-CM (ICD, Tenth Revision, CM) codes from the death certificate. For the current study, 34,179 hospitalized events occurring between study visit 4 (February 1, 1996, to January 30, 1999) and December 31, 2008, were evaluated.

Data Source: Electronic Medical Record Creatinine and Urine Output Data

Serum creatinine and urine output data were available in an electronic medical record for 3710 hospitalizations occurring between 2002 and 2008 in a single health system in Washington County, Maryland. Advantages of this system include a single laboratory measuring both inpatient and outpatient serum creatinine and an inpatient nursing protocol mandating routine capture of fluid intake and output. The majority of hospitalizations (67% over the study period) among ARIC study participants recruited from Washington County occurred in this health system. Hospitalizations for participants who began long-term dialysis (determined by linkage with the US Renal Data System [USRDS]) >30 days before hospital discharge date were not considered AKI, irrespective of electronic medical record serum creatinine values and urine output. Kidney transplant recipients were included. In total, 1970 hospitalizations had a preceding outpatient creatinine in the year before admission, and 1839 of these had available urine output data.

Data Source: Hospital Chart Review

To evaluate kidney events occurring in all study centers and across a greater time span, 665 hospital charts were requested. Charts were selected using a random-number generator within 10 strata of ICD-9-CM codes (Supplemental Table 1), oversampling for records containing codes for kidney disease and, among those with codes for dialysis, for a participant’s first dialysis hospitalization. For each requested hospitalization, the admission history and physical, nephrology consult (if applicable), discharge summary, and inpatient laboratory measures were abstracted, deidentified, and transferred to a central location for adjudication. Urine output from these hospitalizations was not available.

Kidney Event Definitions

For billing code data, an ICD-9-CM code of 584.x or an ICD-10-CM code of N17.x in any of the abstracted codes was considered AKI (Table 1). Hospitalizations with an AKI code and a billing code for dialysis (ICD-9-CM codes: V45.1, V56, 39.95, 54.98; ICD-10-CM codes: Z99.2, Z49, Z45.2) were considered to be AKI requiring renal replacement therapy (AKI-RRT, a subset of stage 3). For electronic medical record data, AKI was defined by the KDIGO creatinine-based criteria (AKI_cr) and KDIGO combined creatinine- and urine output–based criteria (AKI_{cr_uop}), with baseline creatinine defined as the average outpatient creatinine measured 10–365 days before admission (15). The KDIGO urine output criteria were approximated conservatively as <20 ml/hr×24 hours because admission weights were not available and the interval of urine collection varied. For hospital chart review, AKI was adjudicated according to the KDIGO creatinine-based criteria, with baseline creatinine defined as the prehospitalization creatinine, determined from the following (in order of preference): nephrology consult, admission or discharge note, or lowest inpatient serum creatinine (if judged adequate by adjudicating physician).

Table 1.

Definitions of AKI by data source

Type of Review	Definition	Hospitalization Sample Size (n)
ICD-9-CM review
AKI billing codes	584.x (ICD-9-CM) or N17.x (ICD-10-CM) in any position	34,179
Chart reviewa
AKIcr	50% change from outpatient sCr, or ≥0.5-mg/dl increase if peak sCr is >4 mg/dl, or 0.3-mg/dl increase in ≤48 hr, or AKI mention and initiation of RRT	546
Electronic medical record reviewb
AKIcr	50% change from outpatient sCr, or ≥0.5-mg/dl increase if peak sCr is >4 mg/dl, or 0.3-mg/dl increase in ≤48 hr	1970
AKIcr_uop	Outpatient baseline definition above or urine output <20 ml/hr×24 hr	1839

ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification; ICD-10-CM, International Classification of Diseases, Tenth Revision, Clinical Modification; AKI_cr, AKI by the creatinine-based criteria; sCr, serum creatinine; AKI_{cr_uop}, AKI by the creatinine- or urine output–based criteria.

^aBaseline sCr was taken as prehospitalization sCr when available, determined from (in order of preference): nephrology consult, admission or discharge note, or lowest inpatient serum creatinine (if judged adequate by adjudicating physician)

^bOutpatient baseline defined as average outpatient serum creatinine 10–365 days before admission (15); ESRD cases (determined by linkage with US Renal Data System) were not considered AKI regardless of change in creatinine of urine output.

Chart Adjudication Procedure

Of the 665 charts requested, 546 (82%) were located. Charts were more likely to be available in the later years (71%, 87%, and 92% from 1996–2000, 2001–2004, and 2005–2008, respectively; P<0.001). Charts were adjudicated for the presence and stage of AKI by two independent physician reviewers (M.E.G./B.M. or M.E.G./S.W.) blinded to billing code. Initial agreement between reviewers was 91% (κ=0.82; P<0.001). After clarification and resolution of errors, consensus was reached on all charts.

Agreement between Gold Standard Sources

The agreement between chart-adjudicated AKI_cr and electronic medical record AKI_cr was 84% (κ=0.67; P<0.001) for the 36 charts with data from both sources. Rereview of discordant records revealed inconsistencies in the adjudicated baseline creatinine (preadmission creatinine was noted only through documentation from treating physician) and the baseline creatinine from the electronic medical record.

Statistical Analyses

Billing code–identified AKI was compared with AKI_cr and AKI_{cr_uop}. For comparison with chart review, analyses were adjusted for sampling technique, weighting by the inverse probability of being selected within the ICD-9-CM code strata (e.g., total number of hospitalizations within each stratum divided by number of charts reviewed in each stratum) (Table 2). For comparison with electronic medical record data, no weighting was performed because we did not sample these data. SEMs were calculated using a clustered sandwich estimator to account for potential within-participant correlation; validation was performed at a hospitalization level. Because the incidence of AKI is often compared by patient characteristics and over time, the sensitivity and specificity of AKI billing codes compared with AKI_cr within the chart review sample were evaluated by the following factors: era (1996–2002, 2003–2008), participant age (<65, ≥65 years), sex, race (white, black), baseline diabetes (yes, no; assessed at last attended clinic visit), and baseline CKD (eGFR<60 ml/min per 1.73 m² using the CKD-Epidemiology Collaboration creatinine equation [16]). Interactions were tested by logistic regression of AKI defined by billing code on the factor of interest among hospitalizations with (sensitivity) and without (specificity) AKI_cr; P<0.05 was considered to represent a statistically significant difference.

Table 2.

Chart review: International Classification of Diseases, Ninth Revision, Clinical Modification nonoverlapping strata and weights within the Atherosclerosis Risk in Communities study population

Stratum	ICD-9-CM Description	Reviewed (n)	Adjudicated AKIcr (n)	Total (n)	Weight
1	AKI only	32	29	686	21.4
2	AKI+CKD	35	33	547	15.6
3	AKI-D	22	21	31	1.4
4	AKI-D+CKD	58	54	86	1.5
5	CKD only	66	24	2927	44.3
6	CKD-Da	151	35	1041	6.9
7	Dialysis onlya	13	8	57	1.8
8	Transplant	6	2	32	5.3
9	CVD only	105	17	15,416	146.8
10	None of the above	58	10	13,356	230.3
Total		546b	233	34,179

AKI_cr, AKI by the creatinine-based criteria; AKI-D, AKI with receipt of dialysis codes; CKD-D, CKD with receipt of dialysis codes; CVD, cardiovascular disease code only.

^aCombined weight: separate weights were used for first and all other hospitalizations.

^bOut of 655 requested.

To evaluate the implications of using different definitions of AKI, the proportions experiencing in-hospital and 30-day mortality following billing code–identified AKI, AKI_cr, and AKI_{cr_uop} were evaluated in the electronic medical record data sample. The odds of in-hospital and 30-day mortality associated with each AKI definition were compared using seemingly unrelated estimation (17). All analyses were performed using Stata/SE software, version 11.2 (Stata Corp., College Station, TX).

Results

Prevalence of AKI Billing Codes Overall

There were 34,179 hospitalizations among 10,056 ARIC study participants during the study period; 1354 (4.0%) hospitalizations among 980 participants had a billing code for AKI, and 117 (0.3%) hospitalizations among 108 participants had a billing code for both AKI and dialysis (AKI-RRT). Twenty billing code–identified AKI hospitalizations (10 of which were AKI-RRT) occurred more than 30 days after the USRDS-supplied ESRD date. Overall, the number of billing codes recorded increased over time, from a median of 6 in 1996 to 8 in 2008 (maximum in all years, 26).

Validity of AKI Billing Codes Using Chart Review

Nearly half (45.4%) of the 546 reviewed hospitalizations were from the earlier study era (1996–2002), 83.0% were in participants aged 65 and older, 35.0% were in African Americans, and 48.0% were in women. Physician adjudication classified 233 cases as AKI_cr, of which 145 were stage 2–3 by KDIGO criteria (113 were AKI-RRT). Of the AKI-RRT events, 72.6% were among persons with CKD stage 3 or worse, and 69.0% of patients did not recover. In weighted analysis using adjudicated AKI_cr as the gold standard, the sensitivity of ICD-9-CM codes for AKI was 17.4%; specificity was 99.6% (Table 3, first column). Positive and negative predictive values were 92.0% and 81.8%, respectively. ICD-9-CM sensitivity improved to 40.3% if only stage 2–3 AKI was considered (negative predictive value, 99.9%). Sensitivity and specificity of AKI-RRT codes were 36.5% and 99.9%, respectively.

Table 3.

Validation of AKI billing codes compared with AKI based on Kidney Disease Improving Global Outcomes (KDIGO) creatinine criteria and KDIGO combined creatinine and urine output criteria

AKI Definition	AKIcra	AKIcrb	AKIcr_uopb
Data source	Chart review	Electronic medical record	Electronic medical record
Sample size (n)	546	1970	1839
Billing code AKI
Events (n)	233	361	609
Sensitivity (%)	17.4 (11.6 to 23.1)	17.2 (13.2 to 21.2)	11.7 (8.8 to 14.5)
Specificity (%)	99.6 (99.3 to 99.9)	98.5 (97.9 to 99.1)	98.9 (98.2 to 99.5)
PPV (%)	92.0 (85.9 to 98.2)	72.1 (62.4 to 81.8)	83.5 (75.4 to 91.7)
NPV (%)	81.8 (76.5 to 87.2)	84.1 (82.2 to 86.1)	69.3 (66.9 to 71.7)

Ranges in parentheses are 95% confidence intervals. AKI_cr, AKI by the creatinine-based criteria; AKI_{cr_uop}, AKI by the creatinine- or urine output–based criteria; PPV, positive predictive value; NPV, negative predictive value.

^aValidation measures are weighted for probability sampling.

^bWashington County site only, 2002–2008.

Validity of AKI Billing Codes in Electronic Medical Record Data

There were 1970 hospitalizations with a preceding outpatient serum creatinine value in the electronic medical record. Slightly more hospitalizations occurred in the later years (9.3% during 2002 versus 16.9% during 2008). Given the more recent time period, most hospitalized patients (92.3%) were age 65 years or older; 48.9% were women. Because hospitalizations were solely in Washington County, Maryland, <1% of the patients were black. There were 361 AKI events (18.3%) by the KDIGO creatinine criteria (AKI_cr) and 609 events (of 1839 with available urine output [33.1%]) by the KDIGO combined creatinine and urine output criteria (AKI_{cr_uop}). Sensitivity of AKI billing codes was 17.2% compared with AKI_cr and 11.7% compared with AKI_{cr_uop}; specificity was 98.5% and 98.9%, respectively (Table 3, second and third columns).

Variation in Validity over Time and by Patient Factors

The validity of AKI billing codes compared with adjudicated chart review was assessed over time and by patient sex, age, race, diabetes, and CKD status (Figure 1). Sensitivity significantly improved between 1996–2002 and 2002–2008 (from 9.7% to 24.4%; P=0.01). The sensitivity of AKI billing codes was 7.3% in persons <65 years of age and 20.8% in those age≥65 years (P=0.03); this persisted after adjustment for era. Race was not associated with billing code sensitivity (18.0% among whites versus 15.7% among blacks; P=0.7). Specificity differed only by diabetes status (99.8% among patients without diabetes versus 98.8% among patients with diabetes; P=0.03). The sensitivity of AKI-RRT codes did not appear to vary over time, age, diabetes, or CKD status, but there was a difference in sensitivity by race (27.5% among whites versus 53.1% among blacks; P=0.02).

Figure 1.

The sensitivity and specificity of AKI billing codes vary according to patient characteristics and hospitalization year. Validated against chart review. Figure depicts point estimates and 95% confidence intervals. *P<0.05.

Comparison of AKI Stage and Adverse Outcomes by Method of Identification

Among patients with AKI by billing code, 34.9% had stage 3 by the KDIGO creatinine criteria (Table 4). Far fewer had stage 3 among those with AKI_cr (19.7%) and AKI_{cr_uop} (11.5%). Similarly, in-hospital mortality was higher in billing-code AKI than AKI_cr and AKI_{cr_uop} (41.2%, 18.7%, and 12.8%, respectively) (Table 5). Trends in 30-day post-AKI mortality followed the same pattern (51.8%, 23.9%, and 18.6%), as did the odds ratios for in-hospital and 30-day post-discharge mortality (compared with non-AKI hospitalizations). In subgroup analysis, this difference was greater in the earlier era than the later era (odds ratio for in-hospital mortality for AKI billing codes, AKI_cr, and AKI_{cr_uop}, respectively, 30.2, 7.6, and 8.5 for 2002–2004 and 12.3, 9.3, and 5.9 for 2005–2008). There was no significant difference between odds ratios for AKI_cr and AKI_{cr_uop}. The cross-tabulation between AKI creatinine and urine output criteria showed the highest mortality among patients meeting both criteria, followed by those meeting creatinine criteria and then those with isolated oliguria (Supplemental Table 2).

Table 4.

AKI events identified by billing codes, serum creatinine, and combined creatinine and urine output criteria: stage (by creatinine criteria) and presence of oliguria

Variable	Billing Code AKI	AKIcr	AKIcr_uop
Stage (by creatinine criteria) (%)
No AKI	27.9	–a	43.0
Stage 1 AKI	20.9	62.3	35.3
Stage 2 AKI	16.3	18.0	10.2
Stage 3 AKI	34.9	19.7	11.5
Urine output <20 ml/hr×24 hr (%)
Oliguria	47.2	31.7	61.1

Electronic medical record sample; staging is based solely on serum creatinine criteria. AKI_cr, AKI by the creatinine-based criteria; AKI_{cr_uop}, AKI by creatinine- or urine output–based criteria.

^a0% by definition.

Table 5.

AKI events identified by billing codes, serum creatinine, and combined creatinine and urine output criteria: in-hospital and 30-day mortality

Variable	Billing Code AKI	AKIcr	AKIcr_uop
In-hospital mortality
AKI (%)	41.2	18.7	12.8
No AKI (%)	3.9	2.6	2.1
Odds ratio (95% CI)	17.1 (10.0 to 29.1)	8.6a (5.6 to 13.1)	6.8a (4.3 to 10.7)
30-day mortality
AKI (%)	51.8	23.9	18.6
No AKI (%)	8.2	7.0	6.0
Odds ratio (95% CI)	12.1 (7.4 to 19.8)	4.2a (3.0 to 5.9)	3.6a (2.6 to 4.9)

Electronic medical record sample. AKI_cr, AKI by the creatinine-based criteria; AKI_{cr_uop}, AKI by the creatinine- or urine output–based criteria; 95% CI, 95% confidence interval.

^aP<0.05 for comparison with coefficient for billing code AKI.

Discussion

This community-based study critically assessed current epidemiologic methods for identifying AKI. The sensitivity of AKI billing codes was limited (<20%), particularly with use of the combined criteria of change in creatinine or urine output to define AKI. Sensitivity estimates were slightly higher in more severe disease; yet even among patients with stage 2–3 AKI (>100% increase in serum creatinine), billing codes captured only 40% of cases. Billing codes indicating dialysis-requiring AKI, generally thought to be quite accurate (13), demonstrated high specificity (>99%) but low sensitivity (37%). Also notable was the variation in billing code sensitivity: by era and patient age for AKI in general and by race for dialysis-requiring AKI. This implies that studies investigating trends in AKI over time (or trends in AKI-RRT by race) may be biased.

Many studies—including some using serum creatinine to identify AKI—have reported that AKI incidence has increased over time (1,7,9,11,18–20), and several note that trends in AKI incidence might be affected by changes in billing practice. For example, the 2012 USRDS Annual Data Report points to discrepant trends in billing codes for AKI and AKI-RRT (the former increasing, the latter decreasing) as evidence for “code creep,” assuming that the threshold for dialysis in AKI has not changed (18). We confirm that billing practice has changed, with milder cases identified in recent years, a finding perhaps driven in part by the increased numbers of billing codes used. Applying our estimates of billing code validity to USRDS rates of billing code–identified AKI (0.2% in 1995 and 1.6% in 2007, or a 700% increase) would suggest an underestimation of rates and an overestimation of the relative increase (from 1.0% in 1995 to 4.0% in 2007, or a 300% increase) (18).

Our study expands upon previous validation studies (7,13,21–25). Waikar and colleagues reported that billing code–identified AKI had low sensitivity (35.4%) but high specificity (97.7%) compared with a 100% creatinine increase in inpatient creatinine (13). Using the current creatinine-based definition (which includes a 50% or 0.3-mg/dl increase) (6), an improved estimate of baseline creatinine, and a more geographically diverse population, we found that sensitivity was lower (17%), a difference attributable to the expanded AKI definition, since sensitivity compared with stage 2–3 AKI (>100% increase in serum creatinine) was similar (13). The sensitivity of AKI-RRT codes (36.5%) was much lower than the previous estimate (90.4%) (13), which may be due to differences in study design, such as sampling technique, or the uncertainty regarding AKI classification in patients with advanced CKD. Finally, we observed that billing codes capture a different case mix than that of KDIGO, with a higher proportion of stage 3 disease and a stronger association with short-term mortality, although these differences have attenuated over time. Trends in long-term outcomes after billing code–identified AKI should be interpreted cautiously.

To our knowledge, our study is the first to validate AKI billing codes against a definition incorporating both serum creatinine and urine output data (AKI_{cr_uop}). Compared with AKI_{cr_uop}, AKI billing codes performed poorly, but whether this stems from flawed billing practice or a flawed gold standard is uncertain. There remains debate over whether to include isolated oliguria in the AKI definition; this may translate to poor billing code capture. On the other hand, our data span the entire inpatient population and are subject to all the inaccuracies inherent in clinical data collection. We found that isolated oliguria was relatively common—even at our more conservative approximation of the KDIGO definition—similar to a study in intensive care patients (a setting where urine output measurement is thought more accurate) (26). In contrast, another intensive care study found very few cases of oliguria (defined as urine output<400 ml/d) without a concomitant rise in serum creatinine (27). The implication of isolated oliguria remains uncertain: in the intensive care study, it was associated with a higher rate of death than AKI without oliguria; in our study, the rate was lower (26). This difference may reflect error in output measurement outside the intensive care unit, or it may reflect differences in underlying rates of mortality, whereby patients in intensive care have a higher mortality before an observed change in serum creatinine. Similar to a previous study (28), however, we found that those with AKI by both creatinine and urine output criteria experienced the highest rate of in-hospital mortality, suggesting potentially useful risk implications.

Previous validation studies have not routinely adjusted for sampling frame. In our chart review study, unweighted estimates would grossly overestimate sensitivity and underestimate specificity, given the oversampling for hospitalizations with kidney events. The use of inverse probability weighting allows for generalization to the entire ARIC population. However, insofar as billing practices in the communities studied are dissimilar to those in a population of interest, our results may not be generalizable. Our electronic medical record data were also from a single center, although validation measures using these data were very similar to those obtained using chart review, which encompassed multiple hospitals and all four ARIC communities.

Our study highlights the uncertainty regarding the AKI “gold standard.” Event definitions and provider practice have evolved (5,6), and there are differences between chart review and electronic medical record abstraction. Even standardized chart review is highly dependent on provider recognition, documentation, and data ascertainment. When a diagnosis hinges on small changes in serum creatinine, the estimation of “baseline” creatinine is critical—a process, at least in the clinical setting, that is somewhat subjective (15,29–31). Using admission serum creatinine (sometimes the best available approach for a clinician) significantly underestimates AKI incidence (31), whereas using remote outpatient values may misclassify CKD progression as AKI. On the other hand, electronic medical records may have trouble distinguishing AKI events from laboratory errors or the receipt of dialysis; in addition, not all systems integrate inpatient and outpatient laboratory measures.

In summary, using a community-based sample of middle-to-older aged adults, we describe a validation study of AKI billing codes. Sensitivity was low, particularly when urine output was included in the gold standard definition, but it increased in recent years. Insofar as specificity is most important in the mitigation of study bias (32), the use of billing codes for studies of associations may be reasonable. On the other hand, the significant variability in sensitivity over time and by participant age warrants caution in other contexts.

Disclosures

M.E.G., B.M., S.W., and S.H.B. have no relevant financial relationships to disclose. Dr. Coresh has consulted for Amgen and Merck and has an investigator-initiated grant from Amgen. S.S.W. reports serving as a consultant to CVS Caremark, BioTrends Research Group, Harvard Clinical Research Institute, and Takeda; providing expert testimony for GE Heathcare and Salix; and receiving investigator-initiated grants from Otsuka, Merck, Genzyme, and Satellite Healthcare. The authors have no conflicts to report pertaining to the manuscript content.

Some of the data reported here have been supplied by the USRDS. The interpretation and reporting of these data are the responsibility of the authors and in no way should be seen as an official policy or interpretation of the US Government.