eResearch in acute kidney injury: a primer for electronic health record research

Abstract

Acute kidney injury (AKI) has a significant impact on patient morbidity and mortality as well as overall health care costs. eResearch, which integrates information technology and information management to optimize research strategies, provides a perfect platform for necessary ongoing AKI research. With the recent adoption of a widely accepted definition of AKI and near-universal use of electronic health records, eResearch is becoming an important tool in AKI research. Conducting eResearch in AKI should ideally be based on a relatively uniform methodology. This article is the first of its kind to describe a methodology for pursuing eResearch specific to AKI and includes an illustrative database example for critically ill patients. We discuss strategies for using serum creatinine and urine output in large databases to identify and stage AKI and ways to interpolate missing values and validate data. Issues specific to the pediatric population include variation in serum creatinine with growth, varied severity of illness scoring systems and medication dosage based on weight. Many of these same strategies used to optimize AKI eResearch can be applicable to real-time AKI alerts with potential integration of additional clinical variables.

INTRODUCTION

Acute kidney injury (AKI) is a significant public health issue, with an increasing incidence and significant associated deleterious effects [1]. Numerous studies have described the consequences of AKI, including prolonged hospital stay, increased health care costs, morbidity and mortality [2, 3]. Historically, AKI research has suffered from the lack of a unifying definition and suboptimal strategies to assess large cohorts [4]. After development of a consensus definition for AKI, the identification and staging has been simplified. In 2012, the Kidney Disease: Improving Global Outcomes (KDIGO) guideline adopted the RIFLE (Risk, Injury, Failure, Loss, End Stage) and AKIN (AKI Network) criteria for AKI diagnosis and staging and harmonized the criteria for both adults and children [5] (see Table 1). With a validated and near-universally accepted definition of AKI [6, 7], research in this area has grown tremendously.

Table 1.

KDIGO AKI staging

Stage	Serum creatinine	Urine output
1	1.5–1.9 times baseline OR ≥0.3 mg/dL increase	<0.5 mL/kg/h for 6–12 h
2	2.0–2.9 times baseline	<0.5 mL/kg/h for ≥12 h
3	3.0 times baseline or Increase in serum creatinine to ≥4 mg/dL or Initiation of renal replacement therapy or In patients <18 years, decrease in eGFR to <35 mL/min/1.73 m2	<0.3 mL/kg/h for ≥24 h

The stage of acute kidney injury (AKI) is defined by the highest stage from either serum creatinine or urine output criteria. eGFR, estimated glomerular filtration rate. KDIGO, Kidney Disease Improving Global Outcomes.

Within the same era, ‘big data’ research has blossomed. Big data research generally refers to a large, complex repository of data, frequently utilized for predictive abilities, refined analyses and risk stratification models [8]. The incredible growth in machine learning and computational ability in the past several decades can be utilized with eResearch, which is the use of information technology and information management to optimize research strategies [9]. The push for collaborative research and effective use of information technology has driven the adoption of eResearch [10], which is particularly useful in health care informatics.

eResearch is an optimal platform for AKI research. With a definition mapped to numerical values often readily available in the electronic health record (EHR), we are not only able to identify and stage AKI, but can also utilize additional information in the EHR to understand confounding factors like comorbidities, medication administration, hypotension, infection and demographic data. Despite standard definitions for AKI, implementation of the criteria to EHRs is not as straightforward as one might guess. The goal of this article is to discuss how to utilize eResearch and information technology in AKI-based research, focusing on the critically ill population. We will describe an illustrative database while addressing database development, AKI identification and staging, management of covariates and perspectives across adult medicine and pediatrics.

HISTORICAL CONTEXT

Much of what has been studied in AKI using EHRs has been analyzed via administrative coding data. Unfortunately, numerous sources of bias exist with administrative data, including physician-dependent and institution-dependent coding practices, evolving clinician awareness and documentation of ‘renal dysfunction’ without distinguishing AKI from chronic kidney disease (CKD) [11, 12]. Of course, administrative data may still prove useful, especially if documented AKI is the subject of the research. Still, it should be used in conjunction with laboratory and clinical data [13, 14]. Studies have shown poor sensitivity (<25%) in identifying AKI using administrative data in both adults [14] and children [15]. Administrative data are more likely to identify higher-stage AKI, which does correlate more highly with mortality [14], although it limits understanding of the clinical phenotype (oliguric versus nonoliguric) and true disease prevalence [12]. With readily available information in the EHR, we are better equipped to assess AKI rates, risk factors and outcomes [9].

FROM DATA MINING TO REAL-TIME DATA ANALYTICS

There are a variety of ways in which EHR information can be utilized in eResearch. In addition to epidemiologic and retrospective analyses, investigators have developed real-time AKI alerts and promising tools for clinical trial recruitment and monitoring [13]. We developed an AKI alert that has been implemented across all University of Pittsburgh Medical Center (UPMC) hospitals in western Pennsylvania [16]. The alert aides in clinical decision making in two ways. First, for each patient encounter, the system will scan and identify the most appropriate baseline serum creatinine. Second, the system identifies time-based changes in a patient’s creatinine level over the course of hospitalization. We recently analyzed outcomes for >500 000 patients over the year preceding and 2 years following the implementation of the alert. Both hospital mortality {adjusted odds ratio 0.76 [95% confidence interval (CI) 0.70–0.83], P < 0.001} and length of stay [incidence rate ratio 0.91 (95% CI 0.89–0.92), P < 0.001] decreased after initiation of the alert for patients with AKI, while no effect was seen for non-AKI patients [17]. However, a recent randomized controlled trial including 2393 patients failed to show improvement in patient-centered outcomes with an automated electronic alert for AKI [18]. Together, these studies suggest that the effect size expected from an ‘informational alert’ (one in which clinical guidance is limited to event detection without management recommendations) is likely to be small. In order to produce larger effects, alerts will need to include specific actions. In the pediatric literature, an EHR-based trigger tool was developed to screen children who had significant nephrotoxic medication exposure, which subsequently prompted the clinical team to monitor more closely for development of AKI [19, 20]. It has proven to be effective in lowering AKI rates, with a 42% reduction in AKI intensity [20]. This use of EHRs has significant potential influence on future patient care.

While data from EHRs provide much information that can facilitate large cohort-based AKI eResearch, there are important limitations that need to be addressed. It is vital to understand what data are available as well as what is potentially missing in the medical record to determine the validity of the analyses. Identification of what influences data recording aides the researcher in deriving rules to address missing, incomplete and/or questionable data. While data validation on a small scale can attempt to ameliorate systematic errors, the possibility still exists for imperfect approximation of renal function as well as patient characteristics. Additionally, depending on the algorithm used, worsening CKD on admission to the hospital may be misclassified as AKI, with one report documenting a 14% misclassification rate [21]. Having a sensitive screening measure will inevitably lead to false identification of cases. When these limitations are accounted for and understood, EHR-based research can be very valuable.

AKI RESEARCH UTILIZING EHRS

UPMC Pittsburgh

One of the most susceptible populations to AKI includes those who are critically ill [1]. Identifying changes in renal function is most readily apparent in a highly monitored environment such as the intensive care unit (ICU), since AKI staging is based on serum creatinine and urine output. Despite the relative simplicity of AKI criteria, the array of information necessary for eResearch in AKI is still quite significant. The 2000–08 High-Density Intensive Care (HiDenIC-8) database was developed to study multiple different questions in critically ill patients, but has been used most extensively to study AKI. HiDenIC-8 includes all patients who were admitted to an ICU within the UPMC system from 2000 to 2008, >45 000 patient encounters. HiDenIC-8 merges data from several sources (see Figure 1) containing demographics, diagnoses, billing codes, laboratory results, surgical procedures and various text elements such as reports, clinical notes and discharge summaries. These data are combined with data from the Eclypsis database for physiologic information that includes vital signs, medications, fluids, mechanical ventilation, feeding, oxygen, details of renal replacement therapy and transfusions. An honest broker also obtained information from the US Renal Data System and the National Death Index and merged these data with the HiDenIC-8 database. The final data set is deidentified for use in analyses.

FIGURE 1.

Data acquisition for HiDenIC database. HiDenIC, high-density intensive care; UPMC, University of Pittsburgh Medical Center; CARe, Center for Assistance in Research using eRecord; BDMC, Biostatistical and Data Management Core; C3N, Center for Critical Care Nephrology.

Subsequently a version of HiDenIC was created for the pediatric population (Peds HiDenIC) and includes all patient encounters from children >60 days of age admitted to the pediatric ICU or cardiac ICU at the Children’s Hospital of Pittsburgh of UPMC between 2010 and 2014 (see supplementary data, supplementary table 1). This database merged information from two EHRs and includes demographic and admission data, administrative codes, procedure and surgical information, medication administration, physiologic variables, laboratory values, mechanical ventilation and microbiology results.

KDIGO AKI staging

The 2012 KDIGO criteria, the most current definition of AKI, describes staging for increasing severity of AKI (see Table 1). Identification and appropriate assessment of AKI using large databases such as HiDenIC requires the availability of baseline serum creatinine values, in-hospital serum creatinine values and accurate documentation of urine output. Essentially, once AKI is determined, it can be staged using four methods: (i) percentage increase in serum creatinine above reference value (Stages 1–3), (ii) decrease in urine output (Stages 1–3), (iii) initiation of renal replacement therapy for AKI (Stage 3) or (iv) increase in serum creatinine by 0.3 mg/dL within 48 h (Stage 1).

Serum creatinine

AKI staging is dependent on establishing a reliable reference serum creatinine with which to compare measured values during the hospital or ICU stay [22]. Ideally, baseline serum creatinine values are assigned using measurements obtained in the 6 months–1 year prior to admission, utilizing the median value to sensor outliers. If a baseline serum creatinine is not available, either a back-calculated value or admission serum creatinine is used as the reference serum creatinine (see Figure 2).

FIGURE 2.

Assigning reference creatinine in adults. CKD, chronic kidney disease; SCr, serum creatinine.

While using baseline serum creatinine measured prior to admission is preferred, it certainly has limitations. Outpatient testing of serum creatinine can be triggered by acute illnesses or monitoring of chronic diseases, which introduces potential bias. Patients with available values may not be representative of the general population, and comparing inpatient to outpatient values may add to error if different assays were used to measure serum creatinine—although this risk has been reduced in recent years by introduction of the isotope dilution mass spectrometry (IDMS) standard.

If serum creatinine values are not available prior to admission, another strategy is the assumption of a low–normal estimated glomerular filtration rate (eGFR) of 75 mL/min/1.73 m² and back-calculation of the associated serum creatinine using the Modification of Diet in Renal Disease (MDRD) study equation [23]:

$$\text{eGFR }(\text{mL}/\text{min}/1.{73\mathrm{m}}^2)\hspace{0.17em}=\hspace{0.17em}186*{\text{ SCr}}^{-1.54}*{\text{ Age}}^{-0.203}*1.{21}^{\text{African}-\text{American}}*0.{742}^{\text{Female}}$$

This is a necessary practice in order to analyze patients who could have community-acquired AKI on admission to the hospital but no known baseline serum creatinine value. This strategy may overestimate the incidence of AKI if prior CKD has been missed or underestimate AKI if the assumed low–normal eGFR is too low (which may be an issue in healthy young adults and children). When comparing the use of the MDRD equation to serum creatinine–based methods, the MDRD method generated 78–90% reliable results and most often overestimated the level of serum creatinine (therefore underestimating AKI) in adults <25 years of age [24]. Three additional constraints are generally placed on this approach. First, if there is any history of CKD, back-calculation cannot be used. Second, if the admission creatinine is lower than the back-calculated creatinine, the admission value should be used. This helps avoid the overestimation of serum creatinine. Third, the pattern of serum creatinine over the following days should be consistent with AKI. For example, a patient admitted with a serum creatinine of 1.5 mg/dL and a back-calculated creatinine of 1.0 mg/dL might have AKI. However, if the creatinine remains 1.5 mg/dL, this diagnosis is unlikely. Conversely a creatinine that continues to increase or comes down after admission is much more consistent with AKI.

Lastly, the admission creatinine may be the most appropriate reference creatinine to use when no baseline creatinine is available and the patient has a history of CKD or if the admission creatinine value is lower than the back-calculated value. It may also be the best reference in cases of elective surgery or other scenarios where the admission is not prompted by acute illness. Unfortunately, the admission serum creatinine can be an unreliable measure of baseline renal function in cases of community-acquired AKI or if AKI is developing on presentation to the hospital.

Urine output

AKI is best assessed and staged when using the combination of urine output and serum creatinine criteria [25], but unfortunately in a majority of studies, particularly in noncritically ill patients, urine output data are excluded from the AKI stage. Its use is dependent on accurate urine collection as well as documentation in a patient’s chart, which is more likely to be done in a critical care setting. The sheer volume of urine output data can be enormous and managing the data as well as missingness can be time consuming. We generally recommend the use of a rolling 6-h window for urine output in the first 72 h of ICU admission to classify AKI stage in conjunction with serum creatinine values. While ideally urine output should be used in combination with serum creatinine to determine AKI stage at all time points, limiting the use of urine output to the first 72 h is more feasible and captures the majority of AKI since it often presents at or near ICU admission.

Interpolation

Standard methods for interpolation exist for missing values based on whether the values are missing at random (MAR), missing not at random or missing completely at random. Many clinical data sets incur the problem of one or more of the above categories of missing values. For example, the data may be missing from the data set entirely due to a ‘true’ missing value, the data may be missing due to an error or the data may be missing at the point of analysis and found at a later time point. Whatever the reason may be, the impact of the missing values and the reasons why they may be missing often depend on many factors and on the type of disease studied. As researchers, we hope to aim for MAR, however, there may be other categories of missinginess in the data set [26].

Interpolation for urine output and creatinine has some unique features. For creatinine, the known kinetics of the marker help with interpolation. Over the short term, creatinine can usually be interpolated linearly across missing days. At early time points after AKI, an absolute increase in serum creatinine (as opposed to percentage increase) can be appropriate to define AKI. Regardless of underlying renal function, the time to reach an absolute increase in serum creatinine of 0.5 mg/dL is the same after AKI [27], suggesting the appropriate use of linear interpolation between two time points within a 24–48-h time period. Generally we do not recommend interpolating serum creatinine over >48 h.

Ideally serum creatinine is corrected for fluid overload if feasible. This is most important within the first several days of ICU admission or within the postoperative period, when fluid resuscitation is common. Serum creatinine may be diluted with fluid overload, leading to missed or delayed identification of AKI [28, 29]. Correction of serum creatinine for fluid balance can be done according to the following formula [30]:

$$\begin{array}{l}\text{Corrected SCr}\hspace{0.17em}=\hspace{0.17em}\text{measured creatinine }\\ *[1\hspace{0.17em}+\hspace{0.17em}(\text{accumulated net fluid balance}/\text{total body water})],\\ \\ \text{where total body water }=0.6*\text{ weight }(\text{kg})\end{array}$$

For urine output, the principle for interpolation is that volume is usually not lost but rather, what is not recorded at one time point is usually available at the next. For example, urine output that is not recorded at Hour 3 is often carried over into the recorded volume for Hour 4. In such cases, urine volume can be distributed over the missing time points. The limit of interpolation is a matter of judgement. In general, 6-h totals are needed for AKI ascertainment and we therefore recommend limiting interpolation to 6 h. We determined a standardized approach for interpolation of hourly urine output and addressing missing values up to 3 h. If the difference between any two documented urine output values is ≤3 h then, and only then, back-interpolation (distribution) is performed. Otherwise, values are considered missing within that time frame (see Table 2).

Table 2.

Urine output hourly interpolations

Example patients	UOP, Hour 1	UOP, Hour 2	UOP, Hour 3	UOP, Hour 4	UOP, Hour 5	UOP, Hour 6	Eligibility for inclusion after interpolation
Patient 1, actual observation	10	–	–	–	40		Eligible
Patient 1, interpolation	10	10	10	10	10
Patient 2, actual observation	10	–	–	30	30		Eligible
Patient 2, interpolation	10	10	10	10	30
Patient 3, actual observation	10	–	40	10	50		Eligible
Patient 3, interpolation	10	20	20	10	50
Patient 4, actual observation	10	–	–	–	–	50	Not eligible
Patient 4, interpolation	10	–	–	–	–	50

This table illustrates how missing urine output data from four example patients would be interpolated to determine hourly urine output rates. A 6-h rolling window can be used to identify a decrease in urine output that would qualify as AKI. If the difference between any two observations is ≤3 h then, and only then, back-interpolations would be done. Otherwise values are considered missing within that time frame.

UOP, urine output (mL).

Validation

Data validation focuses on consistency with real clinical information, which includes cross-system consistency, cardinality, logistical designs, numeric ranges, missingness, uniqueness, security vulnerability and anonymization, as well as referential integrity. Validation of primary data is a necessary and vital step prior to predictive analytics. Validation methods vary across different data types given the differences in relative importance to the primary outcome and the susceptibility for error. For AKI, creatinine and urine output require multiple rigorous clinical adjudications. We recommend manual validation of a minimum of 100 randomly selected patient records using at least two adjudicators [16]. Overall, the approach is similar to prospective clinical studies where an adjudication committee determines the presence of AKI [31].

Covariates

Additional variables available in the HiDenIC database allow statistical modeling to address potential confounders. Comorbidities can be identified using International Classification of Diseases (ICD) codes, accepting a certain degree of variability based upon clinician recognition of the comorbidity and entry into the EHR. With the transition from ICD-9 to ICD-10 codes in 2015, mapping diagnoses between the versions needs to be done as part of the validation. In the adult population, comorbidities can be combined with patient age to develop a Charlson Comorbidity Index [32], which can serve as a marker for the complexity of underlying health-related issues that often affects the risk for development of AKI. Additionally, various severity of illness scoring systems are available such as the Acute Physiology and Chronic Health Evaluation (APACHE), Simplified Acute Physiology Score (SAPS), Sequential Organ Failure Assessment (SOFA), Mortality Probability Model (MPM) and Multiple Organ Dysfunction Score (MODS). These can be calculated using physiologic variables, demographic data and laboratory values in the database, which can be useful for descriptive purposes as well as severity adjustment.

Hemodynamic variables including blood pressure and cardiac output are also important, especially in the ICU. In addition to the use of vasopressors as a marker of severe hypotension, a hypotensive index has been developed that integrates the duration and depth of systolic blood pressure <90 mmHg in the first 24 h after ICU admission [33]. Lastly, since sepsis is one of the leading causes of AKI, a feasible definition needs to be applied using EHR data. Since administrative codes tend to underestimate the diagnosis, suspected sepsis has been defined as the ordering of blood cultures and antibiotics within 24 h of each other [34]. Utilization of the EHR can develop these covariates to enhance analyses and generalizability of the results.

PEDIATRIC-SPECIFIC ISSUES

In contrast to AKI research focused on the adult population, research in pediatrics has historically not been as robust, though there are promising international pediatric AKI studies under way [35]. Major contrasts in adult and pediatric eResearch include different associated comorbidities, varying baseline serum creatinine values in growing children, alternate eGFR equations and weight-based dosing of medications. Many of the same principles can be applied to pediatric eResearch with some flexibility and modifications.

Reference serum creatinine

While identifying an appropriate reference serum creatinine in adults is challenging, it is amplified in the pediatric population given that serum creatinine trends upwards as a child grows. In adults, if serum creatinine values prior to hospital admission are available, it is reasonable to assign a baseline serum creatinine from a median value over the past year. In children, we would recommend limiting this time frame to 6 months given the possibility of a more rapidly changing value.

While back-calculations of a reference serum creatinine are generated using the MDRD equation in adults, eGFR in children can be approximated with the modified Schwartz formula, which includes a term to adjust for a patient’s height [36]. The modified Schwartz formula is as follows:

$$\text{eGFR}\hspace{0.17em}=\hspace{0.17em}0.413*(\text{Height in cm})/(\text{Serum creatinine in mg}/\text{dL})$$

It is important to note that if height is missing in the EHR for a specific ICU encounter, back-calculation of a reference serum creatinine is not possible and therefore the admission serum creatinine is assigned as the reference value.

In the adult literature, back-calculating a reference serum creatinine is done using an eGFR of 75–100 mL/min/1.73 m² [24], whereas in the pediatric literature eGFR is often 100–120 mL/min/1.73 m² [37]. Missingness is more prevalent in children with regard to baseline serum creatinine measurements [23]. Depending on the reference serum creatinine used, AKI incidence and outcome associations can differ [37]. While assuming an eGFR of 120 mL/min/1.73 m² is less biased with a 25% absolute error, using an eGFR of 100 mL/min/1.73 m² in database research may be more conservative to avoid overdetection of AKI, although it carries up to 50% absolute error [37].

Severity of illness scoring

Several severity of illness scoring systems are used in pediatrics, with the Pediatric Index of Mortality 2 (PIM2) score most widely accepted and validated [38]. This score incorporates physiologic data, lab values, mechanical ventilation, preceding surgery and high- and low-risk diagnoses to calculate a risk of mortality for the ICU admission [39]. While this score is often used in prospective studies, it has yet to be validated in studies using EHR data. The issue arises primarily from time of assessment; the PIM2 score was developed as a bedside tool to be used within the first hour of admission to the ICU. When reviewing EHR data, it is less likely that all the information needed to assess the risk of mortality will be recorded within the first hour of ICU admission, and the time frame needs to be expanded to at least the first 4 h of admission.

Medications

In children, assessment of medication administration in relation to AKI is extremely important, as nephrotoxicity is a leading cause of AKI in hospitalized children [40]. In the critically ill population, it is often difficult to discern the degree to which nephrotoxic medications play a role in the development of AKI given the complexity of these patients. The prevalence of medication-associated AKI in critically ill children may very well be underrecognized and underreported. The use of large databases such as Peds HiDenIC may aid in assessment of medication-associated AKI, and in the pediatric population, analyzing dose–response relationships as well as timing would be vital for supporting an association between medication administration and development of AKI. In children, medications dose is often based on weight. Therefore, appropriate documentation of a weight in the EHR would be necessary to accurately assess dose.

DATA INTEGRITY AND SECURITY

Data integrity and security are vital components of conducting eResearch. Creating policies and procedures for good data stewardship is necessary in collaboration with multiple individuals. One of the single most important documents that one may want to create is a data dictionary, depicting the ‘data’ behind the data by clearly delineating mapping, codes and variable definitions. Establishing good data governance principles and securing any protected health information (PHI) are vital aspects to successfully conducting eResearch. We recommend using a secure cloud-based server with restricted access or a similarly secure data repository to safely store any PHI. The inclusion of dedicated information technology staff members should be considered to ensure the proper use and dissemination of the data by including role-based access control, encryption and authentication of data. Finally, it should be mentioned that data integrity must be maintained and judged to be tenable, reasonable, deidentified and secure before performing analyses, including data mining and predictive modeling techniques.

CONCLUSIONS AND FUTURE DIRECTIONS

eResearch provides a useful, efficient and accurate platform to study AKI. If data are validated and handled appropriately while accepting some limitations inherent to database research, EHRs can be successfully leveraged to gain information in an understudied field. As eResearch in AKI becomes more robust with the use of EHRs, there are great opportunities for continued refinement of data accrual, assessment and inclusion. It has been made readily apparent that an increase in serum creatinine and a decrease in urine output are late markers of AKI. Promising urinary biomarkers are being actively studied and may be incorporated into early AKI detection in the future, which could prove useful in real-time AKI alerts. Additionally, integration of clinically relevant information into AKI detection, including time course (development of acute kidney disease or CKD), histological features if available, clinical context and recovery status could prove useful in all aspects of eResearch as well.

FUNDING

This study was supported by the National Institutes of Health (T32DK091202).