Use of height-independent baseline creatinine imputation method with renal angina index

Abstract

Background

The Renal Angina Index (RAI) is a validated screening tool used at 12 h of pediatric intensive care unit (PICU) admission to predict severe acute kidney injury (AKI) on day 3 of PICU stay. A measured or height-imputed baseline serum creatinine (SCr) is required for AKI diagnosis and RAI calculation, yet these are often lacking. We assessed an age-based, height-independent baseline SCr calculation and compared the RAI values employing this method to their historical counterpart.

Methods

An electronic algorithm was implemented to generate RAI score for patients admitted to our PICU. We reviewed 157 consecutive patient records from May 2017, until we cumulated 100 with a valid RAI calculation. We compared RAI scores using the age-based SCr imputation method of Pottel to the historical RAI. Our primary outcome was a difference in the rate of RAI fulfillment (≥8) reclassification between methods.

Results

Of the first 100 patients, 27 had measured baseline SCr and 73 used height imputation. Only two patients had RAI reclassified with the Pottel method (one in each direction). Being small for age or older were associated with ≥25% overestimation of the baseline SCr in 20 patients with the Pottel method compared with height imputation. 15/157 patients had a falsely positive RAI due to lack of measured baseline SCr and height.

Conclusion

The age-based method to estimate baseline SCr offers a viable height-independent alternative for RAI calculation. While less precise than a height-based approach, this lack of precision rarely leads to reclassification of patient RAI status.

Introduction

Acute kidney injury (AKI) is associated with increased mortality and morbidity in hospitalized patients [1, 2], as well as increasing the risk of CKD [3–5]. The ability to identify patients at high risk of developing severe AKI early is essential to optimize their care [6–9]. Since AKI treatment is currently supportive, prevention and early implementation of bundle-based AKI improvement are valuable approaches to mitigate deleterious outcomes associated with AKI [2, 3, 10, 11]. The busy setting with significant workload of ICUs makes it even more crucial to identify which patients will benefit most from additional preventive efforts, and focus health care resources only on those patients.

The Renal Angina Index (RAI) is a validated stratification tool that identifies children at risk for severe AKI on day 3 of ICU admission [9, 12–14]. While the RAI is simple to calculate, reliance on manual calculation of this time-sensitive indicator is inefficient, thereby losing its early stratification advantage. For this reason, we implemented an automatic RAI calculation algorithm at our center in April 2017, extracting data from the electronic medical record (EMR, EPIC^™, Verona, Wisconsin), thereby generating and resulting an RAI score in the EMR for every patient admitted to our PICU.

The lack of a baseline serum creatinine (SCr), often encountered in healthy children, is a limitation of the RAI calculation and AKI diagnosis. To address this problem, a validated method to impute a baseline creatinine is to use one of the known formulas to estimate glomerular filtration rate (GFR) [16]. We have previously used the Schwartz GFR estimation formula for this purpose, assuming a normal eGFR (120 ml/min/1.73m²) and using the patient height on PICU admission to impute a baseline SCr [16, 17]. However, patient height is often lacking in acute setting, potentially leading to RAI miscalculation and misclassification at the time of RAI generation. In our algorithm, a lack of both SCr baseline and height result in a falsely positive RAI score. A high rate of falsely positives results impairs the reliability of a process by decreasing its accuracy.

Pottel developed a height-independent, age-based formula to estimate the GFR in children with chronic kidney disease (CKD). Its performance was comparable to the Schwartz formula in a cohort of patients who underwent direct GFR assessment by chromium EDTA [18, 19]. However there have been limited studies applying this formula outside the CKD population.

The aim of the current study is to evaluate the use of Pottel height-independent equation to impute SCr baselines for RAI screening. To assess its usability in this setting, we compared existing RAI scores that used either a measured SCr or the height-based imputation method, with a new RAI score generated using age-based imputed baselines to determine how often the two approaches result in a discordant RAI classification in the same patient. As a secondary analysis, we evaluated what clinical factors resulted in a discrepancy between the age-based and height-based methods.

Methods

Designs, setting, and subject selection

We retrospectively reviewed each consecutive PICU admission at Cincinnati Children’s Hospital Medical Center beginning May 1, 2017, until we obtained a total of 100 patients with complete data sets, including both a measured SCr between 0 and 12 h of admission and a valid baseline SCr, either measured or imputed with the height-based formula (Fig. 1). These two types of baseline values were then used as the control group since they represent the current clinical practice for establishing patients’ baseline SCr. In cases where both were available, the algorithm employs the measured SCr value first. This study was approved by the Cincinnati Children’s Hospital Medical Center Institutional Review Board with a waiver of informed consent.

Fig. 1.

CONSORT chart of patients included in the primary analysis. 157 consecutive PICU admissions were reviewed from May 2017 until 100 with valid RAI scoring were obtained for the primary analysis. Two patients had no height ever recorded in their EMR and were excluded. 155 patients had at least one height value at any time during their admission, or up to 1 year prior to it, and were used for the secondary analysis (comparing age-based to height-based imputed baselines). 42 patients without a current creatinine value between 0 and 12 h of PICU admission and 15 patient with a falsely positive score, caused by a lack of both measured baseline creatinine and height value within 0-12 h of PICU admission or 1 year prior to it, were excluded from the primary analysis (assessment of the RAI reclassification using only age-based imputed SCr baselines)

Patients admitted in the PICU on May 2017 were identified through our EMR and their charts were reviewed to collect pertinent data (demographics, gender, height, weight, and every elements needed for the RAI calculation). Each patient had a new RAI score generated using an age-based baseline SCr, regardless of the height-based imputed or known measured baseline. As a primary outcome, the RAI results were compared to assess for a change in the RAI score and if there was a reclassification in the fulfillment of the RAI (threshold of ≥8).

As a secondary outcome, every chart was reviewed to find if a height was reported within a year before or after the PICU admission. We calculated a height-based baseline SCr for everyone with enough data and then compared the percentage of difference with the age-based baseline. We empirically defined a discrepancy threshold of ≥25% to explore potential clinical factors that might have led to a greater discrepancy.

Programs, scores, and formulas

The RAI serves as a risk stratification tool to identify patients at high risk of severe AKI in an ICU settings. Patients are considered to fulfill renal angina at an RAI score ≥8. The RAI has an excellent negative predictive value of 92-99% and a good AUC-ROC of 0.74-0.81 for the development of a severe AKI (stage 2 or 3 based on KDIGO criteria), at day 3 of hospitalization, as early as 12 h after admission [9, 12, 13]. The RAI is obtained by a multiplication of the patient “risk” and “injury” score (Fig. 2) [9, 20, 21].

Fig. 2.

Renal Angina Index calculation. The Renal Angina Index is obtained by multiplying the Risk Score and the Injury Score of patients at 12 h of PICU admission. The Risk Score default to 1 for anyone in the PICU, it increases to 3 for solid organ or bone marrow transplant recipients or to 5 if the patient is mechanically ventilated and on pressor. The injury score is attributed a value of 1, 2, 4, or 8 based on the rise of SCr compared with the patient baseline or the %FO at 12 h of PICU admission, whichever value is highest

For the baseline SCr imputation based on patient height, the bedside Schwartz formula had been implemented in our algorithm. As previously demonstrated, this method can be used to acceptably estimate what would be the baseline SCr of a children using its height and assuming a normal eGFR of 120 ml/min/1.73m² [16, 17].

• ζRevised 2009 Bedside Schwartz formula:
  $$\text{eGFR}(\mathrm{ml}∕\min ∕1.73{\mathrm{m}}^2)=0.413\times (\text{height}(\mathrm{cm})∕\mathrm{SCr}(\mathrm{mg}∕\mathrm{dL}))$$
• ζReversed formula used:
  $$\text{Baseline}\mathrm{SCr}(\mathrm{mg}∕\mathrm{dL})=0.413\times (\text{height}(\mathrm{cm})∕120\mathrm{ml}∕\min ∕1.73{\mathrm{m}}^2)$$

For the age-based imputation method, we did not use the eGFR-Pottel formula as a whole, but rather the population-normalized SCr median values derived from their initial analysis. The initial analysis median SCr for age, by Pottel, found a linear relationship in children without difference for gender from 1 month to 14.5 years old, median SCr (mg/ dL) = 0.027 × age (years) + 0.2329 [15, 18, 19]. The author reported those values in adolescents above that threshold. No equation for medians was reported; however, the medians for age and gender were reported as numerical values from 14.5 to 20.5 years old for each year increment (Table 1) [19]. We used both method to impute an SCR baseline throughout all ages, using the median at 20.5 years old even for older patients, since few of them are seen in our PICU.

Table 1.

Population-normalized SCr median baseline for different age and gender

Age (years)	Age (months)	Male	Female
14.5 to <15.5	174 to <186	0.68	0.62
15.5 to < 16.5	186 to <198	0.78	0.68
16.5 to <17.5	198 to <210	0.82	0.70
17.5 to <18.5	210 to <222	0.85	0.71
18.5 to <19.5	222 to < 234	0.86	0.71
19.5 to <20.5	234 to < 246	0.88	0.70

Adapted from the original table of Blufpand et al. [15]

The computer-based algorithm implemented in our institution has been developed locally. A RAI score is generated automatically and populated in the results tab of the patient’s EMR at the 12th hour after admission after assessing the “risk” and the “injury” components of the score. To establish the patient creatinine baseline, the algorithm searches for a SCr measure between 104 and 76 days prior to the admission (90 ±14 days). If no measured serum creatinine is found, the search is broadened to 120 to 60 days prior (90 ±30 days), and the value closest to 90 days prior is chosen. If there is no preexisting lab values in that period of time, the algorithm searches for the latest height recorded either as an inpatient or outpatient during the last 12 months and uses the reversed beside Schwartz formula to impute the estimated baseline SCr (see above). Afterward, it uses the highest SCr available between 0 and 12 h after their arrival in the PICU and determines the relative increase compared with baseline (SCr_actual/ SCr_baseline). The algorithm then attributes an “injury” score to both the increase in SCr and percent fluid overload (%FO) and use the higher of the two in the RAI calculation.

Statistical analyses

The age, height, %FO, measured, and height-based SCr values were non-normally distributed in our study group, hence the description of our population was mainly with median and interquartile range (IQR). The other descriptive statistics are expressed as percentage when qualitative and binary. Chi Square and Wilcoxon signed-rank tests were perform in when assessing the different baseline SCr between the groups and the independence of age and height from the sub-group of increased SCr baseline discrepancy from the rest of the cohort. A Cochrane-Mantel-Haenszel test in a three-way contingency table was performed to ascertain the independency of these latter 2 variables from one another within the sub-group with the high SCr baseline discrepancy. All statistical analyses were performed with R Studio (Version 1.1.453, 2016, Boston, MA). A p value of < 0.05 was considered to be statistically significant.

Results

Patient characteristics

A total of157 consecutive admissions were reviewed between May 1st and 24th 2017 to accumulate 100 patients with complete data. Out of the 157 charts reviewed for newly admitted patients in the PICU in that time, 42 did not have a SCr drawn within the first 12 h of PICU admission and 15 had a falsely RAI+ generated by a lack of both baseline measured SCr and height, leaving 100 patients with sufficient data to ascertain a RAI for the primary analysis (Fig. 1). Of them, 27 had measured baseline SCr and 73 used height imputation for their RAI score calculation. The clinical characteristics of the 100 patients are reported in Table 2. The median age of the cohort was 7.5 years (IQR, 2.1-14.0 years) with a median height of 117.9 cm (IQR 85.4-152 cm) with 17% being under the 3rd percentile of height for age. Out of the 157 patients, 98 had their height measured in the first 12 h of their PICU admission (62.4%) and 141 had a measured height (89.8%) when including those with heights available in the 12 months prior to PICU admission. Seven out of the 100 valid scores were RAI+ (RAI ≥8), all injury scores being increased by the SCr criteria and not by FO.

Table 2.

Population description

	All patients (100)
Age (years)*	7.5 (2.1-14.0)
Gender (F/M)	50/50
Height (cm) *	117.9 (85.4-152)
Height < 3 percentile (%)	17%
RAI+ (%)	7%
FO (%)*	1.1 (-0.4-2.4)
Measured SCr baseline median (mg/dL)*	0.30 (0.26-0.37)
Height-based SCR baseline median (mg/dL)*	0.42 (0.32-0.54)

^*Expressed as median and interquartile ranges (IQR)

Age-based baseline SCr

The median baseline SCr obtained by the age-based method for the 100 patient cohort was 0.44 mg/dL (IQR 0.29-0.61 mg/dL) compared with 0.38 mg/dL (IQR 0.27-0.52 mg/dL) with the original baseline SCr (p = 0.029). Eighty-one RAI scores remained unchanged with the height-independent method, 18 decreased, and one increased. However, RAI fulfillment (RAI ≥8) only differed for two of the 100 patients, with one changing from RAI+ to RAI- and the other changing from RAI- to RAI+ (Table 3).

Table 3.

Age-based baseline SCr and changes in the RAI fulfillment

	All	Previously measured SCr	Previously height-based SCr
n	100	27	73
Originally used SCr (mg/dL)	0.38 (0.27-0.52)	0.3 (0.26-0.37)	0.42 (0.32-0.54)
Age-based SCr baseline (mg/dL)*	†0.44 (0.29-0.61)	††0.36 (0.26-0.50)	††0.47 (0.3-0.62)
Age (years)*	7.5 (2.1-14.0)	4.8 (1.2-9.7)	8.8 (2.3-14.9)
Gender (F/M)	50/50	11/16	39/34
Height (cm)*	117.9 (85.4-152)	102.0 (73.0-131.1)	123.0 (92.0-157.5)
Reclassified RAI (n)
Up	1	1	0
Down	18	6	12
Unchanged	81	20	61
Change in RAI fulfillment (n)
New positive	1	1	0
New negative	1	1	0
Unchanged	98	25	73

^*Expressed as median and interquartile ranges (IQR)

^†Statistically different from the original baseline, p <0.05

^††No statistical difference from their reference group

We assessed patient characteristics for these two cases. The height-independent newly RAI+ patient had CKD, and therefore had an elevated measured baseline SCr that would not be accurately reflected in an age-based method. The newly RAI-had a RAI risk component of 5 and just enough rise in SCr to have an injury component of 2. Yet, his age-based baseline was slightly higher than a previously measured one, his new injury component was only 1, hence not fulfilling the RAI threshold and missing it on the screening test.

Falsely positive RAI results

Out of the 157 charts consecutively reviewed, we noted 15 falsely positive RAI results (9.6% of all patients and 68% of all RAI+ patients). Thirteen were caused by an “infinite rise” in the SCr due to a lack of measured baseline SCr and height (these being used as denominator for the ratio, hence dividing by zero). Applying the age-based baseline SCr to the 13 lacking data, 2 became true RAI positive and 11 became true RAI negative.

Discrepancy between estimation methods

A total of 155/157 patients had a height recorded either up to a year before the admission to the PICU or at some point of their PICU admission. When height-based baseline SCr was imputed for this group, we identified 31 patients with ≥25% difference compared with the age-based imputed SCr results. Patients with discrepant results were older (15.2 (9.7-17.5) vs. 6.1 (1.42-13.1) years, p < 0.0001), yet more likely to be small (77.4% (24/31) vs. 30.9% (48/155) below the 3rd percentile for age, p <0.0001) (see Table 4 in Supplementary Material). To account for the high proportion of small patients in the older group, defined as 14 years of age and older, we performed a Cochrane-Mantel-Haenszel test in a three-way contingency table, confirming the independence of both variables, age and height, in patients with ≥25% discrepancy between the age-based and height-based SCr baseline (χ² = 30.77, p <0.0001).

Discussion

Every automated process has an inherent possibility for error, or failure rates, with the entire process never achieving more accuracy than its least reliable step [22]. Considering the important clinical benefits that an automated tool can bring into day to day practice, it is crucial that processes are analyzed and reviewed to improve their reliability, accuracy, and generalizability.

A baseline SCr is required for both the diagnosis of AKI and its risk screening with RAI; therefore, alternative imputation methods of baseline SCr are needed when no previously measured SCr is available. While being easy and reliable, the height-based method has the disadvantage of requiring a recent patient height in the acute setting; therefore, an age-based method may be more feasible. In our cohort, the age-based method to estimate a baseline SCr led to only a 2% rate of RAI fulfillment reclassification and only one patient out of 100 would have been missed on screening with the age-based, height-independent, method. In addition, only one patient, who had a documented CKD, would have been reclassified as RAI+. This scenario should represent a rare occurrence since the likelihood of admitting patient in the PICU with undiagnosed or unrecognized underlying CKD is low.

Considering that the age-based method does not account for other relevant patient factors (height, ethnicity, etc.), it is more appropriate to use it in patients with standard body habitus. Our finding that patients with greater discrepancy between the age-based and height-based baseline SCr values were more often below the 3rd percentile of height supports this concept. The age-based method seems to overestimate the baseline SCr for patients 14 years of age and older when compared with the current height-based approach. This may be related to the conversion from the linear equation of Pottel to the medians reported for the age and gender per increment of years at that age. Also, back calculating a SCr baseline using the Schwartz formula is less accurate in male adolescents and young adults and may lead to an underestimation of height-imputed baselines [23]. Moreover, some concerns have been raised about a potential underestimation of those baselines by the used of an expected “normal” eGFR of 120 ml/min/1.73m² for back calculation in children, a method validated to balance the potential underestimation in eGFR of healthy individuals [16]. Recent data suggest that normal pediatric GFR is close to 110 ml/min/1.73m², hence the current method could underestimate the height-imputed SCr baselines by up to 10-15% on average [24]. In a screening algorithm, overestimation of true baseline creatinine by imputation methods could potentially lead to an under-recognition or the under-classification of AKI and RAI fulfillment. The RAI construct was developed for high sensitivity and negative predictive value, so those potential errors do not run counter to the intent of the RAI as a screening tool. Since we did not have a high number of measured baseline SCr, we could not assess the impact of ethnicity on the two estimation methods. Both of these formulas are derived from populations with a lower prevalence of African-Americans than seen in the general US population. Since African-Americans are known to have higher normal SCr values, the aged-based algorithm likely underestimates baseline SCr of African-American children [25]. This systematic error has the potential to slightly increase the number patients who exhibit a falsely positive RAI score, but importantly, it would not lead to a high rate of falsely negative RAI.

Despite the limitation of the age-based method and its potentially greater imprecision than the height-based method, it is still a useful complementary tool when both measured baseline SCr and height are lacking. Given the significant number of falsely positive RAI generated in such situation, we have implemented the age-based approach in our algorithm as a 3rd option, after the use of a measured or height-based SCr, to increase its accuracy (Fig. 3).

Fig. 3.

Implementation of Pottel age-based imputation method in the algorithm. The age-based imputation method have been implemented as a 3rd step in the SCr baseline determination algorithm. When a SCr value is available between 60 and 120 days prior to the current admission, the value closest to 90 days is chosen as the baseline. If none is available, the height-dependent approach is used if a height value between 0 and 12 h of the PICU admission, or up to 1 year before it, is present in the EMR. When none of the two methods can be used, the algorithm now defaults to the age-based method of Pottel to impute a baseline SCr

Conclusion

An age-based, height-independent, approach to impute a baseline SCr enables the use of validated screening tools when there is a lack of measured baseline SCr and when a height cannot be obtained in a timely manner. While being less precise in patients without a normal body habitus, it rarely leads to RAI reclassification. When used in an automated setting with an EMR, it serves as a useful complementary method to add to an existing algorithm. It can improve the accuracy of an algorithm by reducing falsely positive results without impairing the quality of detection with more precise methods if the required data are available. However, clinician vigilance should always prevail when taking care of patients that are older, small for their age, or muscle wasted, as this method could overestimate the patient baseline and consequently miss a rise in SCr. Considering there is yet no specific treatment for AKI, the best approach to mitigate its longer-term consequences on morbidity and mortality is still prevention.