Neutrophil Gelatinase-Associated Lipocalin Measured on Clinical Laboratory Platforms for the Prediction of Acute Kidney Injury and the Associated Need for Dialysis Therapy: A Systematic Review and Meta-analysis

Abstract

Rationale & Objective:

The usefulness of measures of neutrophil gelatinase-associated lipocalin (NGAL) in urine or plasma obtained on clinical laboratory platforms for predicting acute kidney injury (AKI) and AKI requiring dialysis (AKI-D) has not been fully evaluated. We sought to quantitatively summarize published data to evaluate the value of urinary and plasma NGAL for kidney risk prediction.

Study Design:

Literature-based meta-analysis and individual-study-data meta-analysis of diagnostic studies following PRISMA-IPD guidelines.

Setting & Study Populations:

Studies of adults investigating AKI, severe AKI, and AKI-D in the setting of cardiac surgery, intensive care, or emergency department care using either urinary or plasma NGAL measured on clinical laboratory platforms.

Selection Criteria for Studies:

PubMed, Web of Science, Cochrane Library, Scopus, and congress abstracts ever published through February 2020 reporting diagnostic test studies of NGAL measured on clinical laboratory platforms to predict AKI.

Data Extraction:

Individual-study-data meta-analysis was accomplished by giving authors data specifications tailored to their studies and requesting standardized patient-level data analysis.

Analytical Approach:

Individual-study-data meta-analysis used a bivariate time-to-event model for interval-censored data from which discriminative ability (AUC) was characterized. NGAL cutoff concentrations at 95% sensitivity, 95% specificity, and optimal sensitivity and specificity were also estimated. Models incorporated as confounders the clinical setting and use versus nonuse of urine output as a criterion for AKI. A literature-based meta-analysis was also performed for all published studies including those for which the authors were unable to provide individual-study data analyses.

Results:

We included 52 observational studies involving 13,040 patients. We analyzed 30 data sets for the individual-study-data meta-analysis. For AKI, severe AKI, and AKI-D, numbers of events were 837, 304, and 103 for analyses of urinary NGAL, respectively; these values were 705, 271, and 178 for analyses of plasma NGAL. Discriminative performance was similar in both meta-analyses. Individual-study-data meta-analysis AUCs for urinary NGAL were 0.75 (95% CI, 0.73–0.76) and 0.80 (95% CI, 0.79–0.81) for severe AKI and AKI-D, respectively; for plasma NGAL, the corresponding AUCs were 0.80 (95% CI, 0.79–0.81) and 0.86 (95% CI, 0.84–0.86). Cutoff concentrations at 95% specificity for urinary NGAL were >580 ng/mL with 27% sensitivity for severe AKI and >589 ng/mL with 24% sensitivity for AKI-D. Corresponding cutoffs for plasma NGAL were >364 ng/mL with 44% sensitivity and >546 ng/mL with 26% sensitivity, respectively.

Limitations:

Practice variability in initiation of dialysis. Imperfect harmonization of data across studies.

Conclusions:

Urinary and plasma NGAL concentrations may identify patients at high risk for AKI in clinical research and practice. The cutoff concentrations reported in this study require prospective evaluation.

Graphical Abstract

Acute kidney injury (AKI) is common and associated with severely increased morbidity and mortality.¹ Current clinical practice guidelines have allocated high priority to risk assessment of AKI performed by clinicians caring for acutely ill patients.² However, current routine biological kidney markers such as serum creatinine (Scr) level and urine output often delay the diagnosis and possible treatment of AKI given that these markers indicate kidney filtration function, which is affected relatively late in the course of kidney injury.³

Markers of kidney tubular damage⁴ and cellular stress⁵ may improve AKI risk prediction.^6,7 Neutrophil gelatinase-associated lipocalin (NGAL) is one of the recently intensely investigated kidney biomarkers indicating early structural damage and patients’ kidney prognosis.^8–10 An increasing number of studies measuring NGAL concentrations on clinical laboratory platforms are available,^4,11,12 with a previous meta-analysis pointing toward more accurate AKI risk adjudication using such platforms compared with NGAL measured on research assays.⁴ However, meta-analytic data for urine and plasma cutoff concentrations to assist decision making in research and practice are not yet available. A novel meta-analysis would need to address confounders of available individual diagnostic studies being able to provide statistical evaluation beyond the area under the receiver operator characteristic (ROC) curve (AUC).^13,14

To address these issues, we performed both a systematic meta-analysis of the literature and individual-study-data meta-analysis of prospective clinical studies using clinical laboratory platforms for measurement of NGAL in urine or plasma for the prediction of AKI, severe AKI, or AKI requiring dialysis (AKI-D). Using aggregated reanalyzed individual study data, this meta-analysis aimed at standardized and multidimensional assessment of cutoff concentrations at 95% sensitivity, optimal combination of sensitivity/specificity, and 95% specificity for these outcome measures.

Methods

Study aims, search strategy, data extraction, and data synthesis were registered with the International Database of Prospectively Registered Systematic Reviews (PROSPERO; registration number CRD42016042735). The Preferred Reporting Items for a Systematic Review and Meta-Analysis of Individual Participant Data (PRISMA-IPD) guidelines were adhered to.¹⁵ Selection of studies was restricted to diagnostic test studies of adult humans investigating AKI or kidney replacement therapy (KRT) in the setting of critical illness related to cardiac surgery or admission to emergency department or intensive care unit using either urinary or plasma NGAL measured on clinical laboratory platforms. Unpublished studies were not included. Extensive methodology of data sourcing, search strategy, and the process of study selection, data extraction, and quality assessment are provided in Item S1.

The individual-study-data meta-analysis used custom-made standardized data sheets requesting data reanalysis on the patient level by the authors of original diagnostic test studies. In brief, authors of relevant studies were requested to exclude patients with known AKI or KRT at admission or NGAL measurement within 24 hours before the diagnosis of AKI or KRT initiation from the evaluation. For each study, the following prespecified NGAL indexes were then recalculated for each outcome measure (separately by specimen type [urine vs plasma]):

• Cutoff concentrations at 3 standardized points on the summary ROC curve, specifically:

  • the cutoff concentration with 95% sensitivity and corresponding specificity,
  • the Youden index (which gives equal weight to sensitivity and specificity),
  • and the cutoff concentration with 95% specificity and corresponding sensitivity;

• Corresponding rates of true-positives, false-positives, false-negatives, and true-negatives, as well as linked risk assessment variables positive- and negative likelihood ratio and diagnostic odds ratio (DOR) within each study with corresponding 95% confidence intervals (CIs)

• The paired sensitivity and specificity (with 95% CI) for NGAL cutoff concentrations derived from individual ROC curves.

The individual-study-level data received from all participating study authors were then pooled and meta-analyzed. Data from studies not providing individual study data were included in the literature-based meta-analysis only.

Study Outcome Measures

The meta-analysis was performed for 3 outcome measures (end points) separately for urine and plasma specimens: AKI, severe AKI (defined as AKI Network [AKIN] or KDIGO [Kidney Disease: Improving Global Outcomes] stages 2 and 3 or RIFLE [risk, injury, failure, loss of kidney function and end-stage kidney disease] stages I [injury] or F [failure]), and AKI requiring dialysis (AKI-D). Predefined subgroup analyses were performed for potential confounders, including patient clinical setting and use/nonuse of urine output criterion for the classification of AKI. To investigate the presence of a potential selection bias, we performed subgroup analyses separately for studies providing and not providing reanalyzed individual study data and graphically displayed the diagnostic accuracy in the subgroups.

Definition of AKI

For the literature-based meta-analysis, estimates were reported separately for each AKI definition (ie, AKIN, RIFLE, and KDIGO). Previous meta-analyses have pooled various definitions of AKI, introducing inherent bias.¹³ Therefore, the patient-level data were reanalyzed according to a standardized consensus AKI definition classified by severity according to the R (risk), I (injury), or F (failure) RIFLE criteria based on increases in Scr level from baseline within 7 days, as well as urine output criteria if available.¹⁶

Statistical Analysis

Literature-based meta-analysis was performed based on estimated AUC values and standard errors, derived from reported CI widths or using Hanley’s method¹⁷ when indicated. We used random-effects models and Mandel-Paule estimators for between-study heterogeneity,¹⁸ quantified in terms of between-study standard deviation (tau) and the relative measure I². Summary CIs were additionally computed using the modified Knapp-Hartung approach¹⁹ to complement estimates for small subgroups.

For individual-study-data meta-analysis, the approach proposed by Hoyer et al²⁰ was used, in which the ROC curve is interpreted as a bivariate time-to-event model for interval-censored data. The resulting bivariate nonlinear mixed-effects model is a single-step approach. Studies were weighted by the respective number of events in each group of study participants who did or did not reach an end point. NGAL concentrations were assumed to follow a log-normal distribution. AUC was estimated (using the trapezoidal rule with a 2-sided 95% bootstrap CI with 1,000 bootstrapped samples), as well as sensitivity and specificity, with 2-sided 95% CIs at specific cutoff concentrations (optimal as indicated by the Youden index, 95% sensitivity, and 95% specificity). CIs for sensitivity and specificity derived from the confusion matrix were calculated using the Wald or Clopper-Pearson method, as appropriate. The DOR was calculated based on sensitivity and specificity.²¹ Predictive values were calculated based on estimated sensitivity, specificity, and prevalence of the study population using Bayes’ theorem.

For visualization of the diagnostic accuracy regarding sensitivity and specificity, we created ROC plots separately for study outcome measures and specimen type.

To obtain an overview of the variability of the provided raw NGAL values, we calculated weighted descriptive statistics separately for the 3 outcome measures and specimen type illustrated as box and whiskers plots with median, quartile, and 10th to 90th percentiles. Wilcoxon signed rank test was used for comparison.

All statistical analyses were performed using SAS (version 9.4; SAS Institute) and the R Environment for Statistical Computing²² with packages “metafor,”²³ “mada” (Meta-Analysis of Diagnostic Accuracy, version 0.5.7), and “meta.”²⁴

Results

A flow chart illustrating in detail the structure of the meta-analysis is shown in Fig 1. Accumulative NGAL was measured in 13,040 patients. Characteristics of included studies and diagnostic laboratory platforms used are shown in Item S2.

Figure 1.

Flow chart of study selection and inclusion includes search performed on February 29, 2020. *One study reported urine neutrophil gelatinase-associated lipocalin (NGAL) data in the literature, only but additionally provided previously unpublished plasma NGAL data for individual-study-data meta-analysis (IDA). ^†The study by Albert et al, 2020, was excluded for IDA because it reports on the same patient cohort as Haase et al, 2013. Abbreviations: AKI, acute kidney injury; Lab, laboratory; LIMA, literature-based meta-analysis; RRT, renal replacement therapy.

Identification of Studies

In total, we included 52 observational studies.^25–76

In the literature-based meta-analysis, we included 20 studies reporting on urinary NGAL^{25–41,44–46} and 36 studies reporting on plasma NGAL,^{31–38,49–76} of which 8 studies reported on NGAL in both urine and plasma.^31–38 In the individual-study-data meta-analysis, we included 30 data sets from 26 studies. Twelve studies reported on urinary NGAL,^35–46 18 on plasma NGAL,^{36–39,47–60} and 4 reported on NGAL in both urine and plasma.^36–39

Quality Assessment

Quality assessment results are provided in Item S3. In brief, for the literature-based meta-analysis, risk of bias and applicability was moderate. The quality of studies providing individual study data was assessed before and after application of standardization criteria. After standardization, Quadas-2 (Quality Assessment Tool for Diagnostic Accuracy, version 2) studies showed improvement for risk of bias of the index test. Funnel plots showed no strong asymmetry, suggesting no indication of small-study effects.⁷⁷ However, the ability of funnel plots to detect publication bias is limited when the number of studies is small⁷⁸ or heterogeneity between studies is present.⁷⁹

Evidence Synthesis: Literature-Based Meta-analysis

Table 1 shows the number of included studies, patients, and events, as well as estimated pooled AUCs. Corresponding forest plots with pooled estimates separated by AKI definition are provided in Figures 2 and 3. In brief, in the literature-based meta-analysis, the AUCs for urinary NGAL were 0.74 (95% CI, 0.69–0.79), 0.73 (95% CI, 0.64–0.82), and 0.74 (95% CI, 0.66–0.82) for prediction of AKI, severe AKI, and AKI-D, respectively. For plasma NGAL, the corresponding AUCs were 0.77 (95% CI, 0.74–0.80), 0.83 (95% CI, 0.74–0.91), and 0.78 (95% CI, 0.74–0.81).

Table 1.

Pooled Diagnostic Accuracy Expressed as AUC in the Literature-Based Meta-Analysis and Individual-Study-Data Meta-analysis

	Literature-Based Meta-analysis						Individual-Study-Data Meta-analysis
	Studies, N	Pts, N	Events	AUC (95% CI)	tau	I2	Studies, N	Pts, N	Events	AUC (95% CI)
AKI End Point
Urinary NGAL	17	5,750	1,342 (23.3%)	0.742 (0.694–0.790)	0.09	81%	12	3,182	837 (26.3%)	0.694 (0.689–0.705)
Plasma NGAL	32	8,435	1,866 (22.1%)	0.773 (0.743–0.804)	0.07	78%	18	3,473	705 (20.3%)	0.762 (0.747–0.773)
Severe AKI End Point
Urinary NGAL	6	2,469	314 (12.7%)	0.726 (0.638–0.815)	0.10	86%	10	2,564	304 (11.9%)	0.749 (0.734–0.763)
Plasma NGAL	4	938	88 (9.4%)	0.825 (0.739–0.911)	0.07	63%	16	2,842	271 (9.5%)	0.802 (0.793–0.811)
AKI-D End Point
Urine NGAL	6	2,488	151 (6.1%)	0.742 (0.663–0.821)	0.08	69%	9	2,966	103 (3.5%)	0.796 (0.790–0.806)
Plasma NGAL	15	4,063	279 (6.9%)	0.777 (0.743–0.811)	0.03	22%	12	2,842	178 (6.3%)	0.859 (0.838–0.864)

Note: The higher number of studies assessed in the individual-study-data meta-analysis as opposed to literature-based meta-analysis relates to the fact that contributing individual-study-data meta-analysis authors provided outcome data not previously reported in the literature.

Abbreviations: AKI, acute kidney injury; AKI-D, acute kidney injury with dialysis; AUC, area under the curve; CI, confidence interval; NGAL, neutrophil gelatinase-associated lipocalin; Pt, patient.

Figure 2.

Literature-based meta-analysis (LiMA): forest plots of urinary neutrophil gelatinase-associated lipocalin (NGAL) predicting (A) acute kidney injury (AKI), (B) severe AKI (sAKI), and (C) AKI requiring dialysis (AKI-D). Overall summary estimates presented as pooled areas under the receiver operator characteristic curve (AUCs); with a 95% confidence interval (CI), results for subgroups defined by AKI definitions (AKI Network [AKIN], KDIGO [Kidney Disease: Improving Global Outcomes], and RIFLE [risk, injury, failure, loss of kidney function and end-stage kidney disease]) are quoted. For each study, the inverse variance weights (in terms of percentage contribution to the overall estimate) are provided.

Figure 3.

Literature-based meta-analysis (LiMA): Forest plots of plasma neutrophil gelatinase-associated lipocalin (NGAL) level predicting (A, located on previous page) acute kidney injury (AKI), (B) severe AKI, and (C) AKI requiring dialysis (AKI-D). Overall summary estimates presented as pooled areas under the receiver operator characteristic curve (AUCs) with a 95% confidence interval (CI), results for subgroups defined by AKI definitions (AKI Network [AKIN], KDIGO [Kidney Disease: Improving Global Outcomes], and RIFLE [risk, injury, failure, loss of kidney function and end-stage kidney disease]) are quoted. For each study, the inverse variance weights (in terms of percentage contribution to the overall estimate) are provided. Abbreviation: RRT, renal replacement therapy.

Evidence Synthesis: Individual-Study-Data Meta-analysis

The estimated pooled AUC of the individual-study-data meta-analysis, as well as patient and event numbers, are shown in Table 1. The AUCs for AKI and severe AKI were similar to the literature-based meta-analysis results. There was a trend for increased AUC with increased AKI severity (Table 1). Summary AUC plots for individual-study-data meta-analysis are provided in Figure 4.

Figure 4.

Individual-study-data meta-analysis (IDA)-derived accuracy of urine and plasma neutrophil gelatinase-associated lipocalin (NGAL) level for prediction of the study end points, (A) acute kidney injury (AKI), (B) severe AKI, and (C) AKI requiring dialysis (AKI-D) illustrated as summed receiver operator characteristic (sROC) curves (red curve) and individual ROC curves (grey) grouped by sample material. Numbers illustrate the area under the ROC curve (AUC) and 95% CI. The 3 pairs of sensitivity and specificity (95% sensitivity, optimal combination of sensitivity and specificity, and 95%specificity) of 1 individual study are connected by a line. Specifically, the sROC curves for AKI are derived from 12 studies regarding urine NGAL and 18 studies for plasma NGAL; for severe AKI, 10 studies regarding urine NGAL and 16 studies regarding plasma NGAL; for AKI-D, the sROC curve is derived from 9 individual ROC curves for urine NGAL and 12 regarding plasma NGAL (Table 1). Abbreviation: RRT, renal replacement therapy.

Descriptive Statistics of NGAL Values Provided for the Individual-Study-Data Meta-analysis

To obtain an overview on the collected raw individual-study NGAL data, we derived weighted descriptive statistics separately for the end points and specimen types. NGAL values increased incrementally with the severity of AKI. The number of outliers was low and represented studies with low patient numbers³⁷ (Item S4).

Meta-analysis of NGAL Cutoff Concentrations and Discriminative Accuracy

Only individual-study-data meta-analysis enabled calculation of NGAL cutoff concentrations, sensitivity and specificity, predictive values, likelihood ratios, and DORs for AKI, severe AKI, and AKI-D (Table 2). Cutoff concentrations calculated and provided from 30 individual study data sets were included in the meta-analysis using the approach proposed by Hoyer et al.²⁰ Cutoff concentrations meta-analyzed from the individual study data also increased incrementally with AKI severity (Table 2).

Table 2.

NGAL Cutoff Concentrations for AKI End Points, Corresponding Sensitivity and Specificity and Predictive Indexes Derived From the Individual-Study-Data Meta-analysis

Biomarker	Criterion	Cutoff, ng/mL	Sensitivity, % (95% CI)	Specificity, % (95% CI)	LR +	LR −	DOR	Prevalence, %	PPV, %	NPV, %
AKI End Point
Urinary NGAL	95% sensitivity	5	95 (91–99)	12 (4–21)	1.08	0.42	2.6	26.3	27.8	87.1
	Youden	81	56 (43–70)	71 (57–85)	1.93	0.62	3.1	26.3	40.8	81.9
	95% specificity	541	20 (10–30)	95 (91–99)	4.00	0.84	4.8	26.3	58.8	76.9
Plasma NGAL	95% sensitivity	71	95 (91–100)	22 (14–31)	1.22	0.23	5.4	20.3	23.7	94.5
	Youden	165	66 (50–81)	73 (63–83)	2.44	0.47	5.2	20.3	38.4	89.4
	95% specificity	311	30 (15–45)	95 (92–98)	6.00	0.74	8.1	20.3	60.4	84.2
Severe AKI End Point
Urinary NGAL	95% sensitivity	12	95 (90–100)	21 (7–35)	1.20	0.24	5.1	11.9	14.0	96.9
	Youden	105	65 (46–84)	71 (55–87)	2.24	0.49	4.5	11.9	23.2	93.8
	95% specificity	580	27 (10–45)	95 (90–100)	5.40	0.77	7.0	11.9	42.2	90.6
Plasma NGAL	95% sensitivity	79	100 (89–100)	33 (17–41)	1.49	0.00	9.4	9.5	13.5	100.0
	Youden	231	67 (46–77)	89 (76–92)	6.09	0.37	16.4	9.5	39.0	96.3
	95% specificity	364	44 (23–55)	100 (91–100)	NA	0.56	14.9	9.5	100.0	94.4
AKI-D End Point
Urinary NGAL	95% sensitivity	26	95 (89–100)	39 (20–58)	1.56	0.13	12.1	3.5	5.3	99.5
	Youden	83	78 (65–91)	67 (49–84)	2.36	0.33	7.2	3.5	7.9	98.8
	95% specificity	589	24 (10–38)	95 (90–100)	4.80	0.80	6.0	3.5	14.8	97.2
Plasma NGAL	95% sensitivity	162	95 (88–100)	59 (41–77)	2.32	0.08	27.3	6.3	13.5	99.4
	Youden	214	87 (73–100)	71 (55–87)	3.00	0.18	16.4	6.3	16.8	98.8
	95% specificity	546	26 (5–47)	95 (90–100)	5.20	0.78	6.7	6.3	25.9	95.0

Note: The identified cutoff concentrations require prospective evaluation.

Abbreviations: AKI, acute kidney injury; AKI-D, acute kidney injury with dialysis; CI, confidence interval; DOR, diagnostic odds ratio; LR +/−, positive or negative likelihood ratio; NGAL, neutrophil gelatinase-associated lipocalin; NPV, negative predictive value; PPV, positive predictive value.

For example, the urinary NGAL cutoff concentration was 12 ng/mL for severe AKI at 95% sensitivity (with specificity of 21% [95% CI, 7%−35%]; DOR, 5.1). At an optimal combination of sensitivity and specificity (Youden index), a cutoff concentration of 105 ng/mL had sensitivity of 65% and specificity of 71% (DOR, 4.5), while at 95% specificity, the cutoff was 580 ng/mL (sensitivity, 27% [95% CI, 10%−45%]; DOR, 7.0; Table 2). The AUC for urinary NGAL was 0.75 (95% CI, 0.73–0.76) for severe AKI (Fig 4).

For plasma NGAL and severe AKI, at 95% sensitivity, the cutoff concentration was 79 ng/mL (specificity, 33% [95% CI, 17%−41%]; DOR, 9.4). A cutoff concentration of 231 ng/mL, with 67% (95% CI, 46%−77%) sensitivity and 89% (95% CI, 76%−92%) specificity (DOR, 16.4), was calculated for the Youden Index. At 95% specificity, the cutoff was 364 ng/mL with 44% (95% CI, 23%−55%) sensitivity (DOR, 14.9; Table 2). The AUC for plasma NGAL was 0.80 (95% CI, 0.79–0.81) for severe AKI (Fig 4).

Subgroup Analysis

Results of prespecified subgroup analyses including: (1) patient clinical setting and (2) studies using the urine output criterion in addition to Scr level increase for AKI definition¹⁶ and those not using the urine output criterion are provided as Items S5, S6, and S7. In brief, cutoff concentrations were lower in studies using the urine output criterion. With increasing AKI severity, we found increasing cutoff concentrations for studies with and without use of the urine output criterion. The highest AUC values were calculated for the emergency department setting.

Discussion

This meta-analysis provides a systematic overview of the literature summarizing data from 52 observational studies to test the predictive accuracy of NGAL level in 13,040 patients at risk for AKI. Discriminative accuracy and cutoff concentrations of urine and plasma NGAL measured on clinical laboratory platforms for prediction of AKI, severe AKI, or AKI-D were assessed using both literature-based meta-analysis and individual-study-data meta-analysis. After addressing several confounders, individual-study-data meta-analysis quality assessment showed improvement for risk of bias of the index test and high applicability. Moreover, using reanalyzed individual-study-level data, individual-study-data meta-analysis enabled derivation and meta-analysis of prespecified NGAL cutoff concentrations for prediction of AKI, severe AKI, and AKI-D. Cutoff concentrations and discriminative accuracy increased with increasing AKI severity and were highest for patients with AKI-D. AUCs were similar for both meta-analyses (greatest difference was 0.08, for plasma NGAL predicting AKI-D). Finally, use or nonuse of the urine output criterion for AKI affected NGAL’s predictive and discriminatory ability.

NGAL has been increasingly measured on clinical laboratory platforms.^4,11 Individual observational studies infrequently reported on statistical indexes other than AUC.¹³ Less than 60% of studies reported on cutoff concentrations, and 85%, on AUCs. However, AUC may be substantially confounded by the heterogeneity of underlying AKI definitions,⁸⁰ and methods and timing of NGAL measurement will render data synthesis and applicability of a literature-based meta-analysis difficult.⁸¹ Notably, no previous publication has demonstrated “perfect” accuracy of NGAL level for AKI^{4,11,13,81–85} or AKI-D⁸⁶ prediction.

A summary of the studies that reported on NGAL cutoff concentrations for AKI found a range from ≥105 to ≥350 ng/mL for adult patients on clinical laboratory platforms but provided no statistical assessment on discriminatory ability or cutoff concentrations.¹¹ Also, a subgroup analysis of a previous meta-analysis⁴ pointed toward more accurate AKI prediction for a cutoff value ≥ 150 ng/mL on clinical platforms compared with measurements on research assays. However, this meta-analysis did not separately report on urinary or plasma NGAL.⁴ Neither study provided cutoff concentrations at high (95%) sensitivity or high (95%) specificity.

NGAL is one of the most extensively investigated renal biomarkers, but addressing the mentioned issues is needed for meaningful interpretation of biomarker test results. Determination of potentially applicable cutoff values in different settings has been recommended to be the next important step in validation of kidney biomarkers aiming at improved patient care.^2,6,87 Therefore, providing specimen-specific NGAL cutoff concentrations measured on clinical laboratory platforms is needed.

Our finding that predictive ability increased with more severe AKI (DOR up to 16) is biologically plausible. However, NGAL level failing to show perfect AKI prediction in the present meta-analysis may be interpreted as a shortcoming of NGAL (index test) or as a shortcoming of Scr (reference test) or both because these tests reflect different types of kidney injury.

Urinary and plasma NGAL may indicate tubular injury before declining filtration function but concentrations and discriminative ability may also be influenced by systemic conditions such as sepsis⁶⁹ or NGAL originating from nonkidney tissues.⁸⁸ In contrast, Scr level may exhibit limited sensitivity and specificity⁸⁹ for accurately estimating rapid changes in glomerular filtration rate⁶³ and may not indicate tubular pathology. Such understanding may point toward possible dissociation of tubular injury and glomerular functional decline precluding NGAL or other tubular biomarkers^5,90,91 from predicting Scr-based AKI with greater accuracy.⁹²

Accordingly, the concepts of subclinical and hemodynamic AKI may help interpret the findings of the present study.⁹³ Subclinical AKI (false-positive test in relation to Scr) may explain adverse outcomes in patients with high NGAL concentrations but without subsequent development of Scr-based AKI.^94–96 In a complementary fashion, patients with Scr-based AKI and low NGAL concentrations have been attributed to hemodynamic AKI (false negative). ⁹⁷ Such scenarios would reduce the accuracy of NGAL in predicting Scr-based AKI.⁸⁹ This is reflected by the finding of our meta-analysis that >30% of patients were identified as potentially having misclassified AKI or non-AKI using an Scr-based AKI definition (23.5% subclinical [NGAL-positive/Scr-based AKI-negative]; 8.0% hemodynamic [NGAL-negative/Scr-based AKI-positive]). The observed proportions of patients with subclinical or hemodynamic AKI are in line with reports from previous studies for NGAL and other kidney biomarkers.^{41,51,94,95,98}

Finally, a recent meta-analysis⁹⁹ based on 891 critically ill patients from 4 studies showed similar accuracy of a test based on the combination of urine concentrations of tissue inhibitor of metalloproteinase 2 and insulin-like growth factor binding protein 7 ([TIMP-2] × [IGFBP7]) in predicting severe AKI compared with that of NGAL reported in the present study. For the high-specificity cutoff of [TIMP-2] × [IGFBP7] (>2.0 ng/mL²/1,000)⁹⁹ and plasma NGAL (>364 ng/mL [present study]), statistical indexes were 93% versus 100% for specificity, 45% versus 44% for sensitivity, 11.4 versus 14.9 for DOR, and 0.84 versus 0.80 for AUC, respectively. However, discussion of whether the mentioned limitations and concepts may also apply to kidney biomarkers other than NGAL is beyond the scope of the present meta-analysis.

We found that the literature-based investigation was limited because most studies reported outcome measures for AKI or severe AKI only, but not for AKI-D, and vice versa, although the omitted data might be calculable from the original studies’ data sets. Reanalysis of individual-study-level data therefore offers meaningful advantages.¹⁰⁰ Aggregation of reanalyzed individual-study-level data for all 3 outcome measures provided the ability to include and meta-analyze outcome data from previously published studies that have not been reported before. Specifically, this individual-study-data meta-analysis implied standardization of outcome data across multiple data sets and enabled uniform synthesis of cutoff concentrations and their predictive indexes from 3 predefined points on the summary ROC curve (95%/high sensitivity, optimal, 95%/high specificity; Items S8 and S9) complementing the AUC for clinical decision making.¹⁴

The mixed-effects approach²⁰ used for this individual-study-data meta-analysis has several advantages compared with existing methods.¹⁰¹ A 1-step approach avoids accumulation of type one error. The number of thresholds does not have to be identical across included studies, concrete values are considered, and the approach is applicable with extreme values. Therefore, individual-study-data meta-analysis allowed for more precise calculation of predictive accuracy by considering various distributions of individual full ROC curves with several pairs of thresholds and bivariate outcome of sensitivity and specificity.²⁰

The individual-study-data reassessment focused on NGAL measurement in advance of AKI diagnosis or commencement of KRT. Patients with known AKI or KRT initiation within 24 hours of NGAL assessment were excluded. These multicontinental literature-based and individual-study-data meta-analyses included a substantial sample size and number of AKI and AKI-D events. Finally, all studies used certified widely available clinical laboratory platforms featuring superior turn-around times and reproducibility comparable to enzyme-linked immunosorbent assay or research kits.¹⁰² However, interassay variability should be taken into account when interpreting results from the various platforms measuring NGAL, which may affect the transferability of results.^102,103

Our analyses were limited to an adult population and did not include unpublished studies potentially influencing the results of systematic reviews.¹⁰⁴ Not all authors who were initially requested to contribute to the individual-study-data meta-analysis responded or provided reanalyzed data. However, in the funnel plots, in which literature-based and individual-study-data meta-analyses were presented together, no pattern potentially indicating systematic differences between the 2 cohorts was apparent (Item S3).

All included studies used Scr level as reference for the diagnosis of AKI. Authors of each study included in the individual-study-data meta-analysis returned data according to the RIFLE criteria. Rarely, data were returned on other AKI definitions, which precluded further analysis. In support of the decision favoring RIFLE criteria, there is literature indicating noninferior discriminative value of the RIFLE classification in predicting adverse kidney-related outcomes compared with AKIN or KDIGO criteria.^105–108 In the literature-based meta-analysis, the AUC was reported separately for RIFLE, KDIGO, and AKIN criteria. The present study was limited by clinical practice variation regarding KRT initiation for AKI.^109,110 Cutoff concentrations and corresponding diagnostic indexes were derived from an individual-study-data meta-analysis, which cannot replace an appropriately powered end point study for cutoff derivation. Therefore, identified cutoff concentrations require further prospective evaluation. Finally, NGAL tests are currently not approved by the US Food and Drug Administration for diagnostic use in the United States.

The present meta-analysis provides urinary and plasma cutoff concentrations for the subsequent development of severe AKI or AKI-D, potentially facilitating more standardized judgment by nonkidney specialists and nephrologists in unclear clinical situations.^7,30 Although acknowledging the heterogeneity of clinical context, limitations of this meta-analysis, and nonperfect match between kidney function and injury markers, nonetheless urinary and plasma NGAL cutoff concentrations may complement the identification of patients at high kidney risk in clinical research and practice. Our findings support the conclusion that patients with NGAL concentrations below the 95% sensitivity cutoff are at very low risk for developing Scr-based AKI within the next 24 hours. Supported by the understanding that AKI occurs on a continuum,² NGAL concentrations between identified 95% sensitivity and 95% specificity cutoffs may call for intensified kidney observation. It is intriguing to speculate that sequential measurements of NGAL and consideration of trends and delta values might be valuable, especially in patients in whom absolute NGAL values do not exceed or fall below the suggested threshold.¹¹¹ However, the present meta-analysis did not address this question. For NGAL concentrations above the identified 95% specificity cutoff, initiation of kidney care bundles or KRT may be considered even before Scr level increases.^112–115 Nonetheless, we acknowledge that in such patients, interventions other than those recommended by the KDIGO guideline² for prevention and treatment of AKI are not reported to improve outcome.¹¹⁶ A complementary direct comparison between plasma NGAL and contemporaneous Scr levels for predicting the necessity of KRT might favor NGAL over Scr level, but the analysis was post hoc and thus results are preliminary.

Finally, further studies beyond assessment of the relationship between kidney biomarkers and Scr-based AKI are needed to refine the assessment of potentially applicable NGAL cutoff concentrations used in conjunction with other clinical and diagnostic findings.⁷

There is continued clinical interest in improved risk assessment and early identification of AKI. Notwithstanding the heterogeneity of clinical context and other limitations of this meta-analysis, derived urinary and plasma cutoff concentrations may complement the identification of patients at high risk for the development of AKI, severe AKI, and AKI-D.

Supplementary Material

Supplemental Data - NIHMS1719441

Item S1: Extensive methods.

Item S2: Results, identification of studies, and study characteristics.

Item S3: Quality assessment.

Item S4: Descriptive assessment of NGAL values for individual-study-data meta-analysis.

Item S5: Subgroup analysis for urine output criterion of the AKI definition (individual-study-data meta-analysis).

Item S6: Extended accuracy analyses (individual-study-data meta-analysis).

Item S7: Literature-based meta-analysis of NGAL diagnostic performance.

Item S8: Invitation and data request sheet for the individual-study-data meta-analysis.

Item S9: Individual study–level data for individual-study-data meta-analysis.

PLAIN-LANGUAGE SUMMARY.

This meta-analysis provides neutrophil gelatinase-associated lipocalin (NGAL) cutoff concentrations for kidney risk prediction. Recent practice guidelines for acute kidney injury (AKI) renewed the importance of the earliest possible detection of AKI and adjustment of treatment accordingly. Literature-based meta-analysis revealed that the predictive value of NGAL measured on clinical laboratory platforms may improve the prediction of AKI risk. NGAL cutoff concentrations in clinical settings have not been sufficient. We performed an individual-study-data meta-analysis that demonstrated results similar to the literature-based meta-analysis regarding NGAL’s discriminative ability. Using an individual-study-data meta-analysis that incorporated confounding variables enabled derivation of cutoff concentrations for NGAL to identify patients at risk for severe stages of AKI, including the associated need for dialysis. Notwithstanding the heterogeneity of clinical context, urinary and plasma concentrations of NGAL may enable identification of patients at high risk for AKI in clinical research and practice.