Impact of Using Biomarkers in Eligibility Criteria in Acute Kidney Injury Randomized Controlled Trials: A Systematic Review and Methodological Meta-Analysis

Abstract

Rationale & Objective

Many recent randomized controlled trials (RCTs) for acute kidney injury have incorporated biomarkers as part of their eligibility criteria. The sample size calculation of these trials is often based on prior studies that did not include such criteria. This meta-analysis evaluates the impact of a biomarker-based enrichment strategy on the rate of anticipated versus observed primary events and explores the effect of integrating such biomarkers on statistical power.

Study Design

We performed a PRISMA-guided systematic review and methodological meta-analysis of RCTs extracted from 6 databases.

Setting & Study Population

The RCTs included patients at risk for or diagnosed with acute kidney injury.

Selection Criteria for Studies

Clinical trials with a biomarker-based eligibility criterion and either a renal or mortality primary outcome, published between 2010 and 2023.

Data Extraction

Data were extracted independently by 2 reviewers.

Analytical Approach

The absolute risk difference between anticipated and observed event rates was measured for all patients and stratified for control and intervention groups.

Results

Fourteen RCTs involving 3,817 patients were included. Biomarkers of interest were neutrophil gelatinase-associated lipocalin, TIMP-2∗IGFBP7, proteinuria, serum albumin, NT-pro-BNP, homocysteine, and uric acid. The pooled absolute risk difference between anticipated and observed event rates was 0.12 (95% CI; 0.06-0.17). In the control and interventional groups, the absolute risk differences were 0.10 (95% CI, 0.01-0.19) and 0.12 (95% CI, 0.06-0.18), respectively. For kidney damage biomarkers, the risk difference was 0.13 (95% CI, 0.06-0.20). The mean preplanned and achieved statistical powers were 0.84 ± 0.07 and 0.43 ± 0.28, respectively, with no improvement when extrapolating the preplanned sample size.

Limitations

Most RCTs were underpowered for their primary outcome. There was a high level of heterogeneity.

Conclusions

There is a difference between expected and observed incidence rates that may be partly attributed to biomarker-based enrichment methods. These findings highlight the need for rigorous validation of biomarkers before their incorporation into the eligibility criteria of RCTs.

Plain-Language Summary

Clinical trials in acute kidney injury increasingly use specific biomarkers to determine patient eligibility. These tests are meant to identify patients who are most likely to benefit from the intervention. We systematically reviewed 14 randomized trials to assess how this approach affects study results. We found that the number of primary clinical events observed often differed substantially from what researchers expected when planning the trials. Many studies ended up with fewer events and lower statistical power than anticipated. In addition, the use of biomarkers led to the exclusion of many patients during screening. These findings suggest that biomarkers should be carefully validated before being used to select patients for acute kidney injury trials.

There is a growing interest in novel approaches for assessing the risk and extent of kidney damage caused by renal disorders beyond traditional functional metrics such as serum creatinine. In acute kidney injury (AKI), numerous biomarkers, such as the neutrophil gelatinase-associated lipocalin (NGAL) and TIMP-2∗IGFBP7 (NephroCheck), have been reported as noninvasive predictors of structural damage.^1,2 They can help clinicians identify patients at risk for AKI following a potential kidney insult as well as determine etiology and possibly predict short and long-term renal prognosis. The 2012 Kidney Disease: Improving Global Outcomes (KDIGO) Clinical Practice Guidelines for AKI highlighted the importance of exploring the role of such biomarkers in early detection, differential diagnosis and prognosis.³ A large body of evidence has since been reported across diverse AKI populations. The 23rd Acute Disease Quality Initiative Consensus Conference further endorsed the integration of biomarkers into clinical practice and research, alongside standard serum creatinine and urine output, to improve risk stratification, diagnosis and management of AKI.⁴

As the paradigm of AKI has shifted over recent decades, some randomized controlled trials (RCTs) have incorporated biomarkers as part of their eligibility criteria. Investigators aim to identify and study specific AKI phenotypes and better select participants who are most likely to respond to a given intervention.^5,6 For example, using a proteinuria-based exclusion criterion in an RCT investigating an intervention for hepato-renal syndrome may help select AKI patients with cirrhosis most likely to benefit from the intervention, while excluding patients with glomerular diseases in whom vasoconstrictive therapies are unlikely to be successful. The integration of biomarkers in RCTs has also been hypothesized to improve effect size and statistical power, thereby reducing the required sample size, which may positively impact the cost and completion time of clinical trials.⁵ However, sample size calculations are typically based on prior studies that did not include such enrichment criteria. The impact of this practice on the occurrence of the primary outcome, compared with the expected rate of events, remains uncertain. Therefore, we conducted this systematic review and methodological meta-analysis to explore the effect of using biomarkers as enrichment eligibility criteria in AKI RCTs assessing renal outcomes.

Methods

This study was reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Guidelines.⁷ The protocol was registered in the PROSPERO International Prospective Registry of Systematic Reviews (CRD42022337990). This work constitutes a methodological meta-analysis evaluating trial design performance—specifically anticipated versus observed event rates—rather than treatment effect pooling. Unlike conventional meta-analyses, which aim to synthesize effect sizes and determine therapeutic efficacy, this approach examines how biomarker-based enrichment eligibility criteria influence outcome incidence and statistical power.

Eligibility Criteria

We searched for nephrology RCTs involving AKI patients with biomarkers as part of the listed eligibility criteria for all participants. Biomarker measurement should not be included in the primary endpoint and were used exclusively for participant selection. Eligible clinical trials examined standard renal outcomes or composite events as their primary outcome. The following outcomes and composites were included: incidence of AKI, creatinine-based variation in the estimated glomerular filtration rate, death from renal cause, all-cause mortality, meeting dialysis criteria with or without subsequent initiation of dialysis, and initiation of dialysis or renal replacement therapy dependence. Accepted composite outcomes were major adverse kidney events, which include the development or progression of chronic kidney disease, the initiation of maintenance dialysis, or death from any cause at 30, 60, or 90 days; or major adverse renal or cardiac events, which include kidney failure with dialysis, AKI with or without dialysis, contrast-associated nephropathy, acute myocardial infarction, angina, stent occlusion/thrombosis, stroke, transient ischemic attack, or death.

Studies involving pediatric populations and abstracts without full published reports were excluded. Anticipated event rates were derived from prespecified sample size calculations reported in each trial and were not recalculated using participant-level data, as individual-level biomarker-screening distributions were unavailable. When sample size calculation was not available in the published manuscript or trial registries (Clinicaltrial.gov or EudraCT EMA), corresponding authors were contacted; studies were excluded if no response was obtained.

Outcomes

The primary outcome of this study was the difference between the anticipated event rate described in the published trial and the observed event rate from reported results. The secondary outcomes of interest included assessment of the impact of biomarker-based eligibility criteria on planned versus observed statistical power and determination of the proportion of otherwise eligible participants excluded solely due to biomarker criteria.

Literature Search

According to the predetermined protocol, the literature search was developed by FC, WBS, and JMC in cooperation with a trained medical librarian. We searched for RCTs published between 2010 and 2023 in 6 databases: MEDLINE, EMBASE, GOOGLE Scholar, EBM Reviews, MedRxiv, and PROSPERO. The search query included medical subject headings and text words related to Clinical Trials, Renal Insufficiency, Dialysis, and Biomarkers. The MEDLINE research query is shown in the supplementary material (Table S1).

Study Selection

The literature search results were uploaded to the Rayyan QCRI web platform for the first-step screening of titles and abstracts. Duplicates and sub-analyses including the same population were excluded. This step was performed independently by 2 reviewers (FC and RA). Following this first screening, full reports of eligible titles or abstracts that appeared to meet the inclusion criteria were retrieved and assessed for eligibility in a second screening step (FC and MS). For studies with missing data, corresponding authors were contacted to obtain additional information; trials were excluded if data could not be retrieved. Discrepancies were resolved through discussion.

Data Abstraction and Quality Assessment

The systemic review management software Covidence was used to extract data from eligible studies by 2 review authors (FC and MS). The following elements were collected: year, country, and language of publication; trial registration number; planned statistical power; predetermined sample size; inclusion and exclusion criteria; type of kidney biomarker used as eligibility criteria, positivity/negativity threshold of that biomarker, number of participants excluded based on the biomarker criteria; control and experimental interventions; target and recruited sample sizes; primary outcome (as per the endpoint used in sample size calculation); anticipated rate of events for control and intervention groups. The risk of bias was assessed on the primary outcome using the Revised Cochrane risk of bias tool for randomized trials (RoB2).⁸

Statistical Analysis

For each study, the risk difference between the anticipated and observed event rates of the primary outcome was calculated and synthetized using a random-effects model (DerSimonian-Laird method). Results are presented as the absolute risk difference given that biomarker-based eligibility criteria may either increase or decrease the occurrence of the primary outcome depending on the AKI population and research question. Absolute risk difference was used to summarize deviation magnitude irrespective of direction, whereas signed risk differences were displayed at the individual-study level to preserve directional information. Analyses were stratified by treatment arm (intervention and control). A subgroup analysis was performed for kidney damage-specific biomarkers.

We assessed the statistical powers of each RCT by comparing the prespecified (planned) power and the achieved (post-hoc) power. We also calculated the hypothetical statistical power based on the observed event rate, assuming the preplanned sample size was achieved. When available, we also reported the number of potentially eligible participants excluded based solely on the biomarker criterion. Table S2 (supplementary material) defines these metrics.

Descriptive analyses are presented using frequencies for categorical variables and means with standard deviation for continuous variables. The α error was set at 5%. A 95% confidence interval was used. Statistical heterogeneity was tested using I² Statistic (0%-30%: might not be important, 30%-60%: moderate heterogeneity, 50%-90%: substantial heterogeneity, and 75%-100%: considerable heterogeneity). Statistical analyses were performed with Revman 5.4. The Robvis online tool⁹ was used to create the risk of bias figure.

Results

Search Results

A total of 14,473 studies were identified from the literature search, of which 13,656 were excluded with the first-step screening of titles and abstracts. An additional 272 records were discarded during second-step screening. Of the 21 records remaining, seven were excluded after a thorough review. Overall, 14 clinical trials published from 2016 to 2021 were included in the analysis involving 3,817 patients. The PRISMA flow diagram of this review, presented in Fig 1, shows the reasons for exclusion.

Figure 1.

Flow diagram.

The characteristics of included RCTs are displayed in Table 1. Biomarkers as part of eligibility criteria were distributed as follows: 4 studies with NGAL,10, 11, 12, 13 3 studies with NephroCheck,14, 15, 16 3 studies with proteinuria,17, 18, 19 1 study with albuminemia,²⁰ 1 study with NT-pro-BNP,²¹ 1 study with plasma homocysteine,²² and one study with plasma uric acid.²³ The primary outcomes of the selected trials included the occurrence of AKI,^11,14, 15, 16^,20 the occurrence of contrast-associated nephropathy,^22,23 reversal of hepato-renal syndrome,^17,19 reduction of the estimated glomerular filtration rate¹⁸ and all-cause mortality (28 or 90 days).^10,12,13

Table 1.

Characteristics of Included Studies

Study	Biomarker	Intervention	Primary Outcome	Anticipated			Observed
				(Primary Outcome)			(ITT Primary Outcome)
				Statistical Power	Control Group Size	Inter-vention Group Size	Control Group Size	Intervention Group Size
Boyer et al,17 (2016)	Proteinuria (24h)	Intravenous terlipressin	Reversal of hepato-renal syndrome	0.9	75	75	99	97
Gocze et al,14 (2018)	NephroCheck	KDIGO care bundle	AKI within 7 days after surgery	0.8	69	69	61	60
Lee et al,20 (2016)	Plasma albumin	Intravenous 20% albumin	AKI within 48 hours after cardiac bypass surgery	0.8	103	103	101	102
Lumlertgul et al,10 (2018)	Plasma NGAL	Early RRT	28-day all-cause mortality	0.8	450	450	60	58
Ma et al,23 (2022)	Plasma uric acid	Oral febuxostat and intravenous hydration	Contrast associated nephropathy 48 hours after contrast administration	0.9	65	65	102	100
Meersch et al,15 (2017)	NephroCheck	KDIGO CT surgery bundle	AKI within 72 hours after cardiac surgery	0.8	138	138	138	138
Nunez et al,21 (2020)	NT-pro-BNP	Loop diuretics with dosage based on plasma levels of CA125	Improvement of renal function	0.8	77	77	81	79
Palevsky et al,18 (2016)	Proteinuria (ACR)	Oral losartan and lisinopril	Reduction of eGFR	0.8	925	925	724	724
Peng et al,22 (2021)	Plasma homocysteine	Oral folic acid	Contrast associated nephropathy 72 hours after contrast administration	0.8	202	202	209	203
Ribitsch et al,11 (2017)	Urinary NGAL	Intravenous saline	Moderate to severe AKI within the first day of admission	NS	108	108	6	4
Schanz et al,16 (2019)	NephroCheck	Nephrology consultation	Moderate to severe AKI within the first day of admission	NS	113	113	46	54
Srisawat et al,12 (2018)	Plasma NGAL	Early RRT	28-day all-cause mortality	NS	131	131	20	20
Wong et al,19 (2021)	Proteinuria (24h)	Intravenous terlipressin	Reversal of hepato-renal syndrome	0.9	100	200	101	199
Zarbock et al,13 (2016)	Plasma NGAL	Early RRT	90-day all-cause mortality	0.8	115	115	119	112

Abbreviations: ACR, albumin-creatinine ratio; AKI, acute kidney injury; eGFR, estimated glomerular filtration rate; ITT, intention to treat; NS, not specified; RRT, renal replacement therapy.

Risk of Bias

The quality assessment and risks of bias are presented in Fig S1 in the supplementary material. Overall, there were concerns regarding bias in the randomization process^10,11 and deviations from the intended intervention^10,11,13,14 in some studies. Some concerns were noted for missing outcome data in one study.¹³ In contrast, all studies were assessed to have a low risk of bias regarding outcome measurement and selection of reported results.

Primary Analysis

In the pooled analysis, the absolute risk difference was 0.12 (95% CI, 0.06-0.17; P < 0.0001) between anticipated and observed event rates. For control groups, the risk difference varied from −0.18,¹⁰ in which more events were observed (58%) than anticipated (40%); to +0.37¹² (observed [45%] vs anticipated [82%]), while the absolute risk difference was 0.10 (95% CI, 0.01-0.19; P = 0.02). For intervention groups, the risk difference varied from −0.32¹⁰ to +0.43.²³ The absolute risk difference was 0.12 (95% CI, 0.06-0.18; P < 0.0001). The analysis revealed high heterogeneity (I²: 87%). The Forest plots in Fig 2 and Fig S2 illustrate these findings.

Figure 2.

Forest plot of the difference between the observed event rate and the anticipated event rate on the primary outcome. Absolute risk difference (with 95% confidence interval) is shown by intervention arm for each study. The pool analysis is based on a random effect model (DerSimonian-Laird method). Cont, control group; Exp, experimental group.

Kidney Damage Biomarkers Subgroup

Fig S3 in the supplementary material shows the subgroup analysis for studies with kidney damage biomarkers (NGAL, NephroCheck, and proteinuria) totalizing 777 participants. In that subgroup, the absolute risk difference between the anticipated and observed event rates was 0.13 (95% CI, 0.06-0.20; P = 0.0001) in the pooled analysis.

Secondary Analyses

Anticipated Versus Observed Statistical Power

Five studies (35.7%)^15,17,21, 22, 23 reached the preplanned sample size, whereas only two^15,18 achieved the minimum prespecified statistical power. The mean anticipated and observed statistical powers for the primary outcome were 0.84 ± 0.07 and 0.43 ± 0.28, respectively. Fig 3 illustrates 3 different scenarios: (1) the achieved statistical power based on the reported sample size and event rate; (2) the anticipated statistical power if the preplanned sample size and anticipated event rate were respected; and (3) the hypothetical statistical power based on the reported event rate assuming the preplanned sample size was achieved. This post-hoc analysis illustrates that none of the underpowered studies would have reached adequate statistical power even if the preplanned sample size had been achieved.

Figure 3.

Illustration of anticipated and observed statistical powers using three scenarios. Red: based on the study results (observed sample size and event rate). Blue: hypothetical statistical power using preplanned sample size and the expected event rate. Green: hypothetical statistical power using the observed event rate and the preplanned sample size. ^aRibitsch and Srisawat studies had small sample sizes, which should be considered when interpreting the results.

Exclusion of Otherwise Potentially Eligible Participants

Eight trials11, 12, 13, 14, 15, 16^,21,22 reported the number of participants who were rejected based on the biomarker criterion. Out of 6,277 screened candidates, 4,227 were excluded, resulting in an overall exclusion rate of 67.3%. Six out of these 8 studies had kidney damage biomarkers (NGAL or NephroCheck), with an overall exclusion rate of 50% (1,329 out of 2,657 screened candidates).

Discussion

This meta-analysis aimed to determine the effect of using biomarkers as an enrichment strategy when designing RCTs for AKI populations. Our study found a substantial difference between the anticipated and observed event rates, which was statistically significant in intervention groups and a subgroup analysis for kidney damage biomarkers. This analysis also revealed a high participant exclusion rate of 67% with a biomarker-enriched eligibility strategy.

The difference between the expected and observed event rates did not follow a specific pattern and varied between the different types of interventions. As shown in Fig 2, some studies had more events than anticipated while others had fewer. This heterogeneity is likely multifactorial and may not be solely attributed to the biomarker enrichment strategy. The objective was not to determine whether biomarker-enriched trials are uniquely susceptible to event-rate misestimation but to evaluate how enrichment strategies, when implemented during eligibility screening, may impact trial performance. The divergence observed cannot be attributed to a unique explanation but may reflect a combination of factors, including an enriched high-risk phenotype with more chance to respond to the intervention, differential risk of withdrawal or retention, and unknown confounders not linked to the intervention or biomarker. Also, included RCTs in this meta-analysis had distinct patient populations with varying risks, clinical characteristics, inclusion and exclusion criteria, AKI definitions, and interventions. Those factors can also lead to a methodological bias resulting in a different event rate than originally expected, as highlighted by previous meta-analyses.^24,25

This analysis incorporated multiple biomarkers that differ in their role and association with AKI, but also in the way that they were measured and used (ie, inclusion vs exclusion criterion). An ideal biomarker used as an enrichment criterion could improve population selection, thus increasing the overall event rate and the chance of confirming the effectiveness of an intervention. This would minimize the needed sample size and theoretically lead to an increase in effect size, improving the statistical power for observing a clinically and statistically significant difference. The study by Meersch et al¹⁵ illustrates this concept; cardiac surgery patients at high risk for AKI were selected based on a kidney stress biomarker, NephroCheck, which was shown to have good accuracy in predicting early AKI incidence in this population. This criterion allowed the investigators to exclude low-risk patients. The AKI event rate was higher compared with a previous trial with a similar study population,²⁶ but with no biomarker-based eligibility criterion. Most importantly, it led to the preferential recruitment of a population susceptible to benefit from the intervention (KDIGO bundle). As a result, this biomarker increased the observed effect size while minimizing the required sample size.

To improve the strategy of integrating biomarker-based eligibility criteria in AKI RCTs, there should be a causal, or at least probable, mechanistic pathway between the biomarker and kidney injury (eg, NGAL).^27,28 Alternatively, it could be shown to be associated with a response to therapy (eg, NT-ProBNP). However, some biomarkers included in this meta-analysis have weak epidemiological associations derived from previous observational studies, such as homocysteine and uric acid with contrast-associated nephropathy.^22,23 They might not truly capture the risk of AKI or the treatment’s effect and thus have little to no impact on population selection. No single biomarker is entirely specific, as even the most strongly associated ones often lack a direct mechanistic basis.²⁷ Determining the optimal biomarker cut-off for predicting the occurrence of kidney injury or a response to treatment is also uncertain and varies greatly between studies, particularly for NGAL.²⁹

Beyond eligibility criteria, biomarkers can serve as surrogate endpoints to support biological plausibility in early phase trials,^30,31 identify subgroups with differential treatment response, including post-hoc stratification,^32,33 detect early toxicity or injury signals,³⁴ and provide early indications of effectiveness to inform adaptive strategies such as sample size re-estimation or population enrichment.

Interestingly, the absolute risk difference observed in this meta-analysis was slightly more pronounced in a subgroup analysis with kidney damage biomarkers. NGAL, a well-established tubular injury marker, was described as having one of the best predictive accuracies for AKI.¹ However, three of the 4 included studies using NGAL as an eligibility criterion had mortality as their primary endpoint^10,12,13 and showed no particular trend between the observed and anticipated event rate. In contrast, all 3 trials involving NephroCheck had AKI incidence or progression as their main outcome of interest.14, 15, 16 Therefore, the impact on the observed effect size when integrating biomarkers to identify high-risk patients may be greater when considering endpoints that are pathophysiologically associated with said biomarker.

The secondary analyses of this study were not intended to interpret therapeutic efficacy but to quantify the potential operational consequences of biomarker enrichment on feasibility and study design, since such enrichment may directly alter screen-fail proportions, endpoints incidence, and ultimately statistical power. Twelve out of 14 studies in this meta-analysis did not reach their predetermined statistical power when performing post-hoc assessment. When adjusting for the preplanned sample size as illustrated in Fig 3, these trials remained underpowered, which may also reflect either an overestimation or a lack of efficacy of the studied interventions. As illustrated by Zarbock et al,¹³ even an adequately powered study reporting statistical efficacy on its primary outcome could still have a lower post-hoc statistical power attributed to a slightly lower effect size than anticipated. In the same vein, a methodological meta-analysis of cardiovascular trials showed that most RCTs overestimate their pretrial event rate and effect size.²⁵ It was also previously shown that highly cited biomarker studies tend to inflate the true effect size and negatively impact the observed statistical power as compared to larger clinical studies and meta-analyses.³⁵

Our study also found that a biomarker-based enrichment method in RCTs is associated with a high exclusion rate among screened candidates. Although outcome incidence among excluded individuals was unavailable, these exclusion rates provide important insight into feasibility constraints. High screen-failure rates can limit the generalizability of trial findings and complicate their translation into clinical practice, especially in settings where biomarker testing is not routinely available. In addition, extensive screening requirements may hinder trial feasibility by increasing the number of patients who must be assessed to achieve target enrollment, thereby adding operational complexity.⁵ To address these challenges, adaptive trial designs have been proposed to improve feasibility and participant inclusivity.³⁶ One such strategy involves initially broadening eligibility, followed by an interim analysis to identify biomarker-positive subgroups with higher event rates.³⁷ In AKI RCTs, this approach may mitigate overestimation of biomarker predictive performance while better aligning statistical power with observed treatment effects.

There are several limitations to this study. First, the title, abstract and keywords of RCTs rarely report biomarkers-based eligibility criteria, the literature search query had to be liberal to ensure the identification of RCTs of interest. As a result, more than 14,400 references had to be screened. Second, the literature search was limited to English publications, which may introduce a language bias and a lack of representativeness in included studies. Some studies were excluded because anticipated event rates were not available, which limits generalizability. Most RCTs had high to moderate risks of bias, especially arising from the randomizing process and deviations from the intended intervention. They were notably underpowered for their primary analysis. However, the objective of this meta-analysis was not to confirm the benefit of a particular intervention but to explore the methodological effects of integrating biomarkers in eligibility criteria. Post-hoc power calculation is often considered misleading, as it could be used to corroborate positive findings while justifying negative ones simply as a result of an inadequate sample size.³⁸ Our objective was instead to explore the effect of the discrepancy between the anticipated event rate at the time of designing the study and the observed final event rate on the overall statistical power. However, we could not establish if the observed difference reflects inadequate statistical power or a lack of effectiveness of the studied intervention. Most studies reported the anticipated event rate and the preplanned sample size in the final publication. We had to extrapolate these data based on references cited in 2 studies,^10,21 as some data were lacking in the final publication or the study protocol. The heterogeneity across all included studies was high, including for the primary endpoint reported, the sample size, the population of interest and the biomarker used. Finally, we could not determine with certainty the relative importance played by a biomarker criterion in the absolute risk difference observed. Nonetheless, we aimed to explore if the use of biomarkers as an enrichment strategy in AKI trials impacted the observed event rate and, despite the above limitations, we achieved this objective.

In conclusion, this study explored the impact of a biomarker-based enrichment strategy when selecting participants in AKI clinical trials. Our findings highlight the need for rigorous validation of biomarkers in populations of interest to determine their potential impact on the event rate before their incorporation as part of eligibility criteria in RCTs.

Supplementary Materials

Supplementary File (PDF)

Figures S1-S3; Tables S1-S2