Performance of GFR Slope as a Surrogate End Point for Kidney Disease Progression in Clinical Trials: A Statistical Simulation

Significance Statement

Randomized controlled trials of CKD traditionally use clinical events that happen late in the course of CKD progression as study end points. Doing this requires large sample sizes and long follow-up periods that can make the studies very costly. The authors use statistical simulations to investigate circumstances in which using the change in GFR over time or GFR slope as a study end point reduces the required sample size or trial duration compared with clinical end points. They found that GFR slope performs better than clinical end points when patients’ initial GFRs are high and the treatment has no acute effect on GFR. The results along with other recent studies suggest GFR is a valid surrogate end point for CKD clinical trials that may allow for more efficient trials and help speed the development of new CKD therapies.

Visual Abstract

Abstract

Background

Randomized trials of CKD treatments traditionally use clinical events late in CKD progression as end points. This requires costly studies with large sample sizes and long follow-up. Surrogate end points like GFR slope may speed up the evaluation of new therapies by enabling smaller studies with shorter follow-up.

Methods

We used statistical simulations to identify trial situations where GFR slope provides increased statistical power compared with the clinical end point of doubling of serum creatinine or kidney failure. We simulated GFR trajectories based on data from 47 randomized treatment comparisons. We evaluated the sample size required for adequate statistical power based on GFR slopes calculated from baseline and from 3 months follow-up.

Results

In most scenarios where the treatment has no acute effect, analyses of GFR slope provided similar or improved statistical power compared with the clinical end point, often allowing investigators to shorten follow-up by at least half while simultaneously reducing sample size. When patients’ GFRs are higher, the power advantages of GFR slope increase. However, acute treatment effects within several months of randomization can increase the risk of false conclusions about therapies based on GFR slope. Care is needed in study design and analysis to avoid such false conclusions.

Conclusions

Use of GFR slope can substantially increase statistical power compared with the clinical end point, particularly when baseline GFR is high and there is no acute effect. The optimum GFR-based end point depends on multiple factors including the rate of GFR decline, type of treatment effect and study design.

CKD is common and harmful, causing kidney failure in its late stages, but with few therapies.¹ Pivotal randomized controlled trials (RCTs) traditionally use ESKD, eGFR<15 ml/min per 1.73 m² or doubling of serum creatinine as clinical end points.² However, these are late events in CKD progression, leading to costly requirements for large sample sizes and long follow-up.

In March 2018, the National Kidney Foundation (NKF), in collaboration with the Food and Drug Administration (FDA) and European Medicines Agency (EMA), sponsored a scientific workshop “Change in Albuminuria and GFR as End Points for Clinical Trials in Early Stages of CKD” to evaluate surrogate end points for trials of CKD progression. This manuscript is part of a series of manuscripts to report the analyses undertaken in conjunction with this workshop. Other manuscripts in this series demonstrate consistent epidemiologic association between GFR slope and subsequent occurrence of the clinical end point,^3,4 as well as strong agreement between treatment effects on the clinical end point and treatment effects on GFR slope in previous CKD RCTs.⁵ However, these manuscripts did not define the circumstances in which application of GFR slope can improve statistical power to allow smaller sample size or shorter duration while controlling the risk of false conclusions of benefit or harm.⁶

Kidney failure, defined by uremic symptoms signifying a need for kidney replacement therapy, occurs when GFR reaches a level of about 5–15 ml/min per 1.73 m², a narrow range relative to normal GFR. For this manuscript, we applied the relationship between GFR and kidney failure in conjunction with models for GFR trajectories based on previous CKD RCTs to perform statistical simulations to determine conditions in which GFR slope can increase statistical power compared with the clinical end point and end points based on 30% or 40% GFR decline,⁷ while preserving a low risk of false conclusions when there is no treatment effect on the clinical end point.

Methods

Simulation Strategy

Our simulations were based on the same models for GFR trajectories that were used in a previous simulation study which evaluated end points based on 30% or 40% GFR decline,⁸ with updates to input parameters reflecting the distributions of these parameters in an expanded database with 47 randomized treatment comparisons.⁵ We overview our methods below; additional details for the methods used for analyses of previous studies and for the simulation models are provided in Supplement Appendices 1 and 2, including Supplemental Tables 1–3. Results of analyses of previous studies used to update key input parameters of the simulations are provided in Appendix 3 including Supplemental Tables 4 and 5, and Supplemental Figure 1.

GFR Trajectories in the Control Group

We simulated GFR trajectories on the basis of a growth curve model in which each subject’s GFR measurements varied randomly about subject-specific linear trajectories defined by random slopes and intercepts.⁹

Long-Term Treatment Effects on GFR Slope

Some treatments may have larger effects on GFR decline among fast progressors with steep GFR slopes than among slow progressors.^10,11 Thus, we simulated three types of long-term treatment effects: (1) a uniform effect, in which the same treatment effect is assumed for all patients, irrespective of their progression rates; (2) a proportional effect, in which the treatment effect is proportional to the rate of GFR decline among patients with negative slopes but the treatment has no effect among patients whose slopes would have been >0 without the treatment; and (3) an intermediate model halfway between the uniform and proportional effect models.

Short-Term Treatment Effects on GFR

Treatments in CKD trials often produce acute effects that differ from the longer term effects of the treatment on GFR slope.^12–15 We address acute effects by allowing the initial mean slopes during the first 3 months after randomization to differ from long-term mean slopes after 3 months. For most interventions, acute effects are thought to persist while patients remain on treatment.¹⁶ However, if the size of the acute effect depends on the GFR level, with larger acute effects at higher GFR,¹⁷ the acute effect will attenuate as GFR declines. Our simulations considered both negative acute effects, in which the treatment produces an early GFR decline, and positive acute effects, in which the treatment leads to an early GFR increase. After 3 months, we considered scenarios in which acute effects attenuate and have no ultimate influence on time to kidney failure, as well scenarios in which acute effects persist without attenuation. We simulated attenuation of the acute effect by using a model in which the acute effect is linearly related to the GFR level, with the size of the acute effect declining to zero when GFR reaches 15 ml/min per 1.73 m². For scenarios where the acute effect attenuates with declining eGFR, we express the acute effects normed to a GFR of 42.5 ml/min per 1.73 m² as:

Our analyses of prior CKD trials indicate both negative and positive acute effects are common, with an overall mean acute effect of 0.15 ml/min per 1.73 m² and an SD of 1.01 ml/min per 1.73 m² after normalization (Supplemental Appendix 3, Supplemental Table 5). Under the assumption of normally distributed acute effects across trials, the normalized acute effects were estimated to fall between −1.25 and 1.25 ml/min per 1.73 m² for 74% of trials, and between −2.50 and 2.50 ml/min per 1.73 m² for 98% of trials.

Relationships of GFR with Kidney Failure and Death

The mortality hazard rate was assumed to be linearly related to the patients’ predicted GFR, with higher death rates at lower GFR. ESKD was assumed to occur when the GFR trajectory first declined below a patient-specific uniformly distributed random threshold between 6 and 15 ml/min per 1.73 m².

Summary of Simulation Inputs

Table 1 summarizes 20 inputs which define the GFR trajectories, short- and long-term treatment effects, and study design.

Table 1.

Input parameters in simulations

Category	Factor	Values Considered in Simulations
GFR decline	Mean long-term slope	−1.5, -3.25, or −5 ml/min per 1.73 m2
	SD of long-term slope	3.0, 3.5, 4.0, or 4.5 ml/min per year/1.73 m2
	Correlation of slope and intercept	−0.03a
	Slope skewness	Slight negative skewness (generalized log γ shape parameter=3)a
	Autocorrelation	0
	Size of residuals	Residual variance=0.67×expected GFR (low variability) or 0.817×expected GFR (moderate variability)a
Acute effect	Mean acute effect	−2.5, −1.25, 0, +1.25, or 2.5 ml/min per 1.73 m2 at baseline GFR=42.5 ml/min per 1.73 m2
	Attenuation of initial acute effect	Linear to 15 ml/min per 1.73 m2 or no attenuation
	Variability of acute effect	Acute effect SD=1 ml/min per 1.73 m2
Long-term treatment effect	Type of long-term treatment effect	Proportional, uniform or intermediateb
	Size of long-term treatment effect	0% or 25%, reduction in the slope for a subject with an average long-term slope in the absence of treatment
Death and ESKD	Death rate per year	0.03375–0.00025×expected GFR (ranges from 0.030 at GFR=15 to 0.01125 at GFR=90 ml/min per 1.73 m2)a
	GFR level associated with onset of ESKD	Uniformly distributed between 6 and 15 ml/min per 1.73 m2
Design and conduct	Accrual period and total follow-up	Short: 1 yr accrual and 1.5 yr further follow-up
		Medium: 1.5 yr accrual and 2.5 yr further follow-up
		Long: 2 yr accrual and 4 yr further follow-up
	Measurement frequency	Months 3, 6, and every 6 mo thereafter
	Mean baseline GFR	27.5, 42.5, or 67.5 ml/min per 1.73 m2
	No. of baseline GFRs	2
	Permanent loss to follow-up rate	2%/yr
	Intermittent missing GFRs	5%

^aSee supplement to reference 8 for additional details.

^bUnder the proportional effect model, the distribution of chronic GFR slopes in the treatment group were generated by simulating the slope of the i^th patient as β_i×{(1−k)1[β_i<0]+1[β_i≥0]}, where β_i denotes the slope the patient would have had without the treatment, k is the proportional reduction due to the treatment among patients whose GFR would have declined without the treatment, and 1[β_i<0] and 1[β_i≥0] are 0–1 indicator variables for negative and non-negative slopes, respectively. Thus the proportional effect model assumes the treatment reduces the magnitude of the slope by 100×k percent among patients whose slope would have been negative without the treatment but has no effect on patients whose slope would have been ≥0 without the treatment. Under the uniform treatment effect model the chronic GFR slopes in the treatment group were generated as β_i−k×mean(β_i) where mean(β_i) is the mean chronic slope without the treatment. Under the intermediate treatment effect model, the chronic slopes in the treatment group were generated as [β_i−(k/2)×mean(β_i)]+[β_i×{[1−(k/2)]1[β_i<0]+1[β_i≥0]}].

Outcomes

We evaluated two surrogate end points based on GFR slope:

1. the chronic GFR slope, defined as the mean rate of change in GFR after 3 months; and

2. the total GFR slope, defined as the mean rate of change in GFR from baseline to a designated time late in follow-up, incorporating both the acute and chronic slopes. We defined the total slope over 2, 3, or 4 years for RCTs with follow-up periods of 2, 2.5–4, and 4–6 years, respectively.

We also evaluated two alternative end points:

1. the time from randomization to a confirmed 30% GFR decline or kidney failure, and

2. the time from randomization to a confirmed 40% GFR decline or kidney failure.

The simulations approximated the clinical end point as time from randomization to a confirmed 57% GFR decline or kidney failure. A 57% GFR decline closely approximates doubling of serum creatinine. For time-to-event outcomes, we generated a confirmatory GFR measurement 1 month after each initial GFR measurement which met the criteria for a possible event.

Analyses of Simulated Data

For each scenario, we simulated 800 independent data sets, each with 500 subjects (250 assigned to treatment and 250 to control). For each simulated data set, we analyzed GFR slope using mixed effects models and the time-to-event outcomes using Cox proportional hazards regression. To mimic real-world applications, the simulated data included complexities involving the timing of ESKD and the GFR slope distribution which were not fully accounted for by the statistical models that were used for analysis.

Estimation of Required Sample Size, Bias, and Type 1 Error

For each scenario, averages and SD of the estimated treatment effects on each end point were obtained for the 800 simulated data sets. We obtained SEM for sample sizes N that differed from 500 as SEM(N)=SEM(500)×, where SEM(500) is the SD of the estimated treatment effects across the simulated data sets. We used the means and SEM for treatment effects from the simulations to compare the statistical power provided by each outcome for simulated scenarios with a benefit of the treatment on time to ESKD, and to compare risk of falsely concluding treatment benefit or harm for scenarios with no treatment effect on time to ESKD.

Hypothesized Relationships

We designed the simulations to explore three expected trade-offs between slope end points and the clinical end point. First, we expected the largest relative advantages for slope analyses in studies with high baseline GFR, due to fewer clinical events when initial GFR is high. Second, we expected negative acute effects (with greater acute GFR decline in the treatment group) to lead to greater reductions in power for the total slope than for the clinical end point. This is because a small acute effect can substantially influence the average treatment effect on the total slope, but would have less relative effect on the large GFR declines required to reach the clinical end point.¹¹ Third, we expected better relative performance of slope end points when the long-term treatment effect is uniform than when it is proportional. The relative advantage of slope compared with time-to-event analysis for uniform treatment effects arises because the mean slope incorporates GFR data from all patients, including slow progressors,¹⁰ whereas analyses of the clinical end point ignore differences in GFR declines among slow progressors who do not reach the clinical end point. By contrast, when the treatment effect is proportional to the rate of GFR decline, the treatment effect is largest for fast progressors who are emphasized in the clinical end point analysis, whereas the effect size for the mean slope is attenuated by small effect sizes in slow progressors.¹¹

Results

No Acute Effect and Uniform Long-Term Effect

Figure 1 displays the sample sizes required to provide 90% power with two-sided α=0.05 for the slope and time-to-event end points if the treatment reduces the mean chronic slope by 25% in the idealized situation where the long-term treatment effect is uniform and there is no acute effect. The figure contains nine panels representing trials with short (left), medium (center), and long (right) duration in patients with low (top), intermediate (middle), and high (bottom) baseline GFR. Within each panel, the horizontal axis corresponds to slow, intermediate, or fast mean rates of GFR decline. In this idealized situation, GFR slope end points consistently support much smaller sample sizes than the clinical end point.

Figure 1.

The chronic and total slope lead to large reductions in required sample size in the ideal setting in which there is no acute effect and the long term treatment effect is uniform. Shown are the total sample sizes in both the treatment and control groups combined that are required by different endpoints to obtain 90% power with two-sided α=0.05 when the treatment reduces the mean chronic slope by 25% and the long-term treatment effect is uniform. The nine panels represent trials with short (2 years; left panels), medium (2.5–4 years; center panels), and long (4–6 years; right panels) duration in patients with low (27.5 ml/min per 1.73 m²; top panels), intermediate (42.5 ml/min per 1.73 m²; middle panels) and high (67.5 ml/min per 1.73 m²; bottom panels) mean baseline GFR. Within each panel the horizontal axis corresponds to slow (−1.5 ml/min per 1.73 m² per year), intermediate (−3.25 ml/min per 1.73 m² per year), or fast (−5.0 ml/min per 1.73 m² per year) mean rates of GFR decline. Required sample sizes >12,800 are indicated by open circles. All required sample sizes assume there is no acute effect.

No Acute Effect, Intermediate or Proportional Long-Term Effect

The superior power of slope end points compared with the clinical end point for uniform treatment effects is reduced when the long-term treatment effect is proportional (Figure 2, Supplemental Figures 2 and 3). Our analysis of previous CKD trials, presented in Supplemental Appendix 3 (Supplemental Figure 1), suggests that long-term treatment effects are usually intermediate between uniform and proportional. When the long-term effect is intermediate, the relative efficiencies of the slope end points are not as high as for uniform effects, but remain consistently greater than one compared with the clinical end point, with highest relative efficiencies occurring when baseline GFR is high or follow-up is short.

Figure 2.

The relative efficiency of the chronic and total slope compared to the clinical end point is reduced when the long term treatment effect is proportional rather than uniform. Shown are the relative efficiencies of the indicated alternative end points compared with the clinical end point in relation to the type of long-term treatment effect when the mean slope in the control group is moderate (−3.25 ml/min per 1.73 m² per year) and there is no acute effect. The relative efficiency for each alternative end point indicates the savings in required sample size (expressed as a ratio of sample sizes) when that end point is used compared with the clinical end point. Relative efficiencies greater than one indicate higher power for the alternative end point than the clinical end point. Within each panel, relative efficiencies are provided for uniform, intermediate, and proportional long-term treatment effects. The panels correspond to trials in which the mean baseline GFR is low (27.5 ml/min per 1.73 m²; top panels), intermediate (42.5 ml/min per 1.73 m²; middle panels), or high (67.5 ml/min per 1.73 m²; bottom panels), with either short (2 years; left panels), medium (2.5–4 years; middle panels), or long (4–6 years, right panels) follow-up. Relative efficiencies could not be accurately computed for trials with high baseline GFR and 2 years of follow-up due to insufficient events for the clinical end point.

Figure 3, the left panel of Table 2, and Supplemental Table 6A detail the performance of each end point for intermediate long-term treatment effects when there is no acute effect. Using the total GFR slope usually allows both sample size and trial duration to be reduced while maintaining the same power as any of the time-to-event end points. For example, when there is no acute effect and mean progression is intermediate or fast, substituting the total slope for the clinical end point allows investigators to reduce follow-up from 4–6 years to 2 years while simultaneously improving efficiency by 17%–63% (corresponding to sample size savings of 14%–39%) across the scenarios considered, including a 29% reduction for the intermediate case with baseline GFR 42.5 ml/min per 1.73 m² and moderate GFR decline.

Figure 3.

The chronic and total slope often reduce the required sample size compared to the clinical end point when there is no acute effect and the long-term treatment effect is intermediate between proportional and uniform. Shown are the total sample sizes in both the treatment and control groups combined that are required for different endpoints to obtain 90% power with two-sided α=0.05 when the treatment reduces the mean chronic slope by 25%. The panels correspond to trials in which the mean baseline GFR is low (27.5 ml/min per 1.73 m²; top panels), intermediate (42.5 ml/min per 1.73 m²; middle panels), or high (67.5 ml/min per 1.73 m²; bottom panels), with either short (2 years; left panels), medium (2.5–4 years; middle panels), or long (4–6 years, right panels) follow-up. Within each panel, the required sample sizes are provided for slow (−1.5 ml/min per 1.73 m² per year), intermediate (−3.25 ml/min per 1.73 m² per year), or fast (−5.0 ml/min per 1.73 m² per year) mean rates of GFR decline. Required sample sizes >12,800 are indicated by open circles. All required sample sizes assume there is no acute effect and that long-term treatment effects are intermediate between proportional and uniform.

Table 2.

Gain in efficiency for total slope and chronic slope compared with the clinical outcome when the long-term treatment effect is intermediate between uniform and proportional

Mean Baseline GFR (ml/min per 1.73 m2)	Mean Slope (ml/min per 1.73 m2 per year)	Total Slope			Total Slope			Chronic Slope
		No Acute Effect			Acute Effect =−1.25 ml/min per 1.73 m2			Acute Effect =−1.25 ml/min per 1.73 m2
		Relative Efficiency		Required N for Clinical Outcome in a 4–6 yr RCTa	Relative Efficiency		Required N for Clinical Outcome in a 4–6 yr RCT	Relative Efficiency		Required N for Clinical Outcome in a 4–6 yr RCTa
		Total Slope in 2 yr RCT versus Clinical Outcome in 4–6 yr RCT	Total Slope in 4–6 yr RCT versus Clinical Outcome in 4–6 yr RCT		Total Slope in 2 yr RCT versus Clinical Outcome in 4–6 yr RCT	Total Slope in 4–6 yr RCT versus Clinical Outcome in 4–6 yr RCT		Chronic Slope in 2 yr RCT versus Clinical Outcome in 4–6 yr RCT	Chronic Slope in 4–6 yr RCT versus Clinical Outcome in 4–6 yr RCT
27.5	−1.5	1.14	1.07	4980	0.37	0.41	7140	1.27	0.76	7140
	−3.25	1.51	1.58	2170	0.82	1.49	2190	1.29	1.81	2190
	−5.0	1.24	1.40	870	1.13	1.53	960	1.32	1.77	960
42.5	−1.5	0.71	1.11	4130	0.28	0.39	5010	0.71	1.68	5010
	−3.25	1.41	1.71	1750	0.26	1.23	1940	1.13	2.32	1940
	−5.0	1.17	1.61	830	0.64	1.59	930	1.26	2.33	930
67.5	−1.5	1.06	1.83	6090	0.46	0.46	8240	1.18	2.83	8240
	−3.25	1.28	2.16	2480	0.16	0.69	2940	1.42	3.65	2940
	−5.0	1.63	2.42	1310	0.09	1.25	1260	1.36	3.29	1260

All calculations assume a 25% intermediate long-term effect. Relative efficiencies are given by the ratio of sample size (N) for the clinical end point over 4–6 yr versus the slope analysis over the indicated follow-up period. Relative efficiencies greater than one indicate that a smaller sample size is required to achieve the same statistical power with the slope outcome over the indicated follow-up period compared with the clinical end point over 4–6 yr. See Supplemental Figure 6 for additional detail including simulation SEM.

^aIncreases in required N for the clinical end point with baseline GFR=27.5 compared with 42.5 ml/min per 1.73 m² are due to assumptions that the acute effect is smaller at lower GFR levels, a higher proportion of slow progressors reaching clinical events with smaller effect sizes under the intermediate long-term treatment effect model, and larger effects of random error in GFR measurements on end points requiring smaller GFR change.

Nonzero Acute Effects

Figure 4 displays the relative efficiency of the candidate surrogate end points versus the clinical end point when baseline GFR is intermediate and normalized acute effects between −2.5 and +2.5 ml/min per 1.73 m² attenuate linearly to zero as GFR declines to 15 ml/min per 1.73 m². The influence of the acute effect on the total slope is greatest when the mean progression rate is slow and/or follow-up is short. For acute effects between −1.25 and +1.25 ml/min per 1.73 m², the relative efficiency of the total slope is consistently higher than that of the time-to-event end points when the mean GFR decline is fast, but is lower than the time-to-event end points when the acute effect is negative and the mean GFR decline is slow. The relative efficiency of the chronic slope exceeds that of the other end points when the acute effect is negative and is lower than the other end points when the acute effect is positive. Supplemental Figures 4 and 5, the right portion of Table 2, and Supplemental Table 6B provide additional comparisons in the presence of a moderate negative acute effect.

Figure 4.

Non-zero acute effects can strongly influence the relative efficiency of alternative end points. The figure displays the relationship of the relative efficiency of different endpoints compared to the clinical endpoint in relation to the acute effect of the treatment when the long-term treatment effect is intermediate between proportional and uniform. Relative efficiencies greater than one indicate higher power for the alternative end point than the clinical end point. Relative efficiencies are provided for trials in which the mean control group progression rate is fast (−5 ml/min per 1.73 m² per year; top panels), intermediate (−3.25 ml/min per 1.73 m² per year; middle panels), or slow (−1.5 ml/min per 1.73 m² per year; bottom panels), with either short (2 years; left panels), medium (2.5–4 years; middle panels), or long (4–6 years, right panels) follow-up. All relative efficiencies assume an intermediate mean baseline GFR of 42.5 ml/min per 1.73 m² and a long-term treatment effect which is intermediate between uniform and proportional. The size of the acute effect (either negative or positive) is assumed to be greater at higher levels of GFR such that the acute effect fully attenuates by the time GFR declines to 15 ml/min per 1.73 m². Open circles indicate relative efficiencies which are >8 or <0.125.

Effect of GFR Slope Variability

Supplemental Figure 6 demonstrates that when the acute effect is zero, the performance of the slope end points improves relative to the clinical end point when variability of GFR slope is small.

Risk of Bias and Type 1 Error

The top panels of Figure 5 display the estimated treatment effects on the 3-year total slope and chronic slope when there is no long-term treatment effect and the acute effect fully attenuates before ESKD for trials with moderate (2.5–4 year) follow-up. There is no treatment effect on the clinical end point in these scenarios, so nonzero estimated treatment effects represent a bias for the surrogate end points. The bottom panels of Figure 5 provide corresponding estimates of the risk of false inference of treatment benefit or harm, which can be interpreted as risk of type 1 errors when the slope end points are used to assess clinical benefit. When the acute effect attenuates as GFR declines, negative acute effects can lead to substantial biases against the treatment for the total slope, and somewhat smaller biases in favor of the treatment for the chronic slope, with the reversed pattern for positive acute effects. The opposite direction of the biases for the chronic and total slopes arises because the biases in the chronic slope result from attenuation of the initial acute effect. Supplemental Figures 7 and 8 show that the bias in the total slope resulting from acute effects is greater when follow-up is shorter or average GFR decline is slower. By contrast, the bias in the chronic slope resulting from attenuation of the acute effect is greater when the average GFR decline is faster, but unrelated to length of follow-up. Supplemental Figure 9 compares the performance of the end points for acute effects that do not attenuate.

Figure 5.

Acute effects which attenuate can lead to bias and risk of false positive results. The figure displays the bias and risk of false positive and false negative conclusions when there is no long-term treatment effect with medium follow-up time. The top panels display the effects of the treatment on the mean total slope to 3 years (left) and the mean chronic slope (right) as a function of the acute effect on the horizontal axis when the acute effect is assumed to increase linearly from a value of zero when GFR=15 ml/min per 1.73 m² to the values indicated on the horizontal axis at a GFR of 42.5 ml/min per 1.73 m² and follow-up is medium (2.5–4 years). The acute effects are then assumed to attenuate linearly as GFR declines during subsequent follow-up, with complete attenuation reached at a GFR of 15 ml/min per 1.73 m². Aside from the attenuation of the acute effect, no long-term effect of the treatment is assumed. In this setting there is no effect of the treatment on the time to ESKD or death, so any nonzero effects represent a bias relative to the treatment effect on the clinical end point. The bottom panels indicate the implications of these biases for the risk of false conclusions of treatment benefit or of treatment harm for a RCT with 1000 total patients. Negative acute effects lead to bias of the total slope against the treatment, with a consequent inflation of the risk of a false conclusion of treatment harm, whereas positive acute effects lead to bias of the total slope in favor of the treatment, with a consequent inflation of the risk of a false conclusion of treatment benefit. The biases and risks of false conclusions of benefit or harm are in the reverse direction for the chronic slope due to attenuation of the initial acute effect.

Expanded Summaries

The performance of each of the end points is compared across a larger collection of scenarios with intermediate long-term treatment effects in searchable Excel spread sheets provided on the JASN website as well as on ckdepi.org. The guide for the Excel spreadsheets is found in Supplemental Appendix 5.

Discussion

The simulations of this report provide the first systematic comparison of the performance of slope and time-to-event end points in CKD RCTs, and extend the results of our companion manuscripts, which evaluate the validity of the total and chronic slope as surrogate end points and establish the size of treatment effects on slopes that confer high probabilities of clinical benefit.⁵ The simulations identify conditions in which each slope end point does and does not improve statistical power while preserving a low risk of false positive conclusions, thereby providing guidance as to when slope-based end points are useful in CKD clinical trials.

We emphasize two sets of findings. First, our simulations demonstrate that the relative performance of slope end points compared with the clinical end point depends on the type of long-term treatment effect, with slope end points often providing superior performance when treatment effects are uniform, and the clinical end point performing better when treatment effects are proportional, i.e., when there is proportionally more benefit in those who progress faster. Our empirical analyses of previous CKD RCTs suggest that long-term treatment effects tend to be intermediate on this spectrum rather than fully uniform or fully proportional. Second, when the long-term treatment effect is intermediate and there is no acute effect, slope end points can usually achieve the same power as the clinical end point or confirmed 30% or 40% GFR declines while substantially reducing both sample size and follow-up. The efficiency gains of slope end points compared to the clinical end point are greatest for trials with higher baseline GFR. These gains are sufficient to substantially speed up drug development and reduce costs. For example, substituting the total slope for the clinical end point allows investigators to reduce follow-up from 4–6 years to 2 years while simultaneously reducing sample size by 14%–39% across all of the no acute–effects scenarios with moderate to fast GFR progression in Table 2. We note that although many previously evaluated interventions have had acute effects, to our knowledge the majority of interventions under consideration for new RCTs are not expected to produce acute effects. Examples include chemokine receptor antagonists, dipeptidyl peptidase 4 inhibitors, glucagon-like peptide-1 receptor agonists, bicarbonate supplementation, metformin, anti-IgG4 monoclonal antibodies, and NADPH oxidase 2 inhibitors.

Slope-based analyses are more complicated when the treatment has an acute effect. A negative acute effect can reduce or eliminate the power advantages of the total slope compared with the clinical end point. In contrast, the chronic slope usually provides greater power than either the clinical end point or alternative time-to-event end points when acute effects are negative. However, notwithstanding the strong trial-level association enjoyed by the chronic slope,⁵ our simulations show that this advantage in power comes with a risk of false conclusions (see Figure 5) which arises because the chronic slope evaluates GFR change from a postrandomization time point after the treatment has already modified the GFR. Attenuation of a negative acute effect as GFR declines can increase risk of a falsely concluding treatment benefit.

Although further methodological work remains to be done on the best way to account for acute effects, we suggest three strategies that may be useful when a negative acute effect is expected. First, if the size of the acute effect is relatively small (e.g., ≤1.25 ml/min per 1.73 m²), our simulations establish that enrichment of the study cohort with fast progressors combined with sufficiently long follow-up often provides reasonable power for the total slope, which exceeds that of the clinical end point. Second, analyzing the treatment effect on total GFR change from baseline to poststudy GFR measurements obtained after discontinuing the treatment should retain most of the power advantages of the chronic slope while minimizing the risk of falsely concluding benefit.¹⁸ Third, artificial censoring of GFRs after treatment discontinuation while using weighting or imputation to mitigate selection bias¹⁹ can limit bias due to reversal of acute effects after discontinuation.

Although negative acute effects have received the most attention, our analyses suggest that positive acute effects are also common (Supplemental Appendix 3). If there is a suspicion that a positive acute effect may be spurious, analyses of time-to-event outcomes or of the total slope which uses a prerandomization baseline GFR may inflate risk of falsely concluding treatment benefit, and in this situation the chronic slope represents the more conservative end point.

The main strength of this study is the application of rigorous models for GFR trajectories and their relationship with ESKD and the conduct of statistical simulations to evaluate the performance of slope-based end points over a wide range of conditions. We note three limitations. First and most importantly, GFR trajectories are highly multifactorial, and our conclusions extend only to the specific input parameters that we considered in our simulations. Second, our simulations were limited to a simplified GFR model with a short acute phase followed by a long chronic phase with linear GFR decline. Third, we focused primarily on scenarios in which the size of the acute effect depends on the GFR level and attenuates to zero as GFR declines to 15 ml/min per 1.73 m² per year. In such scenarios, acute effects do not influence time to ESKD. This type of dependence of the acute effect on GFR level has been observed for certain interventions.^13,17 However, it is possible that in some situations the acute effect persists without attenuation as GFR declines, in which case the acute effect will influence time to ESKD. Such effects would be ignored altogether in analyses of the chronic slope, and overemphasized in analyses of the total slope within the limited follow-up time of a trial because the relative contribution of the acute slope to total GFR change diminishes over time. Further research is needed to evaluate analytic strategies that can be applied when mean GFR trajectories exhibit nonlinearities throughout follow-up.

In conclusion, for treatments without acute effects, slope-based end points can substantially reduce required sample size and duration of follow-up compared with the clinical end point and to alternative time-to-event end points based on 30% or 40% GFR decline, particularly in study populations with elevated baseline GFR. Application of slope-based end points remains useful under certain conditions in the presence of acute effects but requires considerable care. More generally, the choice of the optimum end point and study design needs to be evaluated based on detailed power calculations under the conditions of each randomized trial.

Disclosures

Dr. Greene reports grants from the NKF during the conduct of the study and personal fees from DURECT Corporation, Janssen Pharmaceuticals, and Pfizer, outside of the submitted work. Dr. Vonesh served as a paid biostatistics consultant for the NKF for the expressed purpose of developing statistical models for use in the estimation and comparison of GFR slopes as a surrogate end point in CKD RCTs. He is also serving as a biostatistics consultant to Prometic and Tricida; some of this work entails consulting on the design and analysis of clinical trials in patients with CKD. Dr. Levey reports grants from the NIH and the NKF during the conduct of the study, and funding from Siemens outside of the submitted work. Dr. Coresh has grants from the NIH and the NKF related and unrelated to this research. Dr. Inker, Dr. Levey, and Dr. Coresh have the patent “Precise estimation of GFR from multiple biomarkers” pending. Dr. Inker reports funding to Tufts Medical Center for research and contracts with the NIH, NKF, Dialysis Inc., Retrophin, Omeros, and Reata Pharmaceuticals. She has consulting agreements with Tricida and Omeros. Tufts Medical Center, John Hopkins University, and Metabolon have a collaboration agreement to develop a product to estimate GFR from a panel of markers. Dr. Heerspink reports grants and “other” from AbbVie, other from Astellas, grants and other from AstraZeneca, grants and other from Boehringer Ingelheim, grants and other from Janssen, other from Fresenius, other from Gilead, and other from Merck, outside of the submitted work. Dr. Maes is the National Leader for the GSK Anaemia Studies in CKD: Erythropoiesis via a Novel PHI Daprodustat trial 807 and 808 and the National Leader for the Retrophin trial A Study of the Effect and Safety of Sparsentan in the Treatment of Patients With IgA Nephropathy. Dr. Imai received grants from MitsubishiTanabe Pharma and AstraZeneca. He received lecture fees from Takeda, Boehringer Ingelheim, Dainippon-Sumitomo, and Lilly. He is a consultant of Daiichi-Sankyo. Dr. Wetzels reports grants from Sanofi, Pfizer, and Amgen; and other from Achillion, Shire, and Vifor Fresenius. Dr. Perrone reports personal fees from Palladiobio, UpToDate Kluwers Wolter, and Otsuka SA; and grants from Sanofi-Genzyme, Otsuka, and Kadmon outside of the submitted work. Mr. Tighiouart, Dr. Ying, Dr. Jafar, Dr. Vecchio, and Ms. Herrick have nothing to disclose.

Funding

The study was funded by the NKF. A variety of sources have supported the RCTs included in the CKD-EPI. These funding sources include government agencies such as national institutes of health and medical research councils as well as foundations and industry sponsors listed in Supplemental Appendix 5. The research reported in this publication was supported in part by the National Center for Advancing Translational Sciences of the NIH under Award No. UL1TR002538.

Supplemental Material

This article contains the following supplemental material online at http://jasn.asnjournals.org/lookup/suppl/doi:10.1681/ASN.2019010009/-/DCSupplemental.

Supplemental Appendix 1. Abbreviations, units, and terms.

Supplemental Appendix 2. Protocol summary.

Supplemental Appendix 3. Analyses of previous studies.

Supplemental Appendix 4. Supplemental simulation results.

Supplemental Appendix 5. Guide to Excel spread sheet with expanded results.

Supplemental Appendix 6. Study funding sources.

Background and rationale.

Dataset development.

Supplemental statistical methods.

Supplemental Table 1. Study inclusion criteria.

Supplemental Table 2. Studies pooled by intervention.

Supplemental Table 3. Description of studies.

Supplemental Table 4. Patient characteristics by study.

Supplemental Table 5. Distribution of key input parameters for statistical simulations.

Supplemental Table 6A: Gains in efficiency (with simulation SEM in parentheses) for the total slope compared with the clinical outcome when the long-term treatment effect is intermediate between uniform and proportional and there is no acute effect.

Supplemental Table 6B. Gains in efficiency (with simulation SEM in parentheses) for total slope and chronic slope compared with the clinical outcome when the long-term treatment effect is intermediate between uniform and proportional and there is a moderate negative acute effect.

Supplemental Figure 1. Meta-regression relating ratios of slope SD to ratios of mean chronic slopes between treatment and control groups.

Supplemental Figure 2. Relationship of relative efficiency of alternative endpoints versus type of long-term treatment effect when there is no acute effect and progression is fast.

Supplemental Figure 3. Required total sample size of alternative endpoints when the long-term treatment effect is fully proportional and there is no acute effect.

Supplemental Figure 4. Required total sample size of alternative endpoints when the long-term treatment effect is intermediate between proportional and uniform and there is moderate negative acute effect which attenuates.

Supplemental Figure 5. Required total sample size of alternative endpoints when the long-term treatment effect is intermediate between proportional and uniform and there is moderate negative acute effect which does not attenuate.

Supplemental Figure 6. Relationship of relative efficiency of alternative endpoints versus SD of GFR slopes when there is no acute effect and the long-term treatment effect is intermediate between proportional and uniform.

Supplemental Figure 7. Bias and risk of false positive and false negative conclusions when there is no long-term treatment effect and follow-up time is short.

Supplemental Figure 8. Bias and risk of false positive and false negative conclusions when there is no long-term treatment effect and follow-up time is long.

Supplemental Figure 9. Estimated treatment effects on the chronic and total slopes when there is no long-term treatment effect and follow-up time is intermediate, and the acute effect does not attenuate.