Sample size methods for evaluation of predictive biomarkers

Abstract

Treatment selection biomarkers are those that can be useful in guiding choice of therapy. Just as new therapies require evaluation in appropriately designed clinical trials to determine their benefit, therapy selection biomarkers require evaluation in appropriately designed studies. These studies may be prospective clinical trials or retrospective studies based on specimens stored from a completed clinical trial. Ideally, patient treatment assignments should be randomized, and consideration should be given to an appropriate sample size – either for prospective planning of a new study or access to a sufficient number of stored specimens. Here, we develop a novel sample size method for estimation of a confidence interval of specified average width, for an intuitively appealing previously proposed parameter that reflects the expected benefit of using biomarker-guided therapy relative to a standard-of-care therapy. The estimation approach combines Monte Carlo and regression to result in a procedure that performs well over a range of scenarios. Although derived under a specific Cox proportional hazards regression model, robustness to model violations is demonstrated by evaluation under accelerated failure time and cure models. The sample size method produces adequate or conservative sample size estimates under a range of scenarios. Computer code in R and C++, and applications for Mac and Windows are made available for implementation of the sample size estimation procedure. The method is applied to a real data setting and results discussed.

INTRODUCTION

Biomarkers can provide information about the likely clinical course of a patient’s disease in the context of no therapy or a standard therapy (prognostic), or the likelihood that the patient will benefit from an experimental therapy relative to some other therapy (predictive). Biomarkers can be either prognostic or predictive, or a combination. Predictive biomarkers are also called treatment selection biomarkers¹. Predictive biomarkers are fundamental to precision medicine. In oncology, many trials of new therapeutics involve use or evaluation of treatment selection biomarkers. In this article, we develop sample size methods for studies that aim to evaluate performance of a candidate predictive (therapy selection) biomarker based on a parameter that quantifies expected benefit of using biomarker-guided therapy relative to giving all patients a specified therapy.

Evaluation of a predictive biomarker typically requires data from a randomized trial providing both the biomarker values at baseline and the patient outcomes on all treatment arms. Here the focus is two-arm randomized trials involving standard of care (SOC) therapy (which may be no therapy) and an experimental therapy with effect that might be related to the value of a continuous biomarker. An assumption for the methods presented here is that half the patients in the study being planned will be (or were, in the case of retrospective studies) randomized to SOC and half to experimental therapy. These methods can be applied for sample size planning for retrospective evaluation of a biomarker using stored specimens from treatment trials, or for prospective planning of biomarker evaluations within the context of a randomized controlled treatment trial².

A predictive biomarker is generally defined as one that shows a qualitative interaction³, i.e., “…if one treatment is superior for some subsets of patients and an alternative for other subsets.”⁴ This is different than a standard statistical interaction between treatment arm and biomarker value, so that sample size methods based on these types of interactions⁵ are not appropriate here. Indeed, an interaction could be quantitative, indicating that the biomarker value is related to magnitude of benefit from a new treatment, but all patients benefit from the new treatment relative to standard⁶. Moreover, there may be no statistical interaction present, but a biomarker is so strongly prognostic that it identifies a group with such favorable outcome that no treatment (or no additional treatment) is needed⁷. To properly attune our sample size method to the setting of a qualitative interaction, we base it on a previously proposed parameter that quantifies the expected benefit of biomarker-guided therapy^{1, 8–11}. The proposed parameter has an intuitively appealing interpretation as the decrease (compared to current standard of care, SOC) in the population event rate under optimal biomarker-guided treatment (OBGT). If an event is a death (or recurrence, etc.) before time t₀, then we can write this in terms of survival time T as

$$\Theta =\mathit{\text{Prob}}(T>t_0|\mathit{\text{OBGT}})-\mathit{\text{Prob}}(T>t_0|SOC)$$

where the probabilities are taken over the target population. The OBGT is chosen to maximize the survival probability, so that Θ ≥ 0.

Our approach is based on a Cox proportional hazards model due to its frequent use for analysis of right censored data common in clinical trials. Given that interaction slope parameters in Cox regression are not so readily interpretable, we reparametrize the models in more intuitive terms and provide graphical examples of parameter specifications. Since sample size methods are typically needed at a time when no data, or only minimal preliminary data, are available, our method uses model assumptions to minimize required user input. Robustness of the proposed sample size estimates to these assumptions is extensively examined.

This paper is organized as follows. In Section Methods, we derive novel equations which can be used as an alternative to¹ for calculating Θ in some specific settings, and for estimating Θ unbiasedly when data are right-censored. We also derive equations for reparameterizations of the Cox proportional hazards model that translate clinically intuitive elicited values into Cox model slope and interaction parameter values required for sample size calculation. Using a theorem (proven in Supplement Section 6), we develop a Monte Carlo and regression-based sample size estimation method that we call SWIRL. In Section Results, we evaluate the performance of the sample size method and show it works satisfactorily over a wide range of scenarios. We evaluate the robustness of our approach to violations of the underlying assumptions about the biomarker distribution and the hazard function, specifically looking at Weibull hazard models and cure models. Conclusions and summary are presented in Section Conclusion and details appear in the Supplement. Software accessibility details appear in Supplement Section 1.

METHODS

The parameter of interest or estimand is the improvement in the population probability of surviving past a specified time t₀, such as t₀ = 5 years, under a biomarker-guided therapy selection strategy. When originally proposed in¹, logistic regression methods were used assuming no censoring. When data are right-censored as is common in clinical trials, unbiased estimation of this quantity requires time to event analysis methods that can account for censoring such as Cox proportional hazards regression.

We assume each patient is associated with a single biomarker value from a well-defined assay (see Supplemental Section 2 for further details). An optimal biomarker-guided therapy (OBGT) selection strategy for target population Ƥ is a BGT that maximizes the probability of surviving past the prespecified time t₀. Under the Cox proportional hazards model the hazard has the form:

$$h\left(t\right)=h_0\left(t\right)Exp\left[{\beta }_{1}Y+{\beta }_{2}A+{\beta }_{3}AY\right]$$ (1)

where h₀ is the baseline hazard, Y is the continuous biomarker value, and A is the treatment indicator function. Note that the hazard ratio between the active treatment and SOC arms is constant across time for a given biomarker value; this means that the OBGT does not depend on the timepoint chosen t₀. Indeed, the time t₀ is used in our approach to convert clinically intuitive proportions into Cox regression slope parameter estimates and has little impact on sample size estimation. The optimal classification rule is “assign to A=1” if β₂ + β₃Y < 0 and “assign to A=0” if β₂ + β₃Y > 0.

Define A as the treatment, with A=0 for standard of care, and A=1 for experimental treatment. BGT’s are functions

$$g:\text{Range}\left(Y\right)\to \left\{SOC,\mathit{\text{Experimental Treatment}}\right\}$$

The OBGT will be the BGT that maximizes survival:

$$\mathit{\text{OBGT}}\left(Y\right)=\underset{g_{BGT}\left(Y\right)}{\text{max}}\text{Pr}\left(T>t_0|A=g_{BGT}\left(Y\right)\right).$$

Two extreme situations where the biomarker is not used to guide therapy include giving all patients the SOC or giving all patients the experimental therapy. Let g₀ denote the BGT that assigns all patients to the standard of care (that is, g₀(Y) ≡ 0). If the current standard of care is g₀, the improvement achieved by the OBGT is:

$${\Theta }_0=\underset{y}{\int }\left\{\text{Pr}\left(T>t_0|\mathit{\text{OBGT}}\left(y\right)\right)-\text{Pr}\left(T>t_0|g_0\left(y\right)\right)\right\}f\left(y\right)dy$$

where f(y) is the density of the biomarker in the target population. Similarly, let g₁ denote the BGT that assigns all patients to the experimental or alternative treatment (that is, g₁(Y) ≡ 1, and define:

$${\Theta }_1=\underset{y}{\int }\left\{\text{Pr}\left(T>t_0|\mathit{\text{OBGT}}\left(y\right)\right)-\text{Pr}(T>t_0|g_1\left(y\right))\right\}f\left(y\right)dy.$$

Janes et al.⁹ used the notation Θ to refer to the difference between an OBGT and a specified alternative. Their Θ is equivalent to our Θ₀ if the specified alternative is the standard of care, and it is equivalent to our Θ₁ if the specified alternative is experimental/active treatment.

Reparametrizing the Cox model to facilitate a parameter specification

Consider a plot with the biomarker value on the x-axis and the conditional probability of surviving past time t₀ on the y-axis as shown in Figure 1 (where we assume that the baseline hazard is exponential). The conditional probability on the y-axis may depend on both the treatment assignment and the biomarker value. If we imagine all patients assigned to the SOC, then the resulting y-axis values would produce one line on each plot (the solid lines shown in Figure 1); if, on the other hand, we imagine all patients assigned to experimental treatment, then the resulting y-axis values would produce a potentially different line on each plot (the dashed lines in Figure 1). There is a qualitative interaction (as defined above) if the lines cross within the range of the biomarker values (x-axis in Figure 1).

Figure 1:

Diagram of k1 to k4 examples. Vertical dotted lines represent the 25^th and 75^th percentiles of the biomarker distributions. Cox proportional hazards model assumed with exponential baseline hazard. Biomarker has a normal distribution with mean 0.50 and variance 1/12.

For Cox regression models of the form

$$h\left(t|A,Y\right)=h_0\left(t\right)Exp\left[{\beta }_{1}Y+{\beta }_{2}A+{\beta }_{3}AY\right]$$

(where β₁, β₂ and β₃ are parameters, h₀ is a baseline hazard function, and A is the experimental treatment indicator), the hazard ratio between the experimental treatment and SOC is a monotone function of the biomarker value whenever β₃ ≠ 0. If a qualitative interaction exists, then the hazard ratio is equal to 1 at some point in the range(Y) – specifically, the point is

$$Y=-{\beta }_2/{\beta }_3$$

for these models. At those points on each plot in Figure 1, the two lines cross; for example, in the upper left plot of the figure, the crossing point is 0.50. The optimal treatment, conditional on the biomarker value, also changes at that crossing point from SOC to experimental treatment (or vice-versa).

Clinicians might have some sense of what survival is expected for someone with a “low” (e.g., 25th percentile) value of the biomarker, Y, on the SOC and experimental treatment, and of what is expected for someone with a “high” (e.g., 75th percentile) biomarker value on SOC and experimental treatment.

These observations motivate our definition of the following quantities which will serve as the input to the sample size algorithm:

$$k_1=\text{Pr}(T>t_0|A_0,Y=q_1)$$

$$k_2=\text{Pr}(T>t_0|A_0,Y=q_3)$$

$$k_3=\text{Pr}(T>t_0|A_1,Y=q_1)$$

$$k_4=\text{Pr}(T>t_0|A_1,Y=q_3)$$

Here, q₁ and q₃ are the first and third quartiles of the biomarker across the target population. For example, k₁ and k₃ are the survival probabilities for an individual with a low biomarker value on standard of care and experimental treatment, respectively. Also, k₂ and k₄ are the survival probabilities for an individual with a high biomarker value on standard of care and experimental treatment, respectively

Under the assumption of an exponential baseline hazard and the Cox regression model of Equation 1, the following equations (derived in the Supplement Section 13), can be used to convert the k₁, k₂, k₃, k₄ to the parameter values:

$${\beta }_1=\left\{\begin{array}{cc}\frac{CLL\left(k_2\right)-CLL\left(k_1\right)}{q_3-q_1}& :{\beta }_1\ne 0\\ 0& {\beta }_1=0\end{array}\right\}$$

$${\beta }_2=\frac{q_1\left(CLL\left(k_2\right)-CLL\left(k_4\right)\right)+\left(CLL\left(k_3\right)-CLL\left(k_1\right)\right)q_3}{q_3-q_1}$$

$${\beta }_3=\frac{CLL\left(k_1\right)-CLL\left(k_2\right)-CLL\left(k_3\right)+CLL\left(k_4\right)}{q_3-q_1}$$

$$\lambda =Exp\left[\frac{(CLL\left(k_1\right)q_3-CLL\left(k_2\right)q_1}{q_3-q_1}-\gamma Ln\left(t_0\right)\right]$$

where CLL(x) = Ln(−Ln(x)).

Sample size method

The objective is to develop a sample size method to ensure a desired mean width of a confidence interval for Θ₀ (or Θ₁), given k₁, k₂, k₃, k₄, t₀, and the proportion of events expected to be observed, p_E. The clinical trial design is assumed to randomly assign treatments to patients, and may or may not be stratified by the biomarker. This is the experimental design that is either going to be used on the future prospective study being planned, or the design that was previously used on the clinical trial from which the biospecimens are being acquired. Importantly, these will be the first sample size methods ever developed for this objective. While some have investigated sample size issues for treatment selection studies¹¹ using simulation, none have developed sample size methods or software. Our sample size method is simple, fast, transparent, and easy to implement. These characteristics of our method should enhance study design moving forward.

For constructing the confidence intervals, we examined several bootstrap-based construction methods described in the Supplement Section 3 and settled on the one denoted “mpirical” in⁸. This approach uses the refined bootstrap, which includes bias-correction and can result in confidence intervals that are asymmetric and/or include negative numbers.

The objective for the sample size assessment

We chose to use confidence interval width as the sample size objective because it is a natural and intuitive one. The problem is not amenable to hypothesis testing and power approaches because the null hypothesis H₀: Θ₀ = 0 is not in practice usually of much interest. Our sample size method estimates the minimum sample size needed to ensure a desired mean width for the resulting 95% confidence interval for Θ₀ (or Θ₁).

Let w be the confidence interval width. Define E[w] = μ_w as the population mean width for a particular setting of model parameters, sampling plan, and event rate. In Supplement section 6 it is shown that the relationship between the number of deaths (denoted D) and inverse squared mean width of the confidence interval for Θ₀ (or Θ₁) is approximately linear. In other words,

$$D\approx {\eta }_0+{\eta }_1{\mu }_w^{-2}$$ (2)

For a targeted confidence interval width, we can therefore estimate the number of deaths $\widehat{D}$ required by estimating the parameters η₀ and η₁. Then we can estimate the sample size required with $\widehat{n}=\widehat{D}/\text{Pr}\left(\mathit{\text{not censored}}\right)$. Strictly speaking, the proof shows that η₀ = 0, but we do not suggest forcing the relationship through the origin in the regression analysis.

Sample size estimation

Our sample size method is a hybrid that combines mathematical results with simulation data. It is similar in this respect to simulation extrapolation¹² (SIMEX), common in errors in variables regression, and to learning curve methods¹³ used in machine learning prediction. In all these methods, one simulates new data (or re-samples from existing data) in order to estimate regression parameters that describe the relationship of interest.

The sample size is estimated by first estimating (η₀, η₁) from Monte Carlo simulations at five distinct D values. Linear regression is used to obtain estimates ${\widehat{\eta }}_0$ and ${\widehat{\eta }}_1$ in Equation 2. Then, if we let the user-defined targeted width be denoted ${\mu }_w^{\mathit{\text{target}}}$, we can estimate the number of deaths required as:

$${\widehat{D}}_{\mathit{\text{target}}}={\widehat{\eta }}_0+{\widehat{\eta }}_1{\left[{\mu }_w^{\mathit{\text{target}}}\right]}^{-2}$$ (3)

A subsequent fine-tuning step generates large simulations near the estimate to obtain a final sample size value.

The full sample size algorithm is shown in Figure 2. Importantly, note that once the desired mean width, ${\mu }_w^{\mathit{\text{target}}}$, the values of the k₁, k₂, k₃, k₄, the distributional form of the biomarker (e.g., normal), and p_E (the expected proportion of events/deaths) have been elicited from clinical experts, no other information is required to produce the sample size estimate. The p_E is used to incorporate censoring into the simulations.

Figure 2:

Diagram of sample size algorithm. The user-specified target width is ${\mu }_w^{\mathit{\text{target}}}$. The meaning of the k’s is shown in Figure 1, and p_E is the proportion of events. The Monte Carlo number of events used are D_j and 100 Monte Carlo replicates are used. The Monte Carlo confidence interval widths are $w_{i,D_j}$. The confidence interval widths for the Monte Carlo are obtained by the TreatmentSelection package in R, or our C++ program that uses Cox regression.

A model with deaths as response and inverse squared widths as predictor is fit using least squares. Importantly, note that the regression parameters can be estimated accurately using a small number of distinct values of D, even just two values; this is in contrast to a brute force Monte Carlo, which will require much more time (see Table S8 of Supplement).

Methods for Weibull and Cure models for robustness studies

To evaluate robustness to model violations, we consider Weibull Accelerated Failure Time models and Cure models. When the true data come from these distributions, then the value of the parameter or estimand Θ needs to be calculated so that we can evaluate the performance of our method under these model violations. Development of the formulas necessary to generate data from the models and calculate Θ are shown in the Supplement Sections 8 and 13.

Sposto and Sather¹⁴ suggest the cure model with hazard function

$$\begin{array}{c}{h}_0\left(t\right)=\begin{array}{cc}\lambda {\left[1-\left\{t/T_{\mathit{\text{cure}}}\right\}\right]}^{\gamma }& :0\le t\le T_{\mathit{\text{cure}}}\\ 0& :t>T_{\mathit{\text{cure}}}\end{array}\end{array}$$ (4)

After further mathematical derivations presented in the Supplement Section 8, we developed an algorithm for generating data from the Sposto and Sather¹⁴ model. The algorithm is shown in Supplement Section 8.

Applications

We apply our approach to the “OncoTypeDx-like biomarker” and the “much better biomarker” of¹. DigitizeIt © version 2.5.3 was used to extract quantiles from figures. The 10-year distant recurrence free rate in the B20 trial¹⁵ was 91%, so the p_E was set to 9%.

RESULTS

The formulas for the true values of Θ₀ and Θ₁ presented in Section 11 were verified in simulations presented in the Supplement Tables S2 and S3.

Since sample size and confidence interval widths for Θ₀ and Θ₁ have not been evaluated before in the literature, we first looked at how user-defined inputs affect the calculation. The impact of the user-selected k values, for a wide range of patterns, were evaluated by Monte Carlo. Some results appear in Supplement Figure S3 and S4, which focus on sample size n = 500 and no censoring. The values of the k’s and of Θ₀ did impact the mean widths, sometimes significantly, but there was not a general pattern discernible across the range of settings. Figure 3 shows some examples of the linearity between the inverse squared mean width of the confidence interval and the number of events, which is the basis for the SWIRL method; the linear relationship is extremely strong.

Figure 3:

Exploratory graphs of mean widths for different simulation settings. Some examples of D (number of deaths/events) versus the inverse squared width of the intervals. The relationship appeared linear over all settings considered, based on 1,000 Monte Carlo.

The number of events estimated from the SWIRL approach are shown in Table 1. These results correspond to a normally distributed biomarker in the context of a randomized treatment trial. The estimates range from 115 to 816 for these parameter combinations and target widths. To obtain the corresponding sample sizes, divide by p_E. Table 1 also shows a Monte Carlo evaluation of the adequacy of the SWIRL estimates. In each case, the empirical mean interval width from the simulations is very close to the targeted width. The interquartile range and range are also shown.

Table 1:

Randomized treatment trial sample size estimates based on t₀ = 5, SWIRL method, confidence interval for Θ₀. Also included is evaluation of adequacy of SWIRL method sample size estimates. Biomarker has normal distribution ($\mu =\frac{1}{2},{\sigma }^2=1/12$). Targeted coverage probabilities in bottom row are 95%. Simulations were based on 500 bootstrap and 10,000 Monte Carlo. “Mean width” is the average over 10,000 simulations, and similarly with the “Width IQR” and “Width Range.” “Cov. Prob.” Is the coverage probabilities of the intervals over the 10,000 simulations. “D” is the number of deaths, $\widehat{D}$ the estimated number of deaths, and the total sample size would be n=D/p_E. Θ₀ is the true value based on the formulas we have developed. TreatmentSelection package used to construct confidence intervals (Marshall Brown and Holly Janes, 2017, TreatmentSelection: Evaluate Treatment Selection Biomarkers. R packages version 2.1.1. http://CRAN.R-project.org/package=TreatmentSelection).

Setting	1	2	3	4	5	6	7	8
k 1	0.25	0.25	0.25	0.25	0.6	0.6	0.1	0.1
k 2	0.75	0.75	0.75	0.75	0.4	0.4	0.55	0.55
k 3	0.75	0.75	0.5	0.5	0.5	0.5	0.9	0.9
k 4	0.25	0.25	0.5	0.5	0.5	0.5	0.45	0.45
Target Width	0.2	0.1	0.2	0.1	0.2	0.1	0.2	0.1
SWIRL $\widehat{D}$	173	704	176	679	174	749	205	816
Sim. D	174	704	176	680	174	750	206	816
Mean width	0.2016	0.0987	0.1998	0.0999	0.2033	0.1015	0.2	0.0997
Width IQR	0.1853–0.1990	0.0927–0.1040	0.1792–0.2201	0.0918–0.1075	0.1732–0.2354	0.0907–0.1122	0.1876–0.2100	0.0949–0.1038
Width Range	0.1301–0.4378	0.0724–0.3757	0.0609–0.4132	0.0605–0.2384	0.0286–0.3679	0.0362–0.1649	0.1433–0.5107	0.0779–0.3123
Θ 0	0.2498	0.2498	0.1263	0.1263	0.0595	0.0595	0.4361	0.4361
Cov. Prob.	0.9575	0.9573	0.9674	0.9578	0.9728	0.964	0.9564	0.948

Table 2 shows results demonstrating robustness to model violation when data are generated from different Weibull models, violating the assumption of exponential baseline hazard required for the derivation in Supplement Section 6; since the Weibull model is both accelerated failure time and proportional hazards, the proportional hazards assumption is not violated. We include the exponential case with Weibull shape parameter WP=1 for comparison. In all cases investigated in the table, the mean widths of the confidence intervals are near the targets or conservative and the coverage probabilities are correct or conservative. This shows that our sample size estimates are robust against Weibull model violations involving increasing or decreasing hazards. In some cases when the Weibull parameter is 2, our method is very conservative; this is not too surprising since a Weibull parameter of 2 indicates increasing hazard and so less censoring will be expected before time t₀ under Weibull than under exponential, and in these examples, we are using the TreatmentSelection package which will be impacted by this fact. Specifically, individuals censored before time t₀ are not used by the TreatmentSelection package’s binary regression fit; thus, the effective sample size is larger for the fitted data than it is for the model used for sample size estimation.

Table 2:

Robustness investigation: Weibull model with normally distributed marker (mean=1/2, variance=1/12), and randomized treatment trial design. Estimates based on 1,000 simulations. “WP” is the Weibull parameter. CI widths based on 95% confidence level. Θ₀ calculated from Monte Carlo with n=1 million, and coverage probability estimates based on Θ₀. “CP” is the coverage probability over the 1,000 simulations. “Mean CI width” is the mean confidence interval width over 1,000 simulations. K’s are inputs for the sample size calculation, and also the true K’s associated with the Weibull model. “D” is the number of deaths, and the total sample size would be n=D/p_E. Resulting sample size estimate in terms of deaths (no censoring). TreatmentSelection package (Marshall Brown and Holly Janes, 2017, TreatmentSelection: Evaluate Treatment Selection Biomarkers. R packages version 2.1.1. http://CRAN.R-project.org/package=TreatmentSelection) used to construct confidence intervals.

k 1	k 2	k 3	k 4	D	WP	Θ0	CP	Mean CI width (SD)
0.90	0.45	0.10	0.55	116	0.5	0.075	0.958	0.200 (0.042)
0.90	0.45	0.10	0.55	116	1.0	0.107	0.952	0.189 (0.031)
0.90	0.45	0.10	0.55	116	2.0	0.020	0.682	0.074 (0.079)
0.25	0.75	0.75	0.25	174	0.5	0.151	0.955	0.196 (0.026)
0.25	0.75	0.75	0.25	174	1.0	0.249	0.951	0.203(0.027)
0.25	0.75	0.75	0.25	174	2.0	0.087	0.950	0.117 (0.021)
0.60	0.40	0.50	0.50	174	0.5	0.027	0.980	0.166 (0.043)
0.60	0.40	0.50	0.50	174	1.0	0.060	0.976	0.199 (0.045)
0.60	0.40	0.50	0.50	174	2.0	0.018	0.971	0.076 (0.080)
0.10	0.55	0.90	0.45	206	0.5	0.214	0.944	0.167 (0.169)
0.10	0.55	0.90	0.45	206	1.0	0.357	0.952	0.179 (0.180)
0.10	0.55	0.90	0.45	206	2.0	0.498	0.956	0.182 (0.016)
0.90	0.45	0.10	0.55	424	0.5	0.075	0.964	0.102 (0.104)
0.90	0.45	0.10	0.55	424	1.0	0.107	0.961	0.096 (0.097)
0.90	0.45	0.10	0.55	424	2.0	0.020	0.952	0.036 (0.037)
0.25	0.75	0.75	0.25	704	0.5	0.151	0.968	0.096 (0.010)
0.25	0.75	0.75	0.25	704	1.0	0.249	0.946	0.099 (0.008)
0.25	0.75	0.75	0.25	704	2.0	0.087	0.948	0.059 (0.005)
0.60	0.40	0.50	0.50	750	0.5	0.027	0.969	0.083 (0.018)
0.60	0.40	0.50	0.50	750	1.0	0.060	0.972	0.099 (0.015)
0.60	0.40	0.50	0.50	750	2.0	0.018	0.969	0.041 (0.008)
0.10	0.55	0.90	0.45	816	0.5	0.214	0.958	0.083 (0.006)
0.10	0.55	0.90	0.45	816	1.0	0.357	0.938	0.089 (0.006)
0.10	0.55	0.90	0.45	816	2.0	0.498	0.956	0.091 (0.005)

We also examined a biomarker with a uniform distribution on the unit interval. Results were overall similar, showing robustness to the marginal distribution (normal versus uniform). Interested readers may refer to Tables S6 and S7 in the Supplement.

Supplement Table S1 shows different solutions to the parameters for the cure model of¹⁴. Note that the cure model does violate proportional hazards. For this model, one has to specify the additional cure model parameters T_cure and γ. We generate data from the four cases shown in the table, and results appear in Table 3. The results indicate that our method is adequate to produce confidence intervals with mean targeted width as desired or narrower. Generally, the sample size estimates appear quite conservative, indicated by the fact that the mean widths are below the targeted widths. Of note is the case when Θ₀ = 0.001, where the coverage probability of the⁹ method is inadequate. This seems of minor concern since it is an example of an extreme situation given that Θ₀ ≥ 0 by definition.

Table 3:

Robustness investigation: cure model simulation results based on 1,000 Monte Carlo. Shown are the mean widths and coverage probabilities for the cure model settings. When “Bootstraps” is less than 1000, then not all 1000 Monte Carlo produced valid bootstrap confidence intervals (invalid intervals result from too few observations in one group). Coverage probabilities based on proportion of intervals that cover Θ, where Θ is obtained with data generated from 1,000,000 simulations and application of TreatmentSelection R package.

Case	D	Bootstraps	Width mean	Width Target	Θ0	CP
Case 1	174	1000	0.17	0.20	0.141	0.96
Case 2	174	1000	0.14	0.20	0.089	0.96
Case 3	174	995	0.10	0.20	0.048	0.95
Case 4	176	159	0.05	0.20	0.001	0.66
Case 1	680	1000	0.07	0.10	0.141	0.96
Case 2	680	1000	0.07	0.10	0.089	0.95
Case 3	680	1000	0.05	0.10	0.048	0.95
Case 4	714	785	0.02	0.10	0.001	0.73

Real data-based application results are presented in Table 4. Since the censoring level is expected to be 91% for this population, we used our Cox proportional hazards model method as implemented in the software application we have developed (versus the R program, which uses the TreatmentSelection package so is not appropriate when censoring is high). Based on the extracted datapoints, the settings for the OncoTypeDX-like biomarker were k₁ = 0.15, k₂ = 0.25, k₃ = 0.13, k₄ = 0.35, resulting in Θ₁ = 0.062 where the comparison is between OBGT and all get chemotherapy; the resulting sample size estimate to achieve a confidence interval width of 0.10 was n=8,833. For comparison, TailoRX trial enrolled 10,273. On the other hand, the B14 trial used as part of the validation of the OncoTypeDX score¹⁷ had 651 patients with blocks available who had previously been randomly assigned to Tamoxifen or placebo. The sample size requirement for the “much better biomarker” of¹ with settings k₁ = 0.17, k₂ = 0.30, k₃ = 0.31, k₄ = 0.06 (Θ1 = 0.092) was n=5,822 for a width of 0.10. One can also see in these applications that a higher level of censoring will lead to larger sample size requirements compared to some of the examples in the tables.

Table 4:

Real data-based application.

Biomarker Type	Target width	Sample size	Reduction for “much better marker”
OncoType-DX-like	0.10	8,833	-
OncoType-DX-like	0.20	1,688	-
Much better marker	0.10	5,822	1-(5822/8833) = 34%
Much better marker	0.20	1,133	1-(1133/1688) = 33%

CONCLUSIONS

In this paper we developed sample size methods for forming confidence intervals for a parameter (or estimand) that quantifies the expected benefit of using a candidate predictive biomarker to guide treatment. In particular, the confidence intervals are intended to estimate the increase in the probability of surviving on marker-guided treatment, relative to current standard of care treatment, at some specified time t₀ (such as 5 years), with some precision (confidence interval width). Because users must set the Cox proportional hazard model slope parameter values, we facilitated this process by reparametrizing the Cox model in terms of conditional probabilities of surviving past t₀ given low or high biomarker values, and treatment assignment. These methods will be useful for both planning prospective trials, and retrospective studies of clinical trial specimens stored in biobanks. The sample size methods developed in this paper are being made available as two computer programs (in R and C++) and as an application (Mac and Windows).

Our sample size method uses a novel regression-based sample size approach which we call SWIRL. The method is based on a theorem that shows that for the linear Cox regression models, the inverse squared mean confidence interval width for the parameter Θ is approximately proportional to the expected number of events. The performance of the method was evaluated by Monte Carlo and it was shown to perform very well over the range of sample sizes considered. The SWIRL method is simple and may also be applicable to other settings in which a proof similar to Theorem 1 of this paper can be derived. The approach avoids complicated and often poor-performing asymptotic approximations, and requires a much lower number of simulations than pure Monte Carlo.

We noted that the estimator (denoted Θ) in Janes et al.⁹ assumes binary outcome data, and can be biased if naively applied in the presence of right censoring. Leveraging novel formulas and algorithms for calculating and estimating Θ, we developed a method based on the Cox proportional hazards model that is an alternative approach to their estimator that provides an unbiased estimate of Θ in the presence of right censoring. However, this novel estimator was not the focus of this paper, which instead was sample size. This was partly because for the scenarios we investigated the coverage probabilities of the Janes et al.⁹ estimator appeared reasonably good until the random censoring was well over 50 percent, and partly because the TreatmentSelection package provides user-friendly software for interval construction.

In our sample size methods, we have focused on situations where specimens used for biomarker determination came from trials that randomly assign treatments to patients. We did not examine scenarios outside this scope, such as when samples are from observational studies, or when the proportion of samples actually available from a trial may be small. In these cases, there may be imbalance between the treatment arms that is greater than examined in this paper and there may be other biases affecting observed outcomes on different treatments. Approaches to extend our methods to these settings, including causal modeling, are a topic for future research.

A limitation of our approach is that the user must input the values k1, k2, k3, and k4. These values can be estimated from prior data. If there is little prior data available, one could input different sets of the k values that seem reasonable and pick the largest sample size calculated. As can be seen in Table 1, for example, the number of observed deaths to achieve a confidence interval width of, say, 0.10, varies greatly over the combinations of k values presented (from 423 to 806). Therefore, the k values are essential for obtaining a reasonably accurate sample size estimate. Availability of preliminary estimates of key parameters is a common challenge for sample size estimation, but our hope is that the more intuitive nature of the k values might make them easier to elicit than Cox proportional hazard model parameter values.

DATA AVAILABILITY STATEMENT

Data sharing not applicable to this article as no datasets were generated or analysed during the current study