A Unified Approach to Sample Size and Power Determination for Testing Parameters in Generalized Linear and Time-to-Event Regression Models

Abstract

To ensure that a study can properly address its research aims, the sample size and power must be determined appropriately. Covariate adjustment via regression modeling permits more precise estimation of the effect of a primary variable of interest at the expense of increased complexity in sample size / power calculation. The presence of correlation between the main variable and other covariates, commonly seen in observational studies and non-randomized clinical trials, further complicates this process. Though sample size and power specification methods have been obtained to accommodate specific covariate distributions and models, most existing approaches rely on either simple approximations lacking theoretical support or complex procedures that are difficult to apply at the design stage. The current literature lacks a general, coherent theory applicable to a broader class of regression models and covariate distributions. We introduce succinct formulas for sample size and power determination with the generalized linear, Cox, and Fine-Gray models that account for correlation between a main effect and other covariates. Extensive simulations demonstrate that this method produces studies that are appropriately sized to meet their type I error rate and power specifications, particularly offering accurate sample size/power estimation in the presence of correlated covariates.

1 |. INTRODUCTION

The primary goal of many biomedical studies is the evaluation of a specific variable’s impact on an outcome of interest. For example, researchers may wish to evaluate efficacy of a promising treatment or a biomarker’s effectiveness in predicting future disease occurrence. An example of the latter is given by Zaid et al.¹, which explored possible correlations of 10 biomarkers with clinical outcomes following blood and marrow transplantation. Regression modeling is often employed to evaluate a primary variable of interest while accounting for the influence of other covariates. Moreover, rather than being normally distributed, the primary endpoint in many clinical studies is a binary, count, or time to event outcome, necessitating the use of nonlinear regression modeling to account for covariate influence. Correlation is often known to exist between the main variable and other covariates, such as demographics or health-related factors; this correlation can arise not only in observational studies but also in non-randomized clinical trials.

The main determinant of the power to detect an appreciable effect in common regression models is the sample size of the study. Calculating the required sample size for nonlinear regression models to satisfy type I error rate and power requirements of a study is made increasingly complicated by two factors: the nonlinear relationship between the response and explanatory variables, and the presence of correlation between covariates. The majority of sample size estimation procedures for regression models assume the covariates are uncorrelated. For the logistic, Poisson, Cox², and Fine-Gray regression³ models, formulas based on the asymptotic, normal distribution of the parameter estimate were presented by Hsieh et al.⁴, Signorini⁵, Schoenfeld⁶, and Latouche et al.⁷, respectively. Extending this estimation to accomodate correlated covariates has proven to be much more challenging, though. For generalized linear models, Self and Mauritsen⁸ proposed a method for estimating the required sample size through an iterative procedure; this approach was extended by Liu and Liang⁹ for marginal modeling of the mean response from clustered observations. But, these techniques fail to give closed forms for the sample size estimate or demonstrate superior performance over simpler, less computationally demanding methods. Alternatively, Hanley and Moodie¹⁰ advocated for a simpler, yet general, approach to sample size and power calculation for generalized linear models. Though the authors demonstrate this approach’s application to a variety of applications of these models, including hypothesis testing of mean differences, correlations, risk differences, and risk and odds ratios, its usage in the setting of correlated covariates was covered only for ordinary linear regression, likely due to the sparsity of existing literature rigorously supporting this method in more complex settings.

When correlation exists between covariates, Hsieh et al.¹¹ argued for using a technique that inflates the sample size obtained for uncorrelated covariates by a factor determined by the strength of the correlation. They presented a simulation study supporting this method’s usage with their logistic regression formula and Schoenfeld’s Cox model formula¹², but did not provide theoretical justification. Signorini⁵ gave theoretical support to this inflation method for Poisson regression with normally distributed covariates, while Schmoor et al.¹³ and Latouche et al.⁷ offered derivations supporting the method’s use with Cox, and Fine-Gray models when two binary covariates are at play. Justification for this approach in a more general context, however, is lacking. Our proposed methodology both extends these simpler formulas for logistic, Poisson, Cox, and Fine-Gray regression to admit a more general class of predictors and regression models and provides theoretical support for the sample size inflation technique that adjusts for correlated covariates.

Section 2 introduces a unified approach to sample size determination for generalized linear, Cox, and Fine-Gray regression models that utilizes the inflation factor approach of Hsieh et al.⁴ to account for the presence of correlation. In Section 3, we present results of a comprehensive simulation study that evaluated the performance of this sample size estimation in studies employing realistic sample sizes. Examples of applying the proposed sample size estimation in the design of prospective cohort studies are given in Section 4. Section 5 concludes the paper with a discussion of these findings and potential extensions of the technique.

2 |. METHODS

2.1 |. General Approach

Suppose that the primary objective of a study is to evaluate the impact of a main covariate Z on an outcome Y while controlling for the influence of other covariates X = (X₁, X₂, …, X_p)^T. For instance, Z could denote a binary treatment indicator in a two-arm clinical trial or a continuously measured biomarker’s level in a correlative study. If the parameter β represents the effect of Z on the outcome in a regression model, then a two sided test of H₀ : β = 0 determines whether the main covariate has any impact on the outcome. For many common regression models, there exists an estimator $\widehat{\beta }$ of β such that $\sqrt{n}(\widehat{\beta }-\beta )\stackrel{d}{\to }N(0,v(\beta ))$, where the asymptotic variance v may depend on the true value of β. From this asymptotic distribution, we can derive a test of H₀. The sample size n required for a two sided Wald test of H₀ to detect an effect β = δ with level α and power π is

$$n=\frac{{\left[v{(0)}^{1/2}{\Phi }^{-1}(1-\alpha /2)+v{(\delta )}^{1/2}{\Phi }^{-1}(\pi )\right]}^2}{{\delta }^2},$$

where Φ⁻¹ is the standard normal quantile function. Usually, v (β) is a continuous function of β such that v (δ) ≈ v (0) when δ is close to 0, giving the simpler formula

$$n=\frac{{\left[{\Phi }^{-1}(1-\alpha /2)+{\Phi }^{-1}(\pi )\right]}^{2}v(0)}{{\delta }^2}.$$ (1)

Formula (1) will be used as the basis for sample size calculation for the generalized linear regression, Cox, and Fine-Gray models in the methodology that follows. The main difficulty involved is accurately specifying the variance term v (0), which can be especially challenging when Z and X are correlated. We will show that simple formulas can be applied that account for this correlation.

2.2 |. Generalized Linear Regression Models

First, consider the class of generalized linear models (GLM)¹⁴. Let ${\left\{\left(Y_i,Z_i,{\text{X}}_i\right)\right\}}_{i=1}^n$ be a random sample of outcomes and covariates such that all Z_i and X_i have finite variance. The covariates and outcome are assumed to be related through the generalized linear model

$$h(E[Y\mid \text{X}])={\beta }_0+\beta Z+{\gamma }^T\mathbf{\text{X}}={\eta }^T\text{W},$$ (2)

where h is a link function that is differentiable and invertible, η = (β₀, β, γ^T)^T is the full parameter vector, and W = (1, Z, X^T)^T. The assumptions are satisfied for common choices of link function, including the identity, log, logit, probit, complementary log-log, and Box-Cox transformations. Furthermore, each response variable Y_i is assumed to arise from the exponential dispersion family¹⁵ with parameters θ and ϕ, having a probability mass/density function of the form

$$\text{exp}\left[\frac{y_i\theta -A(\theta )}{\varphi }+c\left(y_i,\varphi \right)\right].$$

Due to the GLM assumption, the parameter θ depends on Z_i and X_i. Using well-known results from Nelder and Wedderburn¹⁴, it can be shown that the maximum likelihood estimator $\widehat{\eta }$ of η converges in distribution to N (0, Σ⁻¹) where Σ = E [W^⊗2q (η, W)] with $q(\eta ,\text{W})=Var(Y\mid \text{W}){\varphi }^{-2}{\left[g^{\prime }\left({\gamma }^T\text{W}\right)\right]}^2$ and g = (A^′)⁻¹ ∘ h⁻¹ (see Supporting Information for a derivation of this result).

We use the multiple correlation coefficient of Z with X, denoted by R, to describe their correlation; this satisfies R² = Cov(Z, X)Var(Z)⁻¹Var(X)⁻¹Cov(X, Z), where R² can be interpreted as the proportion of the variance of Z that is explained by X. It is known that in an ordinary linear regression model, $Var(\widehat{\beta })$ is inversely proportional to Var (Z) (1 − R²). Because of this, the term VIF = (1 − R²)⁻¹ is called the variance inflation factor, as it indicates the proportional increase in $Var(\widehat{\beta })$ that occurs in linear regression compared to the situation where Z and X are uncorrelated. This also indicates the inflation in sample size required to detect a given effect, compared to the correlation-free setting. For linear regression, this result and formula (1) provide the sample size

$$n=\frac{{\left[{\Phi }^{-1}(1-\alpha /2)+{\Phi }^{-1}(\pi )\right]}^2{\sigma }^2}{Var(Z){\delta }^2}\cdot VIF,$$ (3)

where σ² is the residual variance. GLMs exhibit a similar variance inflation effect, shown by the following theorem.

Theorem 1 Assume that model (2) holds and E (Z |X) = a + b^′X for some scalar a and vector b. Then a sufficient sample size for two-sided testing of H₀ : β = 0 at significance level α with power π to detect β = δ is given by

$$n=\frac{{\left[{\Phi }^{-1}(1-\alpha /2)+{\Phi }^{-1}(\pi )\right]}^2}{f_{00}V\text{ar}(Z){\delta }^2}\cdot VIF,$$ (4)

where f₀₀ = E [q (η₀, W)] and η₀ = (β₀, 0, γ^T)^T is the value of η under H₀.

The sample size given for testing a main effect in GLMs is determined by six components: the type I error rate α; the power π; the targeted effect size δ; the variability of the main effect, Var (Z); the correlation between the main effect and other covariates, described by the V I F; and f₀₀, which relates to the response distribution and choice of link function. This sample size is both proportional to the VIF and inversely proportional to the variance of Z, the same as for ordinary linear regression.

The general form of f₀₀ is quite complicated, but it does simplify greatly for common choices of GLMs. When the canonical link function is chosen such that A^′ ≡ h⁻¹, g is the identity function and so $f_{00}={\varphi }^{-2}\cdot E_{H_0}[Var(Y\mid \text{W})]$ depends only on the dispersion parameter ϕ and the residual variance under H₀. For linear regression, f₀₀ = 1/σ² and so formulas (3) and (4) coincide. For logistic and Poisson regression, ϕ = 1 is assumed. Thus, $f_{00}=E_{H_0}[Var(Y\mid \text{W})]=E\left\{\text{exp}\left({\beta }_0+{\gamma }^{\top }\text{X}\right)/{\left[1+\text{exp}\left({\beta }_0+{\gamma }^{\top }\text{X}\right)\right]}^2\right\}=E[P(Y=1\mid \text{X})[1-P(Y=1\mid \text{X})]$ for the logistic model, the expected variance of Y |X. For Poisson regression, f₀₀ = E[exp(β₀ + γ^T X)], which is also the expected number of events for a subject under H₀, and so the sample size relies upon the expected number of events for this model. Specification of the residual variance term Var (Y|W) in f₀₀ is likely to be the most challenging part in applying formula (4). This challenge is not unique to our method, however, but also arises in simulation-based approaches to sample size estimation for GLMs accompanied by increased probability modeling and computational demands. Historical data and/or expert opinion can be utilized to obtain estimates for the regression parameters β₀ and γ as well as a plausible distribution for W in order to specify f₀₀.

Although this article focuses upon specification of a sufficient sample size for the purpose of designing a study to test a main effect, formula (4) can also be applied toward power calculation or to determine the targeted effect size. Situations arise where the sample size may be predetermined and interest lies in computing the power provided to detect a clinically meaningful effect size or, conversely, in finding the minimum effect size that can be detected at a given power level. Our formula may be applied readily in these situations as well: given any two of the design parameters n, π, and δ, the third is readily determined.

Theorem (1) assumes a linear model for Z given X; however, our simulations support the use of formula (4) in situations where the relationship is not linear but instead follows a logistic or log-linear model. This suggests that the assumption could be relaxed and that perhaps the linear relationship only needs to hold approximately near the mean of X, where the majority of its probability density/mass is concentrated. Formula (4) matches the formula suggested by⁴ for logistic regression. For Poisson regression,⁵ offered a similar method for sample size determination but assumed that influential covariates had a multivariate normal distribution. Our method unites these methods under a general framework that permits a broad class of link functions and covariate distributions.

2.3 |. Fine-Gray Regression Model

Similar formulas can be applied for the Cox and Fine-Gray regression models for time to event data. The Fine-Gray model is considered first, since the competing risks setting to which it applies can be viewed as a special case of the single failure type setting in which the Cox model is applicable. Without loss of generality, we model the occurrence of cause 1 events from competing risks data over the study period [0, t*], where t* is the maximum follow-up time considered. Subjects may be right-censored within this period, and it is assumed that censoring happens independently of event occurrence and covariates. For the i^th patient on study, let T_i denote the event time, C_i be the censoring time, Z_i be the main covariate of interest, X_i be the vector of other covariates, and ε_i be the event type. The vectors ${\left\{\left(T_i,C_i,{ϵ}_i,Z_i,{\text{X}}_i\right)\right\}}_{i=1}^n$ are assumed to be a random sample from the target population. The Fine-Gray regression model specifies the subdistribution hazard function ${\lambda }_1^{*}$ through the form

$${\lambda }_1^{*}(t\mid Z,\text{X})={\lambda }_{10}^{*}(t)\text{ exp}\left(\beta Z+{\gamma }^T\text{X}\right)\text{ for }t\le t^{*}$$ (5)

where t is the time on study and ${\lambda }_{10}^{*}$ is the baseline subdistribution hazard. Let ν = (β, γ^T)^T and $\widetilde{\mathbf{\text{W}}}={\left(Z,{\text{X}}^T\right)}^T$. Martens and Logan¹⁶ showed that the estimator $\widehat{\nu }$ of ν presented in Fine and Gray³ is such that $\sqrt{n}(\widehat{\nu }-\nu )\stackrel{d}{\to }N\left(0,{\left({\Sigma }^{*}\right)}^{-1}\right)$ where, under H₀,

$${\Sigma }^{*}={\int }_0^{t^{*}}P\left(C_i\ge u\right)\left[{\Gamma }_2^{*}(u)-{\Gamma }_1^{*}{(u)}^{\otimes 2}/{\Gamma }_0^{*}(u)\right]{\lambda }_{10}^{*}(u)du$$

with ${\Gamma }_j^{*}(s)=E\left\{\left[1-I\left(T_i<s,{ϵ}_i=1\right)\right]{\widetilde{\mathbf{\text{W}}}}_i^{\otimes j}\text{exp}\left({\gamma }^{\prime }{\text{X}}_i\right)\right\}$. Σ* can be expressed in block matrix form as

$${\Sigma }^{*}=\left(\begin{array}{ll}{\Sigma }_{11}^{*}\hfill & {\Sigma }_{12}^{*}\hfill \\ {\Sigma }_{21}^{*}\hfill & {\Sigma }_{22}^{*}\hfill \end{array}\right),$$

where ${\Sigma }_{11}^{*}$ is the scalar (1,1) element of Σ*. Then for large n, $Var(\sqrt{n}\widehat{\beta })\approx v(0)={\left[{\left({\Sigma }^{*}\right)}^{-1}\right]}_{11}$. As with generalized linear models, we can obtain a simpler expression if E(Z|X) is linear with respect to X.

Theorem 2 Assume that model (5) holds and that E(Z|X) = a + b^′X for some scalar a and vector b. Then v(0) ≤ [Var (Z)ψ*]⁻¹VIF, where ψ* = P(T_i ≤ C_i ^ t*, ε_i = 1) is the probability that a subject has a cause 1 event in [0, t*] that is not censored. Subsequently, a sufficient sample size for two-sided testing of H₀ : β = 0 at significance level α with power π to detect β = δ is given by

$$n=\frac{{\left[{\Phi }^{-1}(1-\alpha /2)+{\Phi }^{-1}(\pi )\right]}^2}{{\psi }^{*}Var(Z){\delta }^2}\cdot VIF.$$ (6)

This formula is nearly identical to (4), the sole difference being that f₀₀ is replaced with ψ*, the chance that an event of interest is observed during the study period [0, t*]. This event frequency is generally easier to specify than the residual variance involved in f₀₀ and this specification can be guided by previous studies or clinicians’ input. As with the GLM formula, the simulation study in Section 3 supports using formula (6) even if E(Z|X) is not linear with respect to X. Latouche et al.⁷ offered this formula solely for the special case where Z is binary and uncorrelated with X in the censoring-free and censoring-complete settings (potential censoring times are known for all patients). Because (6) is derived using the variance calculation in Martens and Logan¹⁶, which is valid for censoring-free, censoring-complete, and random right-censoring situations, our formula (6) is valid in all of these settings as well.

2.4 |. Cox Regression Model

In the single failure type setting, Cox regression is often used to evaluate a main variable’s impact on failure risk while adjusting for covariates. We assume that patients are at risk of a single event type and that their observations ${\left\{\left(T_i,C_i,Z_i,{\text{X}}_i\right)\right\}}_{i=1}^n$ comprise a random sample. Moreover, the censoring assumption may be relaxed from independent censoring to “noninformative” censoring, i.e. T and C are independent given Z and X. Letting λ be the hazard function for the failure distribution, the Cox model assumes that this function has the form

$$\lambda (t\mid Z,\text{X})={\lambda }_0(t)\text{ exp}\left(\beta Z+{\gamma }^T\text{X}\right)\text{ for }t\le t^{*},$$ (7)

where t is the time on study, [0, t*] is the study period being considered, and λ₀ is the baseline hazard function. Similar as before, Cox² showed that the Cox partial likelihood estimator $\widehat{\nu }$ of ν exists such that $\sqrt{n}(\widehat{\nu }-\nu )\stackrel{d}{\to }N\left(0,{\tilde{\mathbf{\Sigma }}}^{-1}\right)$ where, under H₀,

$$\tilde{\mathbf{\Sigma }}={\int }_0^{t^{*}}P\left(C_i\ge u\right)\left[{\Gamma }_2(u)-{\Gamma }_1{(u)}^{\otimes 2}/{\Gamma }_0(u)\right]{\lambda }_0(u)du$$

with ${\Gamma }_j(s)=E[I\left(T_i\wedge C_i\ge s\right){\widetilde{\mathbf{\text{W}}}}_i^{\otimes j}\text{exp}\left({\gamma }^{\prime }{\text{X}}_i\right)\}$. From this, we obtain a formula nearly identical to those for the GLM and Fine-Gray models.

Theorem 3 Assume that the model (7) holds and that E(Z|X) = a + b^′X for some scalar a, vector b. Then v (0) ≤ [Var (Z)ψ]⁻¹V I F, where ψ = P (T_i ≤ C_i ∧ t*) is the probability that a subject has an event in [0, t*] that is not censored. Subsequently, a sufficient sample size for two-sided testing of H₀ : β = 0 at significance level α with power π to detect β = δ is given by

$$n=\frac{{\left[{\Phi }^{-1}(1-\alpha /2)+{\Phi }^{-1}(\pi )\right]}^2}{\psi Var(Z){\delta }^2}\cdot VIF.$$ (8)

This formula simply uses the observed event frequency ψ in place of f₀₀ and ψ* for (4) and (6), respectively. Schoenfeld⁶ first introduced this formula for the Cox model in the special case of a binary main variable that is uncorrelated with other covariates. Hsieh and Lavori¹² proposed applying this formula for a Cox model with a nonbinary main variable and/or correlation between the main and other covariates, but did not provide theoretical support for this usage. Our methodology reconciles these methods of sample size estimation for the Cox and Fine-Gray models by extending their usage to general covariate distributions and possible right censoring, while providing a firm theoretical basis for their application.

As noted in Section 2.3 for GLM models, formulas (6) and (8) may also be applied in situations where the sample size is predetermined and investigators instead wish to assess the power provided to detect a targeted effect size or to find the minimum effect size detectable at a given power level. The power π or effect size δ can be expressed directly in terms of the other elements of these formulas using basic algebra.

2.5 |. Event-driven Sample Size Calculation for the Poisson, Cox, and Fine-Gray Models

The generalized linear, Cox, and Fine-Gray regression sample size calculations presented here are all derived from the very general formula in (1), though derivation of the asymptotic variance term v(0) proceeds differently for each. These formulas provide the required number of subjects for a study to test influence of a main effect of interest; but, sample size calculations for the Cox and Fine-Gray models are sometimes performed in terms of the required number of events. The proposed formulas for these models can also be applied to compute the required number of relevant events because, by rearranging terms in (8) and (6), we obtain

$$\begin{gathered} \frac{{\left[{\Phi }^{-1}(1-\alpha /2)+{\Phi }^{-1}(\pi )\right]}^2}{Var(Z){\delta }^2}\cdot VIF=n\psi \text{ for the Cox model and} \\ =n{\psi }^{*}\text{ for the Fine-Gray model,} \end{gathered}$$

where the right-hand side gives the expected total number of events occurring in n patients under H₀. For large n, this expectation will lie close to the observed number of events, and so the right-hand side can be used as the required number of events to perform the specified two-sided test. The required number of events can be similarly specified for Poisson regression. For this model, the term f₀₀ in (4) equals E[exp(β₀ + γ^TX)], the expected number of events per subject under H₀. Then rearranging terms in (4) gives

$$\frac{{\left[{\Phi }^{-1}(1-\alpha /2)+{\Phi }^{-1}(\pi )\right]}^2}{Var(Z){\delta }^2}\cdot VIF=n{f}_{00}\text{ for the Poisson model,}$$

where the right-hand side is the expected total number of events in n subjects under H₀ and can be used as the targeted number of events for the study.

2.6 |. Precision / Confidence Interval-focused Sample Size Calculation

The proposed formulas can also be applied toward a precision-focused sample size estimation in the generalized linear, Cox, and Fine-Gray models, i.e. choosing an appropriate sample size to yield a confidence interval (CI) for β whose width does not exceed a specified value. To do this, we simply modify the formulas by replacing δ by the desired half-width for the CI and replacing the numerator term [Φ⁻¹ (1 − α/2) + Φ⁻¹ (π)]² by Φ⁻¹ (1 − α/2)²; all other terms remain as previously defined. Conversely, the modifications also may be used to determine the width of the CI that will be produced by a given sample size. These results pertain to both the subject-based sample size formulas (4), (6), and (8) as well as to the event-based formulas in Section 2.5.

3 |. SIMULATION STUDY

The theory presented in Section 2 supports application of the proposed sample size determination techniques for studies with large samples. To assess the methods’ performance for more modest, perhaps more realistic sample sizes, a simulation study was conducted. We evaluated whether the proposed formulas produce sample sizes that provide actual power levels that lie close to the targeted levels for detecting a prespecified effect size of interest. For each simulation, we choose a targeted effect size δ for the model under consideration, compute the estimated sample size from our formula, generate data for the regression model under the alternative hypothesis H_a : β = δ, and record whether the fitted regression model rejects H₀.

Simulations were performed for the logistic, Cox, and Fine-Gray regression models. For each model, two values of δ were considered that give minimum estimated sample sizes in the ranges 100–300 and 400–600. Actual power levels targeted by the sample size calculations included π = 80% and 90%. For these models, we considered the following choices of covariates: Z has either a Bernoulli(0.5), standard normal, or standard lognormal distribution; and X is either Bernoulli(0.5) or is a random vector with independent components, whose first two components are Bernoulli(0.5) and third component is standard normal. Values considered for the VIF were 1.0, 1.2, 1.5, or 2.0. If Z was standard normal, the correlation was induced by generating the main covariate as $Z=c{1}_p^T\text{X}+\sqrt{1-p{c}^2}W$, where W ~ N(0, 1) is independent of X, p = dim(X), and the constant c is chosen to produce the specified VIF value. If Z was Bernoulli, it was generated by the logistic model logit $P(Z\mid \text{X})=c{1}_p^T\text{X}$, with c chosen to induce the targeted level of correlation. If Z was lognormal, it relates to X through the log-linear model $\text{log}(Z)=c{1}_p^T\text{X}+\sqrt{1-p{c}^2}W$, where W ~ N (0, 1) is independent of X and c is chosen to induce the targeted level of correlation.

For the logistic model, h(u) = logit(u) and covariate effects were generated according to model (2) with the following possible specifications: β₀ = −log 3, 0, or log 3; β = log 1.4 or log 1.6; and $\gamma =\phi /\sqrt{p}\cdot 1_p$, where φ = − log 3, − log 2, log 2, or log 3. Choices of β correspond to odds ratios of 1.4 or 1.6 for the main variable. γ was specified so that, for a given value of φ, the linear predictor γ^T X would have the same variance under either choice of distribution for X. This produced a total of 1152 possible configurations of simulation parameters, with 10000 replicates simulated for each.

For the Cox and Fine-Gray models, we assume all subjects are followed for a maximum of one year, with those surviving longer being administratively censored at the one year mark. Simulations for the Cox model considered β = log 1.2 or log 1.3 and for the Fine-Gray model considered β = log 1.3 or log 1.4, corresponding to hazard ratios of 1.2 or 1.3 for the primary variable of interest in the Cox model and subdistribution hazard ratios of 1.3 or 1.4 in the Fine-Gray model. Choices of censoring rates for the two models included either no censoring, or exponential censoring at a rate of 50% per year. For the Fine-Gray model, the overall proportion of cause 1 events was specified as either 30% or 60%. In total, the Cox and Fine-Gray models had 768 and 1536 possible configurations, respectively; for each configuration, 10000 studies were simulated.

From each configuration, an empirical estimate for the power provided by the calculated sample size is obtained. The large number of these estimates for each model were then summarized using box plots to evaluate the performance of our sample size specification across configurations. Moreover, it was noted from our simulation results that the only other simulation parameters that impacted the empirical power estimates noticeably were the VIF and the targeted power level, π. Therefore, these box plots aggregate the estimates obtained from common values of VIF and π.

Figure 1 displays these estimates from the logistic model simulations determined by sample sizes computed using (4). For VIF = 1, where the covariates are uncorrelated, the empirical power estimates tend to fall about 1% short of the targeted power level used in the sample size calculation. We attribute this slight shortfall to incomplete convergence of the logistic model’s parameter estimate $\widehat{\beta }$ to its asymptotic distribution. Interestingly, the empirical power increased slightly for VIF > 1 and meets the targeted power, on average. We believe two factors explain this slight boost in observed power levels: first, the larger sample sizes required by the variance inflation technique ensure tighter convergence of model estimates to the asymptotic distribution from which the sample size formula is derived; second, the sample size from formula (4) was shown to be an upper bound, so it may be conservative. The proposed sample size formula appears to perform well by producing sample sizes that provide power close to or exceeding the targeted level in a wide variety of settings.

FIGURE 1.

Estimated power provided for testing the main effect in the logistic regression model by the proposed sample size method: Empirical estimates of the actual power levels provided by logistic regression using the sample size formula in (4) were generated using sample sizes intended to target 80% and 90% power. Box plots summarize the estimates for each targeted power level and variance inflation factor (VIF) considered. Dotted black lines indicate the targeted power levels for the tests.

Power estimates for sample sizes computed for the Cox model via (8) and for the Fine-Gray model via (6) are displayed in Figures 2 and Figure 3, respectively. Similar trends to those for the logistic model are seen: with uncorrelated covariates (VIF = 1), empirical power levels are roughly 1% less, on average, than the targeted levels; however, in the presence of correlation, estimated power levels tend to meet the target for these time-to-event regression models.

FIGURE 2.

Estimated power provided for testing the main effect in the Cox proportional hazards model by the proposed sample size method: Empirical estimates of the actual power levels provided by Cox regression using the sample size formula in (8) were generated using sample sizes intended to target 80% and 90% power. Box plots summarize the estimates for each targeted power level and variance inflation factor (VIF) considered. Dotted black lines indicate the targeted power levels for the tests.

FIGURE 3.

Estimated power provided for testing the main effect in the Fine-Gray regression model by the proposed sample size method: Empirical estimates of the actual power levels provided by Fine-Gray regression using the sample size formula in (6) were generated using sample sizes intended to target 80% and 90% power. Box plots summarize the estimates for each targeted power level and variance inflation factor (VIF) considered. Dotted black lines indicate the targeted power levels for the tests.

One curiosity seen in Figures 1 – 3 is that, though the locations of the box plots are similar across figures and lie close to the targeted power levels, the widths are noticeably larger for the logistic and Fine-Gray models compared to the Cox model. Moreover, these widths are increasing as VIF does. Our working explanation for these results is that, though the empirical power levels generally match the target, there are some configurations for which the actual power levels deviate from the target due to incomplete convergence of the test statistics to their asymptotic distributions. Because the logistic and Fine-Gray models both considered larger numbers of configurations, the higher variability of their empirical power estimates reflects the greater diversity in simulation settings considered compared to the Cox model. Additionally, the VIF adjustment of the sample size under no correlation (VIF = 1.0) exacerbates any deviations of the true power from the targets, producing the ‘fanning out’ of box plot widths in these figures as the VIF grows larger.

For the simulations displayed in Figures 1 – 3, we randomly generated a distinct set of covariates for each replicate from the specified covariate distribution in its configuration. This was done to assess the performance across a wide variety of realized studies. Alternatively, we also ran simulations by generating a single set of covariates for each configuration and simulating outcomes for its 10000 replicated datasets using this common set of predictors. These simulations gave nearly identical empirical power estimates to those displayed in the Figures.

4 |. EXAMPLES

Applications of the proposed methods are demonstrated through the calculation of sample sizes for hypothetical prospective cohort studies investigating possible correlations between biomarkers and development of adverse outcomes following blood and marrow transplantation. One such outcome is veno-occlusive disease (VOD), a rare but severe complication that occurs in approximately 5% of transplant patients and can result in liver failure and a mortality rate estimated to be as high as 80%. A retrospective biomarker study by Zaid et al.¹ found that lower levels of one biomarker, L-Ficolin, were associated with a greater risk of VOD occurrence within 1 year following transplant (p = 0.005), using data from the Blood and Marrow Transplant Clinical Trials Network 0402 trial¹⁷. This association was evaluated using a multivariable logistic regression model from 197 patients with log L-Ficolin levels as the main effect, dichotomized as Low or High using the median value in the cohort. We will use formula (4) to determine an appropriate sample size for a validation study that similarly evaluates L-Ficolin’s impact on VOD onset within the first year in a logistic regression that adjusts for age, Karnofsky score, race, and donor-recipient sex matching, factors that may correlate with VOD risk and/or L-Ficolin concentrations. Because these factors may also be correlated with the biomarker being considered, accounting for this correlation in the sample size determination for this validation study is imperative.

The proposed sample size formula (4) requires specification of the variance of log L-Ficolin, its multiple correlation coefficient R with the other covariates, the expected variance of VOD occurrence f₀₀, the level and power of the test, and the targeted covariate effect. Data from Zaid et al.¹ provides estimates of R = 0.370 and f₀₀ = 0.0654. Because the main effect of log L-Ficolin level is binary with the observed median level from the previous study as the cutpoint, we expect that its variance Var(Z) will be 0.25. A univariate logistic model regressing VOD occurrence on dichotomized log L-Ficolin gave an estimated odds ratio of 4.04 with a 95% CI of (1.09, 14.98). The large width of the CI is attributed mainly to the low VOD frequency of 7.1% in this study. Moreover, the estimates may possibly be biased due to publication bias. Therefore, we compute the estimated sample sizes providing 80% and 90% power to detect a range of plausible effect sizes lying within the lower half of the CI. Plugging these effect sizes and the other parameter estimates into formula (4) gives the sample size estimates listed in Table 1. To illustrate the importance of accounting for correlation of covariates, suppose that the validation study aims to detect an odds ratio of 3.00 for log L-Ficolin with two-sided testing at a 5% level. The respective sample sizes needed to provide 80% and 90% power are 461 and 617 patients. If log L-Ficolin is instead assumed to be uncorrelated with the other covariates, the estimated sample sizes are 398 and 533 to provide 80% and 90% power, respectively. But, if the true correlation matches the estimate 0.370 from the previous biomarker study, the actual power levels provided by these sample sizes are 74.0% and 85.4%, respectively, falling 6.0% and 4.6% short of the targeted levels. Though these are not large deviations from intended power levels, they are appreciable and likely to be considered unacceptable by study investigators.

TABLE 1.

Estimated Sample Sizes from Formula (4) for Detecting Odds Ratios in VOD Example

Power	Targeted Odds Ratio
	1.25	1.50	2.00	2.50	3.00	3.50	4.00
80%	11171	3384	1158	663	461	355	290
90%	14954	4530	1550	887	617	475	388

Another risk that may follow blood and marrow transplantation is the development of chronic graft-versus-host disease (cGVHD). This disease affects roughly 40% of transplant patients, tends to develop 6 months or later post-transplant, and indicates an elevated risk of mortality; therefore, identifying markers that predict its development is important so that therapy can be initiated early to mitigate this risk. The study by Zaid et al.¹ found an association between elevated levels of chemokine (C-X-C motif) ligand 9 (CXCL9) and increased risk of cGVHD onset in a Cox proportional hazards model (p = 0.007). This evaluation dichotomized CXCL9 according to the median value observed in the sample and treated CXCL9 as a time-dependent variable.

However, because death before the development of cGVHD precludes us from observing this disease, onset of cGVHD is a competing risk outcome. Moreover, dichotomization of a continuous predictor can lead to a loss of power for evaluating its effect in a regression model¹⁸. Therefore, for a hypothetical validation study that evaluates the direct impact of CXCL9 on the chance of developing cGVHD within 2 years of transplant, we would use the Fine-Gray model and treat CXCL9 as continuous by using the log biomarker level at day 100 after transplant. Moreover, the model would adjust for the factors age, conditioning regimen, time from diagnosis to transplant, donor-recipient sex matching, recipient CMV status, and Karnofsky performance score, which are suspected to influence the risk of cGVHD from previous research.

The proposed sample size formula (6) requires specification of the variance of log CXCL9, its multiple correlation coefficient R with the other covariates, the probability of observing cGVHD onset by two years, the level and power of the test, and the covariate effect that we wish to detect. We assume that the censoring distribution for the validation study will be the same as that in the original one. Using data from¹, we estimated Var(log CXCL9) = 4.813, R = 0.380, and the probability of observing cGVHD onset by two years = 135/211 = 64.0%. Because this is a continuous main effect of interest, a subdistribution hazard ratio for log CXCL9 of 1.10 will be considered, indicating increasing likelihood of cGVHD as CXCL9 level increases. Using formula (6), the sample size to detect a log CXCL9 subdistribution hazard ratio of 1.10 at a 5% level with 80% power using two-sided testing is 328; detecting this effect with 90% power requires 439 patients. If the correlation of log CXCL9 with the other covariates is not considered by simply assuming VIF = 1, the formula gives sample sizes of 281 and 376 to provide 80% and 90% power, respectively. If the true correlation equals the estimate of 0.380 from the prior study, though, the respective power levels provided by these sample sizes are 73.6% and 85.0%, falling below the targeted levels by 6.5% and 5.0%. Although the multiple correlations of 0.370 and 0.380 considered for the VOD and cGVHD examples are moderate in size, our calculations demonstrate that ignoring their existence may diminish the actual power provided and prevent the study under design from effectively testing the main effect of interest.

5 |. DISCUSSION

This paper proposes succinct formulas for determining a sufficient sample size to detect the effect of a primary covariate of interest in generalized linear, Cox, and Fine-Gray regression models. This methodology supports a broad class of generalized linear models that includes the frequently used logistic and Poisson models as well as less common choices such as probit, exponential, and inverse Gaussian regression. These techniques support nearly arbitrary choices of primary and other covariates, the only requirement being that they have finite variance. Moreover, they can accommodate correlation between the main effect and other covariates. By including these features, the proposed techniques both synthesize and extend a collection of existing sample size formulas in the statistical literature for restricted sets of regression models and covariate distributions. For the Fine-Gray competing risks regression model in particular, the proposed formula represents a substantial generalization of the work of Latouche et al.⁷ to application with a general class of covariates in both the censoring-complete and random right censoring settings.

A simulation study showed that the proposed formulas give sample sizes that come close to attaining the nominal power levels under a wide variety of choices of data distributions and covariate effects. Notably, in the presence of correlation between the primary variable and other covariates, the inflation factor adjustment to the sample size provided a suitable increase, generally yielding empirical power levels that met or exceeded the targeted levels. Application of the formula was demonstrated through sample size determination for biomarker validation studies, using an existing biomarker study’s data as a reference. Because of the observational nature of studies such as this and previous knowledge of other risk factors for the outcome of interest, it is crucial to adjust for these factors in the analysis and account for their possible correlation with the main effect of interest. These considerations also apply to nonrandomized clinical trials, where randomization may not be possible due to ethical or feasibility concerns. This makes our proposed methodology especially valuable in nonrandomized trials as well as observational studies.

The formulas proffered and examples presented focus upon determination of an appropriate sample size for designing a study. Because these formulas demonstrate a clear relationship between the sample size, power, and effect size, they are also useful for determining the power provided by a given sample to detect a specified effect and for specifying the smallest effect that may be targeted at a given power level and sample size. These are particularly useful applications in situations where the sample size may be predetermined or constrained, such as in designing studies of retrospective data as well as prospective studies in rare disease populations, where feasibility considerations restrict the sample size to be modest.

Our sample size / power estimation approach for the Cox and Fine-Gray models may be used to determine either the number of events or the number of subjects (n) required. The factor relating these quantities in formulas is the likelihood of an event of interest being observed in the study, ψ or ψ*. This likelihood depends not only on the event time distribution itself but also on design features including the follow-up period, random censoring distribution, and, for prospective studies, on the accrual period and pattern. Care must be taken in specifying these elements so that ψ (or ψ*), and consequently, n, may be estimated accurately. Historical data on related research studies and the proposed strategy for recruitment of patients and centers can guide accurate specification of the censoring and accrual distributions.

This paper focuses on accuracy sample size/power specification in designing individual studies because a study is the fundamental unit for performing effect size estimation and hypothesis testing. An important problem not addressed here is power and precision calculations for meta-analyses, which aim to aggregate multiple studies’ findings into a more precise assessment of the primary effect of interest. Random or mixed effects models are often employed in this setting to perform this evaluation while accounting for potential heterogeneity of the primary effect between studies. We will explore extending the methodology presented here to accommodate regression models commonly used in meta-analysis, thereby aiding the design of meta-analysis studies as well.

For the examples, we assumed that the biomarkers were measured precisely, without error. But, measurement error on predictor variables can arise in observational studies. Its presence can cause a loss of power in testing the effect of these variables if it is improperly handled. Tosteson et al.¹⁹ and Gilbert et al.²⁰ present schemes for sample size determination when measurement error exists on the main covariate of interest in a logistic regression that can account for correlation between covariates. Future work will investigate how to incorporate the presence of this type of error into our methods for accurate sample size calculation in nonlinear regression models.

In addition to providing accurate sample size determination at the design stage, our technique may also offer accurate computation for the purpose of unblinded sample size reestimation partway through a study. Because limited information may exist at the design phase to guide specification of nuisance parameters such as covariate effects and event and censoring rates, this can be a valuable tool to salvage a study that is underpowered due to misspecification. In particular, Tarima et al.²¹ proposed a method for interim sample size reestimation that uses resampling of interim data to estimate the distribution of nuisance parameters and, correspondingly, to adjust the total sample size to attain the targeted power level. Moreover, our proposed formulas are compatible with their technique; the resampling method computes interim estimates for nuisance parameters and adjusted α and π values that can be plugged into our formulas to obtain an updated sample size that attains the intended type I error and power levels. The validity of their method, like ours, is proven through derivations that rely on asymptotic properties. Therefore, the combined use of our and their methods for sample size recalculation in studies with more modest sizes requires evaluation before this joint application can be recommended for wide scale application. We plan to assess small sample performance of this application of our formulas through an comprehensive simulation study.