Sample Size Calculations in Simple Linear Regression: A New Approach

Abstract

The problem tackled is the determination of sample size for a given level and power in the context of a simple linear regression model. The standard approach deals with planned experiments in which the predictor X is observed for a number n of times and the corresponding observations on the response variable Y are to be drawn. The statistic that is used is built on the least squares’ estimator of the slope parameter. Its conditional distribution given the data on the predictor X is utilized for sample size calculations. This is problematic. The sample size n is already presaged and the data on X is fixed. In unplanned experiments, in which both X and Y are to be sampled simultaneously, we do not have data on the predictor X yet. This conundrum has been discussed in several papers and books with no solution proposed. We overcome the problem by determining the exact unconditional distribution of the test statistic in the unplanned case. We have provided tables of critical values for given levels of significance following the exact distribution. In addition, we show that the distribution of the test statistic depends only on the effect size, which is defined precisely in the paper.

1. Introduction

Multiple regression is one of the core methodologies in statistics. Power computation and sample size determination have become integral part of many research proposals submitted for funding. Funding agencies such as UKRI (UK Research and Innovation) and NIH (National Institutes of Health) have been demanding sample size calculations in all prospective proposals. Regression has a long history dating back to Galton [1]. Horton and Switzer [2] reported that 51% of research articles published in the New England Journal of Medicine during May 2004 have Multiple Regression as one of the methods used. The figure for power analysis is at 39%.

In this paper, we focus on power computation in the context of simple linear regression. The current approach in power computations lacks justification. We will point out difficulties in this setting [3].

Simple linear regression is ubiquitous in pediatric clinical diagnostics. The model sets standards for normal growth in children on several metrics [4]. As an illustration, a pediatrician wants to check whether the lung function of a 13-year-old patient is normal. Data is to be collected on healthy subjects in the age range 12–14 years with response,

Y = FEV (Forced Expiratory Volume)

and predictor,

X = Height,

which is an example of an unplanned experiment.

In order to trust the model, we need to decide on the sample size, which in turn, depends on the level of significance, power, and effect size.

First, we investigate the setting under the simple linear regression paradigm. The model has two entities X, the predictor, and Y, the response variable. It is stated as

$$\mathrm{Y}|\mathrm{X}~\mathrm{N}\left({\mathsf{\beta }}_0+{\mathsf{\beta }}_1\mathrm{X},{ \mathsf{\sigma }}^2\right)$$

for some ${\mathsf{\beta }}_0$, ${\mathsf{\beta }}_1,$ and ${\mathsf{\sigma }}^2>0$. The null hypothesis of interest is ${\mathrm{H}}_0:{\mathsf{\beta }}_1=0$ against the alternative ${\mathrm{H}}_1:{\mathsf{\beta }}_1\ne 0$. What should be the required sample size, n, for a given level of significance α, power 1-β, and at the alternative value A of ${\mathsf{\beta }}_1$. Let (X₁, Y₁), (X₂, Y₂), …, (X_n, Y_n), be a potential sample for the testing problem. Let ${\widehat{\mathsf{\beta }}}_1$ be the least squares estimator of ${\mathsf{\beta }}_1$, i.e.,

$${\widehat{\mathsf{\beta }}}_1=\frac{{\mathrm{S}}_{\mathrm{XY}}}{{\mathrm{S}}_{\mathrm{XX}}}$$

where

$${\mathrm{S}}_{\mathrm{XY}}={\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{n}}\left({\mathrm{X}}_{\mathrm{i}}-\overline{\mathrm{X}}\right)\left({\mathrm{Y}}_{\mathrm{i}}-\overline{\mathrm{Y}}\right)$$

and

$$S_{XX}={\displaystyle \sum }_{i=1}^n{\left(X_i-\overline{X}\right)}^2$$

Let RSS be the residual sum of squares, i.e.,

$$\mathrm{RSS}={\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{n}}{\left[\left({\mathrm{Y}}_{\mathrm{i}}-\overline{\mathrm{Y}}\right)-{\widehat{\mathsf{\beta }}}_1\left({\mathrm{X}}_{\mathrm{i}}-\overline{\mathrm{X}}\right)\right]}^2$$

For testing the null hypothesis H0, the following test statistic is used:

$$\mathrm{T}=\left({\widehat{\mathsf{\beta }}}_1\sqrt{{\mathrm{S}}_{\mathrm{XX}}}\right)/\left(\sqrt{\mathrm{RSS}/\left(\mathrm{n}-2\right)}\right).$$

Under the null hypothesis, conditioned on the X-data, T has a t-distribution with n − 2 degrees of freedom. Under the alternative value ${\mathsf{\beta }}_1$ = A, T has a non-central t-distribution with degrees of freedom n − 2, and non-centrality parameter $\mathsf{\lambda }=\left|\mathrm{A}\right|\ast \sqrt{\mathrm{Sxx} }/\mathsf{\sigma }$.

We reject the null hypothesis if and only if $\left|\mathrm{T}\right|>{\mathrm{t}}_{\mathrm{n}-2,1-\frac{\mathsf{\alpha }}{2}}$ where ${\mathrm{t}}_{\mathrm{n}-2,1-\frac{\mathsf{\alpha }}{2}}$ is such that the area to the left of Student’s t-curve on (n − 2) degrees of freedom is 1 − α/2.

The power formula is given by

$$\mathrm{Power}\left(\mathrm{A}\right)=\mathrm{Pr}\left({\mathrm{Reject} \mathrm{H}}_0|{\mathsf{\beta }}_1=\mathrm{A}\right)=\mathrm{Pr}\left(\left|\mathrm{T}\right|>{\mathrm{t}}_{\mathrm{n}-2,1-\frac{\mathsf{\alpha }}{2}}|{\mathsf{\beta }}_1=\mathrm{A}\right).$$

We can set power equal to 1-β and solve for n. This would work as long as we know what $\mathsf{\lambda }=\left|\mathrm{A}\right|\ast \sqrt{{\mathrm{S}}_{\mathrm{XX}} }/\mathsf{\sigma }$ is. This would require knowledge of the alternative value of β₁, ${\mathsf{\sigma }}^2$, and ${\mathrm{S}}_{\mathrm{XX}}$. We will not know what ${\mathrm{S}}_{\mathrm{XX}}$ is, prior to data collection, in the unplanned experiments. Equivalently, one should spell out what λ is. This is a tall order. Adcock [5] recognized these problems. Some software and textbooks assume that $\left(1/\mathrm{n}\right){{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}{({\mathrm{X}}_{\mathrm{i}}-\overline{\mathrm{X}})}^2$ is known. For example, the software PASS [6] and nQuery [7] proceed this way. To overcome these difficulties, we proceed with deriving the exact unconditional distribution of a variant of T. This requires a knowledge of the distribution of X. Let ${\mathsf{\sigma }}_{\mathrm{X}}^2$ be the variances of X.

Modify the test statistic.

$$\mathrm{T}={\widehat{\mathsf{\beta }}}_1\ast {\widehat{\mathsf{\sigma }}}_{\mathrm{X}}/\widehat{\mathsf{\sigma }},$$ (1)

where ${\widehat{\mathsf{\sigma }}}^2=\mathrm{RSS}/\left(\mathrm{n}-2\right)$ ${\mathrm{and}\widehat{\mathsf{\sigma }}}_{\mathrm{X}}^2={\mathrm{S}}_{\mathrm{XX}}/\left(\mathrm{n}-1\right).$

We obtain the unconditional distribution of T under β₁ = 0 as well as under β₁ = A ≠ 0. We assume X~N (µ_x, ${\mathsf{\sigma }}_{\mathrm{X}}^2$), both parameters unknown. Under this assumption, the distribution of T is derived.

In due course, we will show the distribution of T when β₁ = A ≠ 0 depends only on δ = $\left|\mathrm{A}\right|{\ast \mathsf{\sigma }}_{\mathrm{X}}/\mathsf{\sigma }$, which we can deem as the effect size.

The five-parameter model now is:

$$\mathrm{Y}|\mathrm{X}~\mathrm{N}\left({\mathsf{\beta }}_0+{\mathsf{\beta }}_1\mathrm{X},{ \mathsf{\sigma }}^2 \right)$$

$$\mathrm{X}~\mathrm{N}({\mathsf{\mu }}_{\mathrm{x}}, {\mathsf{\sigma }}_{\mathrm{X}}^2).$$

Note that the vector (X, Y) has a bivariate normal distribution.

The paper is organized as follows. In Section 2, we provide a literature review. In Section 3, we outline the main results. We derive the unconditional distribution of T under the null hypothesis in Section 3.1. In Section 3.2, we calculate critical values using the main results. In Section 3.3, we lay out the sample sizes required for a given level, power, and effect size $\mathsf{\delta } =\left|\mathrm{A}\right|{\ast \mathsf{\sigma }}_{\mathrm{X}}/\mathsf{\sigma }$. In Section 4, we summarize the results and draw conclusions. The computational details along with the R code [8] are presented in the Supplementary Materials.

2. Literature Review

Ryan [3] has pointed out difficulties in power calculations in the environment of simple linear regression. The problem is how we handle the predictor X. Adcock [5] has looked at some possible scenarios. One scenario is that the investigator knows the X_i-values (deterministic) for every sample size n. In such a case, the test statistic

$$\left({\widehat{\mathsf{\beta }}}_1\sqrt{{\mathrm{S}}_{\mathrm{XX}}}\right)/\left(\sqrt{\mathrm{RSS}/\left(\mathrm{n}-2\right)}\right)$$ (2)

is eminently usable for power calculations. Its (conditional) null and non-null distributions have been worked out explicitly. The conditional approach is also followed by Dupont et al. [9], Draper et al. [10], Hsieh et al. [11], Maxwell [12], and Thigpen [13].

As an alternative to the test statistic (2), we can build a test based on the sample correlation coefficient $\widehat{\mathsf{\rho }}$ [14], under the joint normality of X and Y. The null and non-null distributions of the underlying test statistic based on $\widehat{\mathsf{\rho }}$ have been worked out explicitly. In our consulting work, many researchers prefer to use the test based on ${\widehat{\mathsf{\beta }}}_1$. It is a choice between causality and association [3,14,15,16,17,18,19,20,21]. The hypotheses H₀: ${\mathsf{\beta }}_1$ = 0 and H₀: $\mathsf{\rho }$ = 0 under bivariate normality are equivalent, but the test statistics are different. It is easy to determine sample size under the correlation context [14]. However, this sample size cannot be offered for testing the hypothesis on the slope. The power is less. In other words, test hopping is not permissible; i.e., they are two different tests with distinct power functions.

3. Outline of Results

We will now derive the unconditional distribution of ${\widehat{\mathsf{\beta }}}_1$, which will be instrumental in sample size calculations. We use the test statistic T = ${\widehat{\mathsf{\beta }}}_1\ast {\widehat{\mathsf{\sigma }}}_{\mathrm{X}}/\widehat{\mathsf{\sigma }}$.

Under the null hypothesis β₁ = 0, we show that

$${\mathrm{T}}^2~\frac{\left(\mathrm{n}-2\right)}{\left(\mathrm{n}-1\right)}\ast \frac{{\mathrm{W}}_1{\mathrm{W}}_4}{{\mathrm{W}}_2{\mathrm{W}}_3},$$

where ${\mathrm{W}}_1~{\mathsf{\chi }}_1^2$, ${\mathrm{W}}_2~{\mathsf{\chi }}_{\mathrm{n}-1}^2$, ${\mathrm{W}}_3~{\mathsf{\chi }}_{\mathrm{n}-2}^2$ and ${\mathrm{W}}_4~{\mathsf{\chi }}_{\mathrm{n}-1}^2$, with the W_i values being mutually independent. It follows implicitly that

$$\mathrm{T} ~ \sqrt{\left(\mathrm{n}-2\right)∕\left(\mathrm{n}-1\right)}\ast {\mathrm{U}}_1\ast {\mathrm{U}}_2/{\mathrm{U}}_3$$

with ${\mathrm{U}}_1, {\mathrm{U}}_2, {\mathrm{U}}_3$ independently distributed, ${\mathrm{U}}_1~{\mathrm{t}}_{\mathrm{n}-1}, {\mathrm{U}}_2~{\mathsf{\chi }}_{\mathrm{n}-1}$, and ${\mathrm{U}}_3~{\mathsf{\chi }}_{\mathrm{n}-2}$, and where ${\mathsf{\chi }}_{\mathrm{n}-1}$ is the $\mathsf{\chi }$ distribution with $\left(\mathrm{n}-1\right)$ degrees of freedom

We use this result to obtain the critical values of the test based on T, for given levels. For power and sample size computations, we need the distribution of T for any given value of β₁. The distribution depends on the alternative values of β₁, ${\mathsf{\sigma }}_{\mathrm{X}}^2$ and σ². It turns out that the distribution depends only on δ = ${\mathsf{\beta }}_1{\ast \mathsf{\sigma }}_{\mathrm{X}}/\mathsf{\sigma }$, which we can deem as the effect size. The specification of δ facilitates computation of power. Despite all these deliberations, no magic explicit formula for power surfaces. Knowing the distribution of T² when δ is spelled out, the pain is eased a little bit.

3.1. Distributional Results

In this section, we will derive the distribution of T of (1) unconditionally. The following series of steps will give the desired result.

1. Given X₁, X₂, …, X_n, ${\widehat{\beta }}_1$ has a normal distribution with mean ${\beta }_1$ and variance ${\sigma }^2/S_{XX}$ and ${\widehat{\beta }}_1$ and $RSS$ are independent.

2. Uncoditionally, $RSS/{\sigma }^2~{\chi }_{n-2}^2$

3. $S_{XX}/{\sigma }_X^2~{\chi }_{n-1}^2$.

4. $RSS$ and $S_{XX}$ are independent.

More generally, we obtain the distribution of $T=\left({\widehat{\beta }}_1-{\beta }_1\right){\widehat{\sigma }}_X/\widehat{\sigma }$ for a given value of β₁.

The joint density function of ${\widehat{\beta }}_1$ and $S_{XX}$:

$$\begin{array}{l}g\left({\widehat{\beta }}_1,S_{XX}\right)=\frac{\sqrt{S_{XX}}}{\sqrt{2\pi }\sigma }\ast \mathit{exp}\left(-\frac{S_{XX}}{2{\sigma }^2}{\left({\widehat{\beta }}_1-{\beta }_1\right)}^2\right)\ast \frac{1}{Г\left(\frac{n-1}{2}\right)\ast 2^{\frac{n-1}{2} }}\mathit{exp}\left(-\frac{S_{XX}}{2\ast {\sigma }_X^2}\right)\ast {\left(\frac{S_{XX}}{{\sigma }_X^2}\right)}^{\frac{n-1}{2}-1}\ast \left(\frac{1}{{\sigma }_X^2}\right),\\ -\infty <{\widehat{\beta }}_1<\infty , 0<S_{XX}<\infty \end{array}$$

The (unconditional) marginal density of ${\widehat{\beta }}_1$ is given by:

$$\begin{array}{ll}f\left({\widehat{\beta }}_1\right)& =\frac{1}{2^{\left(\frac{1}{2}\right)}\ast 2^{\frac{n-1}{2}}\ast \sqrt{\pi }\ast Г\left(\frac{n-1}{2}\right)\ast \sigma \ast {\left({\sigma }_X^2\right)}^{\frac{n-1}{2}}}{{\displaystyle \int }}_0^{\infty }{S}_{XX}{}^{\frac{n}{2}-1}\ast \mathit{exp}\left(-\frac{S_{XX}}{2}\left[ \frac{{\left(\widehat{{\beta }_1}-{\beta }_1\right)}^2}{{\sigma }^2}+\frac{1}{{\sigma }_X^2}\right]\right)\mathrm{d}{S}_{XX}\\ & =\frac{Г\left(\frac{n}{2}\right)}{\sqrt{\pi }\ast Г\left(\frac{n-1}{2}\right)\ast \sigma \ast {\left({\sigma }_X^2\right)}^{\frac{n-1}{2}}}{\left(\frac{1}{\frac{1}{{\sigma }_X^2}+\frac{{\left({\widehat{\beta }}_1-{\beta }_1\right)}^2}{{\sigma }^2}}\right)}^{\frac{n}{2}}\\ & =\frac{{\sigma }_X}{B\left(\frac{1}{2},\frac{n-1}{2}\right)\ast \sigma }\ast \frac{1}{{\left(1+\frac{{\left({\widehat{\beta }}_1-{\beta }_1\right)}^2\ast {\sigma }_X^2}{{\sigma }^2}\right)}^{\frac{n}{2}}}, -\infty <{\widehat{\beta }}_1<\infty \end{array}$$

Some properties of this density are clear to observe. For example, the distribution is symmetric around the true value ${\beta }_1$. If n = 2, the distribution is Cauchy. In addition,

$$\left(\frac{{\sigma }_X}{\sigma }\right)\ast \left({\widehat{\beta }}_1-{\beta }_1\right)\sqrt{n-1} ~{\mathrm{t}}_{\mathrm{n}-1}.$$

Further, if n > 3, unconditionally,

1. $E\left({\widehat{\beta }}_1\right)={\beta }_1$ and Var$\left({\widehat{\beta }}_1\right)=({\sigma }^2/{\sigma }_X^2)\ast {\left(n-3\right)}^{-1}$;

2. In the conditional set-up,

   $$E\left({\widehat{\beta }}_1|X_1,X_2,\dots ,X_n\right)={\beta }_1$$

   $$\mathrm{Var}\left({\widehat{\beta }}_1|X_1,X_2,\dots ,X_n\right)={\sigma }^2/S_{XX};$$
3. The random variable $U=\left({\widehat{\beta }}_1-{\beta }_1\right){\sigma }_X/\sigma$ has the probability density function:

   $$f\left(U\right)=\frac{1}{B\left(\frac{1}{2},\frac{n-1}{2}\right)}\ast \frac{1}{{\left(1+U^2\right)}^{\frac{n}{2}}}, -\infty <U<\infty ;$$

4. It follows that $U^2~W_1/W_2$, where $W_1~{\chi }_1^2$ and $W_2~{\chi }_{n-1}^2$, with $W_1$ and $W_2$ being independent;

5. Exact distribution of $T^2$: note that $\left({\widehat{\beta }}_1-{\beta }_1\right)/\sigma$ and ${\widehat{\sigma }}_X^2$ are independent;

6.  

   $$\begin{gathered} T^2={\left({\widehat{\beta }}_1-{\beta }_1\right)}^2{\widehat{\sigma }}_X^2/{\widehat{\sigma }}^2=\left\{{\left({\widehat{\beta }}_1-{\beta }_1\right)}^2\ast {\sigma }_X^2/{\sigma }^2\right\}\ast ({\sigma }^2/{\widehat{\sigma }}^2)\ast {\widehat{\sigma }}_X^2/{\sigma }_X^2 \\ ~(W_1/W_2)\ast \left\{\left(n-2\right)/W_3\right\}\ast W_4/\left(n-1\right), \end{gathered}$$

   where $W_3~{\chi }_{n-2}^2$ and $W_4~{\chi }_{n-1}^2$, and with $W_1,W_2,W_3,$ and $W_4$ being independent.
   In short,

   $${\mathrm{T}}^2 ~ \left(n-2\right)/\left(n-1\right)\ast (W_1{W}_4)/(W_2{W}_3)$$ (3)
7. It follows that:

   $$E\left(T^2\right)=\left(n-2\right)/\{\left(n-3\right)\left(n-4\right)$$
8. An alternative form of the distribution [22]:

   $$\begin{gathered} \left(n-1\right)/\left(n-2\right)\ast T^2 ~ (W_1{W}_4)/(W_2{W}_3) \\ ~ Bet{a}_{II}\left\{\frac{1}{2},\frac{n-2}{2}\right\}\ast Bet{a}_{II}\left\{\frac{n-1}{2},\frac{n-1}{2}\right\}, \end{gathered}$$

   where $Bet{a}_{II}$ signifies the beta distribution of the second kind.

3.2. Critical Values

We obtain the critical values of the test based on the test statistic $\mathrm{T}={\widehat{\mathsf{\beta }}}_1{\ast \widehat{\mathsf{\sigma }}}_{\mathrm{X}}/\widehat{\mathsf{\sigma }}$ for three levels of significance. We denote the critical value by ${\mathrm{C}}_{\mathrm{n},\mathsf{\alpha }}$. The critical value ${\mathrm{C}}_{\mathrm{n},\mathsf{\alpha }}$ satisfies the equation:

$$\mathsf{\alpha }=\mathrm{Pr}\left(\left|\frac{{\widehat{\mathsf{\beta }}}_1{\widehat{\mathsf{\sigma }}}_{\mathrm{X}}}{\widehat{\mathsf{\sigma }}}\right|>{\mathrm{C}}_{\mathrm{n},\mathsf{\alpha }}|{\mathrm{H}}_0:{\mathsf{\beta }}_1=0\right)=\mathrm{Pr}\left(\frac{{\widehat{\mathsf{\beta }}}_1^2{\widehat{\mathsf{\sigma }}}_{\mathrm{X}}^2}{{\widehat{\mathsf{\sigma }}}^2}>{\mathrm{C}}_{\mathrm{n},\mathsf{\alpha }}^2|{\mathrm{H}}_0:{\mathsf{\beta }}_1=0\right)$$

Under ${\mathrm{H}}_0$,

$${\mathrm{T}}^2=\frac{{\widehat{\mathsf{\beta }}}_1^2{\widehat{\mathsf{\sigma }}}_{\mathrm{X}}^2}{{\widehat{\mathsf{\sigma }}}^2}~\frac{\left(\mathrm{n}-2\right)}{\left(\mathrm{n}-1\right)}\ast \frac{{\mathrm{W}}_1{\mathrm{W}}_4}{{\mathrm{W}}_2{\mathrm{W}}_3},$$

where ${\mathrm{W}}_1~{\mathsf{\chi }}_1^2$, ${\mathrm{W}}_2~{\mathsf{\chi }}_{\mathrm{n}-1}^2$, ${\mathrm{W}}_3~{\mathsf{\chi }}_{\mathrm{n}-2}^2$ and ${\mathrm{W}}_4~{\mathsf{\chi }}_{\mathrm{n}-1}^2$, with W_i values being independent.

There are two options. One is using the pdf of $\left\{\left(\mathrm{n}-2\right)/\left(\mathrm{n}-1\right)\right\}$ $\ast$ T². Following Jambunathan [22], one can write the pdf of the product U*V of the random variables U and V with U$~{\mathrm{Beta}}_{\mathrm{II}}\left(1/2 ,\left(\mathrm{n}-2\right)/2\right)$, V$~{ \mathrm{Beta}}_{\mathrm{II}}\left(\left(\mathrm{n}-1\right)/2,\left(\mathrm{n}-1\right)/2\right)$, and U and V being independent. The pdf is in the form of a double integral and its evaluation would require the use of a quadrature formula with the attendant errors of approximation. The second option is to determine the distribution of T² by sampling extensively the components that make up T² via Monte-Carlo. We have pursued the second option. The critical values are tabulated in File S1.

One can also obtain the critical value ${\mathrm{C}}_{\mathrm{n},\mathsf{\alpha }}$ via the asymptotic distribution of T. One benefit of our derivation of the exact distribution is that if n is large, and null hypothesis is true,

$$\mathrm{T}~\mathrm{Normal} (0, \left(\mathrm{n}-2\right)/\{\left(\mathrm{n}-3\right)(\mathrm{n}-4)\left\}\right), \mathrm{approximately}.$$

There are several ways to establish the asymptotic normality of T. The exact unconditional distribution of $\sqrt{\mathrm{n}-1}$ $\ast \left({\widehat{\mathsf{\beta }}}_1-{\mathsf{\beta }}_1\right){\ast \mathsf{\sigma }}_{\mathrm{X}}/\mathsf{\sigma }$ is t_n−1, which is asymptotically N (0, 1). Then we use the fact that ${\widehat{\mathsf{\sigma }}}_{\mathrm{X}}$ is consistent for ${\mathsf{\sigma }}_{\mathrm{X}}$ and that $\widehat{\mathsf{\sigma }}$ is consistent for σ. Since we know the variance of T exactly, we use this variance in the description of the asymptotic distribution of T. We can now calculate the critical values, as well as those coming from the exact distribution, following the asymptotic distribution.

In File S1, we report the average critical values C_n,α along with the critical values stemming from the asymptotic theory. A description of these asymptotic critical values is provided below.

Critical values from the normal approximation:

Level	Critical Value Formula	Verbal description in File S1
10%	$1.645\times \sqrt{\left(\mathrm{n}-2\right)/\left\{\left(\mathrm{n}-3\right)\left(\mathrm{n}-4\right)\right\}}$	10% normal
5%	$1.96\times \sqrt{\left(\mathrm{n}-2\right)/\left\{\left(\mathrm{n}-3\right)\left(\mathrm{n}-4\right)\right\}}$	5% normal
1%	$2.576\times \sqrt{\left(\mathrm{n}-2\right)/\left\{\left(\mathrm{n}-3\right)\left(\mathrm{n}-4\right)\right\}}$	1% normal

Comments on File S1: The Normal Critical Value column is explained.

1. Normal critical value 10% = critical value coming from the asymptotic distribution when α = 0.10.

2. Normal critical value 5% = critical value coming from the asymptotic distribution when α = 0.05.

3. Normal critical value 1% = critical value coming from the asymptotic distribution when α = 0.01.

4. Critical value 10% = critical value coming from the exact distribution of T when α = 0.10.

5. Critical value 5% = critical value coming from the exact distribution of T when α = 0.05.

6. Critical value 1% = critical value coming from the exact distribution of T when α = 0.01.

7. When α = 0.10, |Normal critical value 10%: Critical value 10%| ≤ 0.001 for n ≥ 50. One can enjoy the benefit of normal approximation when n ≥ 50.

8. When α = 0.05, |Normal critical value 5%: Critical value 5%| ≤ 0.001 for n ≥ 89. One can enjoy the benefit of normal approximation when n ≥ 89.

9. For α = 0.01, Table 1 is not informative when |Normal critical value 1%: Critical value 1%| ≤ 0.001.

3.3. Sample Size and Power

For a given level α, sample size n, and alternative value of β₁ = A, power is given by

$$\mathrm{Power} \left(\mathrm{A}\right)=\mathrm{Pr}(\left|{\widehat{\mathsf{\beta }}}_1{\ast \widehat{\mathsf{\sigma }}}_{\mathrm{X}}/\widehat{\mathsf{\sigma }}\right|>{\mathrm{C}}_{\mathrm{n},\mathsf{\alpha }} |{ \mathsf{\beta }}_1=\mathrm{A}).$$

Suppose 1 − β is the specified power. For the sample size, we set

$$1-\mathsf{\beta }=\mathrm{Pr}(\left|{\widehat{\mathsf{\beta }}}_1{\ast \widehat{\mathsf{\sigma }}}_{\mathrm{X}}/\widehat{\mathsf{\sigma }}\right|>{\mathrm{C}}_{\mathrm{n},\mathsf{\alpha }} |{ \mathsf{\beta }}_1=\mathrm{A}).$$

and solve for n. We will need the distribution of ${\widehat{\mathsf{\beta }}}_1{\ast \widehat{\mathsf{\sigma }}}_{\mathrm{X}}/\widehat{\mathsf{\sigma }}$, when ${\mathsf{\beta }}_1=\mathrm{A}$. Rewrite

$${\widehat{\mathsf{\beta }}}_1{\ast \widehat{\mathsf{\sigma }}}_{\mathrm{X}}/\widehat{\mathsf{\sigma }}=({\widehat{\mathsf{\beta }}}_1-{ \mathsf{\beta }}_1){\ast \widehat{\mathsf{\sigma }}}_{\mathrm{X}}/\widehat{\mathsf{\sigma }}+{\mathsf{\beta }}_1{\ast \widehat{\mathsf{\sigma }}}_{\mathrm{X}}/\widehat{\mathsf{\sigma }}.$$

The distribution of $({\widehat{\mathsf{\beta }}}_1-{ \mathsf{\beta }}_1){\ast \widehat{\mathsf{\sigma }}}_{\mathrm{X}}/\widehat{\mathsf{\sigma }}$ is described in Section 3.1 and it is free of the parameters of the regression model. Consequently, the random variables $({\widehat{\mathsf{\beta }}}_1-{ \mathsf{\beta }}_1){\ast \widehat{\mathsf{\sigma }}}_{\mathrm{X}}/\widehat{\mathsf{\sigma }}$ and ${\mathsf{\beta }}_1{\ast \widehat{\mathsf{\sigma }}}_{\mathrm{X}}/\widehat{\mathsf{\sigma }}$ are independently distributed. Since $\widehat{\mathsf{\sigma }}$ and ${\widehat{\mathsf{\sigma }}}_{\mathrm{X}}$ are independently distributed,

$$\begin{gathered} {\left({\mathsf{\beta }}_1{\ast \widehat{\mathsf{\sigma }}}_{\mathrm{X}}/\widehat{\mathsf{\sigma }}\right)}^2 {\to }^{\mathrm{d}} {\mathsf{\beta }}_1^2{\ast \mathsf{\sigma }}_{\mathrm{X}}^2/\left(\mathrm{n}-1\right){\ast \mathrm{W}}_5\ast \left(\mathrm{n}-2\right)/{\mathsf{\sigma }}^2\ast 1/{\mathrm{W}}_6 \\ ={\left({\mathsf{\beta }}_1{\ast \mathsf{\sigma }}_{\mathrm{X}}/\mathsf{\sigma }\right)}^2\left(\mathrm{n}-2\right)/\left(\mathrm{n}-1\right){\ast \mathrm{W}}_5/{\mathrm{W}}_6, \end{gathered}$$

with W₅~${\mathsf{\chi }}_{\mathrm{n}-1}^2$, W₆~${\mathsf{\chi }}_{\mathrm{n}-2}^2$, and W₅ and W₆ being independent.

An important fact emerges from these deliberations in that the distribution of $({\widehat{\mathsf{\beta }}}_1-{ \mathsf{\beta }}_1){\ast \widehat{\mathsf{\sigma }}}_{\mathrm{X}}/\widehat{\mathsf{\sigma }}$ + ${\mathsf{\beta }}_1{\ast \widehat{\mathsf{\sigma }}}_{\mathrm{X}}/\widehat{\mathsf{\sigma }}$ depends only on δ = ${\mathsf{\beta }}_1{\ast \mathsf{\sigma }}_{\mathrm{X}}/\mathsf{\sigma }$, which we declare as the effect size.

In short, when β₁ = A ≠ 0, the key steps are:

• T = $({\widehat{\mathsf{\beta }}}_1-{ \mathsf{\beta }}_1){\ast \widehat{\mathsf{\sigma }}}_{\mathrm{X}}/\widehat{\mathsf{\sigma }}$ + ${\mathsf{\beta }}_1{\ast \widehat{\mathsf{\sigma }}}_{\mathrm{X}}/\widehat{\mathsf{\sigma }}$

• with ${\{({\widehat{\mathsf{\beta }}}_1-{ \mathsf{\beta }}_1){\ast \widehat{\mathsf{\sigma }}}_{\mathrm{X}}/\widehat{\mathsf{\sigma }}\}}^2$~$\left\{\left(\mathrm{n}-2\right)/\left(\mathrm{n}-1\right)\right\}\ast ({\mathrm{W}}_1{\ast \mathrm{W}}_4)/({\mathrm{W}}_2{\ast \mathrm{W}}_3),$

• $({\mathsf{\beta }}_1{\ast \widehat{\mathsf{\sigma }}}_{\mathrm{X}}/\widehat{\mathsf{\sigma }}$)²~${\left({\mathrm{A}\ast \mathsf{\sigma }}_{\mathrm{X}}/\mathsf{\sigma }\right)}^2\ast \left(\mathrm{n}-2\right)/\left(\mathrm{n}-1\right){\ast \mathrm{W}}_5/{\mathrm{W}}_6$,

• $({\widehat{\mathsf{\beta }}}_1-{ \mathsf{\beta }}_1){\ast \widehat{\mathsf{\sigma }}}_{\mathrm{X}}/\widehat{\mathsf{\sigma }}$ and ${\mathsf{\beta }}_1{\ast \widehat{\mathsf{\sigma }}}_{\mathrm{X}}/\widehat{\mathsf{\sigma }}$ are independent,

and the distribution of T depends only on n and effect size δ = $\left|\mathrm{A}\right|{\ast \mathsf{\sigma }}_{\mathrm{X}}/\mathsf{\sigma }$.

In spite of all these labors, the distribution of $\mathrm{T}$ is not amenable to direct and simple computation of power.

We simulate the regression model for power computations. Simulations are greatly simplified when we exploit the key nature of the alternative distribution, namely, that it depends only on n and δ. Simulations are reported in the Supplementary Materials. Sample sizes are tabulated in Table 1, Table 2 and Table 3.

Table 2.

Sample Size for Given Effect Size, Power, Level of Significance 5%, Mean of Power in the Validation Step, and its Standard Deviation.

α	ES = β1 ∗ (σx/σ)	Power	n	Mean	Sd
0.05	0.1	80%	790	0.8006	0.0129
		90%	1080	0.9054	0.0088
		95%	1350	0.9557	0.0067
		99%	1850	0.9898	0.0032
0.05	0.2	80%	199	0.797	0.0133
		90%	272	0.9039	0.0094
		95%	330	0.9497	0.0069
		99%	450	0.9891	0.0033
0.05	0.3	80%	91	0.7978	0.0124
		90%	123	0.9028	0.0094
		95%	150	0.9505	0.0067
		99%	220	0.992	0.0028
0.05	0.4	80%	53	0.773	0.0128
		90%	70	0.8966	0.0096
		95%	87	0.9494	0.0071
		99%	121	0.9891	0.0034
0.05	0.5	80%	36	0.8051	0.0124
		90%	48	0.9095	0.0091
		95%	58	0.95	0.0068
		99%	79	0.9888	0.0033
0.05	0.6	80%	26	0.8005	0.0124
		90%	34	0.8985	0.0094
		95%	43	0.9547	0.0066
		99%	59	0.9901	0.0031

Table 3.

Sample Size for Given Effect Size, Power, Level of Significance 1%, Mean of Power in the Validation Step, and its Standard Deviation.

α	ES = β1 ∗ (σx/σ)	Power	n	Mean	Sd
0.01	0.1	80%	1180	0.8026	0.0124
		90%	1500	0.9015	0.0095
		95%	1760	0.946	0.0072
		99%	2440	0.9906	0.0031
0.01	0.2	80%	301	0.8045	0.0121
		90%	388	0.9046	0.0093
		95%	458	0.9529	0.0065
		99%	620	0.991	0.0031
0.01	0.3	80%	136	0.8044	0.0129
		90%	172	0.9012	0.0089
		95%	199	0.9432	0.0071
		99%	265	0.9872	0.0034
0.01	0.4	80%	78	0.8017	0.0124
		90%	95	0.8856	0.0099
		95%	118	0.949	0.007
		99%	158	0.9892	0.0033
0.01	0.5	80%	51	0.8042	0.0126
		90%	64	0.9011	0.0099
		95%	77	0.9464	0.007
		99%	104	0.9891	0.0032
0.01	0.6	80%	37	0.7975	0.0125
		90%	48	0.906	0.0089
		95%	56	0.9485	0.007
		99%	73	0.9874	0.0035

Table 1.

Sample Size for Given Effect Size, Power, Level of Significance 10%, Mean of Power in the Validation Step, and its Standard Deviation.

α	ES = β1 ∗ (σx/σ)	Power	n	Mean	Sd
0.1	0.1	80%	620	0.7993	0.013
		90%	870	0.9027	0.0095
		95%	1120	0.9546	0.0067
		99%	1690	0.993	0.0027
0.1	0.2	80%	161	0.8259	0.0123
		90%	219	0.90007	0.0096
		95%	274	0.949	0.0069
		99%	440	0.0039	0.0024
0.1	0.3	80%	73	0.8017	0.0124
		90%	100	0.9006	0.00995
		95%	124	0.9475	0.0071
		99%	195	0.931	0.0026
0.1	0.4	80%	43	0.8031	0.0126
		90%	60	0.9073	0.0093
		95%	72	0.9518	0.0073
		99%	105	0.9896	0.0031
0.1	0.5	80%	29	0.8045	0.0134
		90%	39	0.9003	0.0099
		95%	48	0.947	0.0068
		99%	69	0.989	0.0034
0.1	0.6	80%	21	0.8	0.0129
		90%	28	0.8961	0.0096
		95%	35	0.9476	0.0069
		99%	52	0.9911	0.003

Comments on Table 1, Table 2 and Table 3:

1. The first column in each table entertains three types of effect sizes: small (0.1, 0.2); medium (0.3, 0.4); and large (0.5, 0.6) [15].

2. The second column in each table lays out the powers entertained.

3. The third column in each table spells out the requisite sample size.

4. The fourth column is the fruit of our effort to validate the sample size. At the ascertained sample size, data are generated under the specifications, power calculated, and power averaged over thousand times.

5. The fifth column records the standard deviation of the thousand powers calculated.

6. We are satisfied that the sample sizes laid out are holding true.

4. Discussion

A simple linear regression is a five-parameter model spelling out causality between two quantitative variables Y and X typified by:

$$\begin{gathered} \mathrm{Y}|\mathrm{X}~\mathrm{N}\left({\mathsf{\beta }}_0+{\mathsf{\beta }}_1\mathrm{X},{ \mathsf{\sigma }}^2\right) \\ \mathrm{X}~\mathrm{N}\left({\mathsf{\mu }}_{\mathrm{X}},{ \mathsf{\sigma }}_{\mathrm{X}}^2 \right) \end{gathered}$$

for some parameters ${\mathsf{\beta }}_0,{ \mathsf{\beta }}_1,{ \mathsf{\mu }}_{\mathrm{X}}$, ${\mathsf{\sigma }}^2>0$, and ${\mathsf{\sigma }}_{\mathrm{X}}^2>0$. The goal is to sample (X, Y) for testing ${\mathrm{H}}_0:{\mathsf{\beta }}_1=0$ versus the alternative ${\mathrm{H}}_1:{\mathsf{\beta }}_1\ne 0$. For determining sample size, we need the level of significance α, power 1 − β, and the effect size $\mathsf{\delta }={\mathrm{A}\ast \mathsf{\sigma }}_{\mathrm{X}}/\mathsf{\sigma }$, where A is the given alternative value of ${\mathsf{\beta }}_1$. The test statistic T used here is the one based on the least squares’ estimator ${\widehat{\mathsf{\beta }}}_1$ of ${ \mathsf{\beta }}_1$.

The regression model, as originally formulated, is a conditional model, i.e., $\mathrm{Y}|\mathrm{X}~\mathrm{N}\left({\mathsf{\beta }}_0+{\mathsf{\beta }}_1\mathrm{X},{ \mathsf{\sigma }}^2\right)$. In practice, in a planned experiment, the experimenter selects $x_1,x_2,. . .,x_n$ of X, and observes one or more Ys from the conditional distribution of Y|x_i for each i. Thus, the sample size n has already been chosen. The statistic $\left({\widehat{\mathsf{\beta }}}_1\sqrt{{\mathrm{S}}_{\mathrm{XX}}}\right)/\left(\sqrt{\mathrm{RSS}/\left(\mathrm{n}-2\right)}\right)$ is used for testing ${\mathrm{H}}_0:{\mathsf{\beta }}_1=0$ against the alternative ${\mathrm{H}}_1:{\mathsf{\beta }}_1\ne 0$. The conditional distribution of the test statistic given the data on X is Student’s t with n-2 degree of freedom under ${\mathrm{H}}_0$, and the distribution is non-central Student’s t with n-2 degrees of freedom and non-centrality parameter $\left|\mathrm{A}\right|\ast \sqrt{\mathrm{Sxx}}/\mathsf{\sigma }$ under ${\mathrm{H}}_1: {\mathsf{\beta }}_1=A$. The alternative distribution can be used to calculate the power of the test at ${\mathsf{\beta }}_1=A$, and nothing more. The entities n and ${\mathrm{S}}_{\mathrm{XX}}$ are already in place, and $\mathsf{\sigma }$ has to be spelled out. The value of A is provided by the experimenter as the one of clinical significance. From the consulting experience of one of the authors, the experimenter usually comes up with value for $\mathsf{\sigma }$ from his/her pilot study.

In some statistical circles [6,7], the non = null distribution is used to calculate the sample size with the desired power, with ${\mathrm{S}}_{\mathrm{XX}}$ remaining the same. This is controversial and discussed in [3,5,13].

We are dealing with unplanned experiments in which both X and Y are sampled together. Unplanned experiments are very common in clinical studies [4]. The effect size, in this context, is the multiple of the alternative value of ${\mathsf{\beta }}_1$ by the ratio of the two standard deviations of the model.

The current practice demands α, 1 − β, A, σ and ${\mathrm{S}}_{\mathrm{XX}}$, which we do not have. Specification of ${\mathrm{S}}_{\mathrm{XX}}$ is avoided by determining the unconditional distribution of

$$T={\widehat{\mathsf{\beta }}}_{\mathbf{1}}\ast {\widehat{\mathsf{\sigma }}}_X/\widehat{\mathsf{\sigma }}.$$

Exploiting the unconditional distribution of T, we calculated the critical values and required sample size. The unconditional distribution under the alternative depends on the effect size δ = ${\mathsf{\beta }}_1{\ast \mathsf{\sigma }}_{\mathrm{X}}/\mathsf{\sigma }$, as well as n and α. As a contrast, popular software such as PASS [6] and nQuery [7] use the conditional distribution of the test statistic T^* given the data on X, for calculating sample size.

An additional feature of our paper is that we provide a comprehensive table of critical values and sample sizes, unlike commercial software.

The main result that the non-null distribution of the test statistic T depends only on the effect size δ has an echo in other inference problems. For example, when testing ${\mathsf{\mu }}_1={\mathsf{\mu }}_2$ under the normality and common variance ${\mathsf{\sigma }}^2$ assumptions, the non-null distribution of the two-sample t-statistic depends only on the effect size $\mathsf{\lambda }=({\mathsf{\mu }}_1-{\mathsf{\mu }}_{2)}/\mathsf{\sigma }$. This result, in spirit, is like ours. We have archived our findings for comments and insights [23].

We trust that the tables provided will help researchers to calculate sample size in the context of simple linear regression in unplanned experiments avoiding the controversies that have been problematic till now. We will continue to study how sample sizes are contrasted between one test based on the slope parameter of the model and one based on the correlation coefficient.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/e25040611/s1.

Author Contributions

The original sample size problem came from the consulting work of M.K.A. The project was designed by M.K.A. The bulk of derivations and computations were done by T.G. This was part of her thesis work. M.B.R. was the mentor. Conceptualization, M.K.A.; methodology, M.B.R.; software, T.G.; simulations, T.G.; writing—original draft preparation, T.G.; writing—review and editing, M.B.R.; supervision, M.B.R. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Funding Statement

This research received no external funding.