Testing disease progression under the proportional reduction in decline in Alzheimer's disease studies

Abstract

To assess the treatment-placebo difference in Alzheimer's disease (AD) trials, saved time measure provides an easy interpretation of the treatment effect in months as an alternative measure to the treatment-placebo difference at the pre-specified visit that is often estimated from the fitted model. The current method to estimate saved time utilizes the disease progression curve of the placebo group, and this method is primarily descriptive. To fill the gap of the statistical inference for saved time, we propose to develop the likelihood ratio test and the score test under the proportional reduction in decline model. One AD trial data set was utilized to compare the proposed tests and the existing Wald-type test with regard to type I error rate and statistical power. We found that the likelihood ratio test and the score test have similar statistical power, but the score test has better control with regard to type I error rate. The two new tests are more powerful than the Wald test when a new treatment has proportional reduction in decline or constant delay, while the Wald test can have a higher statistical power when a constant treatment effect is expected from a new treatment.

1. Introduction

Alzheimer's disease (AD) is characterized by the accumulation of amyloid and tau protein, which ultimately leads to widespread neurodegeneration [1–3,7,18]. Individuals who progress from healthy normal or mild cognitive impairment (MCI) to AD dementia, experience a substantial decline in cognitive and functional abilities over time, which severely impacts their quality of life and independence [7,8,14,29]. With the successful drug development of disease-modifying therapies (DMTs), such as lecanemab and donanemab, patients have more and effective options to treat AD [23,24].

To assess the treatment effect, the traditional linear mixed-effect models are frequently used in data analysis. In such models, the estimated treatment-placebo difference at the pre-specified visit is the parameter of interest. Wang et al. [25] proposed two proportional repeated-measure models by relaxing the linear assumption on disease progression in the traditional linear mixed-effect models as the disease progression may not be linear over the course of the disease progression. Their proposed models were shown to increase statistical power as compared to the traditional models. Later, Raket [10] proposed the progression models for repeated measures (PMRM). That model incorporates a continuous-time extension, enabling the use of actual visit times rather than visit as a categorical variable. It was noted that the PMRM exhibited inflated type I error rates, which could be attributed to the estimation procedure, distributional deviations, and/or the use of resampled historical data, resulting in deviations from assumptions such as residual normality.

The treatment-placebo difference at the pre-specified visit is statistically sound, but it has some limitations, including the difficulty in interpreting the treatment effect size to patients, family members, and care-givers. For that reason, the concept of saved time was developed to provide an estimate in months that a patient would retain his/her independence if that patient starts the treatment soon. Saved time measure provides a clear and consistent metric, measuring the delay in the decline of a patient's cognitive and/or functional abilities if a patient is treated by a new drug. Extending the time during which patients can remain independent and actively engaged in daily activities offers practical benefits in assessing treatment impact [5]. Additionally, saved time allows for more robust comparisons across studies, even when different endpoints are used, thereby enhancing the interpretability and comparability of outcomes across clinical trials [6].

Saved time can be estimated by using (i) the outcome change at the pre-specified time in the treatment group, and (ii) the disease progression curve of the placebo group. Saved time is the length of time between the last visit and the intersection between the curve (ii) and the horizontal line starting from the value at (i): see Figure 1. As the placebo disease progression (PDP) curve is used in estimating saved time, we refer to this method as the PDP method. The PDP method provides a straightforward and interpretable metric that reflects the absolute time delay achieved by a new drug, often expressed in months or weeks. Dickson et al. [5] utilized the PDP method to estimate saved time using data from the donanemab phase 2 trial that investigated efficacy of donanemab in patients with early symptomatic AD with the follow-up time of 76 weeks. The estimated saved time was more than 5 months in the donanemab group by using either the Integrated Alzheimer's Disease Rating Scale (iADRS) or the Clinical Dementia Rating Sum of Boxes (CDR-SB). In addition to the PDP method based on the placebo disease progression curve, Wang et al. [26] recently proposed to use the treatment disease progression curve and the treatment effect size at the last visit to estimate saved time. The treatment disease progression curve allows direct estimation of time delay by comparing progression levels within the drug treatment group.

Figure 1.

Illustration of saved time calculation by mimicking data from the donanemab phase 3 trial.

The current methods to estimate saved time are primarily descriptive and do not include statistical inference such as hypothesis testing. Inferential methods such as the likelihood-ratio (LR) test and the score test would allow researchers to rigorously test hypotheses for the differences in disease progression between the treatment group and the placebo group [11,13,16]. In this article, we propose to develop the LR test and the score test under the proportional reduction in decline model. The LR test and the score test are often associated with good statistical performance with regard to type I error rate control and statistical power [4,12,17,28]. We derived the closed formula for these new tests in this article. These new tests are ready to be used in real AD trials for data analysis.

2. Methods

Suppose a study has K + 1 visits: $V_0,V_1,\dots ,V_K$, where $V_0=0$ represents the baseline visit. For example, a typical DMT AD trial has 18 months with visits scheduled every 3 months: $(V_0,V_1,V_2,V_3,V_4,V_5,V_6)=(0,3,6,9,12,15,18)$ months. In a randomized placebo-control study, we denote the disease progression at time t in the group g as ${\mu }_g(t)$, where g = p, and d is for the placebo group and the drug group, respectively. In this article, we use the disease progression ${\mu }_g(t)$ as the outcome change at time t from baseline, with ${\mu }_{g,k}={\mu }_g(V_k)$ for the outcome change at the follow-up visit k from baseline in the group g, where $k=1,\dots ,K$.

We are interested in comparing the disease progression difference between the two groups under the proportional reduction in decline model. In this model, there is a constant $\lambda >0$ such that

$$\frac{{\mu }_d(t)}{{\mu }_p(t)}=\lambda ,\mathrm{for}t>0.$$ (1)

When a new drug is expected to reduce the disease progression as compared to the placebo, the value λ is likely to be less than ${\lambda }_0=$1. We are going to test the statistical hypothesis:

$$H_0:\lambda \ge {\lambda }_0\text{against}{H}_a:\lambda <{\lambda }_0.$$ (2)

We propose to develop two new test statistics based on the likelihood ratio method and the score method. By using these test statistics, one can calculate the one-sided p-value based on the fact whether the estimated λ is less than 1.

2.1. Relationship between λ and saved time

We assume that the disease progresses linearly between consecutive visits. Thus, the PDP curve is presented as:

$${\mu }_p(t)={\mu }_{p,k-1}+{\beta }_{p,k}(t-V_{k-1})={\mu }_{p,k-1}+\frac{{\mu }_{p,k}-{\mu }_{p,k-1}}{V_k-V_{k-1}}(t-V_{k-1}),t\in (V_{k-1},V_k),$$

where $k=1,\dots ,K$, with the initial condition ${\mu }_0=0$. The disease progression curve for the drug group can be defined similarly.

Saved time denoted by Δ, can be calculated as $\mathrm{\Delta }=V_K-T_e$ (see Figure 1), where $T_e$ is the time that satisfies:

$${\mu }_p(T_e)={\mu }_d(V_K),$$

where ${\mu }_d(V_K)$ is the disease progression at the last follow-up visit in the drug group. The intersection of the PDP curve and the horizontal line at the value of ${\mu }_d(V_K)$ determines $T_e$ and the visit interval that contains the value $T_e$. Suppose that the visit interval is the jth visit interval $(V_{j-1},V_j)$. Then, we have

$$\mathrm{\Delta }=V_K-V_j+\frac{{\mu }_p(V_j)-\lambda {\mu }_p(V_K)}{{\mu }_p(V_j)-{\mu }_p(V_{j-1})}(V_j-V_{j-1})=\mathrm{a\lambda }+b,$$

where $a=-(V_j-V_{j-1}){\mu }_p(V_K)/[{\mu }_p(V_j)-{\mu }_p(V_{j-1})]$ and $b=V_K-V_j+(V_j-V_{j-1}){\mu }_p(V_j)/[{\mu }_p(V_j)-{\mu }_p(V_{j-1})]$. It can be seen that the relationship between Δ and λ is linear. For that reason, we can make statistical inference for Δ by testing the hypothesis in Equation (2) for λ. We propose to develop the following LR test and the score test for λ and Δ.

2.2. Likelihood ratio test

Let $\mathbf{V}=(V_1,\dots ,V_K{)}^T$ denote the vector of all follow-up visits. Suppose ${\mathbf{Y}}_{g,i}=(y_{g,1,i},\dots ,y_{g,K,i}{)}^T$ is the vector of all observed data for the ith patient in the group g that follows a multivariate normal distribution with mean ${\mu }_g$ and variance matrix ${\mathrm{\Sigma }}_g$, where $i=1,2,\dots ,n_g$ with $n_g$ as the sample size in the group g. Then, the complete data set can be denoted as a matrix $\mathbf{Y}=({\mathbf{Y}}_{p,1},\dots ,{\mathbf{Y}}_{p,n_p},{\mathbf{Y}}_{d,1},\dots ,{\mathbf{Y}}_{d,n_d})$. The function ${\mu }_p$ applies component-wise to $\mathbf{V}$ as: ${\mu }_p(\mathbf{V})=({\mu }_p(V_1),\dots ,{\mu }_p(V_K){)}^T.$

The log-likelihood function can be expressed as:

$$\begin{array}{rl}\tilde{l}(\lambda ,{\mu }_p|{\mathrm{\Sigma }}_p,{\mathrm{\Sigma }}_d,\mathbf{Y})& \propto -\frac{n_p}{2}[{\overline{\mathbf{Y}}}_p-{\mu }_p(\mathbf{V}){]}^T{\mathrm{\Sigma }}_p^{-1}[{\overline{\mathbf{Y}}}_p-{\mu }_p(\mathbf{V})]\\ & -\frac{n_d}{2}[{\overline{\mathbf{Y}}}_d-\lambda {\mu }_p(\mathbf{V}){]}^T{\mathrm{\Sigma }}_d^{-1}[{\overline{\mathbf{Y}}}_d-\lambda {\mu }_p(\mathbf{V})].\end{array}$$

More details to derive the log-likelihood function are presented in Appendix 1.

The parameter λ is the primary parameter of interest while ${\mu }_p(\mathbf{V})$ is regarded as a nuisance parameter in the likelihood function. For a given value of λ, the value of ${\mu }_p(\mathbf{V})$ that maximizes the likelihood function can be readily obtained by solving ${\mathrm{\nabla }}_{({\mu }_p(\mathbf{V}{)}^T)}\tilde{l}(\lambda ,{\mu }_p)=0$ as:

$${\hat{\mu }}_p(\mathbf{V})=[n_p{\mathrm{\Sigma }}_p^{-1}+n_d{\lambda }^2{\mathrm{\Sigma }}_d^{-1}{]}^{-1}[n_p{\mathrm{\Sigma }}_p^{-1}{\overline{\mathbf{Y}}}_p+n_d\lambda {\mathrm{\Sigma }}_d^{-1}{\overline{\mathbf{Y}}}_d].$$ (3)

Then, we use the profile likelihood for λ in the statistical inference:

$$\begin{array}{rl}\overline{l}(\lambda )& =\underset{{\mu }_p}{max}\tilde{l}(\lambda ,{\mu }_p)=\tilde{l}(\lambda ,{\mu }_p){|}_{{\mu }_p={\hat{\mu }}_p}\\ & =\tilde{l}(\lambda ,[n_p{\mathrm{\Sigma }}_p^{-1}+n_d{\lambda }^2{\mathrm{\Sigma }}_d^{-1}{]}^{-1}[n_p{\mathrm{\Sigma }}_p^{-1}{\overline{\mathbf{Y}}}_p+n_d\lambda {\mathrm{\Sigma }}_d^{-1}{\overline{\mathbf{Y}}}_d]).\end{array}$$

The MLE of $\lambda$ is obtained by numerically optimizing $\overline{l}(\lambda )$ over λ with a numerical optimization method, such as the optim() function in R. Let ${\hat{\lambda }}_P$ be the MLE of λ from the profile likelihood.

By the Wilks' theorem [27], the likelihood ratio statistic asymptotically follows a ${\chi }^2$-distribution with one degree of freedom since the number of free parameters differs by one (i.e. λ) between the null hypothesis and the alternative hypothesis here. We propose the LR test based on the profile likelihood estimate (referred to be as the LR test) :

$$T_{\mathrm{LR}}=I({\hat{\lambda }}_P>1)\times \sqrt{2[\hat{l}({\hat{\lambda }}_P)-\hat{l}({\lambda }_0=1)]},$$

where $I()$ is the indicator function. The null hypothesis is rejected for a small test statistic value when the LR test statistic $T_{\mathrm{LR}}$ is less than $-Z_{\alpha }$ for a one-sided hypothesis at the significance level of α, where $Z_{\alpha }$ is the $100(1-\alpha )$th percentile value of the standard normal distribution.

2.3. Score test

We utilize the profile log-likelihood function to estimate the score statistic $\frac{\mathrm{\partial }}{\mathrm{\partial }\lambda }\overline{l}$:

$$\begin{array}{rl}\frac{\mathrm{d}}{\mathrm{d}\lambda }\overline{l}& =\frac{\mathrm{d}}{\mathrm{d}\lambda }\tilde{l}(\lambda ,{\mu }_p){|}_{{\mu }_p={\hat{\mu }}_p}=\frac{\mathrm{d}}{\mathrm{d}\lambda }\tilde{l}(\lambda ,{\hat{\mu }}_p)\\ & =n_p{\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T{\mathrm{\Sigma }}_p^{-1}[{\overline{\mathbf{Y}}}_p-{\hat{\mu }}_p(\mathbf{V})]\\ & +n_d[{\hat{\mu }}_p(\mathbf{V}{)}^T+\lambda {\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T]{\mathrm{\Sigma }}_d^{-1}[{\overline{\mathbf{Y}}}_d-\lambda {\hat{\mu }}_p(\mathbf{V})],\end{array}$$

where $\frac{\mathrm{d}}{\mathrm{d}\lambda }{\hat{\mu }}_p(\mathbf{V})=n_d[n_p{\mathrm{\Sigma }}_p^{-1}+n_d{\lambda }^2{\mathrm{\Sigma }}_d^{-1}{]}^{-1}{\mathrm{\Sigma }}_d^{-1}[{\overline{\mathbf{Y}}}_d-2\lambda {\hat{\mu }}_p(\mathbf{V})]$.

Under the null hypothesis with $\lambda ={\lambda }_0$, the score statistic is calculated as

$$\begin{array}{rl}S({\lambda }_0)& =\frac{1}{n_d}\frac{\mathrm{d}}{\mathrm{d}\lambda }\overline{l}{|}_{\lambda ={\lambda }_0}\\ & =\frac{n_p}{n_d}{\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T{\mathrm{\Sigma }}_p^{-1}[{\overline{\mathbf{Y}}}_p-{\hat{\mu }}_p(\mathbf{V})]\\ & +[{\hat{\mu }}_p(\mathbf{V}{)}^T+{\lambda }_0{\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T]{\mathrm{\Sigma }}_d^{-1}[{\overline{\mathbf{Y}}}_d-{\lambda }_0{\hat{\mu }}_p(\mathbf{V})],\end{array}$$

with its variance as:

$$\begin{array}{rl}\mathrm{Var}(S)& =\frac{n_p}{n_d^2}{\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T{\mathrm{\Sigma }}_p^{-1}\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\\ & +\frac{1}{n_d}[{\hat{\mu }}_p(\mathbf{V}{)}^T+{\lambda }_0{\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T]{\mathrm{\Sigma }}_d^{-1}[{\hat{\mu }}_p(\mathbf{V})+{\lambda }_0\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)],\end{array}$$

where the $\frac{{\mathrm{d}}^2{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}{\lambda }^2}=-2n_d[n_p{\mathrm{\Sigma }}_p^{-1}+n_d{\lambda }^2{\mathrm{\Sigma }}_d^{-1}{]}^{-1}{\mathrm{\Sigma }}_d^{-1}({\hat{\mu }}_p(\mathbf{V})+\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }+\lambda \frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda })$, and $\frac{1}{n_d}$ is added to the score statistic to standardize the test statistic which does not affect the statistical inference. See Appendix 2 for more details on deriving the score statistic and its variance.

Under the null hypothesis, the score statistic $S({\lambda }_0)$ follows a normal distribution with mean zero and variance of $\mathrm{Var}(S)$. Then, the score test statistic is:

$$T_{\mathrm{score}}=\frac{S({\lambda }_0)}{\sqrt{\widehat{\mathrm{Var}}(S({\lambda }_0))}},$$

that follows the standard normal distribution asymptotically. The null hypothesis is rejected when $T_{\mathrm{score}}$ is substantially negative: $T_{\mathrm{score}}<-Z_{\alpha }$. To calculate S and $\mathrm{Var}(S)$, we need the values of ${\hat{\mu }}_p$ which is calculated from Equation (3) using ${\lambda }_0$.

3. Results

We first utilized data from the donanemab phase 2 trial to illustrate the application of saved time and the hypothesis testing with no difference between the donanemab treatment group and the placebo group. Then, we conducted extensive simulation studies to compare the two proposed tests and the existing Wald test with regard to type I error and statistical power [6].

The donanemab phase 2 clinical trial was a randomized, placebo-controlled study to investigate the effectiveness of donanemab for patients with early AD [9]. The trial enrolled 257 early symptomatic AD patients, defined by the presence of both tau and amyloid deposits confirmed via positron-emission tomography (PET). Of these patients, 131 were randomly assigned to the donanemab treatment group, and 126 to the placebo group. The primary outcome was the change in the Integrated Alzheimer's Disease Rating Scale (iADRS) at week 76 from baseline. A lower iADRS score indicates greater cognitive and functional impairment.

The iADRS change at each follow-up visit from baseline can be obtained from the published article by Mintun et al. [9]. Its variance at each follow-up visit can be estimated from that article as well. We assumed the correlation of 0.5 to have the variance-covariance matrix. With these summary statistics, we then used the LR method to estimate the proportional reduction in decline estimate λ. We presented the findings in Table 1. The estimated $\hat{\lambda }$ was 0.608 based on the profile likelihood method, indicating that the reduction in decline was more than 39%. The estimated time saving by using the LR method was 5.74 months. For the score method, the estimated score statistic was $\hat{S}({\lambda }_0)=-0.184$ with its variance estimate $\widehat{\mathrm{Var}}(S({\lambda }_0$))= 0.00316. All three methods reject the null hypothesis with very small p-values. The p-values of the LR test and the score test are much smaller than that of the Wald test.

Table 1.

Saved time estimate from the three methods by using the summary statistics from the donanemab phase 2 trial: $\hat{\lambda }=0.608$ with the estimated saved time of 5.74 months.

Methods	Test statistic	P-value
LR	−3.43	0.0003
Score	−3.27	0.0005
Wald	−2.58	0.0049

Note: The p-value was calculated by using asymptotic standard normal distribution.

3.1. Simulation studies

We then conducted simulation studies to compare the proposed LR and score tests and the existing Wald test with regard to type I error (TIE) and statistical power, with sample size from 50 to 400 per group for a study with 6 visits every 3 months. For the placebo disease progression, we studied the following two common patterns: (i) linear disease progression with 2 points worse every 3 months over 18 months; and (ii) reduced decline over time with a total of 12 points decline over 18 months. For the variance of outcome at each visit was assumed to be (i) constant: SD=10, and (ii) increasing SD from $\sqrt{50}$ to $\sqrt{100}$ over 6 visits. The correlation coefficient ρ is assumed to be 0.5 in the simulation studies. The nominal TIE rate is set as $\alpha =0.05$.

Figure 2 shows the TIE rates of the three tests as a function of sample size when the outcome gets worsen by 2 points every 3 months with the standard deviation of 10 at each visit. The TIE is calculated based on the asymptotic normal distribution. In all these cases, the LR and score tests have the TIE being closer to the nominal level than the Wald test. We also calculated the mean squared distance (MSD) to the nominal level as:

$$\sqrt{\frac{\sum _{i=1}^8(\mathrm{TI}{E}_i-0.05{)}^2}{8}},$$

where i from 1 to 8 represents the cases with sample sizes from 50 to 400. The score test has the lowest MSD of 0.0037, followed by the LR test of 0.0056, and the Wald test of 0.0366. The score test has the best control with regard to TIE. Similar results are observed for nonlinear placebo disease progression and other variance-covariance structures. The value of ρ is assumed to be 0.5 in the TIE calculation. This value can be estimated from pilot data. It is essential to investigate the effect of ρ on the TIE value. In Figure 3, we present the TIE rate of the three tests as ρ from 0.2 to 0.8 when the outcome get worse by 2 points every 3 months with the standard deviation of 10 at each visit. It can be seen that the LR test and the score test are robust to the value of ρ while the Wald test's TIE highly depends on the choice of ρ. For the Wald test, a small correlation coefficient is associated with a higher inflated TIE rate. When ρ is very high (e.g. 0.8) and the sample size is large, the Wald test has the TIE being closer to the nominal level.

Figure 2.

Type I error of the three tests as a function of sample size when the outcome get worse by 2 points every 3 months with the standard deviation of 10 at each visit given $\alpha =0.05$.

Figure 3.

Effect of correlation on Type I error of the three tests as a function of sample size when the outcome get worse by 2 points every 3 months with the standard deviation of 10 at each visit given $\alpha =0.05$.

To have a fair statistical power comparison among these tests, we conducted simulations for each configuration (sample size, placebo disease progression pattern, variance-covariance structure) to obtain the 95% quantile from the simulated tests statistic values to control the TIE of each test at 5%. The determined quantile is then used to calculate statistical power.

We presented the statistical power for the three methods in Figure 4 when a new drug treatment is expected to have a proportional reduction in decline as compared to the placebo, having $\lambda ={\mu }_d(t)/{\mu }_p(t)$ from 0.9 to 0.8. For a given λ value, the statistical power is an increasing function of sample size. In the left plot with $\lambda =0.9$, the statistical power of the score method is increased from 18% to 67% when the sample size is from 50 to 400 per group, with the average statistical power of 45%. A similar trend and average statistical power were observed for the LR test. Both the LR test and the score test increase statistical power by more than 11% as compared to the Wald test (33% on average). Similar results were observed from the middle plot ( $\lambda =0.86$) and the right plot ( $\lambda =0.8$).

Figure 4.

Statistical power comparison under the proportional reduction in decline model with $\lambda ={\mu }_d(t)/{\mu }_p(t)$=0.90, 0.86, and 0.80, among the three tests as a function of sample size when the outcome get worse by 2 points every 3 months with the standard deviation of 10 at each visit given $\alpha =0.05$.

In addition to the proportional reduction in decline, we also considered another two common scenarios: (i) constant delay, and (ii) constant treatment effect. When a new treatment is expected to delay the disease progression over time, the disease progression curve of the drug treatment group is shifted horizontally from the control disease progression curve ${\mu }_p(t)={\mu }_d(t\times t_c)$, where $t_c$ is a pre-specified delay factor. In the case of constant treatment effect, there is a constant difference between the control and treatment disease progression over time, having ${\mu }_p(t)-{\mu }_d(t)=z$, where z is a pre-specified constant.

Figure 5 displays the statistical power results for the constant delay scenario with the delay factor values $t_c$=1.12, 1.16, and 1.20. Here, the value of $(t_c-1)\times t$ represents the saved time in months in the treatment group as compared to the control group at time t. The findings from the configurations under the constant delay model were largely similar to those in the proportional decline scenario in Figure 4. In AD symptom trials, the constant treatment effect is commonly observed. We presented the statistical power for the constant treatment effect scenario with three constant z values: 0.10, 0.14, and 0.22 in Figure 6 when the outcome get worse by 2 points every 3 months with the standard deviation of 10 at each visit given $\alpha =0.05$. The Wald test can be more powerful than the LR and the score tests. When $t_c=0.10$, the Wald test can increase statistical power by 9% on average as compared to the score test whose average statistical power is 24% for these cases. As the value of $t_c$ goes up, the treatment effect is large which leads to improved performance in the group at early visits (e.g. 3 months). This could occur in early AD trials but the improvement at early visits is not small [23]. Similar statistical power results are observed for nonlinear placebo disease progression and other variance-covariance structures.

Figure 5.

Statistical power comparison under the constant delay model with the pre-specified delay factor $t_c$=1.12, 1.16, and 1.20, among the three tests as a function of sample size when the outcome get worse by 2 points every 3 months with the standard deviation of 10 at each visit given $\alpha =0.05$.

Figure 6.

Statistical power comparison under the constant treatment effect model with $z={\mu }_p(t)-{\mu }_d(t)=$ 0.10, 0.14, and 0.22, among the three tests as a function of sample size when the outcome get worse by 2 points every 3 months with the standard deviation of 10 at each visit given $\alpha =0.05$.

4. Discussions

The LR test and the score test were developed under the proportional reduction in decline with one parameter λ to capture the relationship between the disease progression curve of the drug treatment group and that of the placebo group. The LR test and the score test had similar statistical power, but the score test was shown to be associated with better TIE control. They are more powerful than the Wald test when a new treatment has proportional reduction in decline or constant delay [20–22]. The Wald test can have a higher statistical power when a constant treatment effect is expected from a new treatment [15,19]. The TIE rate of the Wald test goes down as the correlation between outcomes from different visits (ρ) increases. Under the null hypothesis, when the correlation ρ decreases, the Wald test is more likely to detect a treatment effect even though there is no difference under the null. That could partially cause a high inflated TIE rate of the Wald test when ρ is low.

In order to calculate the proposed LR and score test statistics, we need only the summary statistics, including the estimated disease progression curves and the estimated variance-covariance matrix of the two groups. The summary statistics are often available in the published articles, while the original data are always difficult to obtain to confirm the analysis results when completed data are needed in mixed-effect models. The existing PMRM model to assess proportional reduction in decline requires the complete data to run the mixed-effect model to estimate the proportional reduction parameter. When the complete data is not publicly available, it could be difficult to use this method. In addition, the PMRM model is computationally intensive, while the proposed three tests have closed formulas which requires less than 1% of the computing time using the PMRM model. For one simulated data set, it takes about 20 seconds for the PMRM model, with a few days to complete 10,000 simulations for one simulation case. The function gnls in the PMRM model takes a long time to fit the model. More efficient computing algorithms are needed to fit the PMRM model in simulation studies.

Appendix 1. Log-likelihood function

The log-likelihood function, given the observed data set $\mathbf{Y}$, is

$$\begin{array}{rl}\log L(\lambda ,{\mu }_p;{\mathrm{\Sigma }}_p,{\mathrm{\Sigma }}_d|\mathbf{Y})=& -\frac{1}{2}\sum _{i=1}^{n_p}\{[{\mathbf{Y}}_{p,i}-{\mu }_p(\mathbf{V}){]}^T{\mathrm{\Sigma }}_p^{-1}[{\mathbf{Y}}_{p,i}-{\mu }_p(\mathbf{V})]\}\\ & -\frac{1}{2}\sum _{i=1}^{n_d}\{[{\mathbf{Y}}_{d,i}-\lambda {\mu }_p(\mathbf{V}){]}^T{\mathrm{\Sigma }}_d^{-1}[{\mathbf{Y}}_{d,i}-\lambda {\mu }_p(\mathbf{V})]\}\\ & -\frac{K{n}_p}{2}\log (2\pi )-\frac{K{n}_d}{2}\log (2\pi )-\frac{n_p}{2}\log \left(|{\mathrm{\Sigma }}_p|\right)-\frac{n_d}{2}\log \left(|{\mathrm{\Sigma }}_d|\right)\end{array}$$

where ${\mathrm{\Sigma }}_p$ and ${\mathrm{\Sigma }}_d$ denote covariance matrices for the placebo and treatment groups respectively.

The likelihood is maximized over the parameters λ and ${\mu }_p(\mathbf{V})$. We isolate the parts of the log-likelihood that involve λ and ${\mu }_p(\mathbf{V})$, while the remaining terms are constant, as shown below:

$$\log L(\lambda ,{\mu }_p;{\mathrm{\Sigma }}_p,{\mathrm{\Sigma }}_d|\mathbf{Y})=\tilde{l}(\lambda ,{\mu }_p)+C,$$

where

$$\tilde{l}(\lambda ,{\mu }_p)=-\frac{n_p}{2}[{\overline{\mathbf{Y}}}_p-{\mu }_p(\mathbf{V}){]}^T{\mathrm{\Sigma }}_p^{-1}[{\overline{\mathbf{Y}}}_p-{\mu }_p(\mathbf{V})]-\frac{n_d}{2}[{\overline{\mathbf{Y}}}_d-\lambda {\mu }_p(\mathbf{V}){]}^T{\mathrm{\Sigma }}_d^{-1}[{\overline{\mathbf{Y}}}_d-\lambda {\mu }_p(\mathbf{V})].$$

Here, ${\overline{\mathbf{Y}}}_p=({\overline{y}}_{p,1},\dots ,{\overline{y}}_{p,K}{)}^T$ and ${\overline{\mathbf{Y}}}_d=({\overline{y}}_{d,1},\dots ,{\overline{y}}_{d,K}{)}^T$ represent the vectors of sample means at each visiting times $\mathbf{V}$ for the placebo and treatment groups, respectively.

Appendix 2. Score test

The score statistic is calculated as:

$$S({\lambda }_0)=\frac{1}{n_d}\frac{\mathrm{d}}{\mathrm{d}\lambda }\tilde{l}(\lambda ,{\hat{\mu }}_p),$$

where $\frac{\mathrm{d}}{\mathrm{d}\lambda }\tilde{l}(\lambda ,{\hat{\mu }}_p)=n_p(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }{)}^T{\mathrm{\Sigma }}_p^{-1}[{\overline{\mathbf{Y}}}_p-{\hat{\mu }}_p(\mathbf{V})]+n_d({\hat{\mu }}_p(\mathbf{V}{)}^T+\lambda (\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }{)}^T){\mathrm{\Sigma }}_d^{-1}[{\overline{\mathbf{Y}}}_d-\lambda {\hat{\mu }}_p(\mathbf{V})].$

A.1. Calculation of $\frac{\mathrm{d}}{\mathrm{d}\lambda }\tilde{l}(\lambda ,{\hat{\mu }}_p)$

From the likelihood function, we have

$$\begin{array}{rl}\frac{\mathrm{d}}{\mathrm{d}\lambda }\tilde{l}(\lambda ,{\hat{\mu }}_p)& =\frac{\mathrm{d}}{\mathrm{d}\lambda }(-\frac{n_p}{2}[{\overline{\mathbf{Y}}}_p-{\hat{\mu }}_p(\mathbf{V}){]}^T{\mathrm{\Sigma }}_p^{-1}[{\overline{\mathbf{Y}}}_p-{\hat{\mu }}_p(\mathbf{V})]\\ & -\frac{n_d}{2}[{\overline{\mathbf{Y}}}_d-\lambda {\hat{\mu }}_p(\mathbf{V}){]}^T{\mathrm{\Sigma }}_d^{-1}[{\overline{\mathbf{Y}}}_d-\lambda {\hat{\mu }}_p(\mathbf{V})]).\end{array}$$

We define the following notations:

$$\begin{array}{rl}U& ={\overline{\mathbf{Y}}}_p-{\hat{\mu }}_p(\mathbf{V}),\frac{\mathrm{d}U}{\mathrm{d}\lambda }=-\frac{\mathrm{d}}{\mathrm{d}\lambda }{\hat{\mu }}_p(\mathbf{V})\\ Q& ={\overline{\mathbf{Y}}}_d-\lambda {\hat{\mu }}_p(\mathbf{V}),\frac{\mathrm{d}Q}{\mathrm{d}\lambda }=-[{\hat{\mu }}_p(\mathbf{V})+\lambda \frac{\mathrm{d}{\mu }_p}{\mathrm{d}\lambda }].\end{array}$$

Then, the first part of $\frac{\mathrm{d}}{\mathrm{d}\lambda }\tilde{l}(\lambda ,{\hat{\mu }}_p)$ is

$$\begin{array}{rl}& -\frac{n_p}{2}\frac{\mathrm{d}}{\mathrm{d}\lambda }[{\overline{\mathbf{Y}}}_p-{\hat{\mu }}_p(\mathbf{V}){]}^T{\mathrm{\Sigma }}_p^{-1}[{\overline{\mathbf{Y}}}_p-{\hat{\mu }}_p(\mathbf{V})]\\ & =-\frac{n_p}{2}\frac{\mathrm{d}}{\mathrm{d}\lambda }\left[U^T{\mathrm{\Sigma }}_p^{-1}U\right]\\ & =-\frac{n_p}{2}[\frac{\mathrm{d}{U}^T}{\mathrm{d}\lambda }{\mathrm{\Sigma }}_p^{-1}U+U^T{\mathrm{\Sigma }}_p^{-1}\frac{\mathrm{d}U}{\mathrm{d}\lambda }]\\ & =\frac{n_p}{2}[{\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T{\mathrm{\Sigma }}_p^{-1}U+U^T{\mathrm{\Sigma }}_p^{-1}\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }]\\ & =\frac{n_p}{2}[{\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T{\mathrm{\Sigma }}_p^{-1}U+{\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T{\mathrm{\Sigma }}_p^{-1}U]\\ & =n_p{\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T{\mathrm{\Sigma }}_p^{-1}U.\end{array}$$

The second part of $\frac{\mathrm{d}}{\mathrm{d}\lambda }\tilde{l}(\lambda ,{\hat{\mu }}_p)$ is

$$\begin{array}{rl}& -\frac{n_d}{2}\frac{\mathrm{d}}{\mathrm{d}\lambda }[{\overline{\mathbf{Y}}}_d-\lambda {\hat{\mu }}_p(\mathbf{V}){]}^T{\mathrm{\Sigma }}_d^{-1}[{\overline{\mathbf{Y}}}_d-\lambda {\hat{\mu }}_p(\mathbf{V})]\\ & =-\frac{n_d}{2}\frac{\mathrm{d}}{\mathrm{d}\lambda }[Q^T{\mathrm{\Sigma }}_d^{-1}Q]\\ & =-\frac{n_d}{2}[{\left(\frac{\mathrm{d}Q}{\mathrm{d}\lambda }\right)}^T{\mathrm{\Sigma }}_d^{-1}Q+Q^T{\mathrm{\Sigma }}_d^{-1}\frac{\mathrm{d}Q}{\mathrm{d}\lambda }]\\ & =\frac{n_d}{2}[{({\hat{\mu }}_p(\mathbf{V})+\lambda \frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda })}^T{\mathrm{\Sigma }}_d^{-1}Q+Q^T{\mathrm{\Sigma }}_d^{-1}({\hat{\mu }}_p(\mathbf{V})+\lambda \frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda })]\\ & =n_d({\hat{\mu }}_p(\mathbf{V}{)}^T+\lambda {\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T){\mathrm{\Sigma }}_d^{-1}Q\end{array}$$

Therefore, we have

$$\begin{array}{rl}\frac{\mathrm{d}}{\mathrm{d}\lambda }\tilde{l}(\lambda ,{\hat{\mu }}_p)& =n_p{\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T{\mathrm{\Sigma }}_p^{-1}U+n_d({\hat{\mu }}_p(\mathbf{V}{)}^T+\lambda {\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T){\mathrm{\Sigma }}_d^{-1}Q\\ & =n_p{\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T{\mathrm{\Sigma }}_p^{-1}[{\overline{\mathbf{Y}}}_p-{\hat{\mu }}_p(\mathbf{V})]\\ & +n_d({\hat{\mu }}_p(\mathbf{V}{)}^T+\lambda {\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T){\mathrm{\Sigma }}_d^{-1}[{\overline{\mathbf{Y}}}_d-\lambda {\hat{\mu }}_p(\mathbf{V})].\end{array}$$

A.2. Calculation of $\frac{\mathrm{\partial }}{\mathrm{\partial }\lambda }{\hat{\mu }}_p$

The disease progression ${\hat{\mu }}_p$ can be written as:

$${\hat{\mu }}_p=A^{-1}B,$$

where $A=n_p{\mathrm{\Sigma }}_p^{-1}+n_d{\lambda }^2{\mathrm{\Sigma }}_d^{-1}$, and $B=n_p{\mathrm{\Sigma }}_p^{-1}{\overline{\mathbf{Y}}}_p+n_d\lambda {\mathrm{\Sigma }}_d^{-1}{\overline{\mathbf{Y}}}_d$. The derivative of inverse matrix is $\frac{\mathrm{d}{A}^{-1}}{\mathrm{d}\lambda }=-A^{-1}\frac{\mathrm{d}A}{\mathrm{d}\lambda }{A}^{-1}.$ Then, its first derivative is

$$\begin{array}{rl}\frac{\mathrm{d}}{\mathrm{d}\lambda }{\hat{\mu }}_p& =\frac{\mathrm{d}}{\mathrm{d}\lambda }{A}^{-1}B=-A^{-1}\frac{\mathrm{d}A}{\mathrm{d}\lambda }{A}^{-1}B+A^{-1}\frac{\mathrm{d}B}{\mathrm{d}\lambda }\\ \frac{\mathrm{d}A}{\mathrm{d}\lambda }& =2n_d\lambda {\mathrm{\Sigma }}_d^{-1}\\ \frac{\mathrm{d}B}{\mathrm{d}\lambda }& =n_d{\mathrm{\Sigma }}_d^{-1}{\overline{\mathbf{Y}}}_d\end{array}$$

It follows that,

$$\begin{array}{rl}\frac{\mathrm{d}}{\mathrm{d}\lambda }{\hat{\mu }}_p& =n_d[-2A^{-1}\lambda {\mathrm{\Sigma }}_d^{-1}{A}^{-1}B+A^{-1}{\mathrm{\Sigma }}_d^{-1}{\overline{\mathbf{Y}}}_d]\\ & =n_d{A}^{-1}{\mathrm{\Sigma }}_d^{-1}[{\overline{\mathbf{Y}}}_d-2\lambda {\hat{\mu }}_p(\mathbf{V})].\end{array}$$

A.3. Variance of score statistic

The variance of score statistic is calculated as:

$$\mathrm{Var}(S)=\mathrm{Var}(M+N),$$

where

$$\begin{array}{rl}M& =\frac{n_p}{n_d}{\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T{\mathrm{\Sigma }}_p^{-1}[{\overline{\mathbf{Y}}}_p-{\hat{\mu }}_p(\mathbf{V})]\mathrm{and}\\ N& =({\hat{\mu }}_p(\mathbf{V}{)}^T+\lambda {\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T){\mathrm{\Sigma }}_d^{-1}[{\overline{\mathbf{Y}}}_d-\lambda {\hat{\mu }}_p(\mathbf{V})].\end{array}$$

Since the placebo group observations are independent of the drug group observations, $\mathrm{COV}(M,N)=0$. Thus $\mathrm{Var}(S)=\mathrm{Var}(M)+\mathrm{Var}(N)$.

$$\begin{array}{rl}\mathrm{Var}(M)& =\frac{n_p}{n_d}{\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T{\mathrm{\Sigma }}_p^{-1}\mathrm{Var}({\overline{\mathbf{Y}}}_p){\mathrm{\Sigma }}_p^{-1}\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)\frac{n_p}{n_d}\\ & =\frac{n_p^2}{n_d^2}{\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T{\mathrm{\Sigma }}_p^{-1}(\frac{1}{n_p}{\mathrm{\Sigma }}_p){\mathrm{\Sigma }}_p^{-1}\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)\\ & =\frac{n_p}{n_d^2}{\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T{\mathrm{\Sigma }}_p^{-1}\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda },\\ \mathrm{Var}(N)& =[{\hat{\mu }}_p(\mathbf{V}{)}^T+{\lambda }_0{\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T]{\mathrm{\Sigma }}_d^{-1}\mathrm{Var}({\overline{\mathbf{Y}}}_d){\mathrm{\Sigma }}_d^{-1}{[{\hat{\mu }}_p(\mathbf{V}{)}^T+{\lambda }_0{\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T]}^T\\ & =[{\hat{\mu }}_p(\mathbf{V}{)}^T+{\lambda }_0{\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T]{\mathrm{\Sigma }}_d^{-1}(\frac{1}{n_d}{\mathrm{\Sigma }}_d){\mathrm{\Sigma }}_d^{-1}[{\hat{\mu }}_p(\mathbf{V})+{\lambda }_0\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)]\\ & =\frac{1}{n_d}[{\hat{\mu }}_p(\mathbf{V}{)}^T+{\lambda }_0{\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T]{\mathrm{\Sigma }}_d^{-1}[{\hat{\mu }}_p(\mathbf{V})+{\lambda }_0\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)].\end{array}$$

Finally, we have the variance of $S$ as

$$\begin{array}{rl}\mathrm{Var}(S)& =\frac{n_p}{n_d^2}{\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T{\mathrm{\Sigma }}_p^{-1}\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\\ & +\frac{1}{n_d}[{\hat{\mu }}_p(\mathbf{V}{)}^T+{\lambda }_0{\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)}^T]{\mathrm{\Sigma }}_d^{-1}[{\hat{\mu }}_p(\mathbf{V})+{\lambda }_0\left(\frac{\mathrm{d}{\hat{\mu }}_p(\mathbf{V})}{\mathrm{d}\lambda }\right)].\end{array}$$

Funding Statement

Shan's research is partially supported by grants from the National Institutes of Health: R01AG070849, and R03CA248006.

Disclosure statement

No potential conflict of interest was reported by the author(s).