Power Calculations and Placebo Effect for Future Clinical Trials in Progressive Supranuclear Palsy

Abstract

Background

Two recent randomized, placebo-controlled trials of putative disease-modifying agents (davunetide, tideglusib) in progressive supranuclear palsy (PSP) failed to show efficacy, but generated data relevant for future trials.

Methods

We provide sample size calculations based on data collected in 187 PSP patients assigned to placebo in these trials. A placebo effect was calculated.

Results

The total PSP-Rating Scale required the least number of patients per group (N = 51) to detect a 50% change in the 1-year progression and 39 when including patients with ≤ 5 years disease duration. The Schwab and England Activities of Daily Living required 70 patients per group and was highly correlated with the PSP-Rating Scale. A placebo effect was not detected in these scales.

Conclusions

We propose the 1-year PSP-Rating Scale score change as the single primary readout in clinical neuroprotective or disease-modifying trials. The Schwab and England Activities of Daily Living could be used as a secondary outcome.

Two recent randomized, placebo-controlled clinical trials (clinicaltrials.gov: NCT01110720, NCT01049399) of putative disease-modifying agents (davunetide and tideglusib) failed to show efficacy in progressive supranuclear palsy (PSP)^1-4 but provided relevant insights in trial design in PSP.^2-4 Sample size calculations from natural history PSP studies are difficult to compare because of methodological differences.^5-13 Moreover, there are no available data about placebo effect in PSP. Thus, we provide sample size calculations and placebo estimations based on data from different relevant scales collected in 187 PSP patients and assigned to the placebo arms in the davunetide and tideglusib trials.^2,3

Methods

Study Population and Clinical Assessments

Raw data were obtained from PSP patients of the placebo arms recruited in the davunetide and tideglusib studies with similar inclusion-exclusion criteria^2,3 (supplementary material). Both trials were planned to demonstrate similar effects on the same primary efficacy variable: 37.5% and 40% annual change in the PSP-Rating Scale (PSPRS) total score, respectively. Ethics approval was obtained at each site from the local ethics committee, and all participants gave written informed consent.

Rating scales^14-16 applied are given in the supplementary material. Raw data from the clinical assessments were obtained for the 26- and the 52-week follow-up visits. The PSPRS raw data from the first (week 4 for davunetide, week 6 for tideglusib) and the second follow-up visit (week 8 for davunetide, week 13 for tideglusib) were obtained for the placebo effect calculation.

Statistics

All statistical analyses were conducted using R version 3.1.1.¹⁷

Sample Size Calculation

Individual differences between baseline and follow-up scores after 26 and 52 weeks, respectively, were computed by subtracting the baseline score from the respective follow-up score to obtain the absolute change (ΔY). Only cases for which both baseline and follow-up measurements were available were included in the sample size calculation. Following this, the mean difference and its standard deviation were used to estimate a standardized effect size according to equation (1.1). Finally, obtained standardized effect sizes were used to determine the required sample size per group for a 2-sample t test.¹⁸ All sample size calculations were based on a 2-sided significance level of 5% and a power of 80%. An approximation of the required sample size per group for the Mann–Whitney U test based on the asymptotic relative efficiency was assessed by dividing the sample size for the 2-sample t test by 0.864.¹⁹

$$d=\frac{{\mu }_{\Delta Y}\ast p}{{\sigma }_{\Delta Y}}$$ (1.1)

where d is the calculated standardized effect size, Y is the score of scale, μ_ΔY is the mean of ΔY, p is the percentage of expected improvement considered clinically relevant (eg, 0.25), and σ_ΔY is the standard deviation of ΔY.

Correlation Analysis

Spearman rank correlation coefficient was applied to detect possible correlations between the PSPRS total score and the Schwab and England Activities of Daily Living (SEADL) score at baseline, week 26, and week 52.

Placebo Effect Estimation

There are no established definitions of the placebo effect in PSP. Based on previous definitions in Parkinson’s disease,²⁰ a considerable placebo effect was defined as an individual improvement of at least 50% when compared with the baseline score on a scale in 10% of all participants. Individual relative changes in scores (ΔS) were computed using equations (2.1) to (2.3) and were expressed as percentages. Patients were stratified by percentage of change using 50% as cut-off point. Finally, proportions of patients with and without an improvement of at least 50% when compared with the baseline score were calculated for each scale separately. Confidence intervals of these proportions were estimated using the modified Wald method.²¹

$$if{S}_f>S_{b}then\Delta S=\frac{(S_f=S_b)}{(S_{max}=S_b)}\ast 100$$ (2.1)

$$if{S}_f<S_{b}then\Delta S=\frac{(S_f=S_b)}{(S_b=S_{min})}\ast 100$$ (2.2)

$$if{S}_f=S_{b}then\Delta S=0$$ (2.3)

where S_b is the baseline score, S_f is the follow-up score, S_min is the lowest score on the scale, and S_max is the highest score on the scale.

We further calculated the placebo effect, which was defined as an individual improvement of at least 20% and 30% when compared with the baseline score on a scale in 10% of all participants.

Results

Study Population

A total of 187 PSP (156 davunetide, 31 tideglusib) patients were included in the analysis (84 women, 103 men). The average age of the participants at base-line was 67.35 (7.04) years, and disease duration was <5 years in 153 (80.8%) patients, > 5 years in 20 (10.6%) patients, and unknown in 14 patients (9.6%). Rating scale scores at different time points and group-level 1-year differences are given in Table 1. PSPRS was available at 1-year follow-up in 144 patients, and the annual difference in the total PSPRS score was 11.24 (9.95), in agreement with previous studies.^9,12

TABLE 1.

Scores in the rating scales at baseline, 26 weeks, and 52 weeks (1 year) follow-up and their 1-year difference

Rating scales	Baseline mean (SD)	After 26 weeks mean (SD)	After 52 weeks mean (SD)	One-year difference mean (SD)
Both studies
SEADL	N = 187, 156 derived from the davunetide study 0.54 (0.21)	N = 156, 133 derived from the davunetide study 0.47 (0.22)	N = 141, 120 derived from the davunetide study 0.38 (0.22)	−0.18 (0.18)
PSPRS	N = 187, 156 derived from the davunetide study	N = 156, 133 derived from the davunetide study	N = 144, 123 derived from the davunetide study
Total score	39.59 (10.97)	44.55 (12.49)	49.96 (13.98)	11.24 (9.95)
Bulbar score	2.82 (1.47)	3.17 (1.64)	3.77 (1.80)	1.00 (1.32)
Gait score	10.39 (3.80)	11.79 (4.14)	13.33 (3.96)	3.33 (3.29)
History score	8.50 (3.42)	9.71 (3.68)	10.69 (3.95)	2.44 (3.30)
Limb score	4.90 (2.18)	5.48 (2.53)	6.17 (3.01)	1.42 (2.25)
Mentation score	3.66 (2.66)	4.15 (2.83)	4.83 (3.15)	1.21 (2.88)
Ocular score	9.32 (3.09)	10.33 (3.10)	11.16 (2.91)	1.83 (2.39)
CGIDS	N = 187, 156 derived from the davunetide study 3.99 (0.90)	N = 25, 2 derived from the davunetide study 4.80 (0.91)	N = 147, 120 derived from the davunetide study 4.76 (0.94)	0.84 (0.95)
Only davunetide
VF	N = 156 10.99 (6.35)	N = 128 9.97 (6.33)	N = 1 13 9.12 (6.41)	−2.23 (4.56)
GDS	N = 156 13.14 (6.75)	N = 131 13.75 (7.36)	N = 116 14.01 (7.51)	0.82 (4.89)
Only tideglusib
FAB	N = 29 10.97 (4.49)	N = 22 11.68 (3.94)	N = 18 12.83 (3.94)	0.56 (2.50)
SAS	N = 31 19.58 (8.14)	N = 21 20.14 (11.05)	N = 16 20.56 (8.76)	1.56 (6.64)
UPDRSII	N = 31 21.87 (5.68)	N = 24 23.96 (6.82)	N = 21 28.67 (7.40)	7.43 (5.94)
LVF	N = 31 9.03 (7.00)	N = 22 11.73 (9.77)	N = 19 12.21 (7.17)	2.26 (5.67)
CVF	N = 31 19.23 (10.11)	N = 22 20.23 (10.62)	N = 19 17.84 (8.85)	23.84 (9.05)

Data are given as mean (standard deviation [SD]). N is the total number of patients from both studies (davunetide and tideglusib). SEADL, Schwab and England Activities of Daily Living Scale; CGIDS, Clinical Global Impression of Disease Severity; PSPRS, Progressive Supranuclear Palsy Rating Scale; VF, verbal fluency (F, A, or S words per minute); FAB, Frontal Assessment Battery; SAS, Starkstein Apathy Scale; UPDRSII, Unified Parkinson’s Disease Rating Scale II; LVF, two letter verbal fluency; CVF, category verbal fluency; GDS: Geriatric Depression Scale.

Sample Size Calculations

Table 2 shows sample size calculations required for a 2-arm, 1-year follow-up therapeutic trial without adjusting for an expected dropout rate, and sample sizes for a 2-arm, 26-week trial are given in Supplementary Table 1.

TABLE 2.

Sample sizes required for a 2-arm, 1-year follow-up therapeutic trial to detect 20%, 25%, 30%, 40%, and 50% change

		20% change (c = 0.20)		25% change (c = 0.25)		30% change (c = 0.30)		40% change (c = 0.40)		50% change (c = 0.50)
Rating scales	Difference mean (SD)	Effect Size	Sample Sizea	Effect Size	Sample Sizea	Effect Size	Sample Sizea	Effect Size	Sample Sizea	Effect Size	Sample Sizea
SEADL	−0.177 (0.185)	0.191	430 (498)	0.239	276 (320)	0.287	192 (223)	0.383	109 (127)	0.478	70 (82)
CGIDS	0.84 (0.95)	0.178	498 (577)	0.222	319 (370)	0.267	222 (257)	0.356	126 (146)	0.445	81 (94)
PSPRS
Total score	11.24 (9.95)	0.226	309 (358)	0.282	198 (230)	0.339	138 (160)	0.452	78 (91)	0.565	51 (60)
Bulbar score	1.00 (1.32)	0.152	682 (790)	0.190	437 (506)	0.228	304 (352)	0.304	172 (200)	0.380	110 (128)
Gait score	3.33 (3.29)	0.202	384 (445)	0.253	246 (285)	0.304	172 (200)	0.405	97 (113)	0.506	63 (73)
History score	2.44 (3.30)	0.148	718 (832)	0.185	460 (533)	0.222	320 (371)	0.296	181 (210)	0.370	116 (135)
Limb score	1.42 (2.25)	0.126	990 (1146)	0.158	634 (734)	0.189	441 (511)	0.252	249 (289)	0.315	160 (186)
Mentation score	1.21 (2.88)	0.084	2226 (2577)	0.105	1425 (1650)	0.126	990 (1146)	0.168	558 (646)	0.210	357 (414)
Ocular score	1.83 (2.39)	0.153	671 (777)	0.191	430 (498)	0.230	299 (347)	0.306	169 (196)	0.383	109 (127)
VF	−2.23 (4.56)	0.098	1642 (1901)	0.122	1051 (1217)	0.147	730 (845)	0.196	412 (477)	0.245	264 (306)
FAB	0.56 (2.50)	0.045	7769 (8992)	0.056	4973 (5756)	0.067	3454 (3998)	0.090	1943 (2249)	0.112	1244 (1440)
SAS	1.56 (6.64)	0.047	7095 (8212)	0.059	4541 (5256)	0.071	3154 (3651)	0.094	1775 (2055)	0.118	1136 (1315)
UPDRSII	7.43 (5.94)	0.250	252 (292)	0.313	162 (188)	0.375	113 (131)	0.500	64 (75)	0.626	42 (49)
LVF	2.26 (5.67)	0.080	2460 (2848)	0.100	1575 (1823)	0.120	1094 (1267)	0.160	616 (713)	0.200	395 (458)
CVF	−3.84 (9.05)	0.085	2179 (2522)	0.106	1395 (1615)	0.127	969 (1122)	0.170	546 (632)	0.212	350 (406)
GDS	0.82 (4.89)	0.033	13989 (16191)	0.042	8954 (10364)	0.050	6218 (7197)	0.067	3498 (4049)	0.084	2240 (2593)

Sample sizes are before adjusting for drop-out rate. To adjust for a dropout rate of 26% (eg, the combined dropout rate from both trials), the following formula should be used: sample size/0.74 (eg, 51/0.74 = 69). SD, standard deviation; SEADL, Schwab and England Activities of Daily Living Scale; CGIDS, Clinical Global Impression of Disease Severity; PSPRS, Progressive Supranuclear Palsy Rating Scale; VF, verbal fluency (F, A, or S words per minute); FAB, Frontal Assessment Battery; SAS, Starkstein Apathy Scale; UPDRSII, Unified Parkinson’s Disease Rating Scale II; LVF, two letter verbal fluency; CVF, Category Verbal Fluency; GDS, Geriatric Depression Scale.

^aPer group, based on a significance level of 5% and a power of 80%; approximations of the sample size for the Mann–Whitney U test in parentheses.

Combining the dropout rates of the 2 trials (23% in davunetide, 35% in tideglusib)^2,3 results in a 26% dropout rate (ie, [davunetide dropouts 1 tideglusib dropouts]/[ davunetide ITT population 1 tideglusib ITT population] 5 [50170]/[1391313]). After adjusting for a dropout rate of 26% (calculated sample size/0.74), the sample size for the PSPRS total score was 69 per group (ie, 51/0.74), to detect a 50% reduction of the progression rate.

The results of subgroup analyses for different age groups, disease durations, and SEADL scores showed that excluding patients with a disease duration of >5 years reduced the sample size for the total PSPRS score by approximately 25% (from 51 to 39; Supplementary Tables 2 and 3).

Correlation Analysis

The PSPRS and SEADL scores were highly correlated at baseline (r = −.63, P < .001, N = 187), week 26 (r = −.72, P < .001, N = 152), and week 52 (r = −.71, P < .001, N = 141).

Placebo Effect Calculation

There was no evidence of a placebo effect in any of the evaluated clinical scales according to the definition of an individual improvement of at least 50% when compared with the baseline score on a scale in 10% of all participants (Supplementary Table 4). Further calculations for possible placebo effect, defined as 20% and 30% individual improvement when compared with the baseline score on a scale in 10% of all participants, showed that the Frontal Assessment Battery (FAB), the Starkstein Apathy Scale (SAS), and the Geriatric Depression Scale (GDS) exhibited a placebo effect (Supplementary Table 5). Additional data analysis indicated no statistically significant change over time for the FAB (baseline vs. week 26, P = .69; baseline vs. week 52, P = .60), SAS (baseline vs. week 26 P = .69; baseline vs. week 52, P = .36), or GDS (baseline vs. week 26 P = .13; baseline vs. week 52, P = .07).

Discussion

We analyzed prospective 1-year data of a decline in rating scales in 144 PSP patients, derived from the placebo groups of the davunetide and tideglusib studies. When compared with all the other scales, we found that the total PSPRS score, a disease-specific rating scale capturing deficits in the different functional domains in PSP, required the least number of patients (51/arm) to detect a 50% change in 1 year (1-year-50%), which was further reduced to 39 when including patients with a disease duration of ≤5 years. The PSPRS gait subscore required the least number of patients (63/arm) for detecting a 1-year-50% difference or 53/arm when including patients with a disease duration of ≤5 years. These results differ from a recently published study on 27 PSP patients (eg, 67 patients/arm for a PSPRS total score and 97/arm for a PSPRS gait subscore),¹³ which showed that the PSPRS ocular subscore would require the least number of patients for detecting a 1-year-50% difference. These discrepancies are probably a result of the smaller number of patients and the mono-centre design in contrast to the results reported here.¹³

In terms of scales addressing activities of daily living, the SEADL score, previously used in other clinical trials,^3,12,22 would require 60/arm for a 1-year-50% change if patients with a disease duration of ≤ 5 years are included. Although the UPDRSII activities of daily living scale would require 42 patients per arm only, this analysis was based only on 21 patients, and therefore these results cannot be safely recommended.

All of the scales used to assess cognition or depression showed no ability to deliver adequate results with a reasonable number of patients. This, together with the fact that these scales do not correlate with disease duration or severity, implies that one might omit those in future trials.^23,24 Of note, our results can only be applied to patients with Richardson’s syndrome, and the numbers of needed patients presenting with other PSP-phenotypes²⁵ is unknown.

An important issue in PSP clinical trials is the high dropout rate. High dropout rates in PSP are not surprising given the great motor and cognitive impairment and the rapid decline of PSP patients,²⁵ and this translates into higher numbers of patients that need to be recruited. Therefore, it is crucial to improve the sustainability of PSP patients in studies. Ideally, a shorter study duration would reduce the dropout rate; however, the sample size needed to detect any improvement would be unacceptably high.

In retrospect, the davunetide study was sufficiently powered to detect the 1-year-37.5% expected change, whereas the tideglusib study was not sufficiently powered to detect a 1-year-40% expected change. This, together with the observation that there may be a slowing in the MRI atrophy rate in a subgroup of patients included in the latter study, may imply that tideglusib could warrant further investigation.²⁶

Last, we did not find a considerable placebo effect in PSP, defined as an individual improvement of at least 50% when compared with the baseline score on a scale in 10% of all participants, in any of the scales analyzed in contrast to the well-known placebo effect in PD.²⁷ The mechanism underlying placebo effect is complex,²⁸ and the prefrontal cortex and the basal ganglia are involved, in particular, a substantial release of endogenous dopamine in the striatum has been found in PD patients.²⁹ The widespread and severe postsynaptic degeneration in PSP may be the reason for a lack of placebo effect. When defining placebo effect as a 20% to 30% improvement on a scale when compared with base-line in 10% of all participants, we found this moderate placebo effect to be present for the FAB, SAS, and GDS. However, these scales did not change significantly over time. This, together with the power calculations for the FAB, SAS, and GDS, strengthens the fact that these scales could be omitted from future trials. Moreover, because there is no control group, we cannot rule out that both arms had a similar placebo effect. However, data from natural history studies in PSP have shown a similar decline in the PSPRS and SEADL as the one observed here, and thus this possibility is unlikely.^9,11

In summary, we propose that the total PSPRS score as a single primary efficacy measure for use in future PSP clinical neuroprotective or disease-modifying trials, which requires the least number of patients to detect 1-year-50% change, with included patients having less than 5 years disease duration. The SEADL could be used as a key secondary outcome measure. Last, more sensitive scales could be developed to capture changes in cognitive and neuropsychiatric features of PSP.