Optimizing clinical trial design for multiple system atrophy: lessons from the rifampicin study

Abstract

Multiple system atrophy (MSA) is a fatal neurodegenerative disorder characterized by autonomic failure and parkinsonism/ataxia; no treatment exists to slow disease progression. A number of factors have prevented or compromised trials targeting disease modification. A major hurdle has been uncertainty about the number of patients needed to achieve adequate power. Information based on natural history studies suggested such numbers to be so large that only international multi-center models seemed feasible. When designing the rifampicin trial in MSA we sought to identify and apply strategies that would improve power and reduce the number needed to treat to allow for an oligocenter approach. Strategies included: (1) inclusion/exclusion criteria designed to enroll patients with relatively early, actively progressing disease; (2) minimizing dropouts; (3) pre-defined interim analysis; and (4) approaches to reduce scoring variability. The model allowed for the number needed to treat to be only 50 patients per treatment arm. Ten selected sites managed to reach the recruitment goal within 12 months. The dropout rate was less than 10 %, and the goal of enrolling patients with actively progressing disease was accomplished as reflected by the progression rate in the placebo group. Data from this unfortunately negative trial can now be effectively used to more realistically power future trials. A number of ways to further improve trial design and feasibility have been identified and include rigorous site selection and training, designated primary site investigators, improved error trapping, early site visits, remedial training, and future biomarkers for earlier diagnosis and tracking of disease progression.

The status quo

Multiple system atrophy (MSA) is a rare, progressive, and fatal neurological disorder characterized by autonomic failure and parkinsonism or ataxia [1]. There is no treatment available to slow or halt disease progression. The pathogenesis is unknown, although the recognition that the misfolding and aggregation of α-synuclein in the synucle-inopathies play a pivotal role in disease pathogenesis has led to improved animal models of the disease [2–4]. Experimental MSA animal models have spurred research into the effects of modifications in α-synuclein aggregation and subsequent events including the generation of toxic oligomers, microglial activation, oxidative injury, and prion-like spread [3, 4]. The decomposition of disease pathogenesis into several steps has provided a number of potential points of intervention. A translational view of MSA pathogenesis is shown in Fig. 1. Normal α-synuclein exists in a soluble form, existing as a monomer or tetramer [5]. Pathogenesis is likely related to misfolding of this protein and its aggregation. There are a number of approaches to block aggregation, including rifampicin [6]. Beyond aggregation there is generation of oligomers, microglial activation, and the inflammatory response, which provide additional options for intervention [7]. The inflammatory response, oxidative injury, and cytokine toxicity follow, and this cascade of events leads to blood–brain barrier disruption and pathological changes in oligodendrocytes and specific neurons. Growth factor depletion, especially involving GDNF and BDNF, has been considered to be a key consequence [3, 4].

Fig. 1.

Translational view of MSA pathogenesis

A number of different factors have prevented or compromised randomized clinical trials targeting these different aspects of the disease cascade in MSA. Since MSA is a rare disease, development of treatment options has been economically unappealing, and focus has been primarily devoted to the much more common synucleinopathy Parkinson’s disease (PD). While the two disorders likely share commonality in pathogenesis, MSA is nevertheless unique. In contrast to PD, the primary pathologic lesion is glial cytoplasmic inclusions; Lewy bodies (the hallmark of PD) are usually absent or rare in MSA [1]. Clinical trials on drugs of interest have typically focused on PD with MSA studied as an add-on or afterthought. There has also been considerable uncertainty on powering such studies adequately for MSA. The number needed to treat, estimated based on the course of probable MSA cases in the North American Natural History Study, has been considered to be very large, so that treatment trials in the USA alone have been considered non-feasible [8]. The only workable model seemed to be an industry model involving many sites, multiple countries, and multiple languages with associated logistical, methodological, and financial obstacles. Furthermore, progress has been made in improving the certitude of the diagnosis of MSA—especially since the development of the consensus criteria. That increased certitude, with a focus on the “probable” MSA designation, has been at the price of study entry at a late stage of disease for many patients [9–13].

Designing the rifampicin study

The potential value of rifampicin emanates from the research by Cliff Shults and Eliezer Masliah as part of our MSA program project. Research has been advanced by the development of a transgenic mouse model expressing human α-synuclein under the control of the myelin basic protein promoter; these transgenic mice develop oligoden-droglial aggregates of α-synuclein and motor deficits characteristic of multiple system atrophy [14]. The antibiotic rifampicin was found to inhibit the formation of α-synuclein fibrils and it disaggregates fibrils that have already formed, which made it of particular interest [6]. These findings led to the hypothesis that this drug may delay progression or reverse neurological and autonomic dysfunction in human MSA. Based upon the clinical back-ground described under “Status Quo” adequately powering a study seemed challenging and so we approached the problem by identifying strategies to improve power. There are a number of practical approaches to improve power in study design. We considered and addressed the following:

1. Avoiding late-stage disease

Accurate diagnosis is clearly important in human trials. As mentioned above, consensus criteria have improved the certitude of diagnosis, but the diagnostic accuracy of “probable” MSA is reached at the cost of delaying diagnosis to a later stage of the disease, since “probable” MSA criteria are often met only at a relatively advanced clinical stage [11, 12]. There is a sigmoid progression of clinical impairment and deficits over time as illustrated in Fig. 2. A preclinical stage without apparent functional impairment or clinical features might be a phase when pathologic changes are starting to accumulate without apparent deficits. This phase is followed by a stage of evolving early disease, which is clinically non-specific and escapes current consensus criteria; further development of biomarkers will hope-fully soon allow for earlier diagnosis at that stage. A phase with still steep progression follows, captured by current consensus criteria; this phase represents patients diagnosed at a relatively early stage, fulfilling “possible” MSA criteria or “probable” MSA criteria, with low UMSARS score. Finally, patients reach a late plateau phase that almost invariably meets consensus criteria for “probable” MSA.

Fig. 2.

Sigmoid progression of clinical impairment and deficits over time. A preclinical stage is followed by a stage of evolving early disease, which is clinically non-specific and escapes current consensus criteria. This is followed by a phase with still steep progression, captured by consensus criteria (“possible” MSA and “early probable” MSA), and finally a late plateau phase that almost invariably meets consensus criteria for “probable” MSA

The goal of randomized clinical trials is to enter an adequate number of subjects with a reliable diagnosis of MSA to answer the important clinical question of efficacy. The importance of subject selection was therefore paramount in our minds in adequately powering the study. It was clear that patients would ideally need to be enrolled early enough to capture patients during the steep part of the progression curve before the plateau of late-stage disease is reached, but limited information was available on the natural history of MSA. Available information was based mainly on retrospective analysis. As part of our MSA Program Project, we undertook a prospective natural history study of “probable” MSA. In a study of 67 subjects studied over 12 months, the mean rate of increase in UMSARS I was +0.258 points per month [8]. Preliminary slope estimates on Mayo subjects (N = 38) revealed a mean rate of increase in UMSARS I of +0.375 points per month [10]. The standard deviations of the slope estimates were 0.417 and 0.633, respectively. Such a slope would require an inordinately large number of subjects per study arm to power the study. However, it is known that slope estimates are much steeper in the case of early MSA cases. Geser et al. [15] found in a prospective natural history study that the rate of change in “possible” MSA cases is approximately +0.66 points per month. These rates of change nicely underline the disease progression model in Fig. 2; patients with “probable” MSA are often further advanced and are at the plateau stage when life expectancy is only 2.1 years [10]. In contrast, the phase before that resides in a much steeper part of the curve. We surmised that by selecting subjects who were still ambulant, with an UMSARS score below 17 (omitting question 11 on erectile dysfunction), and either “possible” MSA or “probable” MSA, we would study subjects residing in the steep part of the curve, with a steeper slope of progression. Therefore, for our power calculations, we assumed that the average rate of increase in the UMSARS I score in the placebo group would be +0.66 points per month with a within-group standard deviation of the slope estimate of 0.559. Under these assumptions 47 subjects per group would have given us 80 % power to detect a reduction in rate of progression in the treated group of 50 % compared to the placebo group with α = 0.05. To account for the possibility of dropouts, we postulated a required sample size of 50 patients per group.

2. Minimizing dropouts

We argued that one way to improve power was to reduce the number of dropouts from the typical 20 % to less than 10 %. Patients with late stage MSA often do not survive one year of study [10]. We argued that the selection of patients with earlier disease and lower UMSARS scores, as well as the selection of a small number of highly performing sites could achieve this goal.

3. Reducing variability in scoring

There are a number of variables that account for much of the excess variability in clinical trials. We focused on the following:

a. Oligocenter study

In studies of rare diseases, such as MSA, it is typically necessary to include multiple centers in multiple countries with multiple languages. Many of the sites recruit small numbers of subjects and those often have the largest dropout rates and largest variability. We argued that the selection of a small number of high performing and experienced sites (oligocenter model) should further reduce variability. For the rifampicin study, we selected 10 sites. Half of the sites (n = 5) were led by principal investigators from the MSA program project or the Autonomic rare disease consortium. The other sites were major research centers with extensive experience with both MSA and clinical trials.

b. Training and teaching aids to reduce scoring variability

The use of reliable and well-validated instruments is a pre-requisite for a meaningful and accurate assessment of disease progression. The instruments should also have a scoring system that is unambiguous. For this reason we selected UMSARS I as the primary endpoint and UMSARS II and COMPASS as secondary endpoints, all well-validated instruments [16, 17]. Investigators who were assigned to do the evaluation were required to attend a training session to minimize heterogeneity in scoring. Video recordings of proper UMSARS II scoring were provided and the audience was tested on their scoring. This approach was coupled with an Operator Manual that contained instructions on scoring, and participants were also provided with a copy of the video for review.

c. Optimized scoring

An important source of variability is whether the subject and investigator have knowledge of the prior score for the items of interest. Our position was that we needed to be consistent and it was decided to make that information available. Furthermore, a common source of variability in scoring among experts is to overscore, i.e., finding abnormalities that are not present in an attempt to not miss any abnormality; our investigators were instructed to score only definite abnormalities as this approach has been proven to be proficient in clinical trials of other neurological disorders [18].

d. Infrastructure to capture inconsistencies and handling emerging problems

We provided a management team available to answer questions from the sites on specific subjects. We placed particular emphasis on the handling of particular problems that we anticipated (such as potential hepato-toxicity) in a pre-emptive uniform fashion, so as to optimize patient care and minimize dropouts.

4. Interim analysis

An interim analysis was planned after the first 30 participants had completed the 12-month treatment period. The primary null hypothesis for the futility analysis was that rifampicin reduces the rate of progression by at least 50 % compared with placebo. We tested this hypothesis against the futility alternative hypothesis that rifampicin reduces the rate of progression by less than 50 % compared with placebo. Apart from the benefit of terminating a clearly futile study early, such analysis also provided the DSMB the opportunity to adjust sample size if actual power calculations had proved to deviate markedly from power estimates.

What we achieved

We assembled 10 sites with the goal of recruiting the planned 100 subjects over 24 months. The study was supported by NINDS under the Autonomic Rare Disease Consortium (Principal Investigator: David Robertson) and the MSA Program Project (Principal Investigator: Phillip Low), using the DMCC infrastructure. The actual accrual rate far exceeded the projected rate (Fig. 3) and recruitment was completed within 12 months. Of note is that four sites recruited more than two-thirds of the subjects. This study demonstrated that treatment trials of MSA are feasible and can be efficiently done in the USA under this infrastructure.

Fig. 3.

Planned versus actual recruitment of patients for the rifampicin treatment trial. Recruitment was completed within one year and one year ahead of plan

The goal of halving the number of dropouts from 20 % was achieved. We had 9 dropouts, 5 in the placebo group and 4 in the rifampicin arm of the study.

The third goal was to achieve a group of early subjects with both possible and probable MSA such that the rate of worsening was steeper than what we reported (0.26–0.38 points per month in UMSARS I) in our natural history study [8, 10]. The slope, or the mean rate of change in UMSARS I score, using the dataset from the placebo arm of the rifampicin study was 0.5 points per month (SD 0.5), which is similar to the steeper slope reported by Geser et al. [15] once a presumed placebo effect is factored in. Although the study was disappointingly cleanly negative, data from the placebo arm of our study will enable us to now even more realistically power future studies more realistically. Using these data and assuming an equal SD in the treatment group, we would have required 64 participants per group to detect a difference of 50 % (i.e., a slope of 0.5 points per month in the placebo group vs. 0.25 points per month in the treatment group) with 80 % power and an alpha level of 0.05 based on a two-sample t test (Fig. 4). Required sample sizes for 40 % and 30 % reduction in slope would have been 100 participants per group and 176 participants per group, respectively. In Fig. 4 we also provide estimated sample sizes for 90 % power and for comparison estimated sample sizes based on the data from our prospective North American Study of MSA [8–10]. The corresponding sample sizes per group for 80 % power and 50, 40, and 30 % reduction in slope would have been 165, 258, and 457 participants per group, respectively, which is nearly three times more than for studies modeled after our approach.

Fig. 4.

Sample size estimations for future studies in MSA by effect size. The required sample size is estimated based on disease progression as observed in the rifampicin treatment trial (early disease stage, solid lines) versus the prospective North American Natural History study of MSA (late disease stage, dotted lines). Sample sizes are estimated for 80% (black) and 90 % (gray) power and an alpha level of 0.05 based on a two-sample t test

How can we do better?

The oligocenter study design enabled us to undertake a more detailed analysis regarding site performance. A preliminary analysis revealed marked variation in consistency of scoring among sites. There were not only large differences in the variability of indices of disease progression among study subjects between sites, but also in the variability of such indices within subjects. This is only partially explainable on the basis of disease heterogeneity and suggests at least a component of inconsistency in scoring at some of the sites. There were also a number of other problems that should be fixable with better error trapping, for instance, missing values at various time points or implausibly low scores.

What might be some changes that should be implemented to improve on those observations? The following should be considered:

1. Rigorous site selection and training

At top performing sites, all evaluations were done by an investigator who was experienced in neurological disorders and specifically MSA. Training and experience with scoring functional impairment and deficits in MSA seems therefore a key factor in reducing variability.

2. Designated primary investigator for each site

At top performing sites, all evaluations were done by the same investigator. A single study investigator should improve site performance and remedy of problems in that (a) the management team can directly work with the responsible person; and (b) the same investigator scores all subjects at all time points to ensure more homogenous scoring between and within subjects.

3. Improved error trapping

For instance, baseline values should fall within certain limits and red flags should be generated for missing values or for values that change beyond certain limits between visits. Such change may be correct, but merits checking.

4. Early site visit

A site visit after the first subject has been recorded and scored in addition to random site visits may enhance site performance.

5. Remedial training

An option for remedial training should be available to enhance site performance in case site deficiencies become apparent.

6. Even earlier diagnosis

Further advances in the development of disease-specific biomarkers is fundamental in diagnosing MSA at an early stage, when the disease is still rapidly progressing and more amenable to treatment interventions.

7. Biomarkers of disease progression

Further development of imaging and other biomarkers that can reliably track disease progression may—by adding sensitive and objective secondary outcome measures—increase the ability of detecting effectiveness in future trials.