A clinical trial method to show delay of onset in Huntington disease

Abstract

Objective:

Disease-modifying clinical trials to prevent disease onset are often limited in methods to assess the impact associated with experimental therapeutics. Although some measures demonstrate decline before onset, there are no outcomes accepted by FDA or EMA. This study suggests that sample enrichment can provide a robust method for prevention of disease onset in individuals with premanifest HD.

Methods:

Using HD onset prediction indexes, we calculated the receiver operating curve (ROC) analysis for HD diagnosis within a three-year time frame. We determined optimal cut-points for recruitment and conducted sample size and power calculations to detect varying effect sizes for treatment efficacy in reducing three-year rates of disease onset. Baseline and longitudinal MRI volumes were analyzed for concurrent and predictive biological validation of the sample enrichment algorithm.

Results:

Areas under the curve (AUC) for three HD onset prediction indexes all demonstrated excellent value with the best sensitivity and specificity shown for the multivariate risk score (MRS).

Conclusions:

The enriched target sample size required for a preventive trial in HD using the MRS was only 89 per arm with a study-wide rate of disease onset of over 37% in the untreated group. This approach makes possible a preventive clinical trial of an experimental therapeutic to delay onset of HD with feasible resource costs.

Trial Registration:

PREDICT-HD is registered with www.clinicaltrials.gov, number NCT00051324.

Introduction

Clinical trial successes in Huntington disease (HD) are symptomatic treatments^1–6. Disease-modifying interventions face a number of challenges^7,8. Two recent papers proposed endpoint measures to facilitate such trials for HD. One of the most frequently cited complexities in clinical trials involves the properties of inclusion criteria to capture heterogeneity^{1, 9–14} while maximizing efficiencies in statistical power, time and cost. The purpose of this investigation was to examine sample enrichment for preventive clinical trials with calculations of required sample sizes for the clinically-relevant outcome, diagnosis^{7, 8, 15}.

Huntington disease (HD) is an autosomal dominant neurodegenerative disease caused by a triplet repeat expansion of cytosine-adenine-guanine (CAG) in the first exon of the huntingtin gene¹⁶. The CAG repeat expansion is associated with age of onset and leads to a progressive loss of function in motor, cognitive, and behavioral integrity resulting in death¹⁷. Given that the mutant gene (mHTT) and its resultant protein (huntingtin) are among the most proximal therapeutic targets, options at this level are intensifying^18–22. However, clinical trial approaches to indicate when to treat or how to distinguish changes associated with interventions are lacking. Despite efforts to predict onset, estimated algorithms are imprecise and require large sample sizes to demonstrate delay of the conversion to disease diagnosis. Though many investigators have given up on the idea of preventing the onset of HD the purpose of this study is to examine possible clinical trial enrichment strategies to facilitate the design of feasible preventive clinical trials.

Methods

Study design and participants

The PREDICT-HD study^{43, 44} was a longitudinal natural history study of over 1100 premanifest HD participants conducted in 32 multinational sites from 2002 to 2014. All participants had prior and independent genetic testing and none had received a diagnosis of HD at study entry. Exclusion criteria included other CNS disease, injury, or developmental disorder, or evidence of an unstable medical or psychiatric illness. During the 12-year study, 225 subjects received a prospective diagnosis of HD according to traditional standards established using the Diagnostic Confidence Level (DCL) of the Unified HD Rating Scale⁴⁵ (UHDRS).

Standard Protocol Approvals, Registrations, and Patient Consents

The research protocol was approved by each site’s respective institutional review board and ethics committee, and all participants gave written informed consent and were treated in accordance with ethical standards.

Procedures

Data collection and storage was described in the other reports of the PREDICT-HD study^{23, 43}.

Primary measures used in this research include motor and cognitive sections of the UHDRS and brain imaging volumetrics.

Briefly, the Total Motor Scale (TMS) is a standardized 31-item assessment rated on a scale from 0 to 4 by an examiner certified by the European HD Network. The range for TMS is 0 – 124 with higher scores reflecting greater abnormality. Following completion of the TMS, the rater completes the UHDRS Diagnostic Confidence Level (DCL), a categorical scale ranging from 0 to 4, where 0 corresponds to non-impaired motor functioning, and 4 corresponds to HD diagnosis defined as unequivocal abnormality of extrapyramidal motor signs of HD with ≥ 99% confidence on the part of the motor examiner.

Cognitive assessment included the Stroop Color Word Test (SCWT)⁴⁶ on which Individuals are invited to read words, name colors and then name the ink color of words as quickly as possible (each for 45 seconds). The test measures cognitive processing efficiency and inhibition of automatic cognitive processing. The Symbol Digit Modality Test (SDMT)⁴⁷ is a 90-second task requiring individuals to fill in boxes with a matching number according to a set of symbol-number pairs presented on the same page. The total number of correct items measures speed of cognitive processing, psychomotor accuracy and working memory.

Onset prediction indexes: Despite widespread usage, age of onset is not completely explained by CAG repeat length^25–31. Figure 1³² confirms wide onset age variation for individuals with the same CAG repeat length. The PREDICT-HD investigators developed five statistical models to index the cumulative toxicity of mutant huntingtin at a given age. First, Langbehn³⁵ used retrospective data from chart review to develop an estimate that was validated with 80 prospectively diagnosed patients with an estimated three-year diagnosis rate of 11.3 percent³⁶. Many studies utilized this model²⁴. Next, Zhang³⁷ developed a revision to the model using 225 prospectively diagnosed HD patients and named it the CAG by Age Product, or CAP score. This formula had more widespread appeal since it didn’t involve the computation of an exponential and was easily calculated. Three-year diagnosis rates for this formula ranged by estimated proximity to diagnosis such that individuals with low CAP showed a conversion rate of 1.6% over three years; medium CAP=7.6%; and High CAP= 22.3%. Clinicians quickly observed that individuals’ CAP scores remained imprecise and did not always reflect the manifestation of clinical phenotype observed upon examination. The multivariate risk score (MRS) was developed by incorporating measures of subtle HD clinical phenotype (motor and cognitive impairments) into the CAP formula^{23, 38}. Most recently, a large collaborative effort validated a normalized prognostic index (PIN) for HD using all available data from PREDICT²³, TRACK³⁹, REGISTRY⁴⁰, and COHORT^{41, 42} . The higher these index values, the more progressive state a premanifest HD individual is considered.

Figure 1.

Age of HD onset by CAG repeat length for 225 prospectively diagnosed individuals in the 12-year natural history study entitled PREDICT-HD.

The three indexes (i.e., CAP, MRS and PIN) examined for a preventive trial enrichment algorithm are shown in Table 1. There was no pre-conceived notion about the strength of these formula for three-year prognosis so all three onset prediction indexes were utilized.

Table 1.

CAP, MRS, and PIN indexes and comparison among outcomes using the sample enrichment algorithm for RCT to delay age of onset in Huntington disease

	Prediction Onset Indexes
Characteristics	CAP37 AGE x (CAG – 33.66)	MRS38 −0.282 x AGE + 0.140 x CAG + 0.565 x TMS −0.021 x SDMT + 0.347 x DCL1 + 0.542 x DCL2 + 1.086 x DCL3 – 0.004 x SC + 0.002 x SW – 0.023 x SI – 0.004 x TMS2 – 0.010 x TMS x CAG + 0.009 x AGE x CAG	PIN41 (51 x TMS – 34 x SDMT + 7 x CAP - 883) / 1044
AUC	0.7897	0.8771	0.8559
Optimal Cut-Point	>390.4	>8.914	>0.3757
Sensitivity	0.6778	0.8222	0.8000
Specificity	0.7734	0.8047	0.7578
Three-Year Rate of HD diagnosis (%)	29.5	37.2	31.7
Required sample size per arm in RCT	126	89	114
Target Sample available in Predict	262	248	286

CAP=CAG age product; MRS=Multivariate risk score; PIN=Prognostic index normalized; RCT=Randomized controlled trial; AUC=Area under the curve; CAG=cytosine, adenosine, guanine repeat length; TMS=Total motor score; SDMT=Symbol digit modalities test; DCL=Diagnostic confidence level; SC=Stroop color; SW=Stroop word; SI=Stroop interference

Brain imaging volumes obtained from Magnetic Resonance Imaging (MRI) are well known to show progressive worsening in concert with clinical and functional decline in premanifest HD^{48, 49}. We chose to use MRI volumes of caudate, putamen, globus pallidus, total gray matter and total white matter for each of the four lobes of the cerebral cortex, total subcortical white matter as well as total cerebrospinal fluid (CSF) since these measurements have demonstrated good replicability across studies and reliability over time^{24, 48, 50}.

Statistical analysis

Methods for the Sample Enrichment Algorithm consisted of three steps: 1. For any given onset prediction index for which a higher value indicates a more advanced premanifest HD state, a receiver operating curve (ROC) analysis⁵¹ for three-year HD diagnosis was conducted; 2. The optimal cut-point for each onset prediction index value (i.e., CAP, MRS, PIN) was determined placing equal importance on sensitivity and specificity; 3. The participants classified into the higher onset group for each of the onset prediction indexes suggested the enriched target sample for preventive clinical trials.

Sample sizes were calculated for conducting a randomized controlled trial (RCT) to test the treatment efficacy in reducing three-year rate of HD onset (i.e., conversion from gene-carrier to motor diagnosis) with a given effect size using a two-sided test for proportions at 0.05 significance level powered at 0.9.

Brain volume data were used as a biological disease parameter showing replication in premanifest HD across studies, sites and methods^{24, 52}. First, a two-way analysis of variance (ANOVA) was conducted to determine whether differences were significant between the enriched and the traditional groups on mean baseline structural volume as a percentage of intra-cranial volume (ICV). This analysis was conducted to consider whether the enriched target sample for HD preventive clinical trials showed concurrent/content validity. Next, linear mixed models (LMM)^{53, 54} were fit to assess whether longitudinal progression rate differences (based on annual percentage of brain volume loss) were evident between the enriched groups adjusting for the effects of scanner strength. This analysis was conducted to consider whether the enriched sample for HD preventive clinical trials showed predictive validity with disease-related biological expectations using brain imaging.

Results

Findings of the sample enrichment algorithm to the three onset prediction indexes using PREDICT-HD data are summarized in Table 1. The AUC values for CAP, MRS, and PIN are 0.7897, 0.8771, and 0.8559 respectively, indicating excellent prognostic value for all prediction formulas. A representative ROC curve is shown in Figure 2. In this example, a RCT to delay HD onset using the MRS would require the targeted recruitment of premanifest gene-expansion carriers with a value above 8.914. Optimal cut-points are shown for each index with related sensitivity and specificity values.

Figure 2.

The ROC curve for the three-year HD diagnosis based on the prognostic marker MRS.

The three-year rate of HD diagnosis in the enriched Target samples (Table 1; 29.5, 37.2, 31.7) can be regarded as the baseline diagnosis rate for untreated participants. Figure 3 displays the sample sizes required for each arm of a RCT to determine treatment efficacy in reducing the three-year rate of HD diagnosis with various effect sizes (reduction rates range from 30% to 80%), based on a two-sided test for proportions at significance level 0.05 and powered at 0.90.

Figure 3.

Sample size required in each arm for a randomized treatment-control clinical trial equipped with two-sided test for proportions at significance level 0.05 and powered at 0.9.

Using the most robust published index (i.e., MRS) comparisons of age, sex, CAG repeat length, CAP score, and the baseline clinical measures between an enriched target sample and the traditional premanifest sample recruited into clinical trials showed highly significant differences on all selected demographic, genetic, and clinical measures (ps<0.0001; see supplementary Table A). Participants in the enriched sample were older with greater CAG expansion, showed worsened motor and cognitive impairment, and were rated higher in diagnostic confidence level at study entry. Moreover, individuals in the high group of CAP were more likely to be in the enriched versus the traditional sample (83.2% vs 20.2%).

Baseline MRI ANOVA results with a total of 774 gene-expanded individuals whose data were available at study entry were compared to individuals in the traditional sample (See Table B in supplementary materials). Individuals in the enriched target sample had substantially smaller volumes for Putamen, Caudate and Globus (ps<0.0001) with relative differences of 30.95%, 31.03% and 27.06% respectively. The enriched sample also had 17.59% greater CSF volume on average, compared to the unselected traditional sample (p<0.0001). Although hypothesis testing demonstrated statistical differences between the two groups in many other brain regions, the relative difference between the groups was no more than three percent (See Table B).

The LMM results for MRI volume percentage annual change was conducted to examine brain atrophy over time between the enriched and the typical groups. Because 61 individuals only had baseline MRIs, 713 gene-expanded individuals were analyzed. MRI volumes decreased over time in both samples, though the annual decrease in the enriched sample was significantly greater than in the traditional sample. In particular, differences were highly significant (p<0.0001) in the Putamen, Thalamus and Globus, with relative differences being 27.87%, 40.79%, and 25.09%, respectively.

Discussion

Using data from the 12-year PREDICT-HD study we developed and validated a sample enrichment algorithm that can be used to study any experimental therapeutic to delay or prevent the onset of HD. To date we are unaware of any RCT design to feasibly test the efficacy of an intervention to delay or prevent the onset of HD. Indeed, some authors have dismissed the conversion rate comparison secondary to feasibility limitations in the number of research participants required^{55, 56}. The proposed algorithm can be used with any index or variable the investigators choose to target the best premanifest/prodromal HD participants to recruit into a trial. The algorithm was applied to three disease prognostic markers: CAP, PIN, and MRS and demonstrated its proficiency in detecting the optimal cut point for RCT sample recruitment. Accurate and enriched selection of participants into studies for prevention efficacy is judicious to maximize statistical power, minimize required sample numbers and increase the probability of detecting therapeutic benefits within three years.

Previous clinical trials in premanifest HD were designed for safety and tolerability⁵⁷ and currently active studies are enrolling premanifest participants whose three-year rate of onset is below 10%. Enriched target group selection based on MRS shows a conversion rate of over 37%. As illustrated in Figure 2, the sample size required for conducting a preventive trial for any given effect size in the two-sided 0.05 significant test of proportions powered at 0.9 is much reduced using the proposed algorithm. For example, if 50% reduction in three-year rate of HD onset is the expected therapeutic efficacy in a RCT, the required one-arm sample size using all HD gene-expansion carriers is about 382. Application of an enrichment algorithm to target recruitment, however, shows greatly reduced samples of 126 and 114 for algorithm-based enrichment using high CAP scores and PIN respectively, and shows the greatest benefit and lowest required sample size (n=89) when the algorithm is applied to MRS.

The use of the proposed enrichment algorithm to select study participants for a three-year preventive trial only requires acquisition of the prognostic variables. This study utilized the CAP, MRS, and PIN, which contain the participant’s current age, CAG repeat length and two to four scores from the UHDRS. The total time to obtain the cognitive and motor variables required for calculation of the MRS or PIN is less than 20 minutes. All measures can be obtained at home, clinic or bedside and no additional equipment is needed.

Since many investigators had dismissed the possibility of preventive trials using diagnosis as an endpoint, a great deal of attention has been directed to the development of other biological or clinical markers that can become surrogates^58–60. In theory, for a surrogate endpoint to be an effective substitute for the clinical endpoint, effects of the intervention on the surrogate must reliably predict the overall effect on the clinical outcome. In practice, this requirement frequently fails. Among several explanations for this failure is the possibility that the disease process could affect the clinical outcome through pathways that are not mediated through the surrogate. Additionally, the intervention might affect the clinical outcome by unintended, unanticipated, and unrecognized mechanisms of action that operate independently of the disease process. Although the search for biomarkers, refined clinical markers and eventual surrogate markers for HD is beneficial (and necessary), there are no markers yet qualified for implementation in costly Phase III trials^{7–9, 61, 62}. Excitement generated by novel treatments can lead to clinical trial designs making assumptions without adequate data-driven support. The FDA recently published guidelines for outcome qualification that are likely to increase the probability of the successful outcome of preventive trials for HD⁶³.

Limitations of this research include the concern that onset/diagnosis of insidious neurodegenerative diseases can be arbitrarily dependent upon frequency of clinician visits, expertise of the diagnostician and reliability of the onset outcome. Comparisons of clinical, retrospective research with the prospectively-diagnosed studies using certified motor examiners (such as TRACK and PREDICT), however, demonstrate high consistency among research outcomes using this parameter. Some investigators might be concerned about the three-year limitation of the proposed enrichment sample, although the methods used here can easily be extended to five-year studies should the experimental therapeutics require longer observation. Benefits to longer studies is there will be higher rates of motor onset/diagnosis in five years than in three years, which will yield more power in terms of demonstrating the treatment efficacy in lowering the rate of conversion during the study period.

Conclusions

Recent natural history studies generated rich and essential information to evaluate the disease progression from premanifest healthy individuals through prodromal disease stages to a meaningful endpoint, motor diagnosis. This paper translates knowledge gained from observational natural history studies to critical decision making for preventive clinical trials.