PRION-1 scales analysis supports use of functional outcome measures in prion disease

Abstract

Objectives:

Human prion diseases are heterogeneous but invariably fatal neurodegenerative disorders with no known effective therapy. PRION-1, the largest clinical trial in prion disease to date, showed no effect of the potential therapeutic quinacrine on survival. Although there are several limitations to the usefulness of survival as an outcome measure, there have been no comprehensive studies of alternatives.

Methods:

To address this we did comparative analyses of neurocognitive, psychiatric, global, clinician-rated, and functional scales, focusing on validity, variability, and impact on statistical power over 77 person-years follow-up in 101 symptomatic patients in PRION-1.

Results:

Quinacrine had no demonstrable benefit on any of the 8 scales (p > 0.4). All scales had substantial numbers of patients with the worst possible score at enrollment (Glasgow Coma Scale score being least affected) and were impacted by missing data due to disease progression. These effects were more significant for cognitive/psychiatric scales than global, clinician-rated, or functional scales. The Barthel and Clinical Dementia Rating scales were the most valid and powerful in simulated clinical trials of an effective therapeutic. A combination of selected subcomponents from these 2 scales gave somewhat increased power, compared to use of survival, to detect clinically relevant effects in future clinical trials of feasible size.

Conclusions:

Our findings have implications for the choice of primary outcome measure in prion disease clinical trials. Prion disease presents the unusual opportunity to follow patients with a neurodegenerative disease through their entire clinical course, and this provides insights relevant to designing outcome measures in related conditions.

Prion diseases are invariably fatal neurodegenerative disorders with no therapy that is known to alter the natural history in man.¹ They comprise an etiologically and clinically heterogeneous group, with genetic, acquired, or sporadic etiologies.² Despite promising developments in basic science and experimental therapeutics, several challenges are evident in the development of clinical trial methodology. First, prion diseases are rare, with the implication that recruitment criteria need to be broad in order to achieve acceptable trial size in a reasonable geographic area or time scale. Second, the clinical phenotype is highly variable within and between each etiologic class. Third, there are strong influences on UK family members to reject randomization in trials which include the possibility of allocation to placebo.^3,4 As a result, carefully documented historical control data may be important alternatives or adjuncts to a randomized placebo-controlled trial.

Any trial is dependent upon the relevance of its outcome measurement instruments. Mortality has been the choice of primary outcome in several prion disease trials/studies, but this is problematic for several reasons.^1,5,6 The immediate cause of death in prion disease, usually sepsis or respiratory failure, can be influenced by several factors unrelated to underlying disease progression, such as use of antibiotics to treat life-threatening infections, and supplementary nutrition. As the aim is to halt or reverse neuropsychiatric impairments with a therapeutic intervention, a more relevant trial outcome measure should consider human prion disease progression directly.

METHODS

PRION-1 study design.

PRION-1 was an open-label patient-preference trial in which patients were offered a choice between taking quinacrine, not taking quinacrine, or being randomized to immediate quinacrine or quinacrine deferred for 24 weeks (appendix e-1 on the Neurology® Web site at www.neurology.org). The objective was to obtain data on the effect of quinacrine in human prion disease, from a randomized comparison where acceptable and otherwise from observational comparisons. Full details of randomization, primary outcome, and adverse events in the pilot and main trial phases have been published.³

Patients were assessed at baseline (enrollment) with follow-up at 1, 2, 4, and 6 months and then 3-monthly including a standardized clinical examination, the Glasgow Coma Scale (GCS),⁷ Mini-Mental State Examination (MMSE),⁸ cognitive component of the Alzheimer's Disease Assessment Scale (ADAS-cog),⁹ Brief Psychiatric Rating Scale (BPRS),¹⁰ Clinical Dementia Rating Scale Sum of Boxes (CDR-SB),¹¹ Global Impression of Change (GIC),¹² Rankin scale,¹³ and the Barthel Activities of Daily Living Index¹⁴ (see appendix e-1).

Standard protocol approvals, registrations, and patient consents.

The PRION-1 trial (ISRCTN 06722585) was approved by the Eastern Multicenter Research Ethics Committee. Written informed consent was obtained from all patients (or guardians/carers according to the Mental Capacity Act 2005, UK) participating in the study.

Statistical analysis.

Changes in neurologic rating scales over time to March 30, 2007 (end of PRION-1), death, or loss to follow-up were estimated using mixed models, allowing individual random effects for intercept (enrollment value) and slope (decline). For more detail, sensitivity analyses, and subgrouping, see appendix e-1.

RESULTS

Recruitment and follow-up summary.

The headline results of PRION-1 have been published.³ Here we describe the pertinent recruitment and survival data needed to understand the scales analysis. A total of 107 patients were recruited in the pilot and main phases of the PRION-1 trial. A total of 45 had sporadic Creutzfeldt-Jakob disease (sCJD), 43 inherited prion disease (IPD), and 19 acquired prion diseases (17/19 variant CJD) (note that one fewer patient had variant Creutzfeldt-Jakob disease [vCJD] than reported in³ due to a late reclassification as IPD). Diagnosis was confirmed by tissue biopsy, at autopsy, or by detection of a genetic mutation in 78/107. Seventy-eight of 107 patients died during the PRION-1 trial (to March 30, 2007). Including additional data from patients still alive and in follow-up on March 30, 2007, from the National Prion Monitoring Cohort to date, in total 29/107 (27%) died within 4 weeks of enrollment, 34 (32%) within 6 weeks, 45 (42%) within 16 weeks, and 75 (70%) within 2 years. See appendix e-2 for details of follow-up and data to support lack of efficacy of quinacrine.

Cross-sectional performance of scales at recruitment of symptomatic patients.

Table 1 shows completion rates for the various scales at recruitment for the 101/107 (94%) patients with symptomatic prion disease at enrollment, overall and according to Rankin. Almost universal completion rates were seen for the global clinician-rated scales (Rankin, GIC) reflecting their simplicity. Very high completion rates were also obtained for the Barthel and GCS (taking into consideration that the Barthel was not used in the pilot phase³ and, following the protocol, GCS was not formally recorded if the MMSE was ≥10 but was rather imputed as 15 in main analyses). The CDR-SB was reasonably well completed but required the participation of a carer who may not have been available at the time. Cognitive scales (MMSE, ADAS-cog) were completed in around half and one-third, respectively, either because of aphasia or advanced stage of disease. The BPRS was only completed in 28/76 participants recruited during the main phase because of the need for a lengthy patient interview.

Table 1.

Characteristics of participants and completion of neurologic rating scales at enrollment into PRION-1 and subsequently

Abbreviations: ADAS-cog = cognitive component of the Alzheimer's Disease Assessment Scale; BPRS = Brief Psychiatric Rating Scale; CDR-SB = Clinical Dementia Rating Scale Sum of Boxes; GCS = Glasgow Coma Scale; GIC = Global Impression of Change; IPD = inherited prion disease; IQR = interquartile range; MMSE = Mini-Mental State Examination; sCJD = sporadic Creutzfeldt-Jakob disease; vCJD = variant Creutzfeldt-Jakob disease.

^aOne patient with MMSE = 15 and moderately ill on GIC did not have Rankin formally recorded at enrollment but is included in the moderate disability Rankin group.

^bBPRS and Barthel were not done routinely during pilot phase of PRION-1. Barthel measurements were available for some of the 23 pilot patients.

^cGCS imputed as 15 for all patients with enrollment MMSE ≥10 following the PRION-1 protocol (see appendix e-1).

Figure 1A illustrates the distribution of observed scores at enrollment, categorizing scores as being at the maximum possible (ceiling, best performance), at the minimum possible (floor, worst performance), or within intermediate terciles (low scores are worst for all rating scales; see appendix e-1). Despite low completion rates, most of those able to complete the cognitive and psychiatric scales scored in the upper terciles. These effects were present to a lesser extent for CDR-SB: in contrast, although Rankin was nearly universally completed, almost 40% of participants already registered the lowest score (floor) at enrollment. Overall, therefore, proportions at this lowest level or unable to complete were similar for the Rankin and CDR-SB. The Barthel, GIC, and GCS had the smallest proportion of patients already registering the worst possible score at enrollment. The only scale to show any significant ceiling effect (i.e., several patients with the best possible score at enrollment) was the GCS.

Figure 1. Distribution of scores at enrollment and completion of MMSE over time.

(A) Distribution of observed scores at enrollment by maximum possible score (ceiling = best), minimum possible score (floor = worst), and intermediate terciles (in 101 symptomatic patients). “Unable” means this was given as a reason for noncompletion on the enrollment form. (B) Completion of MMSE, by months after enrollment (in 101 symptomatic patients). This is shown to illustrate the impact of survival and advanced clinical state on a cognitive rating scale in PRION-1. ADAS-cog = cognitive component of the Alzheimer's Disease Assessment Scale; BPRS = Brief Psychiatric Rating Scale; CDR-SB = Clinical Dementia Rating Scale Sum of Boxes; GCS = Glasgow Coma Scale; GIC = Global Impression of Change; MMSE = Mini-Mental State Examination.

Completeness of rating scale observations over time.

In order to compare interventions in terms of changes in neurologic rating scales, it is imperative that as many of the scheduled assessments as possible are made. This requires attendance at scheduled visits and completion of each test, which may become increasingly problematic as the battery increases in size. In PRION-1, 10%–20% of scheduled visits were not made in patients still known to be alive during the first 36 weeks, declining thereafter to about 5%. Early in the trial most missed visits were due to patients with sporadic CJD becoming moribund. Unsurprisingly, completion of all scales was also strongly impacted by mortality (figure 1B). However, rating scales were not always completed in patients attending visits, for similar reasons as at enrollment. Table 1 shows that between 50% and 75% of the individuals enrolled had only 0 or 1 measurement for the CDR-SB and BPRS, whereas 60%–65% of all patients, and even 32%–35% of severely affected patients, had 2 or more measurements of Barthel and GCS.

Change in rating scales over time.

Some general comments can be made about the patterns of decline recorded in PRION-1, although detailed analysis of factors affecting rates of decline are beyond the scope of this manuscript. Individual patient trajectories are shown for all scales (figure 2). First, as expected, rates of decline were highly heterogeneous. Second, patients tend to follow linear patterns of decline, at least until the most advanced stages of disease. At this point several patients appear to reach a preterminal plateau, although it is unclear whether this observation is related to the insensitivity of scales at the advanced stages of disease, or reflects a true disease phenomenon. In figure 2(see parts D, E, F, I), this phenomenon is illustrated by connecting the time of death (red point) denoted at the worst possible score on each scale with the measured value at last visit (orange line). Finally, for a reasonable proportion of patients, the last observed measurement before death is relatively far above the worst possible score on the scale (see figure 2, parts A, B, C, H)—this is a consequence of rapidity of decline in late-stage disease not covered by the visit schedule, inability to complete the scales with more severe disease (hence absence of further measurements), and the fact that death from human prion disease is often not specifically related to underlying neuropsychiatric decline (e.g., through aspiration pneumonia).

Figure 2. Longitudinal profiles of patients according to cognitive (MMSE, ADAS-cog, BPRS), functional (Barthel), and clinician-rated scales (Rankin, CDR-SB), GIC, and combined Barthel/CDR-SB (combination).

(A–I) Gray lines indicate individual patient trajectories. Red points and orange lines connect the last patient scale measurement with the time of death (allocated the worst possible score, except for BPRS, where the worst score observed during PRION-1 [64] was used rather than the worst possible score for the scale [168]; see appendix e-1). These are shown to illustrate the heterogeneity of individual patient trajectories. ADAS-cog = cognitive component of the Alzheimer's Disease Assessment Scale; BART = Barthel; BPRS = Brief Psychiatric Rating Scale; CDR-SB = Clinical Dementia Rating Scale Sum of Boxes; GCS = Glasgow Coma Scale; GIC = Global Impression of Change; MMSE = Mini-Mental State Examination.

Performance of models used to describe change in rating scales over time.

We estimated annual rates of decline overall, and in fast (sCJD)/medium (vCJD and some IPD, see appendix e-1)/slow progressors (Gerstmann-Straussler-Scheinker), mild to moderate disease stratified by Rankin (2–3 or 2–4), using both observed data only (table 2) and including all symptomatic patients but censoring after the first observed worst possible score for each scale and imputing death as the worst possible score (figure 3). In different disease categories we found no evidence of differences in psychiatric/cognitive scores (MMSE, ADAS-cog, BPRS) at enrollment (p > 0.1) although all functional and clinician-rated scales were worst in “fast” and best in “slow” progressors (p < 0.0001, table 2). However, we found evidence for significant variation in the annual rate of decline after enrollment both by severity and disease category for all scales (p < 0.01, table 2) except BPRS, which had problems with completion as noted above. As expected, the annual rate of decline was greater in “medium” compared to “slow” progressors. However, “fast” progressors (sCJD) had smaller estimated declines than “medium” progressors as a consequence of their very low scores at enrollment, meaning that further declines were limited and therefore estimates of rates of decline are unreliable. With this exception, these data support the behavior of the rating scales as expected, and emphasize the need to diagnose sCJD at earlier stages if such rating scales are to be used as outcome measures in this population. Imputing death as the worst possible score increased the rate of decline, by a much greater extent in those scales with lowest completion rates.

Table 2.

Baseline, annual rates of decline, and power in fast (sCJD)/medium (vCJD and other IPD)/slow progressors (Gerstmann-Straussler-Scheinker), stratified by Rankin (2–3/2–4) and imputed model

Abbreviations: ADAS-cog = cognitive component of the Alzheimer's Disease Assessment Scale; BPRS = Brief Psychiatric Rating Scale; CDR-SB = Clinical Dementia Rating Scale Sum of Boxes; GCS = Glasgow Coma Scale; GIC = Global Impression of Change; IPD = inherited prion disease; MMSE = Mini-Mental State Examination; sCJD = sporadic Creutzfeldt-Jakob disease; vCJD = variant Creutzfeldt-Jakob disease.

^aTest for difference between slow, medium, and fast progressors. Mean baseline and annual rates of decline were estimated from mixed models including observed data only (without imputation) with random intercept and slope for each participant. SD = sqrt(n) × SE (mean annual rate of decline).

^bSample size estimates were based on 80% power, and a 2-sided significance level 5%. The severity stratified models were restricted to patients with baseline Rankin 2–3 or 2–4. In the imputed model the worst score was imputed for death (except for BPRS, see appendix e-1) and any subsequent scores after a patient recorded the worst score were discarded.

Figure 3. Linear mixed model fitted to cognitive (MMSE, ADAS-cog, BPRS), functional (Barthel), and clinician-rated scales (Rankin, CDR-SB), GIC, and combined Barthel/CDR-SB (combination).

(A–I) These graphs illustrate the mean declines and observed values for each of the outcome measures that could be used in a clinical trial setting. Data are from the imputation model, i.e., censoring an individual's trajectory after observing the worst score and, if the worst score was not previously observed, imputing the worst possible score at the point of death (see appendix e-1). ADAS-cog = cognitive component of the Alzheimer's Disease Assessment Scale; BART = Barthel; BPRS = Brief Psychiatric Rating Scale; CDR-SB = Clinical Dementia Rating Scale Sum of Boxes; GCS = Glasgow Coma Scale; GIC = Global Impression of Change; MMSE = Mini-Mental State Examination.

Estimates of sample size in a future clinical trial.

Based on the estimated annual rate of decline for each rating scale and its SD from these mixed models, we estimated the total number of participants required in a 2 parallel-group randomized trial with similar measurement frequency to PRION-1 that would provide at least 80% power to detect a 50% relative reduction in annual rate of decline for that scale (2-sided α = 0.05) (table 2). In terms of subgroups, restricting enrollment through inclusion criteria in future trial designs to only “medium” progressors, or to those with Rankin 2–3 or 2–4 at enrollment, provides some advantages in terms of smaller sample sizes to detect reduced declines (table 2), with “fast” progressors and those at most advanced disease stages clearly able to contribute less information (and potentially also with less ability to benefit from any intervention if substantial neurologic damage has already occurred). However, restricting eligibility clearly comes at the expense of slower recruitment, given the restricted patient population. If enrollment is unrestricted, imputing deaths as worst possible score when this has not previously been observed provides greater power than using observed data only, although it is a heuristic approach to the problems described above. For comparison, assuming 25% of patients survive 2 years,³ then a total of 106 participants would be needed to detect an increase in survival from 25% to at least 50% (i.e., a relative 33% mortality reduction) with at least 80% power and 2-sided α = 0.05. This is about the same as the smallest total number needed to detect 50% reduction in declines in the different rating scales.

Proposed combination of Barthel with subcomponents of CDR-SB.

Based on the validity and good performance of Barthel and CDR-SB in the analyses above (table e-1), we explored the utility of combinations of subcomponents of these scales. A proposed 30-point scale based on the entire Barthel with addition of modified memory, orientation, and judgment subcomponents from the CDR-SB has been included in the power calculations for comparison (table 2). The CDR-SB subcomponents were selected on the basis of ease of use and validity in a prion disease patient population, confirmed by visualization of individual patient trajectories for each CDR-SB subcomponent. Further analyses to justify the choice of subcomponents and refine questions will be conducted in the National Prion Monitoring Cohort study.

DISCUSSION

The desired requirements of a rating scale in prion disease are clear. The scale should be related to the disease manifestation and progression, and should be important to patients and their carers. The scale should be able to measure one or both of these accurately by minimizing disease-unrelated variations, such as those due to the particular examiner, the degree of encouragement, or patient fatigue. To facilitate patient assessment, the scale should not take long to complete and the need for specialist training of the examiner should be minimal. A hallmark of prion diseases is their rapid evolution, which necessitates a scale that can perform across a wide severity range. Given that prion diseases are rare, a scale covering a wide range of severity, which is as simple, easy to measure, and reproducible as possible, and not subject to variation in implementation over calendar time, may be important for studies with broad entry criteria, those recruiting internationally, and those with prolonged recruitment periods. In table e-1, we assess the 8 scales used in PRION-1 against these requirements.

Floor effects for cognitive and psychiatric scales were very strong in PRION-1. Serial cognitive/psychiatric measures were rarely obtained in sporadic CJD. Language output impairments have long been recognized as a classic clinical feature of sporadic CJD.¹⁵ The frequency of this defect may overemphasize deterioration measured using the 3 cognitive/psychiatric scales which rely heavily on spoken responses by the patient. The functional and global scales were also impacted to some extent by insensitivity in the advanced stages of disease. The GCS was most resistant to this problem, but suffered from a marked ceiling effect with many patients never scoring below the best possible score, and questionable relevance to progression of the disease. In general, the floor effects were partially ameliorated by imputation of the worst possible score for patients who had died. This also allowed for the inclusion of patients in models for decline who were otherwise excluded because no measurements were obtained. A disadvantage of the imputed model is that it is only based on a single imputation, and the strong impact of survival limits the advance of the outcome measure beyond the use of mortality alone.

Across the disease-stratified, severity-stratified, and imputed models, the more functionally orientated scales (CDR-SB, Barthel) resulted in patient trajectories that appeared to match clinical experience (figure 2), they differentiated well between patients expected to progress slowly or moderately fast, and gave reasonable power in severity and imputed models to detect clinically relevant changes with moderate sample sizes. Decline in prion disease was generally linear in the measurements from these scales. These scales are quick and easy to apply with little training required, validated for use by several health care professions, can be used over the telephone, measure things fundamentally important to carers, and have been validated for use in settings of patient improvement and deterioration (see appendix e-1). Perhaps the most important advantage of the Barthel scale is that the activities it documents require a combination of mental health, cognitive, visual, balance, and other sensory or motor skills, each of which may be focally compromised in the early stages of prion disease.¹⁴ A more specific cognitive or neurologic scale might not appropriately emphasize the impact of severe focal impairment in one of these domains. The Barthel scale has emphasis on mobility rather than cognition, whereas the latter is the most important overall clinical feature of prion disease. Subcomponents of the CDR-SB may be added to the Barthel to increase the emphasis on cognitive skills.

There are some limitations: we did not consider quality of life measures (not included as outcome measures in PRION-1), the effects of symptomatic therapies, or stratification by codon 129 genotype or prion protein type. By using models with patient-specific trajectories, our findings assume that PRION-1 patients are similar to future patients with regards to these additional factors, an assumption which is plausible due to our high recruitment of eligible patients.³

On the basis of these considerations we propose a scale adapted from the Barthel and CDR-SB for use as a primary outcome in future prion disease clinical trials. There are several advantages of such a scale; it appears to have comparable power to using either survival alone or any individual scale; its applicability for telephone use will allow an intensity of data collection that might lead to further improvements in power (although initial simulations suggest that these may be relatively small); and it documents issues of importance for carers. Clearly it would be advantageous to reach an international consensus on outcome measures in this disease. Further development of the scale including subgroup analysis, prediction of survival, interrater reliability, and telephone use, among other issues, will be considered by the ongoing National Prion Monitoring Cohort study. The unusual opportunity afforded by PRION-1, to follow up patients through most of the clinical course to death, makes our conclusions relevant to other neurodegenerative diseases.

GLOSSARY

ADAS-cog

cognitive component of the Alzheimer's Disease Assessment Scale

BPRS

Brief Psychiatric Rating Scale

CDR-SB

Clinical Dementia Rating Scale Sum of Boxes

GCS

Glasgow Coma Scale

GIC

Global Impression of Change

IPD

inherited prion disease

MMSE

Mini-Mental State Examination

sCJD

sporadic Creutzfeldt-Jakob disease

vCJD

variant Creutzfeldt-Jakob disease

AUTHOR CONTRIBUTIONS

Dr. Mead: drafting/revising the manuscript, study concept or design, analysis or interpretation of data, study supervision. M. Ranopa: analysis or interpretation of data, statistical analysis. Dr. Gopalakrishnan: drafting/revising the manuscript, analysis or interpretation of data. Dr. Thompson: drafting/revising the manuscript, analysis or interpretation of data. Dr. Rudge: drafting/revising the manuscript, study concept or design, analysis or interpretation of data, contribution of vital reagents/tools/patients, acquisition of data, statistical analysis. Dr. Wroe: study concept or design, analysis or interpretation of data, acquisition of data, study supervision, obtaining funding. Dr. Kennedy: study concept or design, acquisition of data. F. Hudson: drafting/revising the manuscript, study supervision. Dr. Mackay: drafting/revising the manuscript, analysis or interpretation of data, acquisition of data, study supervision. Dr. Darbyshire: study concept or design, analysis or interpretation of data, study supervision. Dr. Collinge: drafting/revising the manuscript, study concept or design, analysis or interpretation of data, study supervision, obtaining funding. Dr. Walker: drafting/revising the manuscript, study concept or design, analysis or interpretation of data, statistical analysis, study supervision.

COINVESTIGATORS

Trial steering committee—D. Armstrong, I. Chalmers, L. Firkins, F. Certo, J. Collinge, J. Darbyshire, C. Kennard, A. Kennedy (until 2004), J. Ironside, A. MacKay, H. Millar, J. Newsom-Davies, J. Nicholl, J. Stephenson, M. Wiles, S. Wroe. Trial observers—R. Knight, M. Rossor, A.S. Walker (trial statistician), G. Keogh. Data and safety monitoring committee—M. Ferguson Smith, V. Farewell, I. McDonald, R. Collins.

DISCLOSURE

Dr. Mead receives research support from MRC Prion Unit core funding and Wellcome Trust. M. Ranopa, Dr. Gopalakrishnan, Dr. Thompson, Dr. Rudge, Dr. Wroe, Dr. Kennedy, F. Hudson, Dr. Mackay, and Dr. Darbyshire report no disclosures. Dr. Collinge has received speaker honoraria from GlaxoSmithKline; has received research support from UK Medical Research Council and Department of Health and Wellcome Trust; and is Director and shareholder of D-Gen Limited, an academic spin-out company working in the field of prion disease diagnosis, decontamination, and therapeutics. Dr. Walker serves on scientific advisory boards for Tibotec Therapeutics; has received speaker honoraria from Gilead Sciences, Inc.; serves as Associate Statistical Editor for the Journal of Infectious Diseases; and receives research support from GlaxoSmithKline, Gilead Sciences, Inc., UK Medical Research Council (MRC), UK National Institute of Health Research (NIHR), the European Union, and the Rockefeller Foundation.