Expectations and clinical meaningfulness of randomized controlled trials

Abstract

Alzheimer’s disease (AD) clinical trials are designed and powered to detect the impact of a therapeutic intervention, and there has been considerable discussion on what constitutes a clinically meaningful change in those receiving treatment versus placebo. The pathology of AD is complex, beginning many years before clinical symptoms are detectable, with multiple potential opportunities for therapeutic engagement. Introducing treatment strategies early in the disease and assessing meaningful change over the course of an 18-month clinical trial are critical to understanding the value to an effective intervention. With new clinical trial data expected soon on emerging therapeutics from several AD studies, the Alzheimer’s Association convened a work group of experts to discuss key considerations for interpreting data from cognitive and functional measures and what is considered a clinically meaningful benefit or meaningful slowing of this fatal disease. Our expectations of outcomes from therapeutic interventions in AD may need to be modified.

1 |. INTRODUCTION

A great deal of interest has been generated recently concerning the outcomes of randomized controlled trials (RCTs) for the treatment of early-stage mild cognitive impairment (MCI) or mild dementia due to Alzheimer’s disease (AD). The recent accelerated approval of a disease-modifying therapy (DMT) highlighted this discussion in the field pertaining to the concept of clinically meaningful change in outcome. In the era of disease-modifying therapies, the primary outcome is often the change in a measurement that reflects the rate of progression of the underlying disease process, which would be expected to impact, after some time, the slowing of clinical progression. Generally, an acute symptomatic improvement would not be expected from these interventions; rather, a change in the overall rate of progression of clinical symptoms and disability is the expected outcome. However, for several reasons, this can be challenging to measure in early-stage AD by short-duration RCTs and to unequivocally demonstrate at the individual-patient level. In January 2022, the Alzheimer’s Association convened a workgroup of experts to identify how widely used cognitive and functional measures may inform us, from patient and caregiver perspectives, about what is considered a clinically beneficial or meaningful slowing of the progression of AD. The output from this workgroup would be valuable to the scientific community, clinicians, policymakers, payers, regulatory bodies, and patients and their caregivers by reframing our understanding of what constitutes a “clinically meaningful change.”

In a similar vein, we may need to modify our expectations of DMTs. Although our ultimate goal is to halt disease progression, we must acknowledge the reality that slowing the progression of the disease has measurable and meaningful benefits for patients and their families. Given the complex milieu of events surrounding the neurodegenerative process and the clinical disease phase at which we are intervening, a modest clinical diminution of the rate of progression might be all that we can expect, yet it may be clinically meaningful.

2 |. DEGREE OF CHANGE AND IMPACT OF TIME

Treatments for AD may provide only an improvement in symptoms or only slow the rate of decline, although a given drug could have both effects. For a drug that does provide a primarily symptomatic benefit, historically this has been assessed using a measure of cognition (e.g., Alzheimer’s Disease Assessment Scale-Cognitive Subscale) with a global or functional measure as a coprimary to ensure that the effect on the cognitive measure is clinically meaningful. Historically, the field has focused on establishing score values considered clinically meaningful (primarily for symptomatic benefit) on a given cognitive or functional measure. For treatments that slow disease progression, the situation is more complex because two variables must be considered; one is the number of points relative to placebo on a cognitive scale, but the other variable is time, in other words, how long did it take for that difference in cognitive score to be achieved. For a drug that completely stopped disease progression, a clinically meaningful point difference on a cognitive scale could be achieved in as little as 6 months¹; however, a drug could also have a very small effect on disease progression such that the same difference in cognitive score would require 20 years of treatment. Thus, during an RCT for a treatment that slows disease progression, while it remains necessary to achieve statistical significance compared to a placebo on a cognitive measure for the study to be considered positive, another consideration might be the time needed to achieve that difference. A statistically significant difference seen after 6 months would be considered very important, but a statistically significant result of the same magnitude at the end of a hypothetical 60-month study may not be considered clinically meaningful. In addition, the time a person would be treated with the drug in practice would need to be considered and our expectations modified accordingly.

The underlying biological process that forms the underpinning for the disease begins years to decades prior to the onset of clinical symptoms.^2,3 The amyloid-β peptide (Aβ) plays a primary role; it is derived from the processing of the amyloid precursor protein.⁴ Aβ monomers form soluble species (e.g., protofibrils and Aβ oligomers) in addition to larger insoluble species such as the generation of amyloidogenic peptides, protofibrils, fibrils, and, ultimately, amyloid plaques. These insoluble species can serve as a reservoir for Aβ monomers and other potentially toxic soluble Aβ species. As noted earlier, the fact that this process can evolve over decades is important.

Clinical progression reflects the impact of the underlying pathophysiology of the disease on decompensated brain networks following a long asymptomatic phase. This ultimately results from the deposition of amyloid plaques, which can be seen at autopsy or with positron emission tomography (PET) imaging. The development of accurate biomarkers of disease pathology, including molecular PET imaging, cerebrospinal fluid (CSF), and plasma indicators of amyloid and tau brain pathology, affords the opportunity to intervene at the earlier stages of the disease with the anticipated response of preventing or slowing cognitive, functional, and behavioral decline and accumulating disability.^5–8

A challenge exists when translating these two processes into an RCT design. If the pathological unfolding of the disease process takes 1 to 2 decades, RCTs are typically performed over the course of 18 to 24 months (and when initiated at the asymptomatic phase, 4 years or more). As such, if the therapy is designed to halt one aspect of the Aβ/amyloid cascade, the clinical impact of this intervention may be difficult to ascertain. An RCT of 18 to 24 months may not inform us of how much amyloid would have to be reduced or at what point we are intervening in the disease continuum. In addition, how long would a treatment need to be continued to demonstrate a change? And which outcome(s) could be considered meaningful at the group level? These factors are particularly complicated in light of individual heterogeneity/vulnerability and resilience factors of AD-related pathology.

Several disease-modifying therapies under investigation in the early clinical stages of AD may aim to reduce the amyloid plaque burden over the course of 12 to 18 months.^9–12 Again, this biological process, while effective, takes place in a fraction of the time over which the underlying pathophysiology has evolved. Therefore, if the clinical impact of the disease pathology is only beginning to manifest itself, what might we expect if we reduce the amyloid plaque level at this early symptomatic stage of the disease? In reality, any form of slowing of the clinical progression at this early stage of the disease might be clinically meaningful. In the context of this paper, “meaningful” is a description of the “meaning” in a “clinical” patient-centered environment and has serious or important value, usefulness, or purpose for individual patients and their families. Although there is some discussion here about the specific measurement of clinically derived scores via a structured assessment, our definition of clinical meaningfulness may hold face value and utility for patients and their families were they to actually experience this benefit. One important aspect in defining and supporting clinical meaningfulness is understanding individual patient goals as a definition of success. Our understanding of what matters to patients and their families has greatly improved in recent years,^13,14 but we must continue to press for more advances in understanding the patient voice and what is clinically meaningful. Although this concept resonates across stakeholders (e.g., patients, advocates, clinicians, payers, regulators), it has been misapplied when evaluating clinical trial outcomes for new therapies. Previous attempts to identify thresholds of meaningful within-patient progression^15–17 have recently been misinterpreted as necessary thresholds for determining meaningful group-level differences, creating misguided expectations for clinical meaningfulness in contemporary clinical trials. Within-patient change thresholds are not intended to assess the meaningfulness of differences between group-level changes over time and instead may be useful to illustrate meaningful within-patient progression over the course of a clinical trial via supplementary responder/progressor analyses.¹⁸

In addition, clinical meaningfulness may be quite different for a symptomatic therapy compared to a DMT. In the former, we might expect a clinical response that would be apparent to patients and family members, whereas in a DMT, the changes may only become apparent years after the intervention.

One of the instruments used as a primary outcome measure in several current RCTs for AD is the Clinical Dementia Rating (CDR) scale.¹⁹ Other integrated assessments of cognition and function have been and are currently being employed (e.g., Integrated Alzheimer’s Disease Rating Scale [iADRS]²⁰ and AD Composite Score [ADCOMS]²¹), but here the sum of boxes of the CDR (CDR-SB) is further described as a relevant example. This instrument has six domains, with each domain being graded on this scale: 0, 0.5, 1, 2, 3. The CDR total score, or CDR-SB, ranges from 0 to 18 on this instrument,²² and current clinical trials often involve participants at early stages of the clinical progression who are in a CDR-SB range of 0.5–3.0. The CDR-SB should not be mistaken for, or interchanged with, the GLOBAL CDR score, which uses a scoring algorithm to derive a single score of 0 to 3 based on all six domains and is used as a global rating of dementia severity.

Could a 0.5 change in CDR-SB be of value and likely be of meaningful benefit to any individual with early-stage AD? When considering the verbal descriptors of the various domains in the CDR score, one might conclude that a change from a CDR score of 0.5 to 1 in a single domain is likely to be of meaningful value to patients and their families, particularly because patients, families, and care partners highly value preservation or delay in loss of independence and functional abilities.^13,23 For example, in the memory domain, a CDR of 0.5 is described as “mild consistent forgetfulness; partial recollection of events; ‘benign’ forgetfulness.” A rating of CDR 1 in the memory domain is described as “moderate memory loss, more marked for recent events; defect interferes with everyday activities.” Many would argue that this does constitute a meaningful change insofar as the person with AD has progressed from partial recollection of events to a degree of forgetfulness that is now interfering with daily activities. Considering the community affairs CDR domain, transitioning from a 0.5 to a 1 would be consistent with progressing from the descriptor “slight impairment in these (community affairs) activities” to “unable to function independently at these (community affairs) activities, although may still be engaged in some; appears normal to casual inspection”; again, this is a very face-valid indicator of some loss of independence for the patient. Finally, using as another example of a CDR domain that of home and hobbies, progression from a 0.5 to a 1 rating means transitioning from “life at home, hobbies, and intellectual interests slightly impaired” to “mild but definite impairment of function at home; more difficult chores abandoned; more complicated hobbies and interests abandoned,” a clear indication of loss of function and abilities, as well as restrictions in past interests and hobbies. It is reasonable and natural that such losses, at these early stages of mild illness, would be meaningfully felt by most patients, families, and care partners and that delaying such losses for several months, and potentially longer for a minority of patients treated over several years, would be of meaningful value. These differences in a CDR-SB change of 0.5 would constitute the difference between MCI and dementia, or mild neurocognitive disorder and major neurocognitive disorder in DSM-5.^24,25 The field needs to recognize and further validate these outcomes.

Figure 1, a progression model for repeated measures (PMRM) adapted from Raket,²⁶ illustrates the time savings between a CDR-SB change score at a specific time point and the slowing or delay of disease progression. Slowing of disease progression is often provided as a percentage, and since disease progression is not linear, a 25% reduction in worsening compared to placebo could be postulated for an active treatment group. A 25% reduction is frequently cited as an appropriate benchmark for clinical meaningfulness.^10,27 For example, estimating a 25% slowing of progression in the treatment group, at 12 months a 0.5-point difference between the treatment and placebo groups shows the decline in the treatment group delayed for 3 months (i.e., a slowing of decline or disease progression of 3 months). The same 0.5-point difference seen at 18 months shows a slowing of the decline of 6 months, and at 24 months the slowing of decline is a difference of 7.5 months. In early-stage AD, a slowing of the decline of this magnitude over a 24-month span, reflecting over half a year of delay of disease-related clinical progression, would be considered both interpretable and potentially meaningful by patients, families, care partners, and clinicians.

FIGURE 1.

A progression model for repeated measures (PMRM), adapted from Raket,²⁶ illustrates the time savings between a CDR-SB change score at a specific time point and the slowing or delay of disease progression

3 |. CUMULATIVE BENEFITS AND LONG-TERM TREATMENT

A major issue in addressing the measurement surrounding clinical meaningfulness pertains to the concept of “cumulative benefit” with long-term treatment, and the importance of duration of treatment and follow-up when considering a DMT. As discussed earlier, it is probably not reasonable to expect that the removal of a degree of amyloid plaque over the course of 12 to 18 months would produce a significant change in slope (i.e., slowing of disease progression) over that time frame, although it is just one element in a complex process. However, if the intervention actually perturbs the underlying pathophysiological process, the rate of clinical deterioration may, in fact, be slowed (as illustrated in Figure 2), but the realization of this change in slope may not be apparent for several years following the initiation of therapy. In AD, where current technology allows us to detect evidence of its onset one to two decades before death, a 20% to 30% slowing of the disease initiated in the earlier stages of the disease could mean more time in the less impaired and more functional stages of the disease, as well as a delay in the onset of a later stage (i.e., severe) decline. It is not practical to conduct AD RCTs over one to two decades where we may fully appreciate the impact of early intervention. However, there is mounting evidence that a statistically significant change in clinical outcome measures over the course of an 18 to 24-month clinical trial might actually signify a meaningful change, particularly when projected out over succeeding years.^23,29 As a result, perhaps our expectation of RCTs in the AD space needs to be modified to be more realistic with respect to the underlying pathophysiology and its clinical impact. A challenge is introduced regarding the length of the trial needed to produce convincing benefits in the setting of a devastating disease.

FIGURE 2.

Adapted from Assunção et al.²⁸ Widening drug-placebo difference shows that there is a cumulative benefit with long-term treatment, and this is critical to meaningful benefit

To further complicate matters, AD pathophysiology exists in a complex environment of other medical and neuropathological entities. Since aging is the most prominent risk factor for AD and many other neurodegenerative diseases, the combination of AD pathophysiology with other entities is more common than not. As has been outlined by numerous investigators in recent years, the most common scenario, neuropathologically, at the time of postmortem examination of a cognitively impaired person in his or her 70s or 80s involves amyloid, tau, alpha-synuclein, TDP43, vascular disease, and perhaps other considerations such as CSF dynamics.^30–33 Each of these pathological processes can produce cognitive impairment on its own and could contribute synergistically to cognitive decline with the other pathophysiological elements. Therefore, if one grafts the modest clinical impact expected to be observed over the course of an 18-month trial discussed earlier onto the backdrop of multiple other coneuropathologies, the observed clinical impact of a single intervention is reasonably anticipated to be quite modest. This argument is not meant to “lower the bar” of expectations of AD RCTs; rather, it is meant to view them from a realistic perspective. Again, we are no longer dealing with therapies that are designed to acutely improve cognition and/or clinical function but, rather, to modestly slow the rate of further deterioration. If such therapies are continued long term, and if they sustain their effectiveness at the same “modest” levels to reduce clinical decline, they would be expected to show cumulative benefits that become more readily apparent and meaningful over time.

4 |. CONCLUSION

As the AD scientific community and other stakeholders prepare for data read-outs of several large, late-phase, global RCTs of emerging AD treatments, the Alzheimer’s Association convened a work group of experts to discuss an important and timely issue regarding clinical meaningfulness of results from an RCT and the expectations we place on interpretation of these results. AD pathology is complex, with multiple opportunities for therapeutic engagement. Early intervention over the course of a 1- to 2-year RCT has been shown to slow disease progression and provide clinically meaningful benefits to patients and their families. Although current scales used in RCTs may lack the scoring precision necessary to signify robust statistically significant meaningful change in the early stages of AD (e.g., CDR-SB scoring), we cannot further delay the development of, or patient access to, potentially DMTs while developing and adopting new and improved scales. Also, conducting RCTs over 10 years is neither practical nor necessary.

The review of realistic expectations of what is considered clinically meaningful in an RCT does not suggest that the proposed intervention strategies should be abandoned. In fact, they are vitally necessary to make the presumed improvements in cognition and function that are meant to be derived from interventions on the individual pathological elements. However, two conclusions may be drawn from this position. First, combination therapies will be a necessity to augment the benefit shown with monotherapy. Particularly if monotherapy gains widespread clinical use, the most straightforward approach to assessing combination therapy is a trial of an adjunctive therapy, enrolling people on stable amyloid-targeting monotherapy and randomly assigning them to the addition of an anti-tau drug or placebo. The more we learn about the pathophysiology of AD and build on the success of clinical trials that show clear evidence of slowing disease progression in a broad range of mechanisms of action, the greater the likelihood that we will discover a combination of therapeutic interventions that are effective in certain patients and in particular phase(s) of the disease. Just as we treat hypertension today with diuretics, beta blockers, calcium channel antagonists, angiotensin-receptor blockers, angiotensin-converting enzyme inhibitors, and the like to treat one symptom, elevated blood pressure, we will likely need multiple therapeutic interventions to address complex pathological and cognitive issues of aging. Combination therapy in oncology has also been a very successful approach to cancer treatment (e.g., different treatments are most beneficial at a specific stage of cancer), and these approaches could be very efficacious in AD.^34,35 Second, it is critically important to determine the stage of intervention for this putative underlying pathophysiological process. If DMTs are effective at altering the course of an individual pathological element in the cascade, one would presume that the earlier the intervention, the more likely it is to produce a clinically meaningful outcome. If intervening on amyloid at the plaque stage with associated cognitive impairment demonstrates modest clinical benefit, one would hope that moving to the preclinical, cognitively unimpaired stage when the biomarkers for amyloid are present would be most beneficial. This remains to be demonstrated but does offer hope for the field in the development of future therapeutics. While the concepts related to this manuscript may be useful for ongoing clinical trials, they can be helpful in the design of future clinical trials. Furthermore, the concepts related to clinical meaningfulness hold value for a variety of stakeholders, including regulators (although their role is generally to determine whether a drug is effective based on statistical analyses), payors, and prescribing physicians. But most importantly, these concepts will be of value to patients and their families.

In summary, we are at a very exciting and historical point in AD drug development, and there is evidence that we can now go beyond the modest benefit of approved symptomatic therapies to an intervention that changes the underlying disease and slows its progression. We must convey our realistic expectations about the benefits and risks of interventions for cognition, aging, and AD. Since the degenerative features of AD are sufficiently complex, it is unlikely that any single intervention, be it on amyloid, tau, vascular disease, alpha-synuclein, or TDP-43, is likely to have a large clinical effect. Nevertheless, we need to approach each component with biomarkers and therapies to ultimately determine which combination will have the most meaningful effect.

RESEARCH IN CONTEXT.

1. Systematic Review: The Alzheimer’s Association convened a workgroup of experts to discuss key considerations for interpreting data from cognitive and functional measures and what is considered a clinically meaningful benefit or meaningful slowing of Alzheimer’s disease.

2. Interpretation: Introducing a treatment strategy early in the disease and assessing a meaningful change over the course of a relatively brief 18-month clinical trial are critical to understanding the value of an effective intervention.

3. Future Directions: Discussions on what is considered clinically meaningful change during a randomized controlled trial will help define our expectations of outcomes from therapeutic interventions in AD.