Advancing rare disease measurement through the Rare Disease Clinical Outcome Assessment Consortium

Abstract

There is a significant unmet need to develop and evaluate new treatments for people living with one of approximately 8000 rare diseases. Well-known difficulties in conducting clinical trials (e.g., small samples, wide geographic distribution, heterogeneous symptoms) and developing products for these rare indications persist. Identifying outcomes in rare disease clinical trials remains a hurdle that contributes to the challenges for drug and gene therapy development due to uncertainty about what aspects of a condition to measure for safety and efficacy and often with no regulatory approval precedent. To accelerate rare disease treatments by advancing outcomes measurement, the US Food and Drug Administration (FDA) funded a cooperative agreement to establish the Rare Disease COA Consortium (RD-COAC) in 2019. The RD-COAC officially launched on January 1, 2022, with the mission to enable pre-competitive, multi-stakeholder collaboration aimed at identifying scientifically sound tools and methodologies for collecting clinically meaningful and patient-centric outcomes data in treatment trials for rare diseases. The RD-COAC has four complementary workstreams to advance COA measurement for rare disease clinical trials: (1) Rare Disease COA Resource; (2) Advancing COA Measurement Topic-Focused Working Groups; (3) Rare Disease Discussion Sessions for pre-competitive collaboration and shared learnings among RD-COAC members; and (4) Dissemination. This review provides an overview of the RD-COAC’s activities to date, as well as future directions and opportunities to collaborate.

Plain language summary

Establishment of the Rare Disease Clinical Outcome Assessment Consortium

Approximately 10% of people worldwide are impacted by a rare disease, many in early childhood. Many rare diseases are life-altering or fatal, establishing a need to develop and approve effective medications to improve the lives of those living with a rare disease. However, there are substantial challenges in designing, conducting, and interpreting clinical trials in rare disease populations. To meet these challenges, there has been an unprecedented response within the global scientific, biopharmaceutical industry, regulatory agencies, and patient communities to prioritize rare disease research, accelerate insights, and expedite treatment approvals. These efforts are reflected in pre-competitive collaborations, international basic and translational science centers, rare disease research networks and training programs, data harmonization/sharing platforms, global regulatory agency collaborations, government regulation/legislation, and the increased focus on patient involvement in rare disease drug and gene therapy development. To accelerate rare disease treatments by advancing outcomes measurement, the US Food and Drug Administration (FDA) funded a cooperative agreement to establish the Rare Disease Clinical Outcome Assessment Consortium (RD-COAC) in 2019. The RD-COAC officially launched on January 1, 2022, with the mission to enable pre-competitive, multi-stakeholder collaboration aimed at identifying scientifically sound tools and methodologies for collecting clinically meaningful and patient-centric outcomes data in treatment trials for rare diseases. The RD-COAC has four complementary workstreams to advance COA measurement for rare disease clinical trials: (1) Rare Disease COA Resource; (2) Advancing COA Measurement Topic-Focused Working Groups; (3) Rare Disease Discussion Sessions for pre-competitive collaboration and shared learnings among RD-COAC members; and (4) Dissemination. This review summarizes the RD-COAC’s activities to date, as well as future directions and opportunities to collaborate.

Introduction

It is estimated that more than 350 million people worldwide, approximately 10% of the global population, ¹ half of whom are children, live with a rare disease. Of these rare diseases, fewer than 10% have approved treatments. ² In light of this significant unmet need, there has been an unprecedented response within the global scientific, biopharmaceutical industry, regulatory agencies, and patient communities to prioritize rare disease research, accelerate insights, and expedite treatments. These efforts are reflected in pre-competitive stakeholder collaborations,^3–6 international basic and translational science centers, ⁷ rare disease research networks and scholar training programs,^8–10 data harmonization and sharing platforms,^11–13 global regulatory agency collaboration efforts,^14,15 government regulation and legislation specific to rare disease, ¹⁶ and the increased focus on patient involvement in rare disease medical product development. ¹⁷ This significant unmet treatment need has also been met with unprecedented growth in rare disease drug and gene therapy development and approvals.^18,19

However, the well-known difficulties in conducting clinical trials (e.g., small samples, wide geographic distribution, heterogeneous symptoms) and developing products for these rare indications persist, including challenges measuring clinical benefit.^20–22 Identifying outcomes in rare disease clinical trials remains a hurdle that contributes to the challenges for drug and gene therapy development due to uncertainty about what aspects of a condition to measure for safety and efficacy and often with no regulatory approval precedent.^23–25 While different categories of biomarkers can expedite drug and gene therapy development,^26,27 for biomarkers to be accepted for clinical development for regulatory submission, they need evidence that they predict or are expected to be correlated with a clinical outcome. ²⁰ Some clinical outcomes may be measured as clinically meaningful events (e.g., survival); however, it is often the case that clinical outcomes are measured using clinical outcome assessments (COAs).^20,28 Even when well-established predictive and/or prognostic biomarkers are accepted for a rare disease, COAs will continue to contribute valuable information about how persons/people with lived experience feel, function, and survive in response to an investigational product. Yet, the current state of measurement for clinical trial efficacy endpoints is that, for most rare diseases, COAs have not yet been identified, developed, or modified. There is a clear need for thought leadership from subject matter experts in rare disease drug and gene therapy development to advance COA-based endpoint selection methods, implementation, and interpretation.

Rare Disease COA Consortium

Advancing the measurement of rare disease outcomes in clinical drug and gene therapy development trials was identified as a significant unmet need. This led to funding a cooperative agreement in 2019 by the FDA’s Center for Drug Evaluation and Research to establish the Rare Disease COA Consortium (RD-COAC).

The Critical Path Institute’s (C-Path’s) Patient-Reported Outcome Consortium served as an incubator for a Rare Disease COA Subcommittee prior to the maturation of a pre-competitive, multi-stakeholder consortium within C-Path’s COA Program. This pre-consortium period facilitated alignment from FDA, interested biopharmaceutical companies, and rare disease clinical experts on the key priorities for the consortium. During the pre-consortium period (September 2019–December 2021), members of the Rare Disease Subcommittee included representatives from the FDA, C-Path, the National Organization for Rare Disorders, the National Center for Advancing Translational Science, the Patient-Centered Outcomes Research Institute, and 25 biopharmaceutical firms. This subcommittee determined that (1) a domain-focused approach would be used to identify COAs that may be suitable for COA-based endpoints in rare disease clinical trials; (2) clinical measurement needs across multiple diseases would be the focus of the resource rather than taking a disease-specific core outcome set approach; and (3) initial efforts would focus on (non-oncologic) pediatric populations given the measurement complexities in these populations. The pre-consortium subcommittee established the methodology for developing the Rare Disease COA Resource (see discussion below) with the support of three subject matter experts: two pediatric neuropsychologists and a pediatric speech-language pathologist, all with extensive experience administering neurodevelopmental assessments in pediatric rare disease populations.^29,30

The RD-COAC officially launched on January 1, 2022, with the mission to enable pre-competitive, multi-stakeholder collaboration aimed at identifying scientifically sound tools and methodologies for collecting clinically meaningful and patient-centric outcomes data in treatment trials for rare diseases. FDA continues to play an active role in all RD-COAC activities, but funding for the RD-COAC comes from tiered membership fees from biopharmaceutical sponsors with select non-profit and patient advocacy group participation. The RD-COAC has four complementary workstreams to advance measurement science for rare disease clinical trials: (1) Rare Disease COA Resource; (2) Advancing COA Measurement Topic-Focused Working Groups; (3) Rare Disease Discussion Sessions for pre-competitive collaboration and shared learnings; and (4) Dissemination.

Rare Disease COA Resource

The ongoing development of the Rare Disease COA Resource is a central pillar of the RD-COAC efforts and its mission to advance measurement science for rare disease clinical trials. The RD-COAC Resource creates efficiencies for medical product development by freely providing all rare disease stakeholders with information on a published, curated COA library of drug development tools that may apply to a broad range of rare disease drug development programs. While a listing in the Rare Disease COA Resource is not an endorsement that a COA will meet regulatory evidence standards, it is an essential resource for streamlining the identification of COAs that matter to persons/people with lived experience in rare disease studies. By including COAs covering common concepts of interest across many rare diseases, the Rare Disease COA Resource may be used to identify COAs available to measure outcomes of interest in patient registries and natural history studies or clinical development programs.

COAs included in the Rare Disease COA Resource represent the tools that are the most commonly used in current rare disease research and tools that have been published in the literature and were available to examine against evidentiary criteria. The Rare Disease COA Resource, therefore, only captures existing tools and may not include COAs that have been developed recently but currently lack publications to support their use. The Rare Disease COA Resource will be updated periodically to identify new tools that may warrant inclusion. The Resource indicates the level of evidentiary standards and gaps for each COA included, to help guide users as to where there may be limitations to using a standard-of-care COA. In addition, some historical COAs have been included because, although they may not meet all evidentiary standards, they are the standard of care and allow for comparisons between individual rare disease populations and normative data.

The Rare Disease COA Resource is not a core outcome set. Core outcome sets are an agreed-upon, standardized set of measures that should be gathered and reported as a minimum in all clinical research in specific areas of health or healthcare (please reference: https://nationalhealthcouncil.org/blog/blog-core-outcome-sets-what-are-they/). It substantively differs from the FDA COA Compendium as discussed in the methodology section. ³¹

For each domain, multiple COAs have been identified. This is because the COAs selected for individual research programs still need to be carefully considered based on the suitability for the target population, coverage of concepts, applicability of the concept to the anticipated mechanism of action for the investigational product, abilities of each trial population, etc. Good measurement science principles still need to be applied to the selection of the COAs from the Rare Disease COA Resource to ensure the optimal match with research targets can be met.

Rare Disease COA Resource methodology

The methodology guiding the Rare Disease COA Resource included a subject matter expert committee of the RD-COAC comprising clinicians, academics including clinical researchers, industry, NIH, and FDA representatives who collaborated in the development process (Figure 1).

Figure 1.

Rare Disease COA Resource: ongoing development process.

COA, Clinical Outcome Assessment.

Following domain prioritization, selection, and scoping (i.e., clarifying the domain definition, and defining search methodology), a landscape analysis was conducted for COAs via a literature search from published clinical trials, professional meeting presentations, clinical practice guidelines, and natural history studies. This COA review and synthesis method used was an adaptation of a well-established systematic literature review methodology. ³² Criteria were determined for the selection of COAs for an in-depth analysis of available measurement evidence (i.e., gap analysis). A second literature search was conducted to identify and retrieve available evidence for the selected COAs. A consensus process between each Advisory Panel and the Rare Disease COA Resource Development Subcommittee determined the final COAs to include in the Rare Disease COA Resource. Data extraction elements and additional information on criteria used to evaluate COAs identified in the landscape analysis and gap analysis are provided in the Supplemental material. Evidence from this second literature analysis for each COA included in the Resource can be viewed for each tool individually or in comparison across several tools for a given outcome to support COA selection for an individual research program (https://rdcoas.c-path.org).

Rare Disease COA Resource domains

The inaugural broad domain developed for the Rare Disease COA Resource was “pediatric daily function,” comprising more specific functions of gross and fine motor functioning, self-care, and expressive and receptive communication (Table 1). These domains were selected based on input from FDA, biopharmaceutical member firms, and patient advocacy input on what the highest priority domains were. The Rare Disease COA Resource development will be an iterative, ongoing domain-specific development to expand the Rare Disease COA Resource based on voting member prioritization. Research is already underway for the next measurement topics of pain (e.g., pain severity, pain interference) and sleep (e.g., sleep disturbance, sleep impact), still in pediatric non-oncologic populations.

Table 1.

Number of COAs identified by domain.

Domain	COAs identified (n) a
Fine motor function	16
Gross motor function	34
Self-care	15
Communication/language	23

^aCOAs could be classified across multiple domains.

COA, clinical outcome assessment.

Advancing COA measurement in rare diseases: Topic-focused working groups

While each rare disease clinical trial is unique, there are common methodological challenges (e.g., measurement, implementation) and potential solutions across programs. The RD-COAC advances measurement science by identifying solutions for methodological and measurement challenges that consortium members have identified as hurdles to rare disease clinical trial implementation. Completed working groups evaluated strategies for measuring heterogeneous symptoms and impacts in clinical trials and identified regulatory pathways toward approved therapies using those measurement strategies, with a particular focus on how these strategies could be best applied in rare disease trials. ³³ Another working group developed COVID-19 mitigation strategies in pediatric rare disease clinical trials resulting in a public workshop. ³⁴ Ongoing working groups are advancing the science related to (1) crafting endpoints for gene therapy clinical trials, including lessons learned from historical global approvals, (2) the justification and practical applications of qualitative evidence within clinical trials, and other topics with forthcoming publications from these efforts.

Rare Disease Discussion Sessions

The Rare Disease Discussion Sessions series was launched to promote education and share learnings among member firms and other consortium members to expedite innovations in rare disease clinical trial measurement science. This internal learning series capitalizes on the pre-competitive environment by promoting collaborative learning from real-world experiences in rare disease clinical trial research, highlighting new publications relevant to the membership and providing opportunities to work collaboratively. Some of the topics covered have included discussions from subject matter experts on how a “most bothersome symptom” framework could be implemented in a rare disease trial with a focus on the statistical implications, presentations from patient advocacy groups on their registry programs, presentations from NIH colleagues on comparing ability and norm-referenced scores, including growth scale values to interpret results from a COA commonly used in pediatric clinical trials, and presentations from other rare disease consortia programs within C-Path. Materials presented during these meetings are only available to consortium members as a direct benefit of membership.

Dissemination

As a leader in rare disease COA measurement science, a critical workstream for the RD-COAC is the dissemination of work products developed by the RD-COAC for access by all rare disease stakeholders at professional meetings and publications.^{29–30,33–40}

Discussion

The launch of the Rare Disease COA Resource is the first database of its kind. It is a cost-free, publicly available database of COAs that have been selected by consensus from industry, clinical experts, regulatory experts, and patient/caregiver representatives, reviewed against current regulatory evidentiary expectations, and specifically identifies COAs that may potentially support endpoints in clinical trials across multiple rare diseases. The cost (e.g., time, expenditure) of identifying relevant COAs for a single domain per disease can easily exceed $200,000 (USD); thus, the Rare Disease COA Resource may reduce efforts for COA identification by individual pharmaceutical companies, researchers, and patient advocacy groups and provides a selection of COAs that have been selected by a multi-stakeholder group of experts. Furthermore, results of the available evidence and gap analysis for each tool have been formatted in a way that information needed to populate a COA dossier for a regulatory submission can be easily downloaded. The Resource facilitates comparisons across COAs to support the selection of a COA that best suits a specific disease or research program. However, the Resource does not and cannot generate endpoints.

Considering the Rare Disease COA Consortium’s relatively recent launch in January 2022 and the time from clinical trial design to regulatory approval, its impact cannot be measured in approved products. As a form of a COA library, a direct link between the Resource and product approvals is not an expected outcome as it would not be for any COA database. However, since its launch, several hundred individuals representing academia, patient advocacy groups, industry, and other groups have accessed the Rare Disease COA Resource. Continued monitoring and discovery of the impacts of this Resource are an ongoing effort of the RD-COAC.

The consortium endeavors to make outputs from the Rare Disease COA Consortium publicly available through publications, webinars, and speaking engagements at professional conferences. Materials in development by the consortium are confidential until dissemination to ensure thorough internal review and discussion, resulting in products that offer the best recommendations on behalf of the consortium. As a pre-competitive, multi-stakeholder collaboration that includes industry, clinical, academic, regulatory, and patient advocacy stakeholders, consortium members discuss topics relevant to drug developers and rare disease researchers; interact with regulators on non-application specific methodologic challenges; select which domains of interest will be included in expansions of the Rare Disease COA Resource; provide leadership on working groups or other collaborations; and participate in decision-making for future methodological directions.

Conclusion

The demonstrated success of the RD-COAC lies in its ability to bring scientific thought leaders together to enact change in the rare disease research field. The importance of incorporating the patient voice into rare disease clinical trials toward the development of meaningful outcomes necessitates continued collaboration among patient advocacy, regulatory bodies, academia, and government/industry sponsors. While the initial domains covered by the Rare Disease COA Resource focus on pediatric populations, the Resource will be iteratively expanded to ultimately cover domains of interest across both pediatric and adult populations. These efforts will be important given the number of new treatment modalities (e.g., anti-sense oligonucleotides, immunology, and gene therapies) that have the potential to alter the disease course, improve life expectancy, and modify the background therapy of future clinical trials. The RD-COAC will continue to tackle existing methodologic challenges and address new opportunities for COA-based measurement to meet the ongoing evolution of regulatory science. The enduring success of the RD-COAC relies on member support; researchers, patient advocacy groups, and biopharmaceutical industry members can contact the RD-COAC for more information on how to engage and expand meaningful collaborations across the rare disease COA research community.