Cell and Gene Therapy Trials: Are We Facing an ‘Evidence Crisis’?

1. Introduction

Recent advances in cell and gene therapies are offering new, effective treatment options for patients with ultra-rare and life-threatening conditions. For example, in 2017, the United States' Food and Drug Administration (FDA) approved two Chimeric Antigen Receptor (CAR)-T cell therapeutics: tisagenlecleucel (Kimryah) for pediatric and young adult patients with acute lymphoblastic leukemia, and axicabtagene ciloleucel (Yescarta) for patients with relapsed or refractory large B-cell lymphoma [1], [2]. Both of these CAR-T therapies have also been approved in the EU.

Although there is great potential for cell and gene therapies, most of those currently approved for clinical use, particularly CAR-T cell products, were evaluated in pivotal trials of less than 100 patients [3]. Having such limited data means that there may be a larger-than-usual gap between what is known about the safety and efficacy of these therapies and what clinicians need to know to reliably judge their benefits; and what payers needs to know to establish sound reimbursement policies. Integrating cell and gene therapies into routine clinical practice will therefore require robust mechanisms to report, track, and coordinate further evidence development—to ensure that clinicians and payers have access to the evidence they need [4].

In what follows, we outline some of the challenges inherent in cell and gene therapy evidence development. We then close by presenting a novel model of evidence synthesis that we believe could help to overcome these challenges.

2. Smart Trial Designs and Uniformity in Reporting

Randomized Control Trials (RCTs) are widely accepted as providing the most reliable evidence when evaluating the safety and efficacy of a new intervention and for obtaining regulatory approvals [5]. However, only half of the 12 currently-approved Advanced Therapy Medicinal Products (ATMPs) in Europe were tested in RCTs, and many of these used small sample sizes (ranging from 99 to 512). The other half were tested only in single-arm studies (this is according to the European public assessment reports (EPAR) on the European Medicines Agency (EMA) website).

But although it is tempting to insist that all new therapeutics should be evaluated in rigorous RCTs, this may not actually be possible for rare diseases that have few or no effective treatment options available [6]. It is therefore important to consider other measures that can help to improve the strength of evidence for cell and gene therapies [7]. For example, systematic reviews and meta-analyses pool data from multiple studies and registries, and can be used to provide more precise estimates of effect or risk, particularly when results from individual trials are inconclusive.

However, the utility of systematic reviews and meta-analyses is contingent on the quality of trial reporting [8]. Trial reports that do not sufficiently describe their protocols, study populations, interventions, or outcomes must often be excluded from evidence syntheses. Indeed, valid pooling of data across studies is difficult without the complete patient covariate profiles, documentation of patient flow through the study (e.g., explaining drop-outs or reasons for missing outcome data), and information about the delivery, dosage, and follow up procedures. In the particular case of cell and gene therapies, information about the manufacturing process and release criteria are also needed to adequately correlate between clinical outcomes and unique product characteristics.

In Table 1, we summarize a minimal set of questions that must be answered in reports of cell and gene therapy trials to allow for an adequate interpretation of the result and valid pooling of data. Similarly, we would also encourage greater standardization in experimental design in this domain, and would welcome additional guidance from entities such as NICE and the Cochrane Collaboration on best practices for synthesizing information from single-arm studies.

Table 1.

Questions to be addressed when reporting cell and gene therapy clinical trials.

1	Population	What was the condition (or set of conditions and variant definitions) of interest? What were the inclusion/exclusion criteria for recruitment? What were the baseline characteristics of trial participants?
2	Sampling	How were patients identified and selected for the trial? Do the included patients reflect a convenience sample or random sample of the broader patient population?
3	Outcome	What were the primary endpoints of the study? Why were these particular endpoints chosen? How do the chosen outcomes align with the study aims and objectives?
4	Intervention	What interventions were studied (including dose, schedule, manufacturing parameters, co-interventions, etc.)?
5	Randomization	Was there a suitable control group, making it ethical to randomly assign subjects to either control or experimental arms? If not, what other steps were taken ensure valid results were obtained?
6	Analysis	What was the pre-specified hypothesis and analysis plan? How was missing data handled?
7	Adverse events	What were the clinical characteristics—e.g., severity, frequency, and timing—for all recorded adverse events?
8	Setting	How do characteristics of the trial setting—e.g., readiness of the trial unit and level of training of healthcare professionals—impact the clinical outcome of these interventions?

3. Innovative Models for Evidence Mapping of Clinical Trial Data

But although systematic reviews and meta-analyses are powerful tools of evidence synthesis, they have limitations. For example, systematic reviews can take more than a year to complete and publish. This often means that by the time a review finally appears in print, it may already be out-of-date. Thus, for research domains with high amounts of activity, such as cell and gene therapy, the traditional approaches to systematic reviews struggle to keep pace with new evidence generation.

Many traditional meta-analytic approaches are also not well-suited to summarizing heterogeneous bodies of evidence. As we noted above, standardization and reporting requirements can help to reduce some kinds of trial heterogeneity and thereby make meta-analyses more useful. However, there are also reasons why a heterogeneous body of data can be desirable. For example, in the early phases of clinical development, it may be valuable to test a new cell therapy across a range of patient populations in an effort to discover those patients for whom the new therapy offers the best benefit/risk balance [9]. While it may not be valid to pool all this data into single point estimate, it can nevertheless be extremely useful to survey the entire evidence landscape to see how outcomes vary across a diversity of patient populations.

Fortunately, there are emerging methods of evidence synthesis designed to support decision-makers facing heterogeneous data. Evidence mapping is one such method, which systematically depicts a broad (often heterogeneous) body evidence in order to identify knowledge gaps and opportunities for future research [10]. In contrast to traditional systematic reviews and meta-analyses, whose results are typically presented in the form of dense tables and figures, evidence maps are most often designed to be user-friendly and interactive [10].

The Campbell Collaboration, which promotes the use of systematic reviews to inform evidence-based policy and practice, frequently reviews broad and heterogeneous bodies of data and publishes the results of its reviews as evidence maps. We believe this same systematic approach to mapping the diversity of evidence across a domain can (and should) be applied to medical evidence and clinical decision-making—particularly cell and gene therapies. In fact, organizing and presenting evidence maps in an interactive, online format (as Campbell does) could be used to track evidence about new therapies in near real-time—thereby overcoming one of the major drawbacks of existing evidence synthesis approaches.

To conclude, although cell and gene therapies have great potential to revolutionize treatments of an array of diseases, the supporting clinical evidence for long-term safety and efficacy is still limited. We believe that more comprehensive reporting is needed to allow for rapid and reliable evidence development and to promote valid pooling of data. We also believe that innovative tools, such as interactive and dynamic evidence maps, could be used to track the evidence in real-time and allow the field to make better sense of the heterogeneity across the total evidence landscape. Indeed, we think that combining these approaches would represent a significant step forward, increasing confidence in the existing evidence and paving the way to a smooth integration of these innovative treatments into mainstream clinical practice.