Mitigating Deficiencies in Evidence during Regulatory Assessments of Advanced Therapies: A Comparative Study with Other Biologicals

Abstract

Advanced therapy medicinal products (ATMPs) comprising cell therapy, gene therapy, and tissue-engineered products, offer a multitude of novel therapeutic approaches to a wide range of severe and debilitating diseases. To date, several advanced therapies have received marketing authorization for a variety of indications. However, some products showed disappointing market performance, leading to their withdrawal. The available evidence for quality, safety, and efficacy at product launch can play a crucial rule in their market success. To evaluate the sufficiency of evidence in submissions of advanced therapies for marketing authorization and to benchmark them against more established biological products, we conducted a matched comparison of the regulatory submissions between ATMPs and other biologicals. We applied a quantitative assessment of the regulatory objections and divergence from the expected data requirements as indicators of sufficiency of evidence and regulatory flexibilty, respectively. Our results demonstrated that product manufacturing was challenging regardless of the product type. Advanced therapies displayed critical deficiencies in the submitted clinical data. The submitted non-clinical data packages benefited the most from regulatory flexibility. Additionally, ATMP developers need to comply with more commitments in the post-approval phase, which might add pressure on market performance. Mitigating such observed deficiencies in future product development, may leverage their potential for market success.

This study is the first to compare cell and gene therapies’ regulatory submissions to other biological submissions. We observed that at the time of product launch, cell and gene therapies suffer more evidential shortcomings, which increase the risk of market failure and hinder their integration in routine clinical practice.

Introduction

The pharmaceutical industry is shifting focus toward disease areas with high unmet medical needs such as oncology and rare diseases.¹ Advancements in biotechnology have enabled such a shift by introducing novel therapeutic approaches, particularly cell therapies, gene therapies, and tissue-engineered products, known in the European Union (EU) as advanced therapy medicinal products (ATMPs).² To date, 14 ATMPs have received marketing authorization (MA) in the EU; however, 5 have subsequently been withdrawn from the market. Most recently, Zalmoxis was withdrawn in October 2019 after unfavorable results reported from the post-approval phase III clinical trial,³ a requirement for conditional MA, which was obtained in 2016. Reimbursement and commercial issues, limited market demand and manufacturing problems contributed to the other withdrawals.^4,5 It is expected that pharmaceutical development programs generate safety and efficacy evidence that is not only sufficient to support MA decisions but also decisions made by health technology assessment (HTA) agencies and other relevant stakeholders.6, 7, 8, 9 However, such alarming numbers of withdrawals can indicate that there is a gap between the evidence presented for MA and the evidence deemed sufficient for market and patient access.

ATMPs are also biological medicinal products,¹⁰ a family of products extracted from or manufactured from biological sources. These products include monoclonal antibodies, enzymes, and hormones, the majority of which are produced by recombinant DNA technologies (hereafter referred to as other biologicals). After 30 years of experience with recombinant proteins, their development path has become well established.¹¹ In contrast, ATMPs are a more diverse group of products, often with little in common with each other, and many of them are a poor fit for existing development and business models. This situation challenges developers to identify an appropriate development strategy and determine how much evidence is needed to increase the probability of success in acquiring MA and achieving commercial viability.¹²

The expected evidence that should be collected on a therapeutic candidate during its development for inclusion in a MA application (MAA) is laid down in Annex I of Directive 2001/83/EC (hereafter referred to as data requirements). Sections for specific types of therapeutics, such as ATMPs, are provided in the Annex to acknowledge the complexity of these products and guide developers on how to comply with additional requirements, whenever applicable. Moreover, to emphasize the need for flexibility when developing and testing ATMPs, which are very diverse in nature, Annex I encourages the use of a risk-based approach.^10,13 Such risk analysis can be conducted by the applicant to determine the extent of quality, non-clinical, and clinical evidence to be included in the MAA, and to provide scientific justification when deviating from the requirements of this Annex (hereafter referred to as divergence).¹⁰ However, the degree of divergence of ATMPs from the expectations in Annex I and its effect on the sufficiency of the evidence and ability to reach a conclusion on the overall risks and benefits of the product have not been thoroughly investigated.

Previous studies have attempted to investigate the evidence in ATMP submissions through the quantification of objections raised by regulatory authorities during the assessment procedure of MAAs.14, 15, 16, 17 de Wilde et al.¹⁴ and Carvalho et al.¹⁵ relied on the European public assessment report (EPAR), a document published by the European Medicines Agency (EMA) for all submissions that reach the first stage of assessment, whether approved, refused, or withdrawn. Barkholt et al.¹⁶ at the EMA quantified the objections for the first 20 MAAs for ATMPs. The study by de Wilde et al.¹⁴ showed considerable discrepancies in the results compared to the other two studies^15,16 that performed a more thorough analysis, with Barkholt et al. deemed to be the most reliable data source, as they relied on internal EMA data.¹⁶ Nevertheless, to benchmark the suffeciency of submitted evidence for ATMPs, a comparision with more established biological products is needed, as suggested by Bravery et al.¹⁷ This approach can help ATMP developers mitigate deficiencies in evidence by identifying the weaknesses in existing submissions and understanding the impact on post-approval commitments and performance. To our knowledge, no existing research has attempted to assess the sufficiency of evidence presented for ATMPs in MAA submissions against other biologicals, by not only the quantification of objections, but also by identifying areas of regulatory flexibility, where applicants diverged from data requirements in Annex I.

In this study, we conducted a retrospective, head-to-head, nearest neighbor matched comparison of submitted evidence between ATMPs and other biologicals using data extracted from the EPARs. We accounted for several confounding factors that may impact the extent and the source of the evidence expected in the MAA by matching them in both groups. The data requirements provided in Directive 2001/83/EC, Annex I, were clustered into four evidence domains: the manufacturing and quality testing domain, the experimental design and conduct of studies domain, the efficacy and mode of action (MoA) domain, and the safety and toxicity domain. We then employed the quantitative assessment of the objections and divergence in each domain as indicators of evidence sufficiency and compared them between both groups. The differences in the timing of addressing the detected objections between the authorized cohorts were then explored. Finally, we investigated the possible reasons for the observed differences in evidence sufficiency.

Results

Retrieval and Characteristics of ATMP Submissions

Screening of 1,604 submissions (data cutoff, July 1, 2019) in the EMA databases (authorized or refused submissions, 1,382; withdrawn submissions, 222) identified 22 ATMP submissions (Tables 1 and S1). Out of the 22 submissions, 12 were for gene therapy products (55%, including genetically modified cells), 6 were for tissue-engineered products (27%), and 4 were for somatic cell therapy products (18%). Products that contained autologous cells were 11/22 (50%), while 3/22 products contained allogeneic cells (18%). The first submission was for Cerepro in 2005, while the last identified submission was in 2018 for Zynteglo. The average number of ATMP submissions per year was 1.6 (standard deviation [SD], 0.9; range, 0–3). MA was granted to 14/22 submissions (Table 1), 10 of which were full MA (72%), 3 were conditional MA (CMA) (21%), while 1 (Glybera) was authorized under exceptional circumstances (7%). 21/22 (95%) EPARs were available since one product (Raligize) was withdrawn before the end of the first stage of evaluation (day 120), meaning that no EPAR was released. Out of the 14 approved ATMPs, 5 have been subsequently withdrawn. The screening of the EMA databases and selection of the ATMP submissions is depicted in (Figure S1).

Table 1.

Basic Characteristics of the Matched Cohorts

		ATMPs/ Total (N = 22)	ATMPs/ Matched (n = 17)	Other Biologicals/Matched (n = 17)
MAA outcome (%)	authorized	14 (64)	12 (71)	12 (71)
	failed (refused and/or withdrawn)	8 (36)	5 (29)	5 (29)
MA type (%)	full authorization	10 (45)	10 (59)	10 (59)
	conditional marketing authorization	3 (14)	1 (6)	1 (6)
	marketing authorization under exceptional circumstances	1 (5)	1 (6)	1 (6)
	withdrawn (pre-approval)a	7 (32)	4 (24)	4 (24)
	refused	1 (5)	1 (6)	1 (6)
Orphan designation (%)		13 (60)	11 (65)	11 (65)
Disease area (%)	non-hematological malignant neoplasms	7 (32)	5 (29)	5 (29)
	musculoskeletal diseases	4 (18)	4 (24)	4 (24)
	hematological malignant neoplasms	3 (14)	3 (18)	3 (18)
	endocrine, nutritional, and metabolic diseases	2 (9)	2 (12)	2 (12)
	digestive system diseases	1 (5)	1 (6)	1 (6)
	eye diseases	3 (14)	1 (6)	1 (6)
	diseases of blood, blood-forming organs, and certain immune disorders	2 (9)	1 (6)	1 (6)

MAA, marketing authorization application; MA, marketing authorization.

^aWithdrawn refers to the withdrawal of the marketing authorization application before issuing a final opinion from the Committee for Medicinal Products for Human Use (CHMP).

Retrieval and Characteristics of the Matched Biological Products

The same EMA databases were screened to identify suitable matches to ATMPs from other biologicals. In total, 17/21 (81%) ATMPs were matched to other biologicals submissions (Tables 1 and S2) and compared statistically for objections and divergence. In the authorized ATMP cohort, 12/14 (86%) ATMPs were matched to other authorized biologicals. Two products (Zynteglo and Holoclar) could not be matched, as they received a CMA, and biological products with a CMA in the same disease areas (blood diseases and eye diseases, respectively) could not be identified. In the failed authorization cohort, 5/7 (71%) ATMPs were matched. Contusugene Ladenovec Gendux (CLG) and OraNera could not be matched due to the unavailability of other withdrawn biological products for eye diseases and non-hematological malignancies (not orphan), respectively. Of the 17 matched biologicals, 16 were recombinant products (96%), while the remaining product (Oncophage) was an autologous tumor-derived protein-peptide complex (6%). The 16 recombinant products, comprised, nine monoclonal antibodies (56%), three enzymes (19%), three hormones, cytokines, or growth factors (19%) and one coagulation factor (6%). Out of the 12 approved matched biologicals, only 1 has been subsequently withdrawn. The matching characteristics of the ATMPs and the other biologicals are summarized in Table 1.

To examine whether each ATMP and matched biological underwent the regulatory evaluation at a close time frame, the duration between the dates of the regulatory decisions (authorization, withdrawal, or rejection) for each matched pair was calculated. In the authorized cohorts, the average duration between the date of authorization of matched pairs was 15.6 months (SD, 21.8 months; range, 0–67). In the failed cohorts, the average duration between the withdrawal or rejection date of matched pairs was 41.4 months (SD, 30.9 months; range, 11–86).

Comparing ATMP Regulatory Submissions to Matched Biologicals

The available information in the EPARs on the objections raised on the submitted evidence was then extracted and sorted according to the corresponding evidence domains as defined (Table S3). When comparing the authorized matched paired products (n = 24), the total number of the identified objections in the EPARs of the ATMPs was significantly higher (p = 0.013) (Table 2; Figure S2). When comparing the objections in each evidence domain, objections in the experimental design and conduct of studies domain were significantly higher in authorized ATMPs (p = 0.021). Furthermore, a greater number of objections were raised on the evidence of efficacy and MoA in authorized ATMPs (p = 0.031) (Table 2; Figure S2). In contrast, no significant differences were observed in the product manufacturing and quality domain (p = 0.186) or issues related to product safety (p = 0.727) (Table 2; Figure S2). For the failed submissions (withdrawn or rejected, n = 10), no statistically significant differences were found in either the total number of objections or within any of the four domains (Table 2; Figure S3).

Table 2.

Matched Comparison of Objections between ATMP and Biologicals Submissions

Evidence Domain	Differences in Objections between Successful ATMPs and Biologicals Submissions (n = 24)		Differences in Objections between Failed ATMPs and Biologicals Submissions (n = 10)
	Z	p (Two-Tailed)	Z	p (Two-Tailed)
Manufacturing and quality	−1.380	0.186	−0.674	0.625
Experimental design and conduct of the studies	−2.221	0.021∗	−0.674	0.625
Efficacy and MoA	−2.108	0.031∗	−0.137	1
Safety and toxicity	−0.431	0.727	−0.552	0.750
Total number of objections	−2.396	0.013∗	−0.674	0.625

∗p < 0.05. p values were determined by a Wilcoxon signed-rank test.

The impact of the regulatory flexibility on the evidence was evaluated by estimating the degree of divergence from the data requirements and then comparing them between groups. This was achieved by quantifying the studies that were not submitted in the application, as stated in the EPARs. When comparing the authorized cohorts, in total, significantly more divergence was detected in the EPARs of the ATMPs as compared to the other biologicals (p = 0.0001) (Table 3; Figure S4). Divergence in authorized ATMPs was significantly higher than in other biologicals, in the safety and toxicity evidence domain (p = 0.006), as well as in the clinical efficacy and MoA domain (p = 0.0001) (Table 3; Figure S4). Despite the application of more novel technologies and methods for ATMP manufacture and testing as compared to other biologicals, no divergence from the data requirements was detected in this domain. Additionally, no significant difference in divergence was found in the experimental design and conduct of studies evidence (p = 0.063), despite being greater in authorized matched ATMPs than in matched biologicals (Z = −2.081) (Table 3; Figure S4). No statistically significant differences were observed between the failed authorization cohorts (n = 10) (Table 3; Figure S5).

Table 3.

Matched Comparison of Divergence between ATMPs and Biologicals Submissions

Evidence Domains	Differences in the Divergence between Authorized ATMPs and Biologicals Submissions (n = 24)		Differences in the Divergence between Failed ATMPs and Biologicals Submissions (n = 10)
	Z	p (Two-Tailed)	Z	p (Two-Tailed)
Experimental design and conduct of the studies	−2.081	0.063	0	1.000
Efficacy and MoA	−3.070	0.0001∗	−1.633	0.188
Safety and toxicity	−2.669	0.006∗	−1.214	0.313
Total number of divergence	−3.063	0.0001∗	−1.483	0.188

∗p < 0.05. p values were determined by a Wilcoxon signed-rank test.

Distribution of Objections across ATMPs Compared to Matched Biologicals

The distribution of the objections among the products and evidence domains revealed a clear heterogeneity in the distribution within the ATMP cohort (Figure 1). Most of the objections in both groups were concentrated in the manufacturing and quality domain, followed by the experimental design, and then the efficacy and safety domains. The spread of the objections across the products was greater in the ATMPs for most of the data requirements (Figure 1).

Figure 1.

Heatmaps for the Distribution and Number of Objections among the Advanced Therapy Medicinal Products (ATMPs) and Matched Other Biologicals

The data requirements were clustered into four evidence domains (left y axis). The objections were then identified from the European public assessment reports (EPARs) and sorted to the relevant data requirement. The data requirements are arranged (top-downward) in each domain according to the frequency of objections in ATMP submissions. The total number of objections identified in each EPAR is shown on the top x axis. The frequency of objections and concerns across the products in each data requirement is shown on the right y axis of each heatmap.

The most commonly identified objections in ATMP submissions were on compliance with good clinical practice (GCP) and clinical trial protocols (Figure 1, domain II, row 1). Such objections were due to substantial changes in the trial protocols, inadequate documentation of studies, and GCP non-compliance. These issues were not detected as frequently in the EPARs of the other biologicals (Figure 1, domain II, row 1). Another common objection for ATMPs was related to the efficacy results of the main clinical studies (Figure 1, domain III, row 1). Out of the 12 ATMPs with such detected objections, 7 were successful submissions.

Objections in the manufacturing and quality domain were mostly related to validation of the analytical methods, design and control of the manufacturing process, and comparability (Figure 1, domain I, rows 1–3). Most objections in the design and control of the manufacturing of ATMPs were due to deficiencies in microbiological control (8/12, 67% of the products). Other notable manufacturing objections were related to the choice and justification of the analytical methods (Figure 1, domain I, row 5). The most frequent reason for these objections was the choice of the potency assays (8/11, 73%). Objections around characterization and specifications of ATMPs were also common; however, they were slightly more common in other biologicals (Figure 1, category I, row 4). Safety-related objections were not common and were closely similar in both cohorts.

Main Points of Divergence in ATMP Submissions Compared to Other Biologicals

Sources of divergence were primarily identified in non-clinical studies and, to a lesser degree, in clinical studies (Figure 2). The inability to undertake in vivo toxicity studies such as toxicokinetics, reproduction toxicity, local tolerance, and, in some cases, carcinogenicity studies in the ATMP safety and toxicity domain led to a greater number of divergences (Figure 2). Moreover, a full understanding of MoA was not achievable by conducting animal studies, particularly in cell-based product submissions. Difficulties in the application of good laboratory practice (GLP) principles in non-clinical studies of ATMPs has led to the acceptance of non-compliant studies in the submissions, a divergence not seen with other biologicals (Figure 2).

Figure 2.

Average Numbers of Divergences in Each Data Requirement per Submission across Authorized and Failed ATMPs and Matched Other Biologicals

Divergence from the regulatory data requirements for marketing authorization applications laid down in Annex I of Directive 2001/83/EC was assessed through the quantification of omitted studies in the EPARs. Regardless of the approval status, differences in divergence are evident in the non-clinical toxicity studies and clinical pharmacokinetics and biodistribution (PK/BD) studies between ATMPs and other matched biologicals. Error bars represent the standard error of the mean (SEM). (A) Authorized ATMPs and matched other biologicals (Blue). (B) Failed ATMPs and matched other biologicals (Red).

The absence of pharmacokinetics/biodistribution studies in human subjects (Figure 2) resulted in a significantly higher number of divergences for ATMPs (especially those approved). Absorption, distribution, metabolism, and excretion studies are not expected to be conducted in the case of ATMPs, but other studies such as target organ distribution, migration, and persistence were not conducted in human subjects for some of the products. In those cases, the study was not technically possible, and the available non-clinical evidence was considered sufficient. Furthermore, for only 6/17 (35%) of ATMPs, dose-escalation studies were conducted, while for 15/17 (88%) of other biologicals, traditional dose-escalation studies were carried out.

Differences in Solving the Raised Objections between the Matched Cohorts

Raised regulatory objections can be solved during the MAA procedure with the submission of new data, additional analysis, additional risk minimization measures, or modifications of the summary of product characteristics. Where such solutions are not possible during the procedure and the issue does not preclude approval, applicants can be asked to commit to solving the outstanding issues after approval through submission of more data on the quality, safety, or efficacy of the product. When comparing the approaches to address outstanding objections in successful applications, post-approval commitments were more frequent for ATMP submissions than for other biologicals (Figure 3). Further analysis showed that more manufacturing and quality objections for ATMPs were mentioned in the EPAR to be addressed in the post-approval phase as compared to other biologicals (Figure 3). These objections were mostly related to validations of the analytical methods, improving process control, developing new analytical methods, performing further characterization, and tightening of the proposed specifications.

Figure 3.

Differences in When Regulatory Objections Were Addressed between ATMPs and Matched Other Biologicals

Each solved objection was categorized as solved either in the pre-approval or the post-approval stage based on the information in the EPARs. Note the difference between both cohorts in quality data requirements (top of the chart). Note also the categories of long-term safety and efficacy as well as the clinical efficacy results that were addressed more in the case of ATMPs through post-approval approaches. (I) manufacturing and quality testing domain (II) experimental design and conduct of studies domain (III) efficacy and mode of action domain (IV) safety and toxicity domain.

Furthermore, developers of ATMPs committed to more post-approval approaches to address issues related to the pivotal trial results, long-term efficacy and long-term safety, as compared to biologicals (Figure 3). These approaches mainly included the obligation to perform post-authorization safety studies (PASSs) and post-authorization efficacy studies (PAESs) (Figure 3). Additionally, ATMP developers were obliged to collect specific safety and efficacy information through the use of patient registries.

Other Factors Influencing the Sufficiency of Evidence

Possible differences in the development strategy in both cohorts were explored. The nature of the organization that developed the product was considered and divided into two categories: established large biopharma and micro, small, and medium-sized enterprises (SMEs). The use of scientific advice is reported in the EPAR, so those data were also collected. Most of the ATMP submissions came from SMEs, with only 4/17 (24%) of ATMP submissions from large companies, as compared to 15/17 (88%) for other biologicals. Despite ATMPs being more complex products that may require regulatory advice at several stages of development, EMA scientific advice was sought at nearly equal frequency. On average, developers of authorized ATMPs sought EMA scientific advice 3.0 times (SD, 1.3; range, 1–5), while the developers of the other approved biologicals sought scientific advice 3.1 times (SD, 2.0; range, 0–7).

The main clinical studies utilized for the benefit-risk assessment also showed significant differences between the matched authorized cohorts. Single-arm trials were more frequent among authorized ATMPs, with controlled trials being conducted in only 7/12 (58%) of the authorized ATMPs, as compared to 10/12 (83%) of the other biologicals. Furthermore, there was a significant difference in the number of patients in the main clinical trials of the authorized ATMPs, as compared to the other biologicals (Z = −2.510, p = 0.009). On average, authorized ATMP main clinical trials had 158 patients per clinical trial (SD, 160; range, 12–512), while the other biologicals had an average of 434 patients per clinical trial (SD, 431; range, 13–1,197,). Finally, all authorized other biological trials were multicenter trials, while two ATMPs (Glybera and Strimvelis) were single-center trials. Despite not included in the analysis, one authorized ATMP (Holoclar) used a historic clinical case series as the main study for the MA instead of designing and conducting a clinical trial.

Discussion

ATMPs are a new and more complex group of therapeutic products with a wide range of development challenges. To acknowledge the complexity and novelty of ATMPs, the EU medicines directive (Directive 2001/83/EC) provides some specific requirements for their development in Annex I. Previous studies have explored the reasons for the success and failure of ATMPs by evaluating the objections, duration of review, and outcomes against other factors such as orphan status, company size, and use of scientific advice.18, 19, 20, 21, 22 None to date has tried to evaluate the more subtle question of whether the data provided were consistent with Annex I and whether a risk-based approach was used and, more importantly, accepted. The backdrop to this question was the number of ATMPs withdrawn after approval, reaching a staggering 36% (5/14). Those five products had been approved for an average of 3.60 years (SD, 2.30; range, 1.40–6.82), leaving only three ATMPs that were approved for more than 3 years: Holoclar (longest at 4.37 years), Imlygic, and Strimvelis. Given the small numbers of ATMP submissions, a comparator group was needed to benchmark the performance of ATMPs against more established biological products (other biologicals). We were able to match 17 ATMPs to other biological products based on known confounding factors, thus minimizing potential bias in the comparison (Table 2). Our objectives were as follows: (1) to investigate the sufficiency of evidence through the quantification of objections raised by regulatory authorities, (2) to measure regulatory flexibility where applicants diverged from data requirements in Annex I, and (3) to assess whether any identified weaknesses have post-approval implications.

First, we acknowledge the limitations of this analysis. The only public sources of information available are the EPARs; these are edited versions of the EMA internal assessment reports, with confidential details removed, primarily in the manufacturing and quality section.²³ Moreover, some of the solved issues may have been removed from the final reports, leading to a potential underestimation of the objections raised during the evaluation. Furthermore, the EPAR format has been updated to address the needs of HTAs between 2012 and 2015.²⁴ Nevertheless, these limitations were addressed by applying a strict text mining and analysis framework and matching ATMPs and biologicals on the date of the regulatory decision, respectively.

We scored the objections raised during the regulatory assessments of MAA submissions for both ATMPs and other biologicals (Figure 1) and sorted them to the predefined evidence domains. Even though the manufacturing and quality evidence domain had the highest proportion of objections in both groups, as reported by others,14, 15, 16^,25,26 there were no significant differences in this domain between ATMPs and other biologicals. This observation indicates that manufacturing is challenging across all biological medicinal products. For ATMPs, these objections revealed themselves as mostly deficiencies in product characterization and specification, analytical tests and assays and their validation, microbiological controls, and, inevitably, comparability studies for process changes. For instance, some products were requested to undergo further characterization, such as for leukapheresis starting material and the viral vector in the case of the chimeric antigen receptor T cell product Kymriah.²⁷ For other products such as Provenge, Spherox, and Holoclar, it was requested to develop and validate rapid microbiological testing strategies to overcome the 14 days sterility testing issue, as duration of the test might not be suitable for products with a short shelf-life.²⁸ One important objection related to analytical methods was the potency assay that, ideally, should reflect the biological activity of the product.^16,29,30 For Kymriah and Yescarta, in vitro assays successfully revealed the biological activity of the product and the proposed MoA (e.g., level of interferon γ [IFN-γ] produced upon co-culture with the target cells).^16,27 However, potency testing based on surrogate indicators (e.g., cell surface markers expression) for products such as ChondroCelect, MACI, Spherox, and Provenge were more challenging, as meaningful correlations between the biological activity and the surrogate markers had to be established. Interestingly, we observed that more of these objections were solved through post-approval commitments in the case of ATMPs (Figure 3).

The evidence on the design, conduct, and outcome of clinical studies that were submitted by ATMP developers suffered from more objections when compared to other biologicals (Table 2). Clinical trials of ATMPs did not meet the same strict standards for clinical evidence that were applied to other biologicals submissions. Despite matching for the disease area and orphan status, ATMPs had more non-randomized, non-blinded trials and included significantly (p = 0.009) lower numbers of patients, raising serious doubts about the trial outcomes. In the case of study outcomes, the modest effect size in the primary endpoint (Provenge, Kymriah, Alofisel, Zalmoxis) or relying on secondary and sub-analyses to show the efficacy of the product (Glybera, Imlygic) represented the main share of objections. Addressing the urgency of patient needs and countering the spread of unproven therapeutic claims³¹ has prompted regulatory bodies to launch products with limited clinical evidence.⁶ Nevertheless, HTA agencies, including the National Institute for Health and Care Excellence (NICE) and the Institute for Clinical and Economic Review (ICER), acknowledge this flaw and encourage developers to generate additional evidence post-approval.32, 33, 34 It is acknowledged that financial constraints faced by SMEs, which represent the majority of ATMP developers, can have implications on the ability to conduct large (multicenter) clinical trials. Company size has been shown to be a significant factor in a product’s chances of approval; for example, for the period 2004–2007, large companies had an MA success rate of 89%, medium sized companies had 73%, whereas for small companies it was only 48%.¹⁹ Moreover, in the case of fresh autologous products with a short shelf-life, challenges with manufacturing and logistics can limit the number of centers that can be included in the trials.⁴ Lastly, robust clinical trial designs with randomization and blinding for ATMPs addressing life-threatening or debilitating conditions might not be feasible. However, we showed in previous studies that nearly half of the currently marketed products, including products that were approved based on single-arm trials such as Kymriah and Yescarta, planned or already started controlled trials in the post-approval phase.^27,35 Such observations suggest that the submissions based on single-arm trials might be a strategic decision rather than being forced by limiting factors. These strategies for regulatory submissions can lower the motivation of the industry to attain robust trial designs at the time of the submission and reserve the larger, more financially demanding trials after securing the MA.

Divergence from the Annex I data requirements was not detected in the EPARs in the manufacturing and quality domain of either cohort. This may seem surprising, as this is the area where the use of a risk-based approach would be expected to be most evident. However, as mentioned previously, the details of this section of the dossier are, for the most part, confidential, and, consequently, the details in the EPAR are limited. Nevertheless, some of the shortcomings observed in the second and third domains, and accepted by regulators, were more prominent in ATMPs as compared to other biologicals. In the non-clinical data packages of ATMPs, the technical hurdles and the relevance of the animal models constituted the most observed divergence (Figure 2). Furthermore, developers of authorized ATMPs relied more on non-GLP studies in their submissions (Figure 2). It seems likely that this relates to difficulties in complying with GLP for such studies, since the reasons provided by developers were accepted. This issue has prompted the EMA to release a question and answer document in 2017.^36,37 Due to the high species specificity of gene therapies, there is a challenge in having animal models available that mimic the tissue tropism, immune response, as well as the cellular specificity in humans for toxicology and biodistribution studies.38, 39, 40 In addition, the lack of clear primary pharmacological targets for some of the cellular therapies significantly complicates the design and the robustness of the proof-of-principle animal studies.⁴¹

Both clinical and non-clinical biodistribution and other pharmacokinetics as well as non-clinical toxicity studies led to the most divergence for approved and failed products, equally. Such divergence was understandably around twice as common for ATMPs than for other biologicals. In vivo cell tracking in animals can be technically difficult, with human subjects presenting an even greater challenge. As more experience is gained with certain cell types and vectors, some of these aspects might become addressable. Some developers may consider the possibility of bypassing traditional in vivo animal testing as a benefit; however, these limitations in the non-clinical dataset can pose a significant source of uncertainty, when considering the overall risks and benefits of the product. Properly designed non-clinical studies can reduce such uncertainty and support a positive risk/benefit ratio, while their absence can tip the risk/benefit ratio to the negative or might lead to a CMA with significant post-approval commitments. In our attempt to understand the degree to which a risk-based approach offered flexibility to developers or was accepted by the EMA, we observed only one EPAR, for Provenge, to include a clear statement on using such an approach to justify the extent of the non-clinical data. Two other EPARs referred to risk-based approaches for specific aspects, such as the selection of raw materials and shipping qualification. Consequently, it was challenging to draw such a correlation.

Finally, our results further showed that regulatory objections about the long-term safety and efficacy of ATMPs were addressed through post-approval commitments by performing new clinical trials and deposit data from real-world use into designated registries.⁴² Note that ATMP approvals with limited evidence have led to an increased prevalence of exploratory trial designs required to be performed in the post-approval phase, which does not fully mimic the real-world settings.³⁵ By having many clinical and manufacturing objections for ATMPs addressed in the post-approval settings, developers are overwhelmed with regulatory requirements and commitments, which adds a significant financial, organizational, and administrative burden; in turn, this could impede the product performance and market access.

Conclusions

As of October 2019, 5 out of 14 approved ATMPs were withdrawn after approval. Considering that the first ATMP was approved in October 2010, this is particularly disappointing and warrants analysis such as ours to understand the reasons. As the first study to compare ATMPs to established biologicals, our results send a clear signal that regulations offer a reasonable degree of flexibility in order to bring such innovative therapies to the market. This flexibility comes with a caveat, however. ATMP submissions are authorized with more evidential shortcomings as compared to other biologicals, particularly in the submitted clinical outcomes and trial designs. Such observations, coupled with the high divergence in the non-clinical submission package, create a hurdle for regulators to conduct a well-informed benefit-risk assessment. This might challenge our understanding and confidence in the long-term safety and efficacy of these novel products and could also explain why five ATMPs were withdrawn after approval, approximately 5-fold higher than the matched biological cohort. Even though regulators are imposing extensive post-marketing measures on applicants to overcome these shortcomings, such an approach might impose more hurdles on ATMPs in the post-marketing phase. Our observations are a strong indicator that the scientific community needs to rethink the traditional development framework for such products, in order to mitigate potential evidence deficiencies that may jeopardize their market success. After all, the aim is to develop products that can achieve market sustainability and be available to patients in need.

Materials and Methods

Search Strategy

Data on the authorized, rejected, and withdrawn MAA were obtained from the EMA database (https://www.ema.europa.eu/en/medicines/download-medicine-data) (data cutoff, July 1, 2019). Two separate spreadsheets were obtained: one comprised all of the products that have an EPAR since they completed the evaluation process (authorized and refused), while the other datasheet contained withdrawn products that had a withdrawal assessment report. Screening of all the products presented in the datasheets was performed, and all ATMPs were identified. The corresponding administrative information about each product was then collected through accessing the product-specific profile on the EMA website available from the medicine search engine (https://www.ema.europa.eu/en/medicines). The small and medium-size status of the company was searched on the SME register database (https://fmapps.emea.europa.eu/SME/). When the company was not found, the relevant financial annual report for the year of the MA application submission was obtained and the criteria for SMEs as defined by the EMA were applied.⁴³

Pair Matching ATMPs with Other Biologicals

ATMPs (authorized and failed) were matched to other biologicals to compare the differences in the evaluation process. The products were matched on selected confounding factors that can influence the sufficiency of evidence in the EPARs. The selected factors for matching included 1) the MA application outcome (authorized, refused, or withdrawn), 2) the targeted disease which may influence the availability of suitable animal models and the ability to conduct controlled clinical trials (e.g., in case of oncology treatments),⁴⁴ 3) the nature and rarity of orphan indications which can complicate the clinical trial design, and patient recruitment;⁴⁵ 3) weather products were approved under the CMA or authorization under exceptional circumstances provisions where the product dossiers may have deficiencies in their clinical evidence, and 4) the time at which the application was evaluated, since the regulatory policy, legislation, and guidelines evolve over time and, in turn, the data requirements for MA also evolve. Exact match on MA application outcome, orphan designations and the type of MA was initially conducted. A screening for all the resulted potential biologicals matches was then performed, and exact matching on the disease area was achived. Afterward, a greedy nearest neighbor matching was used to match the date of MA application outcome for biological submissions, as described elsewhere.⁴⁶

Defining the Data Requirements and the Evidence Domains for Comparison

The data requirements that should be submitted within the frame of an MA application were defined and retrieved from Annex I of Directive 2001/83/EC of the European Parliament and the council (Table S3).¹⁰ Rather than attaining the traditional categorization of the data requirements that group them according to their source (manufacturing, non-clinical, and clinical data), we opted to categorize the data requirements according to their purpose in the scientific evaluation and the decision-making process. Accordingly, a value tree similar to that described in studies of multi-criteria decision analysis (MCDA) of the HTAs was formulated (Figure 1) (Table S3).47, 48, 49

Based on this approach, the data requirements can be clustered into four main domains manufacturing and quality testing, experimental design and conduct of studies, efficacy and MoA, and safety and toxicity. The first two domains are considered “confidence criteria” and the other two are considered “outcome criteria”. The confidence criteria ensure that the manufacturing process itself does not introduce additional risks (e.g., impurities, contaminations, formulation) and is able to constantly produce a product with a defined set of physicochemical or biological characteristics. Furthermore, they also aim to ensure that the submitted studies were designed, conducted, and documented in the most proper way. Any issues in these criteria will affect the level of confidence in the reported outcome criteria. For instance, manufacturing data that indicate a high batch-to-batch variation will affect the level of confidence in the consistency of the presented preclinical and clinical evidence across the different studies. Also, an underpowered clinical trial affects the level of confidence in the benefits reported from such a trial and whether the results can be reproduced in real-world scenarios.

Definitions

Objections

During the evaluation of an MA application, the applicant receives a list of identified issues in the applications under two categories: first is “major objections,” defined as critical issues that preclude a recommendation for MA;⁵⁰ second is “other concerns,” defined as issues that do not preclude a recommendation of the MA, as it can be solved through modifying the summary of product characteristics, or implementation of risk minimization measures.⁵⁰ However, in case of failure to solve the other concerns, the product cannot be authorized. Since EPARs do not clearly differentiate between major objections and other concerns, all issues extracted from the EPAR are referred to in this article as objections.

Divergence

Any studies stated as a requirement for the MAA in Annex I of Directive 2001/83/EC and that have not been performed by the applicant should be justified. Justifications include the availability of specific guidelines that deem these studies unnecessary for this kind of therapy, through a rational justification from the applicant or by the application of a risk-based approach. We quantified the degree of divergence by collecting the number of studies that were omitted in the EPARs and accepted during the evaluation of the application.

Data Extraction and Statistical Analysis

Data were then collected, sorted, and coded by M.E and verified by M.A. Upon discrepancies regarding extracted text or sorting of the objections and divergence, discussions were conducted to reach an agreement. All of the data were coded and statistically analyzed using SPSS version 25. Means, ranges, and SDs were used for the descriptive statistics. Due to the small sample size, the matched design, and the exploratory nature of the analysis, a non-parametric statistical test was pre-defined. A Wilcoxon signed-rank test was used to estimate the differences in objections and divergence between the matched pairs. Two-tailed p values less than 0.05 were considered statistically significant. Figures were produced by SPSS version 25 and R studio (version 1.2.1335) using the tidyverse package (version 1.3.0).

Author Contributions

Conceptualization, M.E. and MA.; Methodology, M.E., and M.A.; Formal Analysis, M.E., and M.A.; Investigation, M.E., C.A.B. and M.A..; Writing –Original Draft, M.E.,and M.A.; Writing – Review & Editing, M.E., C.A.B., A.K. and M.A..; Visualization, M.E.; Funding Acquisition, M.A.; Supervision, A.K., and M.A.

Conflicts of Interest

The authors declare no competing interests.