Regulatory approval of pharmaceuticals without a randomised controlled study: analysis of EMA and FDA approvals 1999–2014

Abstract

Introduction

The efficacy of pharmaceuticals is most often demonstrated by randomised controlled trials (RCTs); however, in some cases, regulatory applications lack RCT evidence.

Objective

To investigate the number and type of these approvals over the past 15 years by the European Medicines Agency (EMA) and the US Food and Drug Administration (FDA).

Methods

Drug approval data were downloaded from the EMA website and the ‘Drugs@FDA’ database for all decisions on pharmaceuticals published from 1 January 1999 to 8 May 2014. The details of eligible applications were extracted, including the therapeutic area, type of approval and review period.

Results

Over the period of the study, 76 unique indications were granted without RCT results (44 by the EMA and 60 by the FDA), demonstrating that a substantial number of treatments reach the market without undergoing an RCT. The majority was for haematological malignancies (34), with the next most common areas being oncology (15) and metabolic conditions (15). Of the applications made to both agencies with a comparable data package, the FDA granted more approvals (43/44 vs 35/44) and took less time to review products (8.7 vs 15.5 months). Products reached the market first in the USA in 30 of 34 cases (mean 13.1 months) due to companies making FDA submission before EMA submissions and faster FDA review time.

Discussion

Despite the frequency with which approvals are granted without RCT results, there is no systematic monitoring of such treatments to confirm their effectiveness or consistency regarding when this form of evidence is appropriate. We recommend a more open debate on the role of marketing authorisations granted without RCT results, and the development of guidelines on what constitutes an acceptable data package for regulators.

Strengths and limitations of this study.

• This work is the first systematic attempt to identify drugs approved without randomised evidence.

• The in-depth review identifies all evidence for treatments throughout the clinical development programme, not just the pivotal study.

• Because of the different remits and processes of the European Medicines Agency and the Food and Drug Administration, it is not possible to compare across all disease areas.

• Despite identifying treatments without randomised evidence, it is not in the scope of this study to conclude on the appropriateness of approval on the basis of non-randomised data.

• Due to the lack of follow-up studies, it is also not possible to reach a conclusion on the efficacy of these products.

Introduction

The efficacy of pharmaceuticals is usually evaluated for regulatory purposes using randomised controlled trials (RCTs) that, when properly designed and conducted, provide unbiased estimates of treatment effect.¹ However, there are occasions when a therapy is administered to all patients within a trial; this is a frequent step in the development process of pharmaceuticals. While it is not necessarily a preferred approach, the product can be submitted to regulatory agencies for approval following these studies. This can occur with treatments that are new to the market, or for the extension of a licensed treatment to a different patient population, with the argument that clinical equipoise no longer applies. As the treatment is obviously effective, there is no clinical uncertainty, and as such, to withhold the treatment from one group in a trial would be unethical or impractical.^2–4

Uncontrolled clinical studies may take several forms that include aspects of randomisation (figure 1). For example, patients may be randomised to different dosages (or regimens) of the experimental treatment. Other data packages may include a clinical study compared to a historical control, such as a previously published study of similar patients treated with the relevant comparator.⁵ Finally, other treatments may have no control arm or control data, but instead rely on an ‘obvious’ difference, or avoidance of an expected outcome—for example, death from poisoning. The common factor in each of these designs is that the study in isolation is insufficient to demonstrate the effectiveness of the treatment (ie, external data are needed) as it does not contain an internal control arm (either active or placebo) against which outcomes can be compared.

Figure 1.

Types of uncontrolled studies used to support regulatory applications compared to randomised controlled trials.

Following the clinical study programme, the data package for a treatment is submitted to the regulator, who must give their approval before a drug is allowed on the market. In the EU and the USA, there are slightly different paths to approval. Historically, pharmaceutical licensing in the EU was the responsibility of individual countries; however, with the development of the centralised authorisation procedure by the European Medicines Agency (EMA) in 1995, companies submit a dossier to the EMA. The EMA reviews the submission, asks questions to the company as a part of the process, and then issues a decision that is binding throughout the EU.⁶ In the USA, manufacturers submit a New Drug Application (NDA) to the Food and Drug Administration (FDA) which will review the data submitted before making a decision on the request.⁷

The objective of this study was to review new pharmaceutical approvals by the EMA and the FDA, from 1999 to 2014, to understand how frequently applications (and approvals) are made solely on the basis of uncontrolled studies, without either the pivotal study or supportive studies being RCTs in the indication (previous work has focused solely whether the pivotal study contained a control arm⁸). A secondary analysis compared the approval rates of the EMA and the FDA for evidence of differences in the approach taken to comparable applications in approval rate and timing.

Methods

Details of the trials conducted for the licensing application of treatments were identified through publicly available regulatory documents. In the case of the EMA, the main sources of information were the European Public Assessment Reports and the Committee of Human Medicinal Products Assessment Reports, while for the FDA, these were the approved label and the Medical Officer's report.

All sources were taken directly from the websites of the EMA and the FDA. The EMA approval database was downloaded on 8 May 2014 (http://www.ema.europa.eu/ema/), and the FDA ‘Drugs@FDA’ database was downloaded on 15 May 2014 (http://www.fda.gov/Drugs/). These databases were used to explore drug approvals based on uncontrolled trial results from 1 January 1999 to 8 May 2014. This date range was chosen as it started several years after the beginning of the EMA Centralised Procedure (1995), giving time for the process to become established. Each approval was reviewed in turn to identify applications made without supportive RCT results.

Following deduplication, all products not approved by both agencies were searched for individually on the website of the other agency with no date restriction to understand whether they were approved before the date period. Searches were also conducted on the website of the manufacturer, and in PubMed and Google News for information regarding submissions, in order to identify any products that did not appear in the databases use for the main analysis.

The aim of our study was to look at newly approved indications for pharmaceuticals, which led to the exclusion of several types of product, listed in table 1. Applications were compared against the exclusion criteria, and if excluded, the reason for exclusion was noted in a hierarchical fashion. If a regulatory application for a product was made to one agency but did not appear in the results for the other agency, a search with no date restriction on was performed to identify any applications made outside the review window (to ensure all comparable approvals have been included).

Table 1.

Exclusion criteria and rationale

Exclusion criteria	Rationale
Generic drugs	Licensed on the basis of similarity to existing drugs and would result in duplication
Biosimilar drugs	The interpretation of data from trials of biosimilars is likely to be informed by data available regarding the original drug (in mechanism and effect on a condition). As such biosimilar applications cannot be considered as comparable to applications for new drugs for which no external information is available
Diagnostic technologies and medical devices	As there is no therapeutic effect to measure, these are assessed by different criteria
Vaccines	The majority of vaccines are licensed based on well-understood technologies and mechanisms of action and it is therefore to consider their use in isolation
Antimicrobial products	The approval process for antimicrobial products involves different standards, with in vitro demonstrations of efficacy against different bacteria playing a much larger role in the expansion of indications
Blood products and recombinant blood products	If identical to human blood products, the mechanism of action for these products is well understood, and thus the same efficacy trials to prove the concept of the intervention is not needed
Fixed dose combinations of existing products	Unless additional claims are made of the combination product, the evidence base for these products is influenced by that for the original products

The included data were for each approved indication, other approved indications for the treatment, submission and decision dates, disease area and supporting evidence for the approval.

Results

From January 1999 to May 2014, the EMA issued 795 approvals, including 44 indications approved solely on the basis of uncontrolled trials (figure 2). Of the 44 approvals, 8 were extensions of indications for treatments with RCTs in other approved indications, while the remaining 36 approvals were for products in which there were no RCT results in an approved indication. Over the time period, for nine applications without supporting RCT results, the EMA rejected the application or the manufacturer withdrew its submission with a negative recommendation in place.

Figure 2.

PRISMA diagrams—drug approvals based on uncontrolled clinical studies by the FDA and the EMA.

In the same time period, the FDA issued 774 approvals, which included 60 indications approved solely on the basis of uncontrolled studies (figure 2). Of the 60 approvals, 12 were extensions of indications for treatments with RCTs in other approved indications, while the remaining 48 were for products in which there were no RCT results in an approved indication. Only one application made without controlled trial results (temporfin for head and neck cancer) received was not approved.

Deduplicating for approvals granted in both regions, 74 indications were approved over the period by at least one agency on the basis of uncontrolled trials (table 1). Thirty-four approvals (the largest group) were in haematological oncology, 15 in solid tumour oncology, and 15 in rare metabolic conditions (including enzyme replacement therapies). Outside of these areas, eight approvals were granted in poisoning or emergency medicine and two in general haematology. All the approvals that were extensions of the license for existing therapies were in either haematological oncology or solid tumour oncology (table 2).

Table 2.

Drugs submitted to the EMA and the FDA containing only uncontrolled clinical studies

Generic name	Condition	Categorisation	EMA status	FDA status	RCT results available
Abarelix	Prostate cancer	Solid tumour oncology	–	A	No
Alemtuzumab	Chronic lymphocytic leukaemia (CLL)	Haematological oncology	A	A	No
Alglucosidase alfa	Pompe disease	Rare metabolic condition	A	A	No
Alipogene tiparvovec	Familial lipoprotein lipase deficiency	Rare metabolic condition	A	–	No
Anagrelide	Essential thrombocytopenia	Blood count	A	A*	No
Argatroban	Heparin-induced thrombocytopenia	Blood count	–	A*	No
Arsenic trioxide	Acute promyelocytic leukaemia	Haematological oncology	A	A	No
Asparaginase Erwinia chrysanthemi	Acute lymphoblastic leukaemia (ALL)	Haematological oncology	–	A	No
Bendamustine hydrochloride	Non-Hodgkin's lymphoma	Haematological oncology	–	A	Yes
Betaine anhydrous	Homocystinuria	Rare metabolic condition	A	–	No
Bexarotene	Cutaneous T-cell lymphoma (CTCL)	Solid tumour oncology	A	A	No
Bortezomib	Multiple myeloma (MM)	Haematological oncology	A	A	No
Bortezomib	Mantle cell lymphoma (MCL)	Haematological oncology	–	A	Yes
Bosutinib	Chronic myeloid leukaemia (CML)	Haematological oncology	A	A	Yes
Brentuximab vedotin	Hodgkin's lymphoma (HL)	Haematological oncology	A	A	No
Brentuximab vedotin	Systemic anaplastic large cell lymphoma (sALCL)	Haematological oncology	A	A	No
Busulfan	Haematopoietic progenitor cell transplantation (HPCT)	Haematological oncology	A	A	Yes
Carfilzomib	MM	Haematological oncology	–	A	No
Carglumic acid	Chronic hyperammonaemia	Rare metabolic condition	A	A	No
Ceritinib	Non-small cell lung cancer	Solid tumour oncology	–	A	No
Cetuximab	Colorectal cancer	Solid tumour oncology	A	A	No
Cholic acid (Kolbam)	Inborn errors in primary bile acid synthesis	Rare metabolic condition	A	–	No
Cholic acid (Orphacol)	Inborn errors in primary bile acid synthesis	Rare metabolic condition	A	–	No
Cladribine	Hairy cell leukaemia	Haematological oncology	A	–	No
Clofarabine	ALL	Haematological oncology	A	A	No
Crizotinib	Non-small-cell lung cancer	Solid tumour oncology	–	A	No
Dasatinib	CML	Haematological oncology	A	A	No
Dasatinib	Philadelphia chromosome-positive ALL	Haematological oncology	A	A	No
Defibrotide	Veno-occlusive disease	Poisoning	A	–	Yes
Dexrazoxane hydrochloride	Anthracycline extravasation	Poisoning	A	–	No
Ferric hexacyanoferrate (II)	Internal contamination with radioactive caesium or thallium	Poisoning	–	A	No
Gefitinib	Non-small cell lung cancer (NSCLC)	Solid tumour oncology	–	A	No
Gemtuzumab ozogamicin	Acute myeloid leukaemia (AML)	Haematological oncology	S	S	No
Glucarpidase	Toxic plasma methotrexate concentrations	Poisoning	S	A	No
Hydroxocobalamin	Treatment of cyanide poisoning	Poisoning	A	A	No
Ibrutinib	Mantle cell lymphoma (MCL)	Haematological oncology	–	A	No
Ibrutinib	CLL	Haematological oncology	–	A	No
Imatinib mesylate	Chronic myeloid leukaemia (CML)	Haematological oncology	A	A	No
Imatinib mesylate	Gastrointestinal stromal tumours (GIST)	Solid tumour oncology	A	A	No
Imatinib mesylate	Myelodysplastic/myeloproliferative diseases (MDS/MPD) associated with platelet-derived growth factor receptor (PDGFR) gene re-arrangements	Haematological oncology	A	A	Yes
Imatinib mesylate	Soft tissue sarcoma—Dermatofibrosarcoma protuberans (DFSP)	Solid tumour oncology	A	A	Yes
Imatinib mesylate	Philadelphia chromosome-positive acute ALL	Haematological oncology	–	A	Yes
Imatinib mesylate	Aggressive systemic mastocytosis (ASM)	Haematological oncology	S	A	Yes
Imatinib mesylate	Advanced hypereosinophilic syndrome (HES) and/or chronic eosinophilic leukaemia (CEL) with FIP1L1-PDGFR rearrangement	Haematological oncology	A	A	Yes
Ixabepilone	Breast cancer	Solid tumour oncology	S	A	Yes
Lomitapide mesylate	Familial hypercholesterolemia (HoFH)	Rare metabolic condition	A	A	No
Metreleptin	Lipodystrophy due to leptin deficiency	Rare metabolic condition	–	A	No
Nelarabine	T-cell acute lymphoblastic leukaemia/lymphoma (T-ALL/T-LBL)	Haematological oncology	A	A	No
Nilotinib hydrochloride monohydrate	Chronic myeloid leukaemia (CML)	Haematological oncology	A	A	No
Nitisinone	Hereditary tyrosinaemia	Rare metabolic condition	A	A	No
Ofatumumab	CLL	Haematological oncology	A	A	No
Omacetaxine mepesuccinate	CML	Haematological oncology	S	A	No
Paclitaxel	Kaposi's sarcoma	Solid tumour oncology	A	A	Yes
Pasireotide diaspartate	Cushing's disease	Rare metabolic condition	A	A	No
Pentetate calcium trisodium	Internal contamination with plutonium, americium, or curium	Poisoning	–	A	No
Pentetate zinc trisodium	Internal contamination with plutonium, americium, or curium	Poisoning	–	A	No
Pomalidomide	Multiple myeloma (MM)	Haematological oncology	–	A	No
Ponatinib hydrochloride	CML	Haematological oncology	A	A	No
Ponatinib hydrochloride	Philadelphia chromosome-positive acute ALL	Haematological oncology	A	A	No
Pralatrexate	Peripheral T-cell lymphoma (PTCL)	Haematological oncology	S	A	No
Raxibacumab	Anthrax inhalation	Poisoning	–	A	No
Romidepsin	Peripheral T-cell lymphoma (PTCL)	Haematological oncology	S	A	Yes
Sodium ferric gluconate complex	Iron deficiency	Rare metabolic condition	–	A	No
Sodium phenylbutyrate	Urea cycle disorders	Rare metabolic condition	A	A*	No
Sunitinib malate	Renal cell carcinoma	Solid tumour oncology	–	A	No
Taliglucerase alfa	Gaucher's disease	Rare metabolic condition	S	A	No
Temoporfin	Head and neck cancer	Solid tumour oncology	A	–	No
Temozolomide	Anaplastic astrocytoma	Solid tumour oncology	A	A	Yes
Tocofersolan	Vitamin E deficiency due to cholestasis	Rare metabolic condition	A	–	No
Tositumomab; iodine I 131 tositumomab	Non-Hodgkin's lymphoma	Haematological oncology	–	A	No
Trabectedin	Soft tissue sarcoma	Solid tumour oncology	A	–	No
Vismodegib	Basal cell carcinoma	Solid tumour oncology	A	A	No
Vorinostat	CTCL	Haematological oncology	S	A	No
Zinc	Wilson's disease (hepatolenticular degeneration)	Rare metabolic condition	A	A*	No

A, approved; A*, approved prior to 1999; S, submitted but not approved.

Searching for treatments approved by only one agency on the basis of only uncontrolled studies without a date restriction yielded further four approvals, which was used in the comparison of EMA and the FDA approval rates and times. Four treatments approved by the EMA 1999 in our date range were approved prior to 1999 by the FDA with a similar data package.

For applications made in both regions, 44 applications were made to both the EMA and the FDA without controlled results (including the 4 made to the FDA prior to 1999), with the EMA approving 35 and the FDA approving 43. Of the 34 applications approved in both regions, the EMA approval was granted a mean of 13.4 months later (median 6.7, IQR 4.5–17.2 months). This delay consisted of two parts: first, companies submitted to the EMA a mean of 7.4 months later (median 1.5, IQR 0.1–8 months), with 28 of 34 indications submitted to the FDA first. Second, the EMA took an average of 6.3 months longer to complete their review and approve products (median 0.2, IQR 0.1–0.3 months); in comparable approvals, the EMA review was quicker in only 3 of 34 applications leading to 30/34 products reaching the market first in the USA.

In addition to the EMA approving fewer products and taking longer to approve products, five treatments were approved with only uncontrolled studies by the FDA, but approved by the EMA at a later date when comparative results were available (a delay in approval of 21.5 months mean/9 months median).

Discussion

The number of approvals without supporting RCT evidence was in excess of what we had expected, with a mean of ∼5 indications per year approved by either (or both) the EMA and the FDA. We had also expected the majority of approvals to be license extensions of existing products; however, this was not the case—only 19% of approvals were license extensions of products demonstrated to be efficacious in RCTs in other diseases. Although these approvals based solely on uncontrolled data are a relatively small proportion of the total number approvals, in absolute terms they number ∼3 per year by the EMA and 4 per year by the FDA and are worthy of further scrutiny.

The disease areas where uncontrolled studies are used for approval were primarily in oncology, with 49/74 indications (66%) being either haematological or solid tumour oncology. This also corresponds with previous work regarding drug licensing, which shows a lower barrier to oncology drug approval.⁹ This particularly seems to be the case with FDA reviews; of the nine applications to the EMA that were not approved, seven were in oncology, with the EMA highlighting uncertainty regarding the benefit to the risk ratio of the treatments. While the perceived unmet need in oncology has been used as an argument for the use of uncontrolled studies, we are not aware of a similar discussion in the literature regarding rare diseases. In addition, it is also possible that the FDA did not approve other treatments approved by the EMA, and this was dealt with confidential letters¹⁰ (with no press releases issued by the companies concerned). While this is unlikely, it remains an unavoidable limitation of the study.

While there is a considerable overlap in decisions, it appears that the FDA is more willing to approve products on the basis of uncontrolled trials. This difference may reflect the systems of approval or the attitudes of regulators towards risk versus unmet medical need.^{11 12} Other evidence also shows that the two systems produce decisions on different timescales, despite access to the same clinical studies. A difference in approval times has been seen in the approval of tyrosine kinase inhibitors,¹³ cancer drugs¹⁴ and pharmaceuticals in general¹⁵ by the two agencies. The difference may in part be explained by the FDA's extensive use of ‘accelerated approvals’ (results based on a surrogate end point, with confirmatory RCTs conducted subsequently¹⁶), compared to the lesser use of the equivalent EMA process, conditional approval.

The results regarding dates of approvals appear to show that patients in the EU must wait longer for novel treatments. The delay in submitting between agencies is understandable, given that the staff involved (eg, trial statisticians) may be required to work on submissions for both agencies. Companies appear to prioritise the FDA submission, which we have noted but not speculated on. Initiatives to harmonise the processes (eg, the Transatlantic Trade and Investment Partnership) may be helpful in reducing these delays, provided that they do not have other negative externalities.¹⁷

The issue of longer approval times for the EMA is more complex. The longer process in Europe may be due to the organisation required for making decisions for all EU member states (as well as Iceland, Liechtenstein and Norway). A further potential factor in the higher non-approval rate by the EMA (9 vs 1 applications of similar content) and the longer review period is the opportunity for discussion about the content of applications (and resolution of issues) before a dossier is submitted. Companies typically engage after each phase of clinical development with the FDA, as well as before submitting a New Drug Application.¹⁸ In Europe, companies do not necessarily interact with the EMA during the course of development but may take Scientific Advice prior to submitting; however, only 14 of the 34 products approved by both agencies had done so (Personal communication, European Medicines Agency).

Thus far, we have not discussed the appropriateness of approvals without supporting RCT results. The commonly stated factors affecting the availability of evidence are the rarity of the condition and the absence of approved treatments. The appropriateness of these arguments varies as comparative trials under rare conditions are certainly challenging; however, they do not appear to be consistent—for example canakinumab for Cryopyrin-associated periodic syndrome, with a prevalence of <1/100 000, was studied in an RCT of 31 patients. Where there is no approved standard of care in a terminal disease, in some cases (eg, alemtuzumab in relapsed and refractory chronic lymphocytic leukaemia) this justification was used for the conduct of an uncontrolled study, whereas under similar conditions (pixantrone for aggressive relapsed B-cell non-Hodgkins lymphoma) the manufacturer conducted a trial against the treatment of physicians’ choice (from six widely used off-label treatments). Even under the same conditions, there is variability in the evidence base provided by companies—Gaucher's disease is a rare lysosomal storage disorder and here taliglucerase alfa appears in our sample without controlled trial results, yet velaglucerase alfa included an active controlled trial in its development programme.

Even where placebo RCTs are clearly unviable—serious conditions without licensed treatments and a well-established outcome (poisoning is the obvious example); new treatments are often trialled against standard care or as an addition to existing therapy. Such controlled study designs have been used extensively in the evaluation of novel treatments for schizophrenia and, although not without challenges, could be used to assess how a new treatment is performed.¹⁹

While much has been written on what level of effect requires an RCT;⁴ in the treatments we identified, we found that the persuasiveness of evidence is highly variable. For example, imatinib in chronic myeloid leukaemia (CML) showed a response rate of 49% for major cytogenetic response and 30% for complete cytogenetic response in 532 patients, whereas omacetaxine, also in CML, showed an 18.4% major cytogenetic response rate and a 7.9% complete cytogenetic response in a trial of 76 patients. It is clear that there is a role for approval without the support of RCT results; however, what an acceptable data package should consist of does appear to be inconsistent. This is even more apparent where products must be assessed by organisations such as the National Institute for Health and Care Excellence (NICE) who has the role of estimating the difference in outcomes between the new treatment and standard of care; that is, demonstration of efficacy alone is not sufficient.²⁰ If the studies used to gain marketing authorisation for products are also not able to provide evidence for market access (ie, to convince payers that such products represent value for money), then in some cases, the use of uncontrolled trials may be increasingly difficult to justify as they do not provide the evidence needed for the medicine to ultimately treat patients.

The research that has been conducted on treatments approved without RCT results (limited to oncology drugs approved in the USA) indicates that the availability of RCT evidence does not typically affect the likelihood of remaining on the market, with some treatments withdrawn for a lack of efficacy or safety regardless of whether randomised trial evidence was available.²¹ Those concerned about the lack of confirmatory evidence of efficacy will find little solace in this finding. However, the reassessment of newly licensed drugs may increase in importance with initiatives such as adaptive licensing.²² The limited information we collected (on five drugs approved on uncontrolled data by the FDA and then approved by the EMA after the availability of RCT results) suggest that current regulatory assessments may well reach sound decisions in the absence of RCT results, but this is limited by a small and selected sample. Of particular concern, however, is the extensive use of ‘historical controls’ against which to compare outcomes of trials. Previous research has highlighted that comparisons of this type are prone to exaggeration of the treatment effect, with historical controls often performing far worse than equivalent arms in clinical studies.^23–25

Conclusion

Newly licensed pharmaceutical indications are frequently approved without any controlled trial results, particularly in solid and haematological malignancies. While some agents showed such high levels of effectiveness that an RCT is unlikely to have changed any decision, the level of evidence provided by companies to support marketing authorisations does appear to be inconsistent. In the more complex cases (particularly those treatments not approved by one agency), it is not always clear that the conduct of an RCT was impractical.

Although there appears to be a slight difference between approvals in the EU and the USA (with the FDA more likely to grant a positive approval in oncology), the agencies broadly reach the same decisions. However, the systematic differences in approval timing are of particular interest and concern. We therefore suggest that regulatory agencies should continue to harmonise processes where practical (to reduce delays between submissions). In parallel to this, given the frequency at which uncontrolled studies occur, their role in drug approval requires reappraisal, with formal guidance on what represents an acceptable data package; this should then be debated by regulators, companies and physicians.