Pivotal clinical trials of novel ophthalmic drugs and medical devices: retrospective observational study, 2002–2012

Abstract

Objectives

Novel therapeutics are an important part of ophthalmologists’ armamentarium, and the risks and benefits of these therapies must be carefully evaluated. We sought to quantify the characteristics of the pivotal clinical trials supporting the regulatory approval of new ophthalmic drugs and medical devices.

Design

Retrospective observational study.

Setting and data source

Medical review dossiers for new ophthalmic drug and high-risk device approvals released publicly by the US Food and Drug Administration (FDA).

Main outcome measures

Proportion of pivotal trials with randomisation, masking, active or placebo controls and subgroup analyses; total and median number of trial enrollees; and the number of drugs and devices approved with required postapproval studies.

Results

From 2002 to 2012, the FDA approved 11 ophthalmic drugs and 25 devices. The pivotal trials underlying the approvals of ophthalmic drugs in our study cohort enrolled a median of 809 patients. Virtually all drug trials were randomised and masked (91%), of which 7 (70%) used a placebo control. Pivotal trials for ophthalmic devices enrolled 324 patients on average, and significantly fewer trials for ophthalmic devices versus drugs were randomised (16% vs 91%; p<0.001) or masked (12% vs 91%; p<0.001). 8 (32%) ophthalmic devices and 6 (55%) ophthalmic drugs were approved with required postapproval studies.

Conclusions

Ophthalmic therapeutics were approved based on varying levels of evidence. Postapproval studies could be used to confirm or refute early indications of safety and effectiveness of these therapeutics, with the study results accessible to patients and clinicians who need to make informed treatment decisions.

Strengths and limitations of this study.

• This study is the first to characterise the state of trial evidence underpinning the US Food and Drug Administration's (FDA's) approval of new ophthalmic drugs and high-risk devices over a 10-year period. The results are consistent with data for other therapeutic areas.

• All data were obtained from medical review dossiers released to the public by the FDA, representing the key evidence considered by regulators when assessing a new product.

• Our results may not be generalisable to therapeutics that were not approved by the FDA or to lower-risk ophthalmic devices, for which premarket approval is not required.

Introduction

Blindness and visual impairment affect an estimated 285 million people globally, with people 50 years and older representing approximately 82% and 65% of all blind and visually impaired, respectively.¹ Given ageing populations, the public health and economic burden of vision-related disabilities is projected to grow significantly in the coming years.² Advances in translational research, coupled with increasing investor interest in the eye health market, are yielding new therapeutic approaches to help clinicians address visual impairment.^{3 4}

In the USA, before they may reach the clinic, most new drugs and medical devices must be investigated in clinical trials, which allows investigators to test a cause-effect hypothesis for a new product.⁵ Key elements of trial design, such as the use of randomisation and masking, minimise potential sources of bias. By law, the US Food and Drug Administration (FDA) ordinarily requires ‘adequate and well-controlled studies’ demonstrating safety and effectiveness before approving new drugs.⁶ The FDA regulates medical devices according to their level of potential risk. For high risk (or class III) devices, defined as those that support life or that may present an unreasonable risk of illness or injury, the FDA requires that manufacturers demonstrate ‘reasonable assurance’ of safety and effectiveness through the conduct of at least one human clinical trial.⁷

In addition to their role in the regulatory approval process, the results of key preapproval trials are critical for patients and clinicians to adequately weigh the risks and benefits of a specific treatment.^{8 9} The key trials supporting approval, designated by the FDA as ‘pivotal’ trials, can also help identify subpopulations likely to benefit, or not, and that may be adversely affected by therapy, as well as the potential magnitude of benefits and harms. For example, ocriplasmin, a truncated form of the human serine protease plasmin, was evaluated in two randomised placebo-controlled pivotal trials for the treatment of symptomatic vitreomacular adhesion.¹⁰ In a prespecified analysis, the trial investigators found that certain patients (specifically those without an epiretinal membrane) were more likely to have non-surgical vitreomacular adhesion resolution, compared with those with an epiretinal membrane.

Despite their importance, the characteristics of the key trials underlying the approvals of ophthalmic therapeutics have not been defined. Better information about the evidentiary basis on which new ophthalmic drugs and devices are approved could help patients and clinicians in treatment decisions and inform ongoing policymaking efforts to balance streamlining the regulatory approval process¹¹ by ensuring the safety and effectiveness of new therapies.¹² Prior studies have found that the clinical testing and clinical trial design leading to FDA approval can vary widely. Compared with drugs intended for common diseases, drugs for rare diseases (or orphan drugs) were more likely to be approved on the basis of smaller non-randomised non-controlled trials.^{13 14} Similarly, few pivotal trials supporting premarket approval (PMA) of high-risk cardiovascular devices were randomised or masked.^{15 16} The design of pivotal trials may also vary between and within different medical subspecialties.^{17 18} Therefore, in the present study, we sought to quantify the characteristics of the pivotal clinical trials supporting the regulatory approval of new ophthalmic drugs and medical devices over the past decade.

Methods

Data collection

Our study cohort included all new molecular entities and biologics indicated to treat ophthalmological diseases and approved by the FDA between 1 January 2002 and 31 December 2012. We also included ophthalmic devices approved during the study period through the PMA process, which is reserved for high-risk devices (eg, intraocular lenses and implantable pacemakers). The names and indications of all ophthalmic drugs (including biologics) and devices were identified from public domain master lists of approvals published by the FDA. Generic drugs, duplicate or previously approved agents and reformulations were excluded from the study cohort.

On approval, the FDA publishes certain information relating to the novel therapeutic on its website through the Drugs@FDA and Releasable PMA databases.^{19 20} For new drugs, the online dossier includes the approval order, product label and drug approval package, which includes summaries of the agency's medical, chemistry, pharmacology, statistical and microbiology reviews. In particular, the medical review contains the FDA reviewers’ summary and analysis of the clinical evidence submitted by the manufacturer in support of the new drug, including information on the pivotal clinical trials. For PMA devices, the FDA similarly publishes the approval order, label, and a Summary of Safety and Effectiveness Data, which summarises the preclinical and clinical data submitted by the manufacturer. All data were retrieved on 1 December 2013 and 10 May 2014.

Data extraction

From the publicly available FDA review summaries for each ophthalmic therapeutic in our study sample, we identified the pivotal clinical trials that supported the intended indication(s) for which the therapeutic was approved. Using a structured approach we described previously,^{21 22} two investigators (JH and TJH) independently classified (and resolved by consensus, if any discrepancies) each trial as pivotal, in descending order of available information, if it was: (1) designated by the FDA reviewer as a pivotal trial; (2) the only one presented in the clinical investigation section; (3) the most advanced study to be completed (eg, if results from phase 1 and phase 2 trials were presented, the phase 2 trial was considered pivotal) or (4) the only prospective study (eg, if results from a prospective phase 2 study and a retrospective case series were included, the phase 2 trial was considered pivotal).

The key outcome measures in the present study were the proportion of pivotal trials with randomisation, masking and active or placebo controls; total and median number of trial enrollees; and the number of drugs and devices approved with required postapproval studies. For each trial in our study cohort, two investigators (JH and TJH) extracted information on the design of the pivotal trials: whether participants were randomised, the masking scheme (double-masked, single-masked, post hoc review masked, or open-label), the total number of trial enrollees, the number of trial arms and comparators (standard of care, placebo, historical control, single-arm/no control), and the types of subgroup analyses. Post hoc review masking was defined as the masking of patient identity and intervention to independent reviewers of clinical data after trial completion. Historical controls involved the comparison of outcome data with the existing published evidence or the retrospective matching of trial participants with control subjects from a registry. We also collected information on any postmarketing approval commitments from the approval order.

Analysis

We used descriptive statistics to characterise features of the design of pivotal trials among ophthalmic drugs and devices. We used the Fisher's exact test and Wilcoxon-Mann-Whitney test to compare distributions of categorical (randomisation and masking) and continuous (trial size) attributes between drug and device trials. Statistical analyses were performed using Stata (V.12.0, Stata Corp, College Station, Texas, USA), with a two-sided α=0.05.

Results

From 2002 to 2012, the FDA approved 11 ophthalmic drugs (see online supplementary appendix table A1) and 25 devices (see online supplementary appendix table A2). The most common indications for ophthalmic drug approvals were treatment of neovascular age-related macular degeneration (3 of 11; 27%) and treatment of or prevention of itching associated with bacterial conjunctivitis (4 of 11; 36%). Eleven devices (44%) were intraocular lenses.

The ophthalmic drugs in our study cohort were approved based on a median of two pivotal trials, compared with a median of one pivotal trial per device approval. Ophthalmic drug pivotal trials enrolled a median of 809 patients (IQR 463–1417; total 11 317 patients), of which a median of 464 patients received the active intervention (table 1). Ten pivotal trials (91%), corresponding to 10 drug approvals, were randomised. Seven of the 10 randomised studies compared the treatment agent with placebo, and 3 used an active comparator. All 10 randomised studies were also double-masked. One drug, trypan blue, was approved in 2004 as an aid in ophthalmic (cataract) surgery on the basis of a review of published studies, none of which were randomised or controlled.

Table 1.

Characteristics of pivotal preapproval trials for ophthalmic drugs and devices approved in 2002–2012*

Characteristics	Drug approvals (N=11)	PMA devices (N=25)	p Value
Enrollees, median (IQR)
Total enrollees	809 (463–1170)	324 (217–412)	0.005
Treatment arm	462 (230–829)	294 (199–383)	0.10
Randomised, n (%)	10 (91)	4 (16)	<0.001
Comparator, n (%)	11 (100)	16 (64)	0.10
Active comparator	3 (27)	5 (20)	–
Placebo or contralateral	7 (64)	1 (4)	–
Historical control	1 (9)	10 (40)	–
None	0 (0)	9 (36)	–
Masking, n (%)	10 (91)	3 (12)	<0.001
Double-masked	10 (91)	0	–
Single-masked	0 (0)	3 (12)	–
Open-label, n (%)	1 (9)	22 (88)	<0.001

*Owing to rounding, proportions may not sum to 100.

PMA, premarket approval.

Pivotal trials for ophthalmic devices enrolled significantly fewer patients on average than pivotal trials for ophthalmic drugs (median 324 vs 809; p=0.005; total 9022 patients). Pivotal trials for ophthalmic drugs were more likely than those for devices to be randomised (91% vs 16%, p<0.001) and masked (91% vs 12%, p<0.001). Ten (40%) device pivotal trials used historical controls, and five (20%) used an active comparator. Subgroup analyses of treatment effects by age and sex were provided for six (55%) drugs and four (16%) devices.

Six (55%) ophthalmic drugs and eight (32%) ophthalmic devices were approved with required postapproval studies. Three of the six ophthalmic drug postapproval studies were intended to evaluate the risk of adverse events, and two studies were intended to evaluate safety and efficacy in paediatric populations. Information on trial design was available for only one postapproval study: the postapproval study of besifloxacin was required to be randomised and vehicle-controlled, and to enrol at least 300 patients. For devices, six of the eight postapproval studies were intended to evaluate device safety, and two were intended to monitor if the effectiveness of the therapy was attenuated over time. Seven of the eight devices with required postapproval studies also had specific requirements for trial design. For example, the pivotal study for an intraocular lens enrolled 102 patients, and the postapproval study was intended to be a 5-year retrospective study of safety in approximately 2000 patients, including 1000 paediatric patients.

The postapproval studies for three drugs were completed, postapproval studies for two drugs were still pending and the status of one drug postapproval study was not available. For example, according to the FDA database, a postapproval study to determine whether there are any adverse effects of intravitreal administration of pegaptanib on the corneal endothelium is listed with a deadline of 31 July 2008, and is marked as delayed. With respect to ophthalmic devices, the postapproval studies for three devices were completed, and the status of postapproval studies for four devices was not available.

Discussion

Novel therapeutics are an important part of ophthalmologists’ armamentarium, and the risks and benefits of these therapies must be carefully evaluated. Our study showed that most ophthalmic drugs over the past decade were approved on the basis of randomised, controlled and masked pivotal trials, and subgroup information was not frequently available for approved ophthalmic therapeutics. Few pivotal trials for ophthalmic devices were randomised or masked. Only a minority of ophthalmic device approvals were conditional on postapproval studies.

Conducting pivotal trials is a time-intensive and resource-intensive process,²³ and the results from pivotal trials are important for patients and clinicians. In ophthalmology, as in other specialties, these data are often the principal source of information that patients and clinicians rely on to determine whether to use a newly approved therapeutic.²³ In the absence of such factors as randomisation, masking and controls, the true risk–benefit balance of a therapy could be obscured by bias, evidenced by high rates of positive response to sham surgery and oral pharmacological placebos.^{24 25} Without the availability of adequate and rigorous data before these therapies are used or implanted, there is greater risk that patients may be exposed to treatments that are not effective or that are associated with serious safety issues.^{26 27} Equally, patients may elect to decline therapy if they are pre-emptively informed of these risks.²⁸ In addition, subgroup analyses, as in the case of ocriplasmin, could be helpful in directing patients to appropriate therapy. Similarly, improving the diversity of participants in clinical trials to reflect the intended patient population, may help regulators and clinicians better assess the real world performance of these therapeutics.^{29 30}

Regulators have recognised that the appropriate risk–benefit balance may vary, and have demonstrated flexibility in accelerating the approval of therapies for patients with no other treatment options and those with serious and life-threatening diseases. Such flexibility may involve smaller and fewer preapproval studies based on surrogate end points, but, in turn, may also provide less certainty about the therapy's potential risks and benefits.³¹ In these cases, the FDA has required active study after approval, such as through a prospective study or patient registry. Completion of and timely dissemination of the results from these postapproval studies would provide greater certainty to patients and clinicians about the efficacy and safety signals found in premarket trials. For example, in the case of a device indicated for treatment-refractory depression, the randomised postapproval study showed no dose–response for the treatment, leading to changes in the device labelling.³² In addition, longer term observation would help public health officials identify trends in adverse events, and for devices, possible malfunctions and design deficiencies.^{33 34} For example, a global registry of patients undergoing total hip or total knee arthroplasty helped regulators uncover early failure rates of metal-on-metal hip implants.³⁵

Our study revealed that roughly a third of ophthalmic devices were approved contingent on completion of postapproval study commitments. Moreover, for ophthalmic drugs as well as for devices, the design and results of these postapproval studies were not readily transparent. These findings are consistent with other studies of postapproval trials,^{36 37} and highlight the importance of improving the transparency and effectiveness of surveillance initiatives. It has been recommended that the FDA require more postapproval studies for high-risk medical devices and that the FDA explore mechanisms to ensure more timely completion of these studies.³⁸ These recommendations are concordant with current initiatives by the FDA to improve the state of postapproval monitoring of invasive devices. In 2007, the US Congress passed the FDA Amendments Act, which authorised the FDA to require postapproval studies for a prescription drug's approval. In 2012, the FDA committed to develop a comprehensive postapproval surveillance system for medical devices.^{39 40}

In addition, our results may have implications for policymakers given the ongoing scrutiny of differences in regulatory standards between drugs and devices.⁴¹ Following a series of safety controversies,⁴² the European Union is reviewing proposals to overhaul its device regulations, which currently do not require that high-risk devices demonstrate effectiveness prior to their approval.^{21 43 44} Some commentators have argued that undertaking randomisation or masking is technically challenging for agents involving implantation. Others have pointed out that the burden of proof should be higher for implanted devices since they cannot be easily discontinued, and sham procedures could be used to minimise potential placebo effects.^{45 46} For example, investigators conducted two randomised masked sham-controlled trials to evaluate the safety and efficacy of dexamethasone intravitreal implant compared with sham in eyes with vision loss due to macular oedema associated with branch retinal vein occlusion or central retinal vein occlusion.^47–49 These pivotal trials indicated that dexamethasone intravitreal implant could reduce the risk of vision loss.

Our analysis has a number of strengths and limitations. To the best of our knowledge, this study is the first to characterise the state of trial evidence underpinning the approval of new drugs and devices available to ophthalmologists and their patients, spanning a 10-year period. Data were obtained from publicly available documents that may have omitted or not systematically included additional information on clinical evidence, postapproval commitments and other elements that investigators typically use to systematically evaluate study quality. Further research on applying validated instruments of study quality to FDA review summaries would be warranted. However, the online review dossiers are required by rule to include the “summaries, conclusions, and results of all clinical experience or investigations (whether adverse or supportive)…that are relevant to an assessment of the risks and benefits” of the device or drug,⁵⁰ and our study end points have been used in previous assessments of regulatory review dossiers. We also focused on the design of pivotal trials, which are the ones designated by the FDA as the primary source of evidence it uses to make an approval decision. Earlier stage or other supportive clinical trials may have different characteristics. Finally, ophthalmic devices may pose a lower level of risk to patients than ophthalmic drugs, which may explain the differences in clinical trial design leading to approval. However, to minimise inherent differences in risk between drugs and devices, we focused on high-risk implantable devices, which are required by the FDA to undergo the most stringent regulatory pathway (PMA).

Clinicians and patients who need to make informed treatment decisions should keep in mind the varying levels of evidence underpinning new ophthalmic therapies. Randomised controlled trials are widely regarded as the gold standard for clinical investigations. If such trials cannot or will not be required before approval, postapproval studies could be used to confirm or refute early indications of safety and efficacy of these therapeutics.