What is an adequate and well controlled study?

With the pandemic events of 2020, there was a great awareness of evaluation of novel preventative and therapeutic agents for COVID-19 infection. Upon hearing of news of a novel product, the public at large questioned the nature of the evidence. I think people might have ask questions such as “Was it just an anecdotal report in a few patients?“, “Was it a large controlled trial?“, and “Where was it conducted”. To me, these questions all come under the heading of “Was this an adequate and well controlled study?“. As scientists, in particular clinical scientists, we have several criteria for assessing the quality of a clinical study and clinical evidence.

From a historical perspective, I go back to “Koch's postulates”, developed by Robert Koch in the 19th century for establishing the microbiological etiology of infection and disease. These postulates are (1) The microorganism must be found in diseased but not healthy individuals; (2) The microorganism must be cultured from the diseased individual; (3) Inoculation of a healthy individual with the cultured microorganism must recapitulate the disease; and finally (4) The microorganism must be re-isolated from the inoculated, diseased individual and matched to the original microorganism [1]. These can be expanded to apply to consideration of new therapeutics.

For example, in ocular surface disease, you might posit that a particular cytokine is elevated, and a treatment to reduce that cytokine might be effective. This would require steps akin to Koch's postulates. First you might show that patients with disease have elevated cytokine levels in ocular tissues, whereas healthy subjects do not. Then, you might want to show that treatment with your novel agent reduces cytokines in both normal animals and in a model of dry eye disease. You would want to show that treatment in humans (patients and/or subjects) with your novel agent reduces cytokine levels. And then finally, you would want to show that treatment in patients with this treatment decreases the signs and symptoms of disease.

Today, I think that when scientists think of scientific assessment, many think of the Cochrane System. According to their website, a Cochrane Review is “...a systematic review of research in health care and health policy that is published in the Cochrane Database of Systematic Reviews. Cochrane Reviews base their findings on the results of studies that meet certain quality criteria, since the most reliable studies will provide the best evidence for making decisions about health care. Authors of Cochrane Reviews apply methods which reduce the impact of bias across different parts of the review process…” (www.cochranelibrary.com).

The Cochrane method ties into “levels of evidence”, which were originally described in a report by the Canadian Task Force on the Periodic Health Examination in 1979 [2]. As shown in Table 1 , this is a relatively simple 4-level scale. It ranges from expert opinion (low level) to randomized clinical trial (RCT). The levels of evidence were further described and expanded by David Sackett in an article on levels of evidence for antithrombotic agents in 1989 (Table 2 ) [3]. This is a 5-level scale, ranging from case reports (low level) to large, RCTs (plural). These scales have been expanded by many organizations, with a 10-level scale developed by the Centre for Evidence Based Medicine (CEBM) for treatment (http://www.cebm.net, Table 3 ).4.

Table 1.

Canadian task force on the periodic health Examination's levels of evidence*.

Level	Type of evidence
I	At least 1 RCT with proper randomization
II.1	Well-designed cohort or case-control study
II.2	Time series comparisons or dramatic results from uncontrolled studies
III	Expert opinions

RCT = Randomized clinical trial.

Adapted from Canadian Task Force on the Periodic Health Examination [2].

Table 2.

Levels of evidence from sackett.

Level	Type of evidence
I	Large RCTs with clear cut results
II	Small RCTs with unclear results
III	Cohort and case-control studies
IV	Historical cohort or case-control studies
V	Case series, studies with no controls

RCT = Randomized clinical trial.

From [3].

Table 3.

Levels of evidence for therapeutic studies.

Level	Type of evidence
1A	Systematic review (with homogeneity) of RCTs
1B	Individual RCT (with narrow confidence intervals)
1C	All or none study
2A	Systematic review (with homogeneity) of cohort studies
2B	Individual Cohort study (including low quality RCT, e.g. <80% follow-up)
2C	“Outcomes” research; Ecological studies
3A	Systematic review (with homogeneity) of case-control studies
3B	Individual Case-control study
4	Case series (and poor quality cohort and case-control study
5	Expert opinion without explicit critical appraisal or based on physiology bench research or “first principles”

RCT = Randomized clinical trial.

From [4].

Levels of evidence is closely related to our current concepts on “Evidence Based Medicine (EBM)”. Sackett in 1997 wrote EMB “…whose philosophical origins extend back to mid-19th century Paris and earlier, is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence-based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research.”

In a review in 2008, Masic et al. wrote”…“Evidence based medicine is the conscientious, explicit, judicious and reasonable use of modern, best evidence in making decisions about the care of individual patients. EBM integrates clinical experience and patient values with the best available research information. It is a movement which aims to increase the use of high quality clinical research in clinical decision making. EBM requires new skills of the clinician, including efficient literature-searching, and the application of formal rules of evidence in evaluating the clinical literature. The practice of evidence-based medicine is a process of lifelong, self-directed, problem-based learning in which caring for one's own patients creates the need for clinically important information about diagnosis, prognosis, therapy and other clinical and health care issues.” [5].

In the Clinical Trial Design and Regulatory subcommittee of the TFOS DEWS II report, we considered standards for well-controlled studies [6]. We cited the U.S. Congress’ passage of the Kefauver-Harris amendment to the Federal Food, Drug & Cosmetic Act in 1962. That law required approval of a new drug be based upon evidence of effectiveness which is based on adequate and well-controlled clinical studies conducted by qualified experts. The definition of an adequate and well-controlled study was subsequently defined in 21 CFR 314.126. These characteristics are provided in Table 4 . The subcommittee recommended that researchers consider these attributes at the outset of clinical trial design and in selecting sites for the conduct of a clinical trial. Note that subcommittee also understood that not all studies can meet all these requirements. In particular, pilot or early stage clinical trials tend to be small, and of relatively low biostatistical power. These characteristic however are recommended for pivotal, Phase 3 studies.

Table 4.

Definition of an adequate and well controlled study.

(1)Clear statement of the objectives of the investigation.
(2)Uses a design that permits a valid comparison with a control to provide a quantitative assessment of drug effect. (i) Placebo, (ii) Dose- comparison, (iii) No treatment (iv) Active (v) Historical control.
(3)Method of selection of subjects provides adequate assurance that they have the disease or condition being studied.
(4)Method of assigning patients to treatment and control groups minimizes bias.
(5)Adequate measures are taken to minimize bias on the part of the subjects, observers, and analysts of the data.
(6)Methods of assessment of subjects' responses are well defined and reliable.
(7)There is an analysis of the results of the study adequate to assess the effects of the drug. (d)The test drug is standardized as to identity, strength, quality, purity, and dosage form to give significance to the results of the investigation.

The conduct of clinical trials is provided in detail in “Good Clinical Practice” (GCP). Recommendations for the conduct of trials according to GCP are provided in detail in the International Conference on Harmonisation (ICH) “Guideline For Good Clinical Practice (E6, https://database.ich.org/sites/default/files/E6_R2_Addendum.pdf). Extramural studies sponsored by the National Institutes of Health do not necessarily cited ICH E6, however, they effectively create such standards with the study Manual of Procedures.

The European Union (and European Medicines Agency) do not seem to use the same terms as the U.S. FDA. However, they do refer to the “reliability and robustness” of the clinical trial and its data, based on adequate monitoring of the trial; traceability, storage, return and destruction of investigational medicinal products; adequate recording, storage and handling of the clinical trial data; rules on manufacturing, importation and distribution of investigational and auxiliary medical products to clinical trial sites; appropriate labeling of investigational medical products; the quality of the investigational medicinal products; data protection and informed consent; adherence to GCPs; statistical approaches, design of the clinical trial and methodology, including sample size and randomization, comparator and endpoints (Regulation 536/2014, https://ec.europa.eu/health/human-use/clinical-trials/regulation_en).China also follows ICH E6.

In summary, there are many published, accepted standards for the definition of an adequate and well controlled study. These definitions then lead us to the practice of evidence based medicine. This “science of medicine” is often in contradistinction to the “art of medicine”, which is often driven by anecdotal clinical experience. This is especially true in ocular surface disease, in which there is a wide variability of patient signs and symptoms, and thus the need to individualize therapy. Nonetheless, it is adequate and well controlled studies that are used by regulators to approve novel therapies for marketing, so that they then become available for patients.

News from pharmaceutical and medical device companies

Ophthalmic products related to the ocular surface

• •Aldeyra started enrollment in its phase 3 TRANQUILITY study of 0.25% reproxalap ophthalmic solution for the treatment of dry eye disease (December 2020).
• •Aurinia reported from the their Phase 2/3 AUDREY clinical study evaluating voclosporin ophthalmic solution (VOS) for the potential treatment of dry eye syndrome (November 2020).
• •Bausch Health initiated the second of two phase 3 studies evaluating the investigational treatment NOV03 (perfluorohexyloctane) to treat the signs and symptoms of dry eye disease (November 2020).
• •Novaliq started enrollment in its phase 3 clinical trial ESSENCE-2 of CyclASol® (0.1% cyclosporine A in a proprietary formulation) for the treatment of dry eye diseases (December 2020).
• •Oyster Point Pharma submitted an NDA to the U.S. FDA for its OC-01 (varenicline) nasal spray for the treatment of signs and symptoms of dry eye disease (December 2020).

Ophthalmic products not related to the ocular surface

• •Aerie received marketing authorization from the European Commission (EC) for its for Roclanda (netarsudil and latanoprost ophthalmic solution) 0.02%/0.005% for the reduction of elevated intraocular pressure (IOP) in adult patients with primary open-angle glaucoma or ocular hypertension for whom monotherapy with a prostaglandin or netarsudil provides insufficient IOP reduction (January 2021).
• •Aerpio presented results of a Phase 2 program of topical ocular razuprotafib in patients with glaucoma as an additive agent to latanoprost (January 2021).
• •Clearside Biomedical started enrollment in its OASIS phase 1/2a clinical trial of CLS-AX (axitinib injectable suspension) in patients with wet AMD (January 2021).
• •Eyenovia started enrollment in its Phase 3 VISION-1 study of pilocarpine (a proprietary formulation in a proprietary delivery system) for treatment of presbyopia. The firm also submitted a New Drug Application (NDA) to the U.S. FDA for its proprietary fixed combination of phenylephrine 2.5% and tropicamide 1% for diagnostic mydriasis (December 2020).
• •EyePoint announced results from its Good Laboratory Practice (GLP) preclinical toxicology study of intravitreal EYP-1901 (a proprietary delivery system of vorolanib), intended to support evaluation in patients wet age-related macular degeneration (wet AMD, December 2020). The firm also started enrollment in its Phase 1 study of intravitreal EYP-1901 (vorolanib, a tyrosine kinase inhibitor, in its bioerodible Durasert® delivery system) in patients with wet AMD (January 2021).
• •Gemini Therapeutics received Fast Track designation for its GEM103 (a recombinant native complement modulator, full-length recombinant complement factor H protein) (January 2021). The firm also started a Phase 2a trial of GEM103 in patients with wet AMD (February 2021).
• •Genentech/Roche announced results from four Phase 3 programs in which its intravitreal bispecific antibody, faricimab, was evaluated: YOSEMITE and RHINE in patients with diabetic macular edema (December 2020), and TENAYA and LUCERNE in patients with wet AMD (January 2021).
• •Glaukos presented results from a study of its iStent Infinite™ Trabecular Micro-Bypass System in patients with open-angle glaucoma uncontrolled by prior surgical or medical therapy. The firm also announced results with its iDose TR sustained-release travoprost implant in patients with glaucoma (January 2021).
• •Iveena reported results of its Phase 1/2a clinical trial of IVMED-80 eyedrops for the treatment of keratoconus. IVMED-80 is a regulator of lysyl oxidase (October 2020).
• •Kubota Vision entered into a collaborative research agreement for clinical studies of their wearable myopia-control device (November 2020).
• •Nicox and its partner Ocumension Therapeutics started enrollment in a Phase 3 clinical trial in China with Zerviate™ (cetirizine ophthalmic solution) for the treatment of ocular itching associated with allergic conjunctivitis in China (December 2020).
• •Ocular Therapeutix submitted a supplemental NDA to the U.S. FDA for its Dextenza® (dexamethasone ophthalmic insert) for the treatment of ocular itching associated with allergic conjunctivitis (December 2020).
• •Ocuphire completed enrollment in its MIRA-2 Phase 3 study of its Nyxol® (phentolamine) to reverse pharmacologically-induced mydriasis (January 2021). The firm also started its LYNX-1 study evaluating the safety and efficacy of Nyxol® in night vision disturbances (January 2021).
• •ONL Therapeutics will be developing ONL 1204 (a Fas inhibitor) for treatment of retinal detachment, glaucoma and dry AMD (December 2020).
• •PolyActiva completed a Phase I clinical study for its Latanoprost FA SR Ocular Implant for the treatment of elevated introacular pressure (November 2020).
• •REGENXBIO started enrollment in its ALTITUDE study, a Phase II trial to evaluate the suprachoroidal delivery of RGX-314 gene therapy using the suprachoroidal space microinjector for the treatment of diabetic retinopathy (DR, December 2020).
• •SIFI completed enrollment of its Phase 3 trial for evaluation of polihexanide in patients with Acanthamoeba keratitis (November 2020).
• •Stargazer Pharmaceuticals initiated a Phase 2a study of its STG-001 in patients with Stargardt Disease (November 2020).
• •Surface Ophthalmics announced results from a Phase 2 trial for SURF-201 (0.2% betamethasone in a unique vehicle) for the treatment of post-cataract surgery pain and inflammation. The firm also started enrollment in a Phase 2 trial of its SURF-200 (betamethasone in a unique vehicle) for the treatment of acute dry eye (February 2021).
• •Tsubota Laboratory announced results of an exploratory clinical trial of a medical using violet-light technology for the suppression of myopia progression (December 2020).

Gene and cell therapy

• •Atsena Therapeutics will be developing gene therapy for GUCY2D-associated Leber congenital amaurosis (LCA1) disease (December 2020).
• •Foundation Fighting Blindness listed 42 ongoing trials of therapeutics in inherited retinal disease and dry AMD (January 2021).
• •Janssen purchased Hemera Biosciences which is developing a CD59 gene therapy for the treatment of geographic atrophy (December 2020).
• •Lineage Cell Therapeutics completed enrollment in a Phase 1/2a study of OpRegen® (a subretinal investigational retinal pigment epithelium cell therapy) in patients with geographic atrophy (November 2020).

Other news about pharmaceutical and medical device firms

• •AstraZeneca received a positive recommendation from the European Medicine's Agency for its COVID-19 vaccine (January 2021).
• •Several firms recalled oral metformin products due to the detection of N-Nitrosodimethylamine (October 2020 to January 2021).

Regulatory, government, and other research news

• •The United Kingdom National Institute for Health and Care Excellence (NICE) issued a final appraisal determination recommending Novartis' brolucizumab (Beovu®) treatment for wet AMD (December 2020).
• •The U.S. National Institutes of Health encouraged rapid, collaborative reporting of clinical trials results with COVID-19 therapies on clinicaltrials.gov (November 2020).
• •In the relatively few serious allergic reactions to COVID-19 vaccine injections, there may be a connection to polythelyne glycol (PEG), an excipient also used in ophthalmic products (December 2020).
• •The U.S. Department of Health and Human Services (of which FDA is a part) highilighted their significant concerns about companies offering or paying remuneration to health care professionals in connection with speaker programs (November 2020).
• •As dermatologists expand the off-label use of topical dermal timolol (using the ophthalmic product), there is a growing realization of cardiopulmonary adverse events (well known to the ophthalmic community, January 2021) [7].
• •
  The U.S. FDA (November 2020 to January 2021):

  • o
    Approved 53 new drugs and 8 biologics in Calendar 2020. Seven drugs for ophthalmic indications in this period were (in chronological order): Tepezza™ (teprotumumab-trbw), Durysta™ (bimatoprost), Fluorescein/Benoxinate, Upneeq™ (oxymetaxoline), Enspryng™ (satralizumab-mwge), Cystadrops™ (cysteamine), and Eysuvis™ (loteprednol etabonate).
  • o Selected Janet Woodcock, M.D., (currently director of CDER, and who was detailed on Operation Warp Speed) as interim director of FDA.
  • o Terminated a program regarding the removal of unapproved drugs after an approval of the same drug, as well is considering how to deal with “grandfathered drugs” (marketed prior to 1938) [8].
  • o Is reorganizing the office of generic drugs to strengthen its operations and allow the office to meet the evolving needs of generic drug review.
  • o Providing direction on increasing the sterility and other aspects of pharmacy compounded pharmaceuticals.
  • o Announced focus areas for regulatory science.
  • o Published guidances for Developing drugs for treatment of dry eye disease.
  • o Published other guidances include best practices in developing proprietary names for human prescription drug products, biosimilarity and Interchangeability, enhancing the diversity of clinical trial populations, investigational new drug applications for Individualized antisense oligonucleotide drug products [9], and protecting participants in bioequivalence studies for generic drugs during the COVID-19 health emergency.

Declaration of competing interest

Gary D. Novack PhD consults with numerous pharmaceutical firms.