The penumbra of randomized control trials

Abstract

Pre-occupation with randomized control trials as the basis of evidence-based medicine has increasingly shadowed other study designs over the last half a century. These include surveys, case-control studies, and case-cohort studies. They have the potential to overcome several ethical and cost constraints, but depend on the embedding of research in routine practice, emphasis on relevant but limited, accurate, and complete data, harnessing of information technology for this purpose, and epidemiological and statistical literacy among clinicians. Only then will it be possible to nurture and network research-oriented practices by therapeutic areas. Given these, the alternative study designs can pave the way to regulatory reforms that will ultimately benefit the discoverers, approvers and users of health-care tools.

INTRODUCTION

Penumbra is an area of diminished visibility as if under a gossamer veil. The randomized control trial (RCT) seems to cast such a shadow on other study designs, and it is about them that I wish to write today.

What has prompted me to think of this topic is a paper I read last September.[1] It addressed the question, “do anti-human immunodeficiency virus (HIV) drugs increase the risk of diabetes mellitus (DM) in HIV patients undergoing treatment with them?” It revealed that only saquinavir and stavudine, but not other anti-HIV drugs, increased the incidence rate of DM among HIV patients above that in the general population when adjustment was made for various confounding variables. I wondered how many years it would have taken to do a GCP-standard RCT to answer the same question, how much it would have cost, and what ethical issues would have beset its path.

Several highlights of this study engaged my attention. First, it was a case-cohort study rather than a RCT. Second, it was possible for the authors to assemble valid controls and to collect relevant data on them because of the availability of four national databases: The Danish HIV Cohort Study, The Danish Civil Registration System, The Danish National Hospital Registry, and The Danish National Prescription Registry. Third, because the authors could access these databases, they could identify confounders and adjust the outcome results for them. Fourth, they compared incidence rate ratios by the Poisson regression method. All these reminded me of a comment, made in 2001, in a booklet[2] which I consider a fine primer of clinical research.

“Indeed, ongoing action for health, if it does not contain an imbedded program of research, frequently becomes irrelevant, misleading or unnecessarily costly.”

CLINICAL RESEARCH GOALS

This quote emphasizes the importance of making research, or an enquiring attitude, an integral part of routine health-care practice so that it does not become outdated, harmful or unduly expensive. Only such research can deal with real health-care needs: first, by identifying illness, and its impact on individual, family and community; second, by defining its prevalence (extent), incidence (occurrence), importance and urgency; third, by developing drugs, diagnostics, devices and procedures to prevent, diagnose or treat it; and finally, by assessing the benefits, risks, and costs of the health-care tools.

ASSESSMENT RESEARCH

For any assessment research, the model must consider four items of information: the initial state, the exposure or intervention, the outcome or endpoint, and the confounders or concomitant factors. The inference from any such inquiry is beset with three issues: First, bias, either in the selection or measurement, which must be prevented or compensated; second, confounders, which may be known or can be surmised from the previous subject knowledge or reasoning; and third, random variation or chance, which statistical methods can deal with.

LIMITATIONS OF RCT

For over half a century, RCT has become the gold standard for evidence-based medicine, but it has its own limitations. First, the RCT focuses on the efficacy and safety, not on prevalence, incidence, causality or effectiveness. Second, it takes a long time to do and is very expensive. Third, although it enjoys high internal validity, it has limited external validity. Fourth, overemphasis on it casts a shadow on other study designs. Besides, its shortcomings in detecting rare or long-term adverse events are now well recognised. To quote Califf:[3]

“Despite numerous RCTs, performed within a structure of extensive documentation and data collection, serious shortcomings in a number of pharmaceutical therapies were not detected until after the drugs were approved and widely adopted by clinicians.”

Commenting on the external validity or generalizability of RCT, Ware and Hamel say:[4]

“Although RCTs provide essential, high-quality evidence about the benefits and harms of medical interventions, many such trials have limited relevance to clinical practice.”

ALTERNATIVE DESIGNS

Thus, there is a real need to remember and use alternative study designs for clinical studies. They are needed to measure prevalence and incidence of diseases, and to explore causality. They are needed to identify and prioritise health-care needs, and thus, guide the development of preventive, diagnostic, and therapeutic tools. Besides, they can make the assessment of these tools cost-effective and relevant, using the appropriate outcomes, and addressing genuine ethical constraints. Examples of such alternative study designs are cross-sectional surveys; cohort studies, either retrospective or prospective; and case-control studies. However, these pose their own challenges, which include: Sampling strategies for getting “representative” samples; accurate and complete data on the participants; subject knowledge for avoiding bias and for identifying or surmising confounders; and appropriate methods of statistical analyses.

REQUIREMENTS

However, if we wish to consider these alternative designs, we must ponder and focus on the following: First, relevant but limited data, which will facilitate accuracy and completeness; second, standardization of methods, procedures, and records; third, effective use of information technology to minimise drudgery, to automate certain tasks or algorithmic decisions, and to avoid human errors due to tiredness and boredom; fourth, involvement of regulators, patients and doctors, for judgment and value inputs in the planning of studies; fifth, nurturing, and networking of research-oriented practices by therapeutic areas; and sixth, creating epidemiological and statistical literacy (not expertise) among clinicians.

FEASIBILITY

Is such an approach possible? There are people who have tried and believe it is possible. van Weel and de Grauw say:[5]

“Through primary care registration networks and practice based research networks (PBRNs) it is possible to tap-in unselected care of patients and at the same time produce scientifically rigorous data. This enables research that represents the realities of primary care with the valid data.”

Attempts of this nature are being made for example, in the field of otology and neurotology. To quote the words of Tucci et al.:[6]

“PBRNs are the preferred research setting for descriptive/epidemiologic studies and studies that explore the effectiveness of treatments for the disease that are managed in community settings, away from the rubric of the academic medical center.

“We have formed a PBRN that we call the CHEER Network: Creating Health-care Excellence Through Education and Research.”

Relton et al.,[7]have proposed a model for cohort multiple RCTs in practice. It consists of forming patient cohorts with standardised records; identifying patients eligible for studies A, B, etc.; randomly selecting them for informed consent and test intervention; assigning the others to “usual” reference intervention; recording outcomes; and analysing the data by appropriate methods.

POSSIBLE REGULATORY REFORMS

Should such a scenario come to pass in the near future, some regulatory reforms might also enter the realm of possibility. For example, the current phase 3a could be replaced by a period of regulated distribution use and documentation after a new drug completes phase 2, and adequate data are available about its efficacy and safety. The data emerging during this period of supervised usage could be the basis for finalising the labeling for marketing authorisation. And marketing authorization could be synchronised with the beginning of pharmacovigilance and post-marketing surveillance.

Who will benefit from such reforms? First, the discoverers, because of lower cost of clinical research, and development. Second, regulators, doctors, and payers, because of substantial and realistic data for decision making. And finally, patients, because of early access to new, better and safer health-care tools at economical prices.

ANTHEM

I hope young clinical researchers will be encouraged, enabled and empowered by all stakeholders to make such reforms a reality. A day will come, then, when at least some of them will able to hum the following words of Frost:[8]

“I shall be telling this with a sigh

“Somewhere ages and ages hence:

“Two roads diverged in a wood and I-

“I took the one less traveled by,

“And that has made all the difference.”