Improving Post-Approval Drug Safety Surveillance: Getting Better Information Sooner

Abstract

Adverse drug events (ADEs) are an important public health concern, accounting for 5% of all hospital admissions and two-thirds of all complications occurring shortly after hospital discharge. There are often long delays between when a drug is approved and when serious ADEs are identified. Recent and ongoing advances in drug safety surveillance include establishment of government-sponsored networks of population databases, use of data mining approaches, and formal integration of diverse sources of drug safety information. These advances promise to reduce delays in identifying drug-related risks, allowing earlier identification of risks as well as reassurance about the absence of specific risks.

INTRODUCTION

The science of assessing post-approval drug safety is changing rapidly. This paper describes recent regulatory and methodologic advances in drug safety surveillance. These advances promise to reduce the delays that often occur in the identifying important drug-related risks. By reducing these delays, these advances can reduce the clinical and public health burden caused by serious adverse drug events. For new drugs without important risks, these advances will provide earlier reassurance about safety.

CLINICAL AND PUBLIC HEALTH CONSEQUENCES OF ADVERSE DRUG EVENTS (ADES)

The use of prescription drugs continues its decades-long increase, with recent data showing that 90% of Americans age 65+ and 48% of Americans of all ages take at least one prescription drug in a given month (1). Given such widespread medication use, it is not surprising that adverse drug effects (ADEs) have become an important public health concern, accounting for 5% of all hospital admissions (2) and two-thirds of all complications occurring shortly after hospital discharge (3). Common, well-known toxicities of widely-used older drugs (e.g., hypoglycemia from antidiabetic drugs, neutropenic fever from chemotherapeutic agents, intestinal obstruction from opiates) are responsible for a clear majority of serious ADEs (4). However, low-frequency but serious ADEs associated with newer drugs (e.g., ventricular arrhythmia from terfenadine and cisapride, myocardial infarction from rofecoxib and possibly from rosiglitazone) seem to garner much more regulatory attention and news coverage. This may be due to the controversy that often surrounds unfolding evidence of a newly identified adverse effect, the commercial implications to highly marketed and profitable drugs, and/or the observation that people tend to be more accepting of known risks than unknown risks (5). Regardless, post-approval drug safety research is needed both to identify previously unrecognized ADEs and to better quantify and understand well-known ADEs so that their risks can be mitigated. As described below, there are often long delays between when a drug is approved and important adverse effects are identified.

DELAYS IN IDENTIFYING AND ELUCIDATING ADVERSE DRUG EVENTS

Despite the clinical and public health need for such research to identify serious adverse drug events, their identification can be delayed for many years after a drug is approved. In the US for example, 20% of drugs receive at least one new boxed warning (the strongest type of warning that can be placed on a drug’s label) after approval (6), with a median time between approval and issuance of a new boxed warning of 10 years (7). Such long delays can result in serious harm to many thousands of patients. For example, in the US alone, encainide and flecainide caused an estimated 50,000 premature deaths from cardiac arrhythmia (8), and rofecoxib may have caused an estimated 88,000–140,000 cases of serious coronary heart disease (9). To reliably detect rare adverse events, many thousands of patients taking a new drug need to be studied. Given the high per-patient cost of pre-approval trials, it is unrealistic to expect them to enroll a sufficient number of patients to reliably identify all serious adverse effects of drugs, especially uncommon effects, even if they are very important. Therefore, earlier post-approval identification, characterization, and mitigation of these risks is a major public health imperative, both to identify serious adverse effects and to provide reassurance when such effects are absent or vanishingly rare. New information about previously unknown drug effects comes from a variety of different sources, described below.

SOURCES OF INFORMATION UNDERLYING THE IDENTIFICATION OF NEW DRUG SAFETY INFORMATION

Spontaneous adverse drug reaction reporting systems use reports of suspected ADEs submitted by manufacturers, health professionals, and patients to identify signals of potential ADEs. Such systems were developed in industrialized countries in response to the thalidomide disaster of the early 1960s in which women given thalidomide (a sleep agent promoted as being safer than barbiturates) gave birth to infants with phocomelia, a rare but serious birth defect syndrome characterized by often severe limb deformities (10).

Spontaneous reporting systems remain to this day a crucial means of identifying important post-approval drug safety information. Their importance is illustrated by a recent study by Lester and colleagues, who identified the sources of information that led to the 407 drug safety labeling changes that were made in the US in 2010. They identified 500 sources of information that led to these changes. Spontaneous reports were the leading source of such information, accounting for 52% of the sources (7). Other information sources that commonly led to safety-related labeling changes were clinical trials (16%), pharmacokinetic studies (11%), published case reports (6%), and observational pharmacoepidemiologic studies (6%). Spontaneous reporting systems appear to be particularly effective at identifying adverse events that occur rarely in the absence of drug exposure (e.g., rhabdomyolysis), but seem to perform poorly at identifying adverse events that represent an increase in the rate of relatively common events (e.g., myocardial infarction in persons with arthritis) (11). Further, spontaneous reports are by their nature anecdotal, and while useful for identifying potential drug safety signals and as a source of hypotheses about such features as induction period, susceptible subgroups, etc., they are of much more limited value for measuring incidence and inferring causality. While new surveillance systems such as those described below should increase the role of systematically collected healthcare data in identifying and quantifying new adverse effects, spontaneous reporting systems are likely to remain an essential component of drug safety surveillance for the foreseeable future.

CONTINUUM OF EVIDENCE THAT A DRUG CAUSES OR DOES NOT CAUSE AN ADVERSE DRUG EVENT

Figure 1 depicts a conceptual continuum of evidence that any given drug causes or does not cause any particular adverse event. Such a continuum can be said to exist for every drug-adverse event pair. This evidence continuum ranges from evidence that the risk is absent, or at most small, on the left to an established, well-characterized risk on the right. It is important to distinguish strength of the drug-outcome association from strength of the evidence, although the two are related. This relationship stems from the fact that strength of association (often expressed as risk ratio or rate ratio) is widely regarded as an important factor in considering the strength of evidence for a causal relationship (12). Nevertheless, there can be strong evidence for a weak association (e.g., a meta-analysis of rigorous studies showing that oral contraceptives are associated with a 1.24-fold risk of breast cancer (13)) or weak evidence of a strong association (e.g., a non-population-based case-control study showing that drinking three or more cups of coffee per day was associated with a 2.7-fold risk of pancreatic cancer (14), an association that was later refuted)(15).

Figure 1.

Continuum of evidence that a given drug causes a given adverse outcome.

On the far left hand side of the continuum lies evidence against risk. While pharmacoepidemiologic studies can demonstrate that any incremental risk, if present, is within certain numeric bounds, neither they nor any other kind of empiric research can fully disprove the existence of effect. For example, medications used to treat attention deficit hyperactivity disorder (ADHD) produce modest increases in average blood pressure and heart rate (16), which in population-based epidemiologic studies have been associated with an increased risk of cardiovascular events. These hypertensive and chronotropic effects of ADHD medications together with case reports of cardiovascular events occurring in children and adolescents taking ADHD medications led to widespread concern that ADHD medications might increase the risk of cardiovascular events in this population (17). Adding to the concern is the high prevalence of exposure to ADHD medications that is seen in some countries (17). Because of this concern, several large pharmacoepidemiologic studies were performed to assess this potential risk. These studies, which collectively included more than 2.5 million children and adolescents, identified no increased risk of myocardial infarction or stroke associated with use of ADHD medications in children and adolescents (18–20). However, despite their large size, these studies still cannot exclude the possibility of a small or delayed risk, or a risk in an as-yet unidentified subgroup (e.g., those with undiagnosed congenital anomalies). Nevertheless, these studies do provide important reassurance that any incremental risk of cardiovascular events due to short-term use of ADHD medications in the overall population of children and adolescents, if it exists at all, is probably small.

At the far right-hand side of the evidence of risk continuum in Figure 1 lie established, well-characterized risks. However, even risks that are regarded as well-characterized usually need additional research to better understand their frequency, determinants, and mechanisms, and to explore strategies to mitigate them. For example, the anticoagulant warfarin has been used clinically since 1954, and many studies have examined the risk of bleeding while on warfarin and determinants of this risk. Nevertheless, warfarin continues to be widely-used, and bleeding due to warfarin remains common and of enormous clinical and public health importance. Therefore, studies to better understand and reduce the risk of warfarin-associated bleeding continue to be performed. Such studies include examinations of genetic determinants of responses to warfarin (21), drug-drug interactions involving warfarin (22), and the effectiveness of genetically-based warfarin dosing to minimize the risks of bleeding and thromboembolic events (23). Thus, even well-characterized risks often need further study to better understand and mitigate them. This is especially true given the high public health burden of common, serious adverse effects of widely-used drugs.

As described above, it can take many years after a drug’s approval for evidence concerning the existence of an ADE to accumulate, and at any juncture there can be great uncertainty and disagreement as to what the actual level of evidence is. Making regulatory decisions in the setting of such uncertainty and disagreement can be challenging for public health officials who on the one hand wish to minimize the burden of ADEs and on the other avoid dissuading or preventing safe use of the drugs in question. For example, at a Food and Drug Administration Advisory Committee hearing on the issue of whether antiepileptic drugs increase the risk of suicide, committee members expressed concern that warning the public about a potential but unproven risk of suicide might dissuade some patients from taking needed antiepileptic drugs, leading to preventable seizures and their attendant consequences, such as motor vehicle crashes (24). Such seizures would be particularly unacceptable if the potential antiepileptic drug-suicide link was later refuted. Therefore, earlier development of rigorous evidence for or against such risks would greatly benefit regulators, clinicians, manufacturers, and most importantly patients.

A typical scenario for the development of evidence for a particular drug-ADE pair (Figure 1) is to start at “no suggestion or refutation of risk” and move to “suggestion of possible risk” (often based on case reports) to “evidence for risk” to “established risk” and then possibly to “well-characterized risk” (often based on controlled pharmacoepidemiologic studies). This pattern has characterized many recently identified serious ADEs, including terfenadine-ventricular arrhythmia (25, 26), cisapride-ventricular arrhythmia (27, 28), and cerivastatin-rhabdomyolysis (29, 30). Regulatory action has often preceded confirmation and characterization of risk in pharmacoepidemiologic studies, particularly when acceptable therapeutic alternatives were available. With the ongoing development of government-sponsored prospective epidemiologic surveillance systems (described below) that increasingly provide early pharmacoepidemiologic data about potential ADEs, regulatory action in the absence of pharmacoepidemiologic data may soon be less common.

Moving from right to left on the evidence of risk continuum is also possible, although seemingly less common. That is, data can emerge that argue against the existence of a risk that had previously been suggested or believed to be true. For example, metformin is the second member of the biguanide class of antidiabetic drugs, and at the time of approval was feared to cause lactic acidosis because phenformin, the first marketed biguanide, was withdrawn from the market because of this sometimes fatal adverse event (31). However, in the time since metformin has become widely used, evidence has emerged that the risk of lactic acidosis in patients with diabetes treated with metformin is no higher than that risk in similar patients treated with other antidiabetic drugs (32), arguing against an effect of metformin on increasing the risk of lactic acidosis. Further, the use of metformin appears safe even in persons with renal insufficiency (33), a very large population in whom metformin is currently officially contraindicated.

Thus, post-approval information can provide reason for either concern or reassurance. As discussed below, recent and proposed changes to drug safety surveillance promise to reduce the current delay in the generation of this information.

RECENT AND PROPOSED APPROACHES TO IMPROVING POST-APPROVAL DRUG SAFETY SURVEILLANCE

Establishment of Government-Sponsored Networks of Population Databases for Drug Safety Surveillance

When a new drug is approved, identifying rare adverse effects (or conversely, providing reassurance about the apparent absence of such effects) as early as possible necessitates obtaining access to databases that record drug exposures and health outcomes in very large populations. Historically, population databases, such as those recording the experience of people in a particular health care plan, have each been examined individually, most often by academic investigators and/or pharmaceutical companies. More recently, government agencies have established networks of population databases and begun using them to perform medical product safety surveillance. Such networks can be used to prospectively assess the safety of a drug, vaccine, or other medical product as it begins to be used. Prominent examples of such networks are presented in Table 1.

Table 1.

Large networks of government-sponsored networks of population databases used for research and surveillance of drug and vaccine safety.

Sponsor	Name	Populations covered*	Number of individuals in population actively contributing data	Examples of recent evaluations
Health Canada; Canadian Institutes of Health Research	Canadian Network for Observational Drug Effect Studies (CNODES) (58)	Alberta, British Columbia, Manitoba, Nova Scotia, Ontario, Quebec, UK Clinical Practice Research Database	3.3 million in Canada + 11.8 million in the UK	Statins and acute kidney injury (59) PPIs and risk of pneumonia (59) Antipsychotic drugs and hyperglycemia (59)
European Commission	Exploring and Understanding Adverse Drug Reaction by integrative mining of clinical records and biomedical knowledge (EU-ADR) (60)	Italy, Denmark, Netherlands, United Kingdom,	20 million (61)	Nonsteroidal anti-inflammatory drugs and gastrointestinal bleeding (60)
US Food and Drug Administration	Mini-Sentinel (62)	multiple health plans in United States	43 million (63)	Dabigatran and bleeding (42) Agents acting on the renin-angiotensin-aldosterone system and angioedema (37)
US Food and Drug Administration	US Federal Partners’ Collaboration (64)	US Medicare, Medicaid, Department of Veterans Affairs, and Department of Defense	153 million (64)	Safety of antiviral drugs used in the 2009 H1N1 (64) influenza pandemic Dronedaroneand heart failure (65)
US Centers for Disease Control and Prevention	Vaccine Safety Datalink (66)	multiple health plans in United States	9 million (67)

^*Entire populations of these geographic areas are not necessarily covered.

Crucial to the operation of many of these networks is the use of a distributed data model in which the data remain in possession of the data holder, which is often is a health plan, commercial insurance company, or other non-governmental entity (34). This distributed model is in contrast to a centralized data model in which data holders transfer a full copy of the data to a central repository. A major advantage of a distributed data model is that data holders retain physical possession and control over the data. This ameliorates issues related both to patient privacy and to the proprietary value of the individual-level data. An additional advantage of a distributed data model is that it ensures that the data holders, who are most familiar with the environment in which the data were produced, have maximal opportunity for scientific input into the planning, conduct, and reporting of safety evaluations. In some distributed data models, each data holder creates a separate copy of its data in a standard format. This approach is known as a common data model (35, 36). Use of a common data model allows statistical analysis programs to be written centrally and executed peripherally, with aggregated results (which can be either devoid of personal identifiers or be restricted to only highly summarized personal identifiers, depending on the analytic approach) provided to the coordinating center (37). Writing statistical programs centrally improves consistency, reduces the opportunity for error, and confers operational efficiencies. The development of analytic methods that accommodate distributed data environments is an active area of methodologic research (38, 39), as is the development of methods for safety surveillance that accommodate health care data that accrue over time (40, 41).

One early example of a post-approval surveillance activity using a government-sponsored network was an assessment of the risk of serious bleeding in users of dabigatran, an anticoagulant approved in Europe and Canada in 2008 and the US in 2010. In response to a large number of spontaneously reported episodes of serious and fatal bleeding in users of dabigatran, Mini-Sentinel (a pilot project sponsored by the FDA to create an active surveillance system—the Sentinel System—to monitor the safety of FDA-regulated medical products) rapidly compared the frequency of serious bleeding between new users of dabigatran and new users of warfarin in a set of health plans. The rapid evaluation found that rates of serious intracranial and gastrointestinal bleeding in dabigatran users were no higher than the corresponding rates in users of warfarin (42). Although this evaluation was not randomized and did not control for potential confounding factors, it did provide relatively early reassurance that real-world use of dabigatran was not associated with higher bleeding rates than warfarin, in accordance with the results of a prior randomized trial (43). In follow-up to this initial evaluation, FDA is conducting additional assessments that will control for measured potential confounding factors (42).

Use of Data Mining to Identify Potential Adverse Drug Events

We are currently experiencing an explosion in the application of data mining (i.e., the use of computational processes to discover patterns in large data sets) to predict, identify, and explain drug effects. Data mining approaches use a wide variety of source data, including collections of anecdotal adverse drug event reports (42), coded and free-text health care data (44–46), published biomedical papers (47), curated drug information sources (48), and internet message boards (49, 50). One family of data mining approaches used to predict and explain drug action is the construction of biological networks linking drugs with other entities including genes, metabolites, microRNA, proteins, etc. through pairwise links reflecting current knowledge and/or new empiric findings (51). Construction of such networks is a key tool of systems pharmacology, an emerging field that uses empiric observation and computation to develop an understanding of drug action across multiple scales of complexity (52). Naturally, many potential relationships identified through network analysis and other data mining approaches may be spurious rather than reflect biologically-driven relationships. Therefore, potential relationships identified through data mining require confirmation that they are biologically and clinically meaningful, and elucidation of the mechanisms of such relationships. That said, data mining approaches hold promise for reducing the delay in identification of important ADEs.

Integration of Diverse Sources of Information to Better Predict and Identify Adverse Drug Effects

Many different types of information are used to predict, identify, and explain potential adverse drug effects. This information includes the structural, physiochemical, pharmacokinetic, and pharmacodynamic properties of drugs, and characteristics of the multiple pathways and systems that the drugs and their metabolites interact with. Such information relates to vastly different biologic scales (e.g., molecules, tissues, organs, physiologic systems, whole organisms, populations) and derives from a wide variety of methods (e.g., in vitro studies, animal studies, human biomarker studies, randomized trials examining health outcomes, spontaneous reporting systems, pharmacoepidemiologic studies).

Biologic plausibility is often considered when evaluating whether a given drug safety signal (such as one arising from spontaneous reporting systems) indicates a true causal relationship between a drug and a health outcome. Biologic plausibility is assessed and expressed implicitly and qualitatively, in contrast to the results of clinical and epidemiologic studies of health outcomes, which are expressed quantitatively (e.g., risk ratio, risk difference). Integration of biologic plausibility together with the results of health outcome studies is also done qualitatively, with biologic plausibility often assessed in response to the emergence of unexpected drug-outcome associations. As pointed out in the 2012 Institute of Medicine (IOM) report entitled Ethical and Scientific Issues in Studying the Safety of Approved Drugs (53), drug safety would be improved by the development of approaches to express biologic plausibility quantitatively, and further through quantitative incorporation of biologic plausibility together with the results of health outcome studies. As further pointed out by the 2012 IOM report, a Bayesian approach to quantifying biologic plausibility and incorporating biologic plausibility with health outcome data holds promise. Such an approach might begin with the elicitation from biological scientists of a mechanistically-based prior probability that a drug causes a given outcome or a distribution of such probabilities if (as seems likely) a single satisfactory estimate cannot be elicited. This distribution of prior probabilities would then be combined explicitly and mathematically with the results of health outcomes studies as they emerge. Such results would be expressed as a Bayes factor, which is the ratio of the posterior probability that a causal relationship exists relative to the probability that one does not (54). Multiplying the mechanistically-based prior probability by the Bayes factor derived from health outcome studies would yield a posterior probability (with a credible interval, the Bayesian analogue of a confidence interval) that the drug causes the outcome. This posterior probability could inform (but not replace) regulatory decision-making and later serve as a prior probability when subsequent mechanistic and health outcome data emerge.

Conceivably, a set of mechanistically-based prior probabilities that a new drug causes a set of health outcomes commonly associated with drugs (e.g., liver injury, myocardial infarction, ventricular arrhythmia, venous thromboembolism, pancreatitis, etc.) could be estimated when a drug is approved and updated as new mechanistic and health outcomes information emerges.

A number of challenges need to be addressed to make such an approach practical. One challenge is the uncertainty in identifying which biologic scientists should provide mechanistically-based probabilities. The concept of “biologic plausibility” is broad and ill-defined, and it is unclear who would give the best prior probabilities. It seems likely that experts in one area relevant to a drug’s mechanism may have little knowledge about others. Further, because many or most of the experts in the pharmacology of a new drug may have participated in that drug’s development, it may be difficult to identify experts with the requisite knowledge who do not possess disqualifying intellectual and/or financial conflicts of interest.

Experts being asked to provide mechanism-based prior probabilities should presumably be provided with standard summaries of the most relevant information. The ideal content and format of such information also needs study, and deciding what is “relevant” in this context may be difficult. Further, much of the relevant biologic information may be considered confidential and proprietary.

In addition, while there is a modest literature examining approaches to eliciting Bayesian prior probabilities (55–57), there is no agreed-upon standard, nor even criteria as to how different approaches should be assessed and compared.

Determining the best approach for eliciting prior probabilities is made challenging by the lack of a gold standard prior probability. While one could consider using “settled” cases (e.g., rofecoxib and myocardial infarction) to develop and test elicitation methods, it would be unrealistic to ask experts to forget what they know about the health outcome data.

Other challenges include how to express the full range of the results of health outcome data as a single Bayes factor. Pharmacoepidemiologic studies often yield multiple results that can inform causal judgments, including the overall association, associations with alternatively-defined outcomes, analysis of dose-response, duration-response, subgroups, and sensitivity to various assumptions. However, current Bayesian methods assume that a single metric expresses the strength of the evidence for the null vs. alternative hypotheses. Thus, approaches to deriving Bayes factors that more fully the broad range of study results are needed. Further, the potential for Bayes factors to incorporate potential sources of error such as selection bias, confounding, and information bias needs further development.

Finally, it is unclear who will provide the considerable resources needed to implement and maintain such a system for all drugs or even all new drugs, given that there is often a near-constant flow of new information for rapidly emerging safety issues.

Despite these challenges and knowledge gaps, quantitative incorporation of biologic plausibility with health outcome data is a promising approach to reduce the delay in identifying important drug safety issues.

SUMMARY AND CONCLUSIONS

The use of prescription drugs is continues to grow, and adverse drug effects have enormously important clinical and public health consequences. The discovery of new adverse events is often delayed for many years after a drug is approved, resulting in many thousands of deaths and cases of serious injury. At any point in time, the evidence for or against a given drug-outcome association can be plotted on a continuum of certainty, and this evidence changes over time. A number of ongoing and proposed changes to drug safety surveillance promise to reduce the delay in identifying these risks or, conversely, providing reassurance. These changes include establishment of government-sponsored networks of population databases, use of data mining, and explicit quantitative integration of diverse sources of drug safety information, including mechanistic information. These changes promise to reduce the delay in identifying serious adverse drug events and thus reduce their considerable clinical and public health burden.