Signals and Noise in Drug Safety Analyses

In this issue of Diabetes Care, our editorial team has decided to “push the envelope” by openly addressing a safety issue in drug development and monitoring that is gaining interest and provoking controversy. Traditionally, we have not felt it was our charge to comment on moral issues or political decisions related to aspects of medical care or delivery. Rather, it has been our goal and privilege to “survey the current landscape of clinical, research, and health care changes” and “to provide the most up-to-date information for our readers” (1). On one previous occasion we felt it was necessary to voice our opinion as an editorial on academic freedom (2), which we considered critical for the sake of scientific dialogue and collaboration across national borders. At present, we feel it is our charge to bring to the forefront another contentious topic that directly concerns the care of people with diabetes: the ongoing debate on the risks versus benefits of incretin-based therapies. Thus, in this issue we present four articles providing background material and specific arguments on this controversy.

As is currently well known and accepted, many parties have a stake in drug development, its regulation, and the subsequent use and monitoring of approved agents. Manufacturers of drugs and devices have obvious financial incentives to develop and market their products. We charge the regulatory agencies with guiding the process, approving safe agents for clinical use, and assuring long-term surveillance of their effects. The charge of the medical and scientific communities is to assist the pharmaceutical industry in developing new therapies, testing them, and advising how they can best be used in clinical practice. However, as outlined in a commentary by Dr. Eric P. Brass in this issue of Diabetes Care (3), the input of advisory committees is not always clear-cut in the approval regulatory process. The charge of regulatory agencies includes monitoring the long-term safety of drugs that have attained approval for clinical use, especially those for which specific adverse events have been suspected. When a drug is released, clinicians must judge how best to use it and balance the safety profile, among other available therapies, with the aim of providing the best “personalized” treatment for individual patients. Unfortunately, one can argue that the current systems to register efficacy and systematically report adverse events are largely inefficient or incomplete.

Recently, another force has emerged in this process: the media. The intensity of interest in drug choices has been increased by direct-to-consumer advertising, but uncertainty about possible adverse effects from some agents is now at the forefront of any discussion. In some cases, well-publicized allegations of risk from specific agents have provoked widespread anxiety and distress. Regulatory agencies are under immense pressure to approve or not to approve agents. Some of the ensuing debates have been acrimonious. Based on these realities, our editorial team has concluded that leading medical journals such as Diabetes Care have a responsibility to provide leadership while supporting clear thinking, good science, and civility in the ongoing dialogue. Our goal is to provide our readers with the most up-to-date information to assist them not only in managing their patients but also in understanding the public conversation on incretin-based therapies. We believe that the weaknesses of the current pharmacovigilance system call for renewed attention and improvement. For example, prospectively collected data for longitudinal epidemiologic analysis and well-performed patient-level meta-analyses of large, randomized, long-term medical outcome trials would do much to support evidence-based decisions. Toward this end, we hope to stimulate new ideas and new methods of addressing important questions concerning therapies now in development.

In this issue, we begin our discussion on drug safety with an invited review from Prof. Clifford J. Bailey (4). Large randomized interventional trials, population-based epidemiology, and medical statistics are relatively young sciences that have been made possible by the recently acquired power to store and analyze huge collections of data using computers. We are still learning how best to use them. Methodologic disputes about ways to measure medical outcomes, identify trends, and assess statistical relevance are common and lead to difficulties for clinical researchers, regulatory agencies, practicing physicians, and patients alike. Thus, it is important to understand the complexities of assessing new or current products in regard to both benefits and risks. The review by Bailey elegantly describes recent signals that have been associated with diabetes therapies, and his narrative illustrates the difficulties when ascribing causality and when evaluating absolute risk, predictability, prevention, and containment. His article emphasizes that individual clinical trials are necessarily restricted for patient selection, numbers, and duration and that they can introduce allocation and ascertainment bias. Further, clinical trials often rely on biomarkers to estimate long-term clinical outcomes that may not necessarily correlate with hard end points. As discussed by Bailey, reports of small numbers of fatalities or other serious illnesses associated with specific drugs have led to high levels of alarm and, as in the case of rosiglitazone, sometimes the withdrawal of agents from use. Determining whether such associations are the results of chance, confounding factors, or the drug itself is a statistical problem. Beyond that, deciding how to respond to such information is a policy issue. All drugs (and foods and beverages) have some capacity for harm. Consuming too much alcohol or too many pizzas over a chronic period obviously can result in adverse effects on human health, and even drinking too much water can cause illness. Penicillin can cause fatal allergic reactions, and aspirin can lead to death from gastrointestinal bleeding. Both agents might have difficulty passing regulatory review in a zero-tolerance regulatory atmosphere. The resolution of this problem lies in regarding risk always in the context of benefit. Of course, the benefits of treatments are no less challenging to quantify than the harms.

Even when we have relevant data from clinical trials outlining the signals and benefits, how to act upon this information is not always clear. To illustrate this point, the aforementioned commentary in this issue by Brass (3) addresses the approval process for new therapeutic drugs in the U.S. He describes the charge of advisory committees such as the Endocrinologic and Metabolic Drugs Advisory Committee (EMDAC), which “reviews and evaluates data concerning the safety and effectiveness of marketed and investigational human drug products for use in the treatment of endocrine and metabolic disorders, and makes appropriate recommendations to the Commissioner of Food and Drugs” (3). Unfortunately, there is concern that diabetes clinical trial experts may be excluded from these committees because of real or perceived conflicts of interest related to previous participation in the design and/or conduct of any industry-sponsored study. As such, one can argue that we are not taking advantage of additional valuable expertise specific to the drugs being assessed. Brass notes that the U.S. Food and Drug Administration's (FDA's) Division of Metabolism and Endocrinology Products (DMEP) is responsible for the review of most drugs used for indications relevant to the endocrinologist and coordinates the meetings of the EMDAC. In his commentary, Brass describes how the assessment of these drugs requires integration of preclinical and clinical data to formulate a benefit-risk assessment. Clearly, the assessment of the benefit-risk for drugs is increasingly recognized to be a challenging task, and formal tools have been proposed to make this process more effective. It is clear that the FDA makes safety the highest priority, and we fully endorse that approach. Thus, it is of interest to compare recommendations made by the EMDAC and those of the DMEP. We doubt that many readers are aware of the level of concordance between EMDAC votes and the subsequent actions of the DMEP and the ultimate fate of the drug. Brass’s comments on this point are enlightening, to say the least.

Finally, Bailey’s review and Brass’s commentary set the stage for the specific case of incretin therapies. The Point-Counterpoint debate in this issue (5,6) bears the overarching title, “A Critical Analysis of the Clinical Use of Incretin-Based Therapies.” It was not a hard decision to focus on this topic. On the one hand, incretin-based medications have proven to be effective glucose-lowering agents. The glucagon-like peptide 1 (GLP-1) receptor agonists demonstrate, depending on study populations, an efficacy comparable to insulin treatment, and they appear to do so with minimal hypoglycemia and often favorable effects on body weight. In addition, there are data showing that dipeptidyl peptidase 4 (DPP-4) inhibitors also improve glycemic control with weight-neutral effects and with less risk of hypoglycemia than is caused by sulfonylureas. Thus, these agents appear to address unmet clinical needs. However, as highlighted above, important concerns have been raised about the long-term safety of these agents. In addition to preclinical data, some randomized controlled trials of incretin-related therapies have reported small imbalances on the frequency of pancreatitis as have isolated case reports after recent exposure to incretin-related therapies. The concern grew exponentially with the reports from the MedWatch surveillance system (the FDA Safety Information and Adverse Event Reporting Program), and the controversy expanded further with the conflicting reports extracted from insurance claims and prescription databases not designed to prospectively capture adverse events. Specifically, based on such reports, it has been suggested that both of these classes of agents have the potential to promote acute pancreatitis, to initiate histological changes suggesting chronic pancreatitis including associated preneoplastic lesions, and potentially, in the long run, pancreatic cancer. Other issues relate to a potentially increased risk of thyroid cancer. We recognize that the data are complex and perhaps conflicting, so in keeping with our aim of providing a balanced view, we present a two-part Point-Counterpoint discussion. Prof. Edwin A.M. Gale and colleagues describe findings supporting their opinion that we should reconsider the use of incretin-based therapies because of growing concerns of unacceptable risks (5). In the narrative following this contribution, Dr. Michael A. Nauck provides a defense of incretin-based therapies, suggesting that the benefits clearly outweigh the concerns of risk (6). Both narratives are well written, make excellent points, are well referenced, and insightful.

So, where do we stand after this exercise? We believe the excellent articles in this issue of Diabetes Care support the notion that the drug development process is complex, involves many parties, and should be guided by science and professional discussions rather than inflammatory exchanges in the public media. Furthermore, when available information is limited, many different opinions on these risks and benefits are possible, but caution should be exercised to avoid premature or unwarranted positions. A commitment to the long-term evaluation of all treatments, extending well beyond initial regulatory approval, is needed. An important long-term goal should be the identification of individuals or groups of individuals particularly suited for—or at risk from—treatment with widely used therapies.

We do not speak on behalf of the American Diabetes Association, which has allowed us to provide a balanced forum. Nevertheless, whether or not incretin-related therapies increase the risk for pancreatitis and pancreatic cancer, we believe there are pertinent questions that need to be properly addressed. Several ways to improve pharmacovigilance are proposed below.

• 1)Given that the current ongoing randomized cardiovascular (CV) outcome trials with GLP-1 receptor agonists (n = 5) and DPP-4 inhibitors (n = 4) already have a pancreatitis adjudication process, these studies would benefit from a separate process for identifying and adjudicating cancer events. The sponsors of the CV outcome trials are encouraged to agree to pool patient-level data from the tens of thousands of participants in these trials for meta-analysis by independent experts.
• 2)Creation of a blue ribbon independent expert committee including leading cancer epidemiologists, biostatisticians, oncologists, gastroenterologists, cardiologists, and diabetologists is strongly encouraged. If provided access to all patient-level data from the long-term CV outcome trials, the group could thoroughly report the incidence of pancreatitis, pancreatic cancer, thyroid cancer, or any other relevant adverse outcomes. The most relevant professional organizations could take the lead in convening and supporting such a committee.
• 3)An alliance between all incretin-related manufacturers to sponsor the implementation and conduct of large, long-term, prospective epidemiological studies specifically designed to assess the occurrence of pancreatitis and pancreatic cancer would be very helpful. Countries with reliable registries for cancer and diabetes morbidities as well as electronic medical systems where reliable demographic and morbid data are available, including pharmacy data, will be best equipped to conduct such large-scale prospective epidemiologic studies.
• 4)Finally, we suggest the current pharmacovigilance system based on the spontaneous reporting (MedWatch) be re-evaluated, with the goal of improving the quality of the data collected and the means of adjudicating and better processing the captured information.

In closing, we will restate our belief that in reaching conclusions and establishing policies regarding the assessment of drug safety, all the parties involved (i.e., manufacturers of drugs, regulatory agencies, medical and scientific communities, and patients) have much to gain from a transparent and complete evaluation process. We hope the articles assembled in this issue of Diabetes Care will help all readers to judge for themselves the evidence presented regarding the incretin-based therapies and to appreciate the challenges of the drug development and monitoring process. We welcome comments on whether we have achieved our objectives.