Adverse events related to biologicals used for patients with multiple sclerosis: a comparison between information originating from regulators and information originating from the scientific community

Abstract

Background and purpose

Clinical decision making is facilitated by healthcare professionals’ and patients’ adequate knowledge of the adverse events. This is especially important for biologicals used for treating multiple sclerosis (MS). So far, little is known about whether different information sources report adverse events consistently.

Methods

Biologicals authorized by the European Medicines Agency for the treatment of MS were included in this study. Information on adverse events derived from phase 3 clinical trials from European Public Assessment Reports (EPARs) and from scientific publications was compared.

Results

In the study, eight biologicals used for the treatment of MS were included for which the EPAR and/or scientific publication reported a total of 707 adverse events. Approximately one‐third of the adverse events was reported in both the EPAR and scientific publication, one‐third was only reported in the EPAR and one‐third only in the scientific publication. Serious adverse events and adverse events that regulators classified as ‘important identified risk’ were significantly more often reported in both sources compared to adverse events not classified as such (respectively, 38% vs. 30% and 49% vs. 30%). Adverse events only reported in the EPAR or in the scientific publication were, in general, not described in the benefit–risk section or abstract, which were considered to be the most important sections of the documents.

Conclusions

This study showed that there is substantial discordance in the reporting of adverse events on the same phase 3 trials between EPARs and scientific publications. To support optimal clinical decision making, both documents should be considered.

Introduction

Regulators have approved several biologicals to treat patients with relapsing and progressive multiple sclerosis (MS) during the last decade. Although these biologicals improve clinical symptoms and reduce relapse rates and disease progression, serious adverse events (SAEs) can occur. The detection of the adverse events (AEs) of these drugs may be complicated as these AEs can mimic the clinical expression of MS. For example, the early symptoms of encephalitis associated with the use of daclizumab include aphasia, confusion and disorientation, which are symptoms similar to those associated with a serious MS relapse [1]. Encephalitis was therefore first interpreted as a worsening of the disease and as lack of efficacy of the drug instead of a SAE [1].

Healthcare professionals and patients can use different sources of information to obtain knowledge about the efficacy and safety profile of a drug in order to guide clinical decision making. At the time of approval, knowledge about the efficacy and safety profile is mainly based on the findings of the phase 3 randomized clinical trials that supported marketing approval. The results of these clinical trials are (publicly) available in various information sources. One of these information sources is peer‐reviewed scientific publications where investigators report the results of the clinical trials. These scientific publications were an important source of evidence for the development of the European Clinical Guideline on the pharmacological treatment of people with MS [2]. Another source is the publicly available European Public Assessment Report (EPAR). The European Medicines Agency (EMA), which is the regulatory authority in Europe responsible for evaluating marketing approval applications, publishes the EPAR; it provides an overview of the assessment procedure, including an assessment of the conducted clinical trials [3].

Although these two information sources reflect information obtained from the same clinical trials, the choice of the clinical findings that are extracted from these trials and the attention given to those clinical findings can differ. However, one might expect that the most important information generated from the clinical trials is reported in both documents. Several studies have assessed synergies between the reporting of efficacy and safety information from clinical trials by regulatory authorities and in scientific publications [4, 5, 6, 7, 8, 9]. These show that there are large differences in reporting between these two types of information sources. For example, de Vries et al. [5] showed that, for antidepressants, 79% of the scientific publications provided incomplete information on SAEs compared to data obtained from the US Food and Drug Administration, and 63% did not mention SAEs at all. Another study on insomnia medication showed that scientific publications from studies identified in the EPAR reported reliably on the primary end‐points but less reliably or not at all on the safety of the drug [6].

Since SAEs have occurred in clinical practice for biologicals used for MS, clinicians should have a comprehensive view of the safety profile to support clinical decision making. Therefore, the aim of this study was to provide, for biologicals used in MS, an analysis on which AEs from clinical trials are reported in the EPARs and the corresponding scientific publications, and whether these differ.

Methods

Study drugs and information sources

In this study, biologicals that were or had been approved by the EMA for the treatment of MS (as of 31 December 2018) were included. The EPARs were retrieved from the EMA website (www.ema.europa.eu). The corresponding scientific publications of the phase 3 randomized clinical trials that supported approval of the product were identified using PubMed and the webpage clinicaltrials.gov. The full text of the scientific publication was obtained from the scientific journal concerned. Whether the scientific publications corresponded with the clinical trials described in the EPARs was verified by comparing the identifiers used in the EPARs and scientific publications (e.g. the clinicaltrials.gov identifier), the study design and the number of patients included. Furthermore, a cross‐check with the Cochrane review on immunomodulators and immunosuppressants for relapsing–remitting MS was performed [10].

For each product, information on the year of approval, number of clinical trials supporting the approval of the product, and mechanism of action from the EPAR was retrieved.

Adverse events

For both information sources, the reported AEs for each product were compared. For the EPAR, the analysis was limited to the sections reporting on the safety information from the clinical trials and the benefit–risk discussion, whereas for the scientific publications all sections (including appendices, if applicable) were taken into account.

The AEs reported for the product were identified and characterized using the Medical Dictionary for Regulatory Affairs (MedDRA^®) [11]. MedDRA^® is a validated standardized terminology used to facilitate the exchange of information on AEs, and it is used, amongst other things, in the communication of information from clinical trials between industry and regulators. MedDRA^® is hierarchically structured. The lowest, and most specific, level reflects how an AE is reported in practice. Each of these lower level terms is linked to one preferred term. Multiple lower level terms can fall within one preferred term, as they may include synonyms or different word forms for the same expression. For example, the lower level terms ‘multiple sclerosis exacerbation’ and ‘multiple sclerosis flare’ fall within the same preferred term ‘multiple sclerosis relapse’. For this study, the consistency in the reporting of AEs was assessed by comparing the AEs on the preferred term level. The AEs were also grouped according to the highest level of the MedDRA^® hierarchy, namely the System Organ Class level.

In addition, various characteristics of the reported AEs were assessed as follows.

• Attention: An assessment was made of where in the text the authors described the AE. For the EPAR, it was assessed whether the regulators described the AE in the concluding section that reports how the benefits are weighted against the risks. For the scientific publications, it was assessed whether researchers described the AE in the abstract, main body of the text, a table and/or an appendix.

• Seriousness: Adverse events were categorized as a SAE if the authors specifically described the AE as being serious or if an AE was listed on the important medical events list of the EMA [12]. An SAE is an AE that results in death, is life‐threatening, requires hospitalization or prolongs existing hospitalization, results in persistent or significant disability, or is a birth defect. This definition is also included in the guidelines for scientific publications.

• Regulatory importance: Adverse events were categorized as regulatory important if regulators included these as important risks in the risk management plan (RMP). A separate chapter of the EPAR describes the RMP, including the important identified risks. Regulators include safety issues as important identified risks in the RMP if these have been causally associated with the product, should be further characterized after marketing approval, and are likely to have an impact on the benefit–risk balance [13]. As the EMA introduced RMPs in 2005, this information could not be included for the products authorized prior to 2005.

Data analysis

Whether the EPAR and scientific publication report consistently on AEs for the same biological was assessed by comparing these on the preferred term level. In the EPAR, when the authors referred to a pooled analysis of data, it was considered to be consistently reported if the AE was reported in at least one of the scientific publications. The frequencies of AEs that were consistently reported in both the EPAR and scientific publication, those that were only reported in the EPAR, and those that were only reported in the scientific publication were calculated.

Relative risks (including 95% confidence intervals) were calculated to assess the association of the characteristics of the AE described above and the consistency in reporting of the AE in both the EPAR and scientific publication.

Statistical analysis was performed using R statistical software version 3.6.0 (R Core Team, Vienna, Austria).

Results

As of 31 December 2018, the EMA had approved nine biologicals for the treatment of MS. From these nine products, one [Extavia^® (interferon‐β‐1b)] was excluded from the analysis as the company used the same dossier of the already available Betaferon^® for the marketing approval. Although the company has taken Zinbryta^® (daclizumab) off the market in March 2018, it was included in the analysis as only the information available at the time of regulatory approval was taken into account. As a result, eight biologicals were included in this study (Table 1). For all the products, the results of the phase 3 clinical trials were published in the scientific literature.

Table 1.

Biologicals authorized by the European Medicines Agency (EMA) for the treatment of multiple sclerosis included in this study

Product name	Active substance	Year of EMA approval	Number of trials supporting the approval	Mechanism of action
Betaferon®	Interferon‐β‐1b	1995	1	Immunomodulating cytokine
Avonex®	Interferon‐β‐1a	1997	1	Immunomodulating cytokine
Rebif®	Interferon‐β‐1a	1998	1	Immunomodulating cytokine
Tysabri®	Natalizumab	2006	2	Anti‐α4‐integrin
Lemtrada®	Alemtuzumab	2013	2	Anti‐CD52
Plegridy®	Peginterferon‐β‐1a	2014	1	Immunomodulating cytokine
Zinbryta®	Daclizumab	2016 (withdrawn 2018)	2	Anti‐CD25
Ocrevus®	Ocrelizumab	2018	3	Anti‐CD20

Consistency in reporting of AEs

The EPARs and/or the scientific publications reported 707 AEs. A comparable number of different AEs was reported for the interferons Avonex^® (n = 23), Rebif^® (n = 38) and Betaferon^® (n = 33), whereas a considerably higher number was reported for the peginterferon product Plegridy^® (n = 103). For the monoclonal antibodies, the number of AEs ranged from 108 for Ocrevus^® to 174 for Lemtrada^®.

Overall, the proportion of AEs consistently reported in both the EPAR and scientific publication was 35%. Amongst the interferons, the proportion ranged from 27% for Betaferon^® to 35% for Avonex^® (Fig. 1). For the monoclonal antibodies, the proportion of AEs consistently reported in both information sources ranged from 29% for Tysabri^® to 42% for Zinbryta^® (Fig. 1). Of the 707 reported AEs, 222 AEs (31%), of which 116 were SAEs, were only described in the EPAR and not in the scientific publication. Accordingly, a total of 239 AEs (34%), of which 123 were SAEs, were described only in the scientific publication. Whether more AEs were described in either the EPAR or the scientific publication differed per product. For example, for Plegridy^®, 63% of the AEs were described only in the EPAR, whereas for Tysabri^®, 53% of the AEs were described only in the scientific publication.

Figure 1.

Venn diagrams displaying the number of AEs that were described in the EPAR and scientific publication

Of the 222 AEs that were described only in the EPAR, 35 (16%) were described in the section discussing the benefit–risk balance. Of the 239 AEs described only in the scientific publication, four AEs (2%) were described in the abstract of the scientific publication. The AEs were most often described in a table (50%) or the text (35%).

Serious AEs were significantly (P < 0.05) more often consistently reported in both the EPAR and scientific publication compared to non‐serious AEs (38% vs. 30%, relative risk 1.23, 95% confidence interval 1.00–1.52) (Table 2). Also, AEs that regulators classified as important identified risk were significantly more often consistently reported in both documents compared to those that authorities did not classify as such (49% vs. 30%, relative risk 1.65, 95% confidence interval 1.34–2.03).

Table 2.

Consistency in reporting of the adverse events stratified by seriousness and regulatory importance of the adverse event

	Consistently reported in both EPAR and scientific publication, n (%)	Relative risk (95% confidence interval)
Serious adverse event (n = 386)	147 (38%)	1.23 (1.00–1.52)
Non‐serious adverse event (n = 321)	99 (30%)	Reference
Adverse event classified as important identified risk (n = 188)	92 (49%)	1.65 (1.34–2.03)
Adverse event not classified as important identified risk (n = 425)	126 (30%)	Reference

EPAR, European Public Assessment Report.

Nature of the reported AEs

In line with the known safety profile of the products, most AEs were infections and infestations (n = 145, 21%), followed by investigations (n = 94, 13%) and general disorders and administration site conditions (n = 70, 10%). For these categories, the consistency in reporting of the AEs ranged from 39% for infections and infestations to 49% for general disorders and administration site conditions.

The pattern of reporting SAEs in specific categories differed per product. For Avonex^®, Betaferon^® and Rebif^®, it was not possible to observe any differences as a limited number of SAEs were reported. For Plegridy^®, it was observed that five SAEs, classified as neoplasms benign, malignant and unspecified (including cysts and polyps), were only described in the EPAR. For the monoclonal antibodies, additional SAEs were reported in the EPAR and scientific publication that were related to the mechanism of action (i.e. infections and infestations) besides the SAEs that were reported in both documents. However, it was also observed that SAEs in specific categories (e.g. vascular disorders, neoplasms benign, malignant and unspecified) were only described in either one of the documents.

Discussion

The current study provided a comparison of AEs reported in EPARs and scientific publications. Overall, approximately one‐third of the AEs was consistently reported in both the EPAR and scientific publication, one‐third in the EPAR only, and one‐third in the scientific publication only. The results indicate ample discordance in the reporting of AEs between EPARs and scientific publications. However, the AEs that were reported in the EPAR or scientific publication only were, in general, not described in the most important sections of the documents, i.e. abstract or benefit–risk section. Also, SAEs and events that regulators classified as important identified risks were more often consistently reported. Therefore, both documents probably reflect the safety information that is key to the benefit–risk of the product and clinical decision making, whereas a complete overview of the AEs is lacking. This might have implications for the information presented in the clinical guidelines, including the guidelines for treatment of MS, as these are mainly based on the information that is described in the scientific publications [2]. It is recommended that information from the regulators be incorporated during the development of clinical guidelines. However, the EPAR may also not reflect the complete safety profile of the product, as approximately one‐third of the AEs was only reported in scientific publications. As the EPAR is a reflection of the assessment procedure, the regulators may have given specific attention to AEs that were of major concern during the assessment.

The proportion of AEs that was consistently reported was comparable amongst the products. However, whether the proportion of AEs reported in either one of the documents was higher for the EPAR or scientific publication differed per product. When looking into the nature of the AEs that were only reported in one of the documents, it was observed that these were mostly in line with the consistently reported AEs and the AEs directly linked to the mechanism of action. However, it was also observed that for some products the authors did not report on a specific type of AE, whereas the authors of the other information source did.

In line with previous studies that compared information from EPARs with scientific publications, there are differences in the information provided by the regulators and the authors of scientific publications. However, the proportion of safety information missing in the scientific publications was lower in our study compared to a previous study that performed a high‐level comparison (comparing specific AEs for insomnia medication) of safety data, which reported missing safety data in eight of the 15 scientific publications [7]. Also, a study that assessed reporting of SAEs in scientific publications of antidepressants found that 63% of the scientific articles did not mention any SAEs [6]. These differences may be explained by the difference in the nature of the products that were included as, for example, more SAEs are associated with monoclonal antibodies used for treating MS than with the use of insomnia medication. Given these differences and as it was observed that the pattern of reporting of AEs between EPARs and scientific publications differed per product, the results may not be generalizable to other (types of) products.

For this study, all AEs that were reported at least once were considered for the included biologicals in the EPARs or scientific publications. As a causality assessment on the AEs was not performed, AEs were included that may not have been associated with the product. Also, the extraction of the AEs from the text might have been sensitive to interpretation in some cases where the authors did not specifically state whether the AE had been reported for the product under study or whether the AE was considered to be serious. However, this was minimized through consensus amongst the authors on the interpretation of different scenarios reported in the EPARs and scientific publications.

An in‐depth comparison of AEs reported in the two information sources is provided and these data are put into perspective. Also, several studies considered the information from regulators as the reference information source. However, within this study it is shown that scientific publications also contribute to a complete overview of the AEs. These observations need further research on how to align the information in both sources more consistently.

Substantial discordance was observed in the AEs reported on the same phase 3 trials of biologicals for MS in information originating from regulators (described in the EPAR) and the scientific community (described in scientific publications). To support optimal clinical decision making, healthcare professionals and patients should consider both documents.

Disclosure of conflicts of interest

The authors declare no financial or other conflicts of interest.