Abstract
Aims
To describe and characterize the first cohort of Post‐Authorization Safety Study (PASS) protocols reviewed under the recent European pharmacovigilance legislation.
Methods
A systematic approach was used to compile all publicly available information on PASS protocols and assessments submitted from July 2012 to July 2015 from Pharmacovigilance Risk Assessment Committee (PRAC) minutes, European Medicines Agency (EMA) and European Network of Pharmacovigilance and Pharmacoepidemiology (ENCePP) webpages.
Results
During the study period, 189 different PASS protocols were submitted to the PRAC, half of which were entered in the ENCePP electronic register of post‐authorization studies (EU‐PAS) by July 2015. Those protocols were assessed during 353 PRAC reviews. The EMA published only 31% of the PRAC feedback, of which the main concerns were study design (37%) and feasibility (30%). Among the 189 PASS, slightly more involved primary data capture (58%). PASS assessing drug utilization mainly leveraged secondary data sources (58%). The majority of the PASS did not include a comparator (65%) and 35% of PASS also evaluated clinical effectiveness endpoints.
Conclusions
To the best of our knowledge this is the first comprehensive review of three years of PASS protocols submitted under the new pharmacovigilance legislation. Our results show that both EMA and PASS sponsors could respectively increase the availability of protocol assessments and documents in the EU‐PAS. Protocol content review and the high number of PRAC comments related to methodological issues and feasibility concerns should raise awareness among PASS stakeholders to design more thoughtful studies according to pharmacoepidemiological principles and existing guidelines.
Keywords: ENCePP, new pharmacovigilance legislation, PASS, PRAC, transparency
What is Already Known About This Subject
The implementation of new pharmacovigilance legislation in 2012 demanded a proactive focus on risk management, including the conduct of PASS.
While PASS‐imposed conditions or special obligations on marketing authorizations represent a fraction of the PASS contingent, they must follow more stringent requirements in terms of submissions and registration as compared to non‐imposed PASS.
The EMA increased public access to information as part of the effort to increase transparency.
What This Study Adds
This is the first comprehensive review of three years of PASS protocol submissions since the inaugural PRAC meeting.
Although the EMA have significantly improved the availability of PASS information, there is limited access to PRAC feedback and only half of the PASS were entered in the EU‐PAS register.
There is a general lack of granularity on PASS methodological and feasibility considerations which are below expectations given the existing pharmacoepidemiology guidelines to design robust studies.
Introduction
Implemented in July 2012, the 2010 European Pharmacovigilance Legislation 1, 2 was the biggest change to the regulation of human medicines in the European Union (EU) since 1995 3.
The new legislation sets forth a firm legal foundation for Post‐Authorization Safety Studies (PASS) and established new guidelines 4, from protocol development to final study reporting under the oversight of the Pharmacovigilance Risk Assessment Committee (PRAC). A PASS is defined as any study relating to an authorized medicinal product conducted with the aim of identifying, characterizing or quantifying a safety hazard, confirming the safety profile of the medicinal product, or of measuring the effectiveness of risk management measures.
The regulators recognize the need for early access to certain medicinal products provided that, at the time of marketing authorization, the risk–benefit balance is positive, despite the fact that some uncertainties could remain. In those circumstances, the regulators may impose a PASS as a condition of the marketing authorization or specific obligation (category 1 and 2) 5. In other cases, PASS may be planned as a requirement in the Risk Management Plan (RMP) as an additional pharmacovigilance activity either to address safety concern(s) or to evaluate the effectiveness of risk management measures (category 3) 4. In addition, to improve transparency, the Marketing Authorization Holder (MAH) should make study information available in a dedicated electronic register of post‐authorization studies (EU‐PAS), maintained by the European Network of Pharmacovigilance and Pharmacoepidemiology (ENCePP) coordinated by the European Medicines Agency (EMA) 6. The ENCePP aims to uphold high standards throughout the research process based on the principles of robust methodologies, transparency and scientific independence. PASS following those principles can obtain an ENCePP seal 7, 8, which signifies that they have adhered to all the requirements and methodological principles underpinning the ENCePP code of conduct.
Since the introduction of these new requirements, a broad spectrum of publicly available data on PASS have emerged, including information on study regulatory, methodological and operational considerations. Although a recent publication examined the characteristics and follow‐up of post marketing studies attached as specific obligations to conditional marketing authorizations 9, to the best of our knowledge, this is the first time the PASS landscape has been drawn, since the implementation of the new pharmacovigilance legislation using publicly available data sources. Therefore, the objective of this study is to describe the PASS landscape during the first three complete years of the new pharmacovigilance legislation by characterizing the purpose and methodology of the studies. It also aims to give a critical perspective on the level of publicly available information on the PASS review process and PRAC feedback.
Methods
A review of all monthly PRAC meeting minutes 10 was performed to build a complete data set on all PASS assessed by the PRAC from its inaugural meeting on July 2012 through July 2015, in chronological order. A systematic approach was used to compile all the information in a Microsoft Excel™ database. Multiple occurrences of the same PASS protocol over the three‐year period corresponded to different rounds of review of the same protocol.
The following information was retrieved from the PRAC meeting minutes when available: name of active substance, MAH, whether it was an imposed or non‐imposed PASS, PRAC assessment outcome (endorsement/objection or need for revision/administrative procedural information/unknown). PRAC reasons for objection/revision were further categorized by area of concern (study objectives and endpoints/study design/data source and population/data collection and management/study variables/study size/data analysis/milestones and timelines/feasibility and bias considerations/other/missing).
When no information on PRAC final outcome was available through the review of consecutive PRAC meeting minutes, the following rules were applied to ascertain whether or not the protocol was endorsed: the PASS was found in the EU‐PAS or more than one year had elapsed since the last assessment of that PASS protocol. For all protocols that were considered approved, the number of rounds and the duration of PASS assessment (number of months elapsing since first and last presentation date) were estimated.
The EMA website provided details on medicinal products authorized by a centralized procedure (marketing authorization date, orphan drug status) and Assessment Reports 11 in which to find the reason why the PASS was initiated (e.g., condition of the marketing authorization, required in the RMP, following initial marketing authorization or a variation to the marketing authorization such as an extension for a new indication). It also provided details on referral procedures which could have originated PASS.
The EU‐PAS register was searched to identify if the PASS was registered and to retrieve the following variables: PASS focus (investigate safety concerns/assess effectiveness of risk minimization measures/drug utilization study, as per guidelines 1, 4), type of data collection (primary/secondary), study design (longitudinal/transversal), study population inclusion‐based criteria in terms of type of exposure (disease/single medicinal product/multiple medicinal products) and focus in a specific type of population (pregnancy/paediatric/healthcare providers).
An additional set of variables was also collected for a subset of PASS for which the protocol was available: geographic scope (Europe only/Europe and American continents and other regions), sample size, informed consent requirement, follow‐up duration when applicable, subgroups of special interest for the analysis (paediatric/pregnancy/elderly/hepatic/renal/cardiovascular impaired/others), use of a comparator (other treatments/unexposed to the medicinal product of interest/external data source/other/none), inclusion of effectiveness endpoints (yes/no), use of patient reported outcome (PRO) instruments (yes/no).
Analysis
Descriptive analyses were performed among (i) all the PASS protocols submitted, (ii) the subsample of PASS with protocol documents available, and (iii) among the full consecutive PRAC comments published in the minutes.
Categorical variables were summarized by the number and percentage (%) of PASS in each category excluding missing data. Continuous variables were summarized using descriptive statistics (mean, standard deviation, median, quartiles, minimum and maximum values).
Comparisons using Chi square and Student's t‐tests were performed (α = 5%) for categorical and continuous variables. All analyses were performed on actual data, with no imputations for missing data. Proportions were calculated excluding missing data from the denominator.
Results
Between July 2012 and July 2015, 189 PASS protocols were assessed by the PRAC (Table 1) during 353 submissions (including resubmissions). As shown in Figure 1, the number of submissions increased from 2012 to 2015, with more than 75% of the PASS protocols in the analysis having been submitted in the latter two years. Overall, approximately one‐third (31%, n = 58) were PASS imposed to the marketing authorization. The remainder (n = 131) were non‐imposed PASS, the majority of which (89%, n = 116) were required in the RMP (category 3 PASS). The proportion of PASS addressing a medicinal product with orphan drug status was higher among the imposed PASS compared to the non‐imposed PASS (22% vs. 9%, P = 0.013). Most PASS had a unique sponsor with only 12% PASS identified as jointly sponsored, the majority of these imposed after a referral procedure (63% of the joint PASS).
Table 1.
Regulatory and methodological overview of the PASS protocols submitted from July 2012 to July 2015, n = 189
| PASS Status | Imposed (n = 58) | Non‐imposed (n = 131) | Overall (n = 189) | Chi‐square test (P‐value) |
|---|---|---|---|---|
| Regulatory aspects | n (%) | n (%) | n (%) | |
| Condition to MA (Cat. 1) | 28 (48.3) | — | 28 (14.8) | — |
| • Initial MA | 18 (64.3) | — | 18 (64.3) | — |
| • After MA renewal | 2 (7.1) | — | 2 (7.1) | — |
| • After MA variation a | 8 (28.6) | — | 8 (28.6) | — |
| Specific obligation (Cat. 2) | 7 (12.1) | — | 7 (3.7) | — |
| Referral | 23 (39.7) | — | 23 (12.2) | — |
| RMP (Cat. 3) | — | 116 (88.5) | 116 (61.4) | — |
| • Initial requirement | — | 85 (73.3) | 85 (73.3) | — |
| • RMP update (paediatric indication extension) | — | 7 (6.0) | 7 (6.0) | — |
| • RMP update (extension to new condition) | — | 13 (11.2) | 13 (11.2) | — |
| • RMP update (new dose/ route of administration) | — | 3 (2.6) | 3 (2.6) | — |
| • Other RMP requirements b | — | 8 (6.9) | 8 (6.9) | — |
| Other c | — | 15 (11.5) | 15 (7.9) | — |
| Orphan Drug Status | 13 (22.4) | 12 (9.2) | 25 (13.2) | 0.013 |
| Joint PASS d | 10 (24.4) | 6 (6.6) | 16 (12.1) | 0.004 |
| Registration in the EU‐PAS | 34 (58.6) | 59 (45.0) | 93 (49.2) | 0.085 |
| • Protocol available | 12 (35.3) | 28 (47.5) | 40 (43.0) | 0.254 |
| • ENCEPP seal | 2 (5.9) | 2 (3.4) | 4 (4.3) | 0.568 |
| Study specifics | ||||
| PASS focus e | ||||
| • To investigate safety concerns | 42 (72.4) | 98 (74.8) | 140 (74.1) | 0.729 |
| • Drug utilization study | 23 (39.7) | 42 (32.1) | 65 (34.4) | 0.311 |
| • Assess effectiveness of risk minimization measures | 19 (32.8) | 29 (22.1) | 48 (25.4) | 0.122 |
| Data collection f | ||||
| • Primary | 25 (56.8) | 56 (58.3) | 81 (57.9) | 0.866 |
| • Secondary | 19 (43.2) | 40 (41.7) | 59 (42.1) | |
| Study Design g | ||||
| • Longitudinal follow‐up | 42 (89.4) | 68 (76.4) | 110 (80.9) | 0.068 |
| • Transversal/ cross‐sectional | 5 (10.6) | 21 (23.6) | 26 (19.1) | |
| Study population inclusion based criteria h | ||||
| Type of exposure | ||||
| • Disease | 6 (11.8) | 7 (6.7) | 13 (8.4) | 0.319 |
| • Multiple medicinal products | 13 (25.5) | 20 (19.2) | 33 (21.3) | |
| • Single medicinal product | 32 (62.7) | 77 (74.0) | 109 (70.3) | |
| Special populations focus | ||||
| • Pregnant women | 1 (4.5) | 8 (14.3) | 9 (11.5) | 0.226 |
| • Paediatric population | 5 (22.7) | 9 (16.1) | 14 (17.9) | 0.491 |
| • Healthcare providers | 3 (13.6) | 14 (25.0) | 17 (21.8) | 0.274 |
MA, marketing authorization; PASS, Post‐Authorization Safety Study; RMP, Risk Management Plan.
For each variable, the percentage was estimated excluding missing values. Categories are mutually exclusive unless otherwise specified.
Variations due to PSUR results/extension of indication to paediatric/extension of indication
For example, renewal of MA, change to manufacturing process
Considered to be category 4 by exclusion
More than one Marketing Authorization Holder sponsored the study. Missing data: imposed = 17 (29.3%), non‐imposed = 40 (30.5%); total = 57 (30.2%)
Categories not mutually exclusive
Primary: collection of data specifically for the study; Secondary: use of data already collected for another purpose. Missing data: imposed = 14 (24.1%), non‐imposed = 35 (26.7%); total = 49 (25.9%)
Longitudinal: involves collection of variables at least two points in time; Transversal: involved collection of variables at a certain point in time. Missing data: imposed = 11 (19.0%), non‐imposed = 42 (32.1%); total = 53 (28.0%)
Common inclusion criteria: in terms of exposure (patients with a certain disease irrespective of medicinal products, restricted to single medicinal product exposure/ prescription or exposure/ prescription of more than one predefined medicinal product/ treatment modalities. Missing data: imposed = 7 (12.1%), non‐imposed = 27 (20.6%); total = 34 (18.0%)
Special population focus: Inclusion restricted to special groups of interest (children, pregnant women). Missing data: imposed = 36 (62.1%), non‐imposed = 75 (57.3%), total = 111 (58.7%)
Figure 1.
Number of PASS protocols reviewed monthly (new submissions and resubmissions)
By July 2015, half of the 189 PASS (49%) were entered in the EU‐PAS register with a higher proportion among the imposed PASS compared to the non‐imposed PASS (59% vs. 45%, P = 0.085). Almost half of the EU‐PAS entries had the protocols available (43%). Very few of the PASS registered (4%) had an ENCePP seal.
With regard to the PASS focus, approximately one‐third of the 189 PASS (31%) combined at least two of the three PASS focus categories under the same protocol, in particular combining drug utilization and assessment of the effectiveness of the risk minimization measures.
Among the 189 PASS, slightly more PASS involved the collection of original primary data (58%). The PASS with at least one objective related to study drug utilization were mainly using secondary data collection approaches (58%), P = 0.005.
More than two‐thirds (70%) of the 189 PASS study population focused on patients exposed to a single medicinal product, while the remainder targeted more than one medicinal product for comparative safety reasons. A minority of studies (8%) included disease‐exposed patients independent of the treatment of interest.
With regard to the study population, 12% of the overall PASS were conducted as pregnancy registries, while 18% included paediatric patients only. In addition, 22% of PASS targeted healthcare professionals (n = 17), the vast majority (94%) aiming to assess effectiveness of risk minimization measures and/or drug utilization.
Table 2 presents the characteristics of the subsample of 57 PASS with available protocol documents. There were no statistically significant differences between the distributions of the variables – regulatory reason for PASS initiation, PASS focus, data collection, study design and study population inclusion‐based criteria – between this subsample and the overall 189 PASS study population. Therefore the subsample was considered representative of the overall PASS population.
Table 2.
In depth characterization of PASS protocols for which protocol document was available (July 2012–July 2105, n = 57)
| n (%) | |
|---|---|
| Objectives | |
| • To investigate safety concerns | 43 (75.4) |
| • Drug utilization study | 23 (40.4) |
| • Assess effectiveness of risk minimization measures | 18 (31.6) |
| Data collection | |
| • Primary | 38 (66.7) |
| • Secondary | 19 (33.3) |
| ° Chart abstraction | 8 (42.1) |
| ° Claims, database, EHR | 7 (36.8) |
| ° Existing registry | 4 (21.1) |
| Geographic scope a | |
| • Europe only | 44 (77.2) |
| • Europe and Americas only | 7 (12.3) |
| • Europe and/or Americas and other regions | 6 (10.5) |
| Study population inclusion‐based criteria | |
| Type of exposure | |
| • Disease | 4 (7.0) |
| • Multiple products | 7 (12.3) |
| • Single product | 46 (80.7) |
| Special populations focus | |
| • Pregnant women | 3 (5.3) |
| • Paediatric population | 8 (14.1) |
| • Healthcare providers | 9 (15.8) |
| Sample size b | |
| • Patient based populations (n = 49) Min, Q1, Median, Q3, Max | 30, 300, 1000, 2000, 280 000 |
| • HCP based populations (n = 9) Min, Q1, Median, Q3, Max | 30, 125, 289, 625, 1320 |
| Use of informed consent | |
| • Primary data research PASS | 33 (89.2) |
| • Secondary data research PASS | 7 (36.8) |
| Duration of follow‐up (years) (n = 40) Min, Q1, Median, Q3, Max | 0.003, 1.0, 3.0, 5.0, 15.0 |
| Analysis subgroups of special interest c | |
| • Elderly | 11 (19.3) |
| • Pregnancy | 9 (15.8) |
| • Paediatric | 6 (10.5) |
| • Hepatic impaired | 10 (17.5) |
| • Renal impaired | 9 (15.8) |
| • Cardiovascular comorbidities | 9 (15.8 |
| Existence of comparator d | |
| • No | 37 (64.9) |
| • Yes | 20 (35.1) |
| ° Other treatments | 8 (40.0) |
| ° Unexposed | 5 (25.0) |
| ° External data source | 5 (25.0) |
| ° Pre and post a certain outcome of interest | 4 (20.0) |
| Effectiveness endpoint (yes) | 20 (35.1) |
| Use of PRO (yes) | 8 (14.0) |
HER, Electronic Health Record; HCP, Healthcare Professional; PASS, Post‐Authorization Safety Study; PRO = Patient Reported Outcome; Q, quartile.
For the variables also present in Table 1, the same footnotes apply.
Countries from the American continents included in more than one protocol are: United States, Canada, Brazil, Argentina, Mexico and Puerto Rico. Other regions included Africa and Middle East (n = 4), Asia (n = 3) and Oceania (n = 1)
Three PASS included both patients and HCPs. Number of patients and HCPs are calculated in the respective category
Identification of special groups of interest from the objectives or the analysis sections
Two PASS used patients unexposed to the medicinal product of interest and also external data source as comparators
Most PASS (77%, n = 44) were conducted in European countries only, with half of those in five or fewer countries.
Among the 19 (33% of the subsample) PASS using secondary data collection, 58% leveraged electronic health records (EHRs), insurance or social care databases or existing registries, while the remainder used onsite medical chart review. Among the PASS with longitudinal design, the median patient follow‐up was 3 years (Q1, Q2: 1.0, 5.0; n = 40). The median sample size was 1000 (Q1, Q2: 300, 2000; n = 49) patients and 289 healthcare professionals (Q1, Q2: 125, 625; n = 9). Ten of the 11 studies with broader patient exposure inclusion‐based criteria (disease or multiple medicinal product cohorts) had samples larger than the overall median (1000 patients).
The vast majority of the PASS protocols only mentioned descriptive analysis. Among the 57 PASS protocols, only 2 (4%) explicitly formulate a hypothesis to test. Survival analysis and statistical modelling were present in approximately one‐fifth of the protocols.
Analysis of subgroups of interest such as paediatrics, pregnancies, elderly or patients with comorbidities was mentioned in approximately one‐third of PASS protocols. The majority of the PASS did not include a comparator (65%). Comparators included other treatments, patients not using the medicinal product being studied, external data sources or comparison before and after the occurrence of an event of interest. In addition to the safety endpoint analyses, 35% of the protocols included clinical effectiveness endpoints as secondary or tertiary objectives. Patient reported outcome (PRO) measurements were included in 14% of the protocols and included assessments of symptoms, burden of disease and quality of life.
Overall, the level of detail provided in PASS protocols was rather limited, most of the time lacking the critical reasoning behind the methodological decision, the acknowledgement of study limitations, considerations to support the practical feasibility of the method and its robustness to obtain valid study conclusions.
Overall, PRAC comments were available in the PRAC meeting minutes for approximately one‐third (31%) of the 353 submissions: 56% of 130 submissions of imposed PASS protocols had PRAC comments available vs. 16% of 223 submissions of non‐imposed PASS protocols, P < 0.001. Missing information on PRAC assessment outcomes in the minutes significantly increased between July 2012 and July 2015 (P < 0.001).
Figure 2 shows the distribution of the areas of the protocol identified among the PRAC comments as responsible for the objection/need for revision of the protocol, irrespective of imposed or non‐imposed status. The most common objections were related to inadequate study design (37%), feasibility of the study (30%), which included concerns with selection bias, concerns with the appropriateness of the data sources and/or population, and an insufficient analytic plan (22% each). There was no significant difference between the distribution of comments of imposed and non‐imposed PASS.
Figure 2.
Methodological issues raised by the PRAC and documented in public PRAC meeting minutes (July 2012–July 2015)
To describe PRAC process metrics, we estimated that the PRAC review process had been completed for 100 PASS protocols among the 189 in our study period. When looking at the number of rounds of review and time elapsed since first and last appearance, approximately one‐third (31 of the 100 PASS) appeared only once in the minutes and the PRAC comments were not conclusive on whether they were approved at that round. The vast majority of those (n = 21) were category 3 non‐imposed PASS (i.e. required in the RMP which could have followed a parallel review process not captured in our analysis), representing 40% of the non‐imposed PASS. Therefore, we do not present metrics on the PRAC review process for the non‐imposed PASS. For the imposed PASS protocols appearing more than once (84% of the 37 imposed PASS considered approved), our data suggests a decrease in the median number of rounds of review from three to two and an average (standard deviation [SD]) decrease in time of review from 10 months (SD = 5) to 4 months (SD = 2) between the first and the third year of our review.
Discussion
To the best of our knowledge this is the first comprehensive review of three years of PASS protocol submissions since the inaugural PRAC meeting in July 2012. Although the EMA and the PRAC have significantly improved transparency of those studies, still limited data on protocol feedback and protocol documents in the EU‐PAS are available for researchers and PASS stakeholders. The high number of PRAC comments related to methodological concerns, as well as the lack of details in feasibility considerations and operational plan within PASS protocols, further highlight the need for PASS sponsors to design more feasible and methodologically sound studies according to existing guidelines.
The establishment of PRAC oversight, specific procedures and timelines for PASS submission, the publication of the monthly meeting minutes and the endorsement of the EU‐PAS register as the official database for PASS registration, represent major achievements of the new legal framework 12. They echo and partially address the needs for more transparency and regulatory oversight on PASS, which was identified as a need prior to the new pharmacovigilance legislation 13. Nevertheless, our results show that, although these activities have streamlined PASS communications and assessment, PRAC assessment comments were available in the publicly available meeting minutes for only one‐third of the submissions. The number of available PRAC comments was higher in the first year of the new legislation but decreased during the following two years. In addition, although the registration of PASS in the EU‐PAS register is strongly recommended, our results show that only half of the PASS covered in the study period were registered by July 2015. This result has to be balanced with the fact that registration is legally binding and subject to financial penalties only for imposed PASS at the time of final study report 4. Although we observed an increase in PASS registration over the past year: 43% in July 2015 compared to 26% in July 2014 14, few documents such as protocols were available. The regulators should keep emphasizing the importance of this registration and the recent and upcoming updates on the guidelines or functionalities of the EU‐PAS register will be important levers 15.
Although only imposed PASS must legally be submitted to the PRAC and have a specific procedure and timelines for assessment, our results confirm that sponsors of non‐imposed PASS are following the same procedures, with more than two‐thirds of the total PASS in our analysis being non‐imposed. The PASS protocol is submitted for review and within 60 days the conclusion of the assessment shall be issued 1. Visibility on the submitted protocols that are actually approved could be further improved as we were able to identify only very few protocols endorsed by the PRAC from the different data sources. Our data suggest a decrease in the median number of rounds of review from three to two between the first and the third year of our review for the imposed PASS protocols submitted to the PRAC.
One other finding of our study is that more than half of the PASS submitted related to new marketing authorization applications. This is not entirely unexpected as there is still uncertainty about the safety profile of a new medicine when it is first launched although the expected risk–benefit balance is positive. The PASS were typically sponsored by a single MAH. Joint sponsorship was rare and mostly seen in PASS resulting from a decision after a referral procedure as those apply to multiple medicinal products with the same active substance or a class of products 16. EU initiatives, such as the IMI ADVANCE project 17 and the EMA patient registry pilot 18, represent good opportunities to expand joint efforts leveraging new governance models.
A post‐authorization study is classified as a PASS when the main aim for initiating the study includes the quantification and assessment of risks or their absence (either known or for which there are uncertainties), to assess patterns of drug utilization that add knowledge on the safety profile of the medicinal product or to measure the effectiveness of risk management activities 4. Approximately one‐third of the PASS in our cohort combine at least two of these objectives. In addition, the definition of a PASS is not constrained by the type of design chosen 4. Nevertheless, our results show that PASS using secondary data collection approaches are more frequently leveraged to assess drug utilization. Direct abstraction of information collected as part of routine care at physician practice was the most common approach to use data already collected for other purpose in the PASS.
The use of electronic databases in pharmacoepidemiology has increased in the past decade as they confer advantages such as increased speed, overcome limitations of recall and reporting bias and lower costs 19. The ENCePP Inventory of Databases provides a catalogue of data sources available in EU research organizations to serve as a hub in which researchers can identify a potential resource for their investigation 20. An ongoing EMA initiative 18 will aim to increase visibility of existing data sources and support their use in a cross‐border setting for both public health and research needs. While the use of databases is increasingly popular, validation studies have been encouraged to ensure validity of study results 21. In addition, since PASS are often multicountry studies, additional considerations on how to combine different databases are important and have been subject to intense research over the past years 22, 23, 24.
The majority of the PASS were restrictive in the exposure criteria used to define the study population, as more than two‐thirds focused on the exposure to the medicinal product of interest only. The vast majority of the analyses presented in the protocols we assessed were descriptive, and only one‐third included a comparison group. Comparative safety could be encouraged especially given the large amount of methodological guidelines available. PASS should be designed according to strong scientific and methodological pharmacoepidemiological principles. The ENCePP Guide on Methodological Standards in Pharmacoepidemiology offers such comprehensive scientific and methodological guidance 25. One other interesting finding of our review is that one‐third of the PASS also assessed benefit endpoints in addition to the safety ones. In particular, as prospective PASS using primary data collection are generally very costly long‐term studies, it is understandable that they represent good opportunities for MAHs to capture routine effectiveness and real‐world information for other stakeholders such as Health Technology Assessors. However, adding extra objectives is usually discouraged if they jeopardize meeting safety objectives or are too burdensome for the study conduct and may possibly lead to Ethics Committee (EC) protocol rejection 26. Two recent studies 27, 28 also highlighted the variety of regulatory requirements by geography, and that early determination of those are key to ensuring a successful implementation of the PASS. It is hoped that the upcoming 2016 new clinical trial regulation 29 will contribute to more harmonized decisions among European ECs with regard to the classification of PASS as interventional or non‐interventional. However, the introduction of ‘low interventional clinical trials’ could result in many PASS assessments no longer being led by the PRAC.
The recent review of post‐marketing studies imposed as specific obligations to the licence of conditionally authorized medicines in the EU raised concerns over the timely completion of studies. It suggests that critical ethical and logistical challenges faced by the study sponsors may be compromising study execution and completion within the timeframe expected by the regulators, who may be less aware of the operational barriers faced by the MAHs 9.
Irrespective of the status of PASS (imposed or non‐imposed), comments and requested revisions of protocols were consistent with previous results 30, 31. The results from the assessment of PASS in the first cohort of EU RMP, in 2009, further highlighted that one‐third of the PASS did not include EU populations, therefore limiting the generalizability of the results. Our results showed that among the 57 PASS protocols examined, all included at least one EU country and more than three quarters were performed exclusively in the EU. However, inadequate study design to fulfil the PASS objectives is still one of the most common methodological issues along with the weakness of the analytic plan, or the lack of proper selection of the right data sources to conduct the PASS. This is consistent with the finding that the vast majority of the studies are purely descriptive, rarely accounting for potential confounders and insufficiently discussing bias such as selection bias from poor selection of a data source. The choice of the best pharmacoepidemiological approach should be tailored to the research question to be addressed 32. Given the specific definition of PASS in the context of RMP, the adequacy of PASS methodology would have to be assessed on a case‐by‐case basis. However, we observed, in general, a limited level of detail in the protocols, which is not in line with the major guidelines in the area (Good Pharmacoepidemiology Practices [GPP], Good Pharmacovigilance Practices [GVP]), all emphasizing the need to include details on the reasoning behind the methodological decisions and feasibility considerations 4, 33. The findings were corroborated by the high number of PRAC comments related to those aspects.
Although the possibility to track numerous data on all single PASS submitted since July 2012 is the major strength of our review, the amount of missing and incomplete information correlated to the improved, but still limited, transparency from both EMA and PASS sponsors, is also its main limitation. In particular, tracking of the different PASS in the successive meeting minutes was complex and impacted our ability to draw robust conclusions on assessment timelines and methodological issues raised by the PRAC. It is hoped that the recent technical upgrade of the EU‐PAS register 34 as well as the updated version of GVP module VIII 4 will increase registration of PASS protocols in the EU‐PAS register, providing more comprehensive details on those studies in the near future.
Conclusion
The public availability of a large variety of information on PASS has so far provided critical insights into the design and the conduct of PASS under the new European pharmacovigilance legislation. While transparency from both regulators and MAH could be further improved, it will be essential to keep supporting and encouraging cooperation between the different PASS stakeholders for the development of transparent and methodologically sound studies fully aligned with the other risk management strategies. The pool of knowledge collaboratively generated will strengthen the EU capacity to deliver better and safer therapies.
Competing Interests
Pierre Engel and Marieke L De Bruin are Steering Group members of the European Network of Pharmacoepidemiology and Pharmacovigilance (ENCePP). The authors declare no other relationships or activities that could appear to have influenced the submitted work.
No external funding was received and therefore no external group had any involvement in the design and carrying out of the study, collection, management, analysis and interpretation of data, or preparation, review and approval of the manuscript.
Contributors
PE and MFA had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. PE, MFA, MLDB, KS, SB and NAD were responsible for the study concept and design. PE and MFA acquired the data, which was analysed and interpreted by PE, MFA, MLDB, KS, SB and NAD. PE and MFA drafted the manuscript. All authors critically revised the manuscript for important intellectual content. PE and MFA carried out the statistical analysis. The study was supervised by NAD.
Engel, P. , Almas, M. F. , De Bruin, M. L. , Starzyk, K. , Blackburn, S. , and Dreyer, N. A. (2017) Lessons learned on the design and the conduct of Post‐Authorization Safety Studies: review of 3 years of PRAC oversight. Br J Clin Pharmacol, 83: 884–893. doi: 10.1111/bcp.13165.
References
- 1. European Commission . Directive 2010/84/EU of the European Parliament and of the Council of 15 December 2010 amending, as regards pharmacovigilance, Directive 2001/83/EC on the Community code relating to medicinal products for human use. OJ EU 2010; L/348: 74–99 [online]. Available at http://eur‐lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2010:348:0074:0099:EN:PDF (last accessed 10 October 2016).
- 2. European Commission . Regulation (EU) No 1235/2010 of the European Parliament and of the Council of 15 December 2010 amending, as regards pharmacovigilance of medicinal products for human use, Regulation (EC) No 726/2004 laying down Community procedures for the authorisation and supervision of medicinal products for human and veterinary use and establishing a European Medicines Agency, and Regulation (EC) No 1394/2007 on advanced therapy medicinal products. OJ EU 2010; L348: 1–16 [online]. Available at http://eur‐lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2010:348:0001:0016:EN:PDF (last accessed 10 October 2016).
- 3. European Medicines Agency . Pharmacovigilance legislation [online]. Available at http://www.ema.europa.eu/ema/index.jsp?curl=pages/special_topics/general/general_content_000491.jsp&mid=WC0b01ac058058f32d (last accessed 10 October 2016).
- 4. European Medicines Agency . Guideline on good pharmacovigilance practices (GVP) – Module VIII: Post‐authorisation safety studies (Rev 2). Doc. Ref. EMA/813938/2011 Rev 2; August 2016. [online]. Available at http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2012/06/WC500129137.pdf (last accessed 10 October 2016).
- 5. European Commission . Commission Regulation (EC) No 507/2006 of 29 March 2006 on the conditional marketing authorisation for medicinal products for human use falling within the scope of Regulation (EC) No 726/2004 of the European Parliament and of the Council. OJ EU 2006; L/92: 6–9 [online]. Available at http://ec.europa.eu/health/files/eudralex/vol‐1/reg_2006_507/reg_2006_507_en.pdf (last accessed 3 May 2016).
- 6. European Medicines Agency . EU PAS Register Guide. Doc. Ref. EMA/613603/2012; September 2016 [online]. Available at http://www.encepp.eu/encepp_studies/documents/EUPASRegisterGuide.pdf (last accessed 10 October 2016).
- 7. European Medicines Agency . How to apply for the ENCePP Study Seal – Guide. Doc. Ref. EMA/615023/2012; July 2016 [online]. Available at http://www.encepp.eu/publications/documents/ENCePPStudySealGuide.pdf (last accessed 10 October 2016).
- 8. Blake KV, Devries CS, Arlett P, Kurz X, Fitt H. Increasing scientific standards, independence and transparency in post‐authorisation studies: the role of the European Network of Centres for Pharmacoepidemiology and Pharmacovigilance. Pharmacoepidemiol Drug Saf 2012; 21: 690–696. [DOI] [PubMed] [Google Scholar]
- 9. Hoekman J, Klamer TT, Mantel‐Teeuwisse AK, Leufkens HG, De Bruin ML. Characteristics and follow‐up of post‐marketing studies of conditionally authorized medicines in the EU. Br J Clin Pharmacol 2016; 82: 213–226. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. European Medicines Agency . PRAC: Agendas, minutes and highlights [online]. Available at http://www.ema.europa.eu/ema/index.jsp?curl=pages/about_us/document_listing/document_listing_000353.jsp&mid=WC0b01ac05805a21cf (last accessed 3 May 2016).
- 11. European Medicines Agency . European public assessment reports: background and context [online]. Available at http://www.ema.europa.eu/ema/index.jsp?curl=pages/medicines/general/general_content_000433.jsp (last accessed 3 May 2016).
- 12. Arlett P, Portier G, de Lisa R, Blake K, Wathion N, Dogne JM, et al. Proactively managing the risk of marketed drugs: experience with the EMA Pharmacovigilance Risk Assessment Committee. Nat Rev Drug Discov 2014; 13: 395–397. [DOI] [PubMed] [Google Scholar]
- 13. Blake KV, Prilla S, Accadebled S, Guimier M, Biscaro M, Persson I, et al. European Medicines Agency review of post‐authorisation studies with implications for the European Network of Centres for Pharmacoepidemiology and Pharmacovigilance. Pharmacoepidemiol Drug Saf 2011; 20: 1021–1029. [DOI] [PubMed] [Google Scholar]
- 14. Engel P, De Bruin ML. Lessons learned on the design of European Post Authorisation Safety Studies (PASS): review of 24 months of PRAC oversight. Presented at: ENCePP Plenary Meeting, November 2014 [online]. Available at http://www.encepp.eu/publications/documents/4.2_Lessons_learned_on_PASS.pdf (last accessed 3 May 2016).
- 15. ENCePP Secretariat . Minutes – ENCePP Steering Group Meeting: 4 June 2015, 10.00 to 17.00, chaired by Peter Arlett. Doc. Ref. EMA/372715/2015; June 2015 [online]. Available at http://www.encepp.eu/publications/documents/Minutes_ENCePPSG_4Jun15.pdf (last accessed 15 November 2016).
- 16. Bouvy JC, Huinink L, De Bruin ML. Benefit–risk reassessment of medicines: a retrospective analysis of all safety‐related referral procedures in Europe during 2001–2012. Pharmacoepidemiol Drug Saf 2016; 25: 1004–1014. [DOI] [PubMed] [Google Scholar]
- 17. Kurz X. Looking towards the future: brainstorming on funding mechanisms for PAS. Presented at: ENCePP Plenary Meeting, November 2015. [online]. Available at http://www.encepp.eu/publications/PlenaryMeetingReports.shtml (last accessed 3 May 2016).
- 18. European Medicines Agency . Initiative for patient registries. Doc. Ref. EMA/176050/2014, September 2015. [online]. Available at http://www.ema.europa.eu/docs/en_GB/document_library/Other/2015/10/WC500195576.pdf (last accessed 10 October 2016).
- 19. Hall GC, Sauer B, Bourke A, Brown JS, Reynolds MW, Lo Casale R. Guidelines for good database selection and use in pharmacoepidemiology research. Pharmacoepidemiol Drug Saf 2012; 21: 1–10. [DOI] [PubMed] [Google Scholar]
- 20. The European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP) . ENCePP Resources Database [online]. Available at http://www.encepp.eu/encepp/resourcesDatabase.jsp (last accessed 10 October 2016).
- 21. Ehrenstein V, Petersen I, Smeeth L, Jick SS, Benchimol EI, Ludvigsson JF, et al. Helping everyone do better: a call for validation studies of routinely recorded health data. Clin Epidemiol 2016; 8: 49–51. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Klungel OH, Kurz X, de Groot MC, Schlienger RG, Tcherny‐Lessenot S, Grimaldi L, et al. Multi‐centre, multi‐database studies with common protocols: lessons learnt from the IMI PROTECT project. Pharmacoepidemiol Drug Saf 2016; 25 (Suppl 1): 156–165. [DOI] [PubMed] [Google Scholar]
- 23. Gini R, Schuemie M, Brown J, Ryan P, Vacchi E, Coppola M, et al. Data extraction and management in networks of observational health care databases for scientific research: a comparison of EU‐ADR, OMOP, Mini‐Sentinel and MATRICE strategies. EGEMS (Wash DC) 2016; 4: 1189. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Bazelier MT, Eriksson I, de Vries F, Schmidt MK, Raitanen J, Haukka J, et al. Data management and data analysis techniques in pharmacoepidemiological studies using a pre‐planned multi‐database approach: a systematic literature review. Pharmacoepidemiol Drug Saf 2015; 24: 897–905. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. The European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP) . Guide on Methodological Standards in Pharmacoepidemiology (Revision 4). Doc. Ref. EMA/95098/2010, July 2015. [online]. Available at http://www.encepp.eu/standards_and_guidances/documents/ENCePPGuideofMethStandardsinPE_Rev4.pdf (last accessed 10 October 2016).
- 26. Ramirez I. Navigating the maze of requirements for obtaining approval of non‐interventional studies (NIS) in the European Union. Ger Med Sci 2015; 13: Doc21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Pan I, Walton A, Dieck G. Post Authorization Safety Studies (PASS): application of prospective observational study methodology, 2014. [online]. Available at http://www.ubc.com/sites/default/files/library/icpe_2014_abstract_‐_pass_prospective_obs_studies_0.pdf (last accessed 10 October 2016).
- 28. Stein D, Payne, K . Retrospective Chart Review Studies: key considerations for fulfilling safety reporting requirements, 2014. [online]. Available at http://www.ubc.com/sites/default/files/library/retrospective_chart_review_studies_safety_reporting_req_abstract.pdf (last accessed 10 October 2016).
- 29. European Commission . Regulation (EU) No 536/2014 of the European Parliament and of the council of 16 April 2014 on clinical trials on medicinal products for human use, and repealing Directive 2001/20/EC. OJ EU 2014; L/158: 1–76 [online]. Available at http://ec.europa.eu/health/files/eudralex/vol‐1/reg_2014_536/reg_2014_536_en.pdf (last accessed 3 May 2016).
- 30. Engel P, Bulliard M, Parmenter L, Starzyk K, Dreyer N. Lessons learned on the design of European Post Authorisation Safety Studies (PASS): review of 18 months of PRAC oversight. Pharmacoepidemiol Drug Saf 2014; 23 (Suppl 1): S621. [Google Scholar]
- 31. Giezen TJ, Mantel‐Teeuwisse AK, Straus SM, Egberts TC, Blackburn S, Persson I, et al. Evaluation of post‐authorization safety studies in the first cohort of EU Risk Management Plans at time of regulatory approval. Drug Saf 2009; 32: 1175–1187. [DOI] [PubMed] [Google Scholar]
- 32. Strom BL, Kimmel SE, Hennessy S, eds. Pharmacoepidemiology, 5th edn. Chichester: John Wiley & Sons, 2012. [Google Scholar]
- 33. International Society for Pharmacoepidemiology . Guidelines for Good Pharmacoepidemiology Practices (GPP) [online]. Available at https://www.pharmacoepi.org/resources/guidelines_08027.cfm (last accessed 10 October 2016).
- 34. European Medicines Agency . EU PAS Register upgrade July 2016 – What's new? [online]. Available at http://www.encepp.eu/encepp_studies/documents/EUPASRegisterUpgrade_WhatsNew.pdf (last accessed 14 October 2016).