Abstract
Background
Additional risk minimization measures (aRMMs) are required for some pharmaceutical products when routine risk minimization measures (i.e., product labeling) are deemed insufficient. Measures often include educational materials, such as paper brochures, leaflets, and/or alert cards that provide information to healthcare professionals and patients on the key risks associated with a product and risk minimization actions to take should particular signs or symptoms arise. Paper-based educational aRMMs have several limitations. They do not present information in an interactive manner, and their update and distribution can be costly and often complex. Measuring how effective they are in achieving their aims can also be difficult. Digital methods offer design and delivery flexibility, easier updating processes, opportunities to increase engagement with important information, as well possibilities for tracking distribution, receipt, and potentially understanding of the materials. Pharmaceutical companies have started to look to digital methods as an option for educational aRMMs, alongside paper materials.
Objectives
Research into healthcare professionals and patient needs and preferences, as well as the general acceptability of digital educational options is needed to establish a baseline. This was an exploratory study intended to provide initial insights on the acceptability of digital aRMMs and to inform future research directions.
Methods
Digital concepts for educational aRMMs, one for healthcare professionals and one for patients, were evaluated with 30 healthcare professionals and 20 patients in six countries through 1:1 Zoom calls, with responses recorded in a structured Qualtrics-based survey. Criteria for selecting the six countries included local familiarity with aRMMs as well as interest in and capability to deliver a potential digital aRMM program by the sponsoring company’s affiliate teams. Of the healthcare professionals, 19 were rheumatologists and 11 were dermatologists. 16 patients had rheumatologic (rheumatoid arthritis, psoriatic arthritis, ankylosing spondylitis) conditions and four had atopic dermatitis. These conditions were chosen as they aligned to potential therapeutic areas where the sponsoring company may have the opportunity to use a digital aRMM. Participants were given an overview of the concept as well as the opportunity to interact with it directly via the “control screen” function in Zoom before questions were posed.
Results
The results demonstrated that the majority of healthcare professionals (87%) and all patients interviewed would prefer website-based or app-based delivery, respectively, of aRMM information instead of, or alongside traditional paper-based approaches, with only 13% of healthcare professionals and no patients expressing a preference for paper-only communication.
Conclusions
Given new options offered by digital technology, its widespread use in many fields, and the importance of patient safety as a topic, there is an imperative for pharmaceutical companies and regulators to work together to establish a way forward for the use of digital options for aRMMs. This study is limited in its generalizability but offers some ideas for future research directions.
Supplementary Information
The online version contains supplementary material available at 10.1007/s40290-021-00415-7.
Key Points
|
Digital methods of delivering educational additional risk minimization measures that go beyond simple provision of PDFs of paper materials are likely to offer improvements over traditional paper-based programs such as: • Rapid distribution of up-to-date information once it has been approved; • Provision of feedback on the number of users and user engagement with content; • Additional features such as inclusion of rich media, reminders, interactive knowledge checks; and tailoring to user needs and preferences; • Option to connect with data from various sources (e.g., telemetry devices, mobile devices, wearables) to enable tracking of adverse events (risks). |
| The results of this exploratory qualitative survey study show that healthcare professionals and patients may be ready to use digital options for educational additional risk minimization measures, either alongside or instead of paper. |
| Given the benefits offered by digital technology, its widespread use in many fields, and the importance of patient safety as a topic, pharmaceutical companies and regulators should work together to establish a way forward for the use of more sophisticated digital options for educational additional risk minimization measures. |
Introduction
For most medicinal products, which include medications and medicine-device combination products, adverse events or risks are adequately addressed using routine risk minimization approaches. In the European Union, these comprise the product label (known as the Summary of Product Characteristics) for healthcare providers and the product information leaflet for patients, as well as product packaging, including instructions for use, and pack design features. These are all intended to provide prescribers and patients with information on a product’s benefits and risks, and ways to minimize the risks. In certain circumstances, however, additional risk minimization measures (aRMMs) are required by regulatory authorities such as the Pharmacovigilance Risk Assessment Committee in the European Union to maintain a positive benefit-risk profile. The implementation of aRMMs by the Marketing Authorization Holder can become a condition of approval [1–3] and the effectiveness of the aRMMs put in place should also be demonstrated [2].
Educational materials are just one category of additional risk minimization, other categories include controlled access and pregnancy prevention programs. Educational aRMMs are based largely on paper materials (sometimes referred to as “tools”), such as printed healthcare professional (HCP) or patient guides, which are distributed to stakeholders such as HCPs, pharmacists, and patients/caregivers. These materials are focused on specific risks and include accompanying risk minimization information/instructions. Per the regulations, these materials [1, 2, 4] must be separate from any promotional materials.
This article summarizes an exploratory study of a digital approach to educational aRMMs (which goes beyond the provision of simple websites that re-state the information contained within the paper materials and provide downloadable versions of the paper materials), and HCP and patient feedback on a digital aRMM concept.
Methodology
Concept Tool
The design of digital educational aRMM prototypes (hereafter referred to as digital concept(s) [HCP or patient focused]) was demonstrated as an Adobe XD concept. Adobe XD is a graphical design software tool (https://www.adobe.com/uk/products/xd.html) that allows “wireframe” graphical designs to be shown as “clickable” interactive demonstrations. We refer to “Product-X” in the digital concepts and did not refer to any specific pharmaceutical company-supported medicinal product to maintain a blinded design and to comply with BHBIA guidance on market research-type activities [6].
HCP-Directed Tool
The HCP digital concept was designed as an interactive website (Fig. 1), which included the following:
Interactive risk messaging: description of key risks associated with Product-X presented as videos and graphics.
Prescribing guidance (similar to a checklist): covers points to check before prescribing Product-X for the first time and at subsequent consultations (e.g., laboratory parameters, vaccination status) (Fig. 2).
HCP-patient discussion guide: provides guidance on what HCPs may wish to cover when discussing Product-X with patients.
Handy contact list: where to e-mail/call for further information, or to report an adverse event.
Digital HCP alert card: key risk and risk minimization information for emergency room physicians who may encounter a patient taking Product-X.
Clinical scenarios: knowledge checks based on patient scenarios, designed to reinforce messages about the key risks and associated risk minimization information/instructions by presenting them in realistic clinical settings.
Frequently asked questions: list of common questions and their answers with a search functionality.
Resources: easy access to materials for download, such as the Summary of Product Characteristics, PDF versions of paper-based educational aRMMs, and other resources an HCP may find helpful.
Fig. 1.
Example of a website "home page” for the healthcare professional digital concept. AEs adverse events
Fig. 2.
Healthcare professional prescribing guidance screen
Patient-Directed Tool
The patient digital concept was designed as an app (for mobile phone/smart phone) that included the following:
Interactive risk messaging: description of key risks associated with Product-X, presented through use of videos and graphics (Fig. 3).
Option for tailoring presentation of messaging based on basic demographic information (e.g., age, sex, body mass index).
Option for appointment reminders, or reminders for recommended vaccines.
Handy contact list: where to e-mail/call for further information, or to report an adverse event.
Patient card: a downloadable version of the patient card, which is a paper-based patient educational aRMM. Can be easily accessed on the mobile app to provide to emergency room physicians or other physicians involved in the patient’s care.
Confirm your knowledge: scenario-based knowledge checks, intended to reinforce understanding of the key risks by putting them into everyday contexts (Fig. 4).
Frequently asked questions: list of common questions and their answers with a search functionality (Fig. 5).
Resources: materials for download, such as a patient information leaflet or other resources a patient may find helpful.
Fig. 3.
Example of digital patient concept, key risks section
Fig. 4.
Example of digital patient concept, patient knowledge check
Fig. 5.
Example of digital patient concept, patient frequently asked questions (FAQs)
Research Design
We targeted HCPs in the rheumatology and dermatology therapeutic areas, and patients with a rheumatologic disorder (rheumatoid arthritis, psoriatic arthritis, or ankylosing spondylitis) or the dermatologic disorder atopic dermatitis. These conditions were chosen as they aligned to potential therapeutic areas where the sponsor company may have the opportunity to use a digital aRMM.
The study was carried out in six countries (Australia, Canada, The Czech Republic, Germany, Mexico, Switzerland) during September and October 2020. Key criteria for selecting these countries included the need to provide a global perspective and their use of the European Union guidelines on good pharmacovigilance practices (GVP) for risk management [1, 2, 4] (hereafter referred to as GVP-RM) or regulations that align with these, as part of these regulations include aRMMs. The Czech Republic and Germany follow GVP as they fall within the remit of the European Medicines Agency. The Australian guidelines are largely aligned with GVP (see [11] for an example), and Health Canada and Swissmedic have implemented elements of GVP [12, 13]. The pharmacovigilance system in Mexico includes the use of a risk management plan and draws upon elements of European systems [14]. Additionally, the countries selected were aligned with AbbVie local affiliates with an interest in and capability to potentially support the delivery of digital aRMMs.
The tool was presented in the English language only, and we therefore recruited participants who were comfortable interacting in English. There were some exceptions to this, with the Swiss interviews being planned in German from the outset, with a thorough briefing of the German-speaking moderator in advance, to allow him to describe the tool and objectives in German, and interviews carried out in local languages for some of the German and Czech participants, as it was noted during the study that some individuals were not comfortable in English.
Adept Field Solutions, Ltd. was engaged to recruit participants. Their network of HCPs and patients constitutes individuals who are comfortable with discussing their conditions/areas of practice and who are conversant with field-based research. Thus, the opinions of the participants may not reflect the general HCP and patient populations for rheumatological and dermatological conditions. Further detail on participant selection is included in the Electronic Supplementary Material (ESM), where the screening approach is described.
As product risks are managed through the Risk Evaluation and Mitigation Strategy in the USA [10], which differs from GVP-RM, US HCPs and patients were not recruited for this study. However, the goal of both the Risk Evaluation and Mitigation Strategy and the GVP-RM is the same—to reduce certain risks associated with products, and therefore the findings of this study can also be applied to the USA and other countries that do not follow GVP-RM.
Screening guides developed by the researchers (provided as ESM) were used to select HCP and patient participants and to gain consent for recording the interviews. Criteria for selection included (but were not limited to): length of time in practice, familiarity with key conditions (for HCPs) and length of time living with the relevant condition, use of biologic therapy, and familiarity with aRMMs (for patients).
Interview Plan, Based on COVID-19 Restrictions
Under normal circumstances, a research program such as this would involve face-to-face interviews. However, because of COVID-19 restrictions, hour-long video conference calls were set up for each participant (HCP and patient). Therefore, participants comfortable with using a computer/smartphone/tablet were selected. During the video conference, the HCPs and patients were introduced to the study using a PowerPoint presentation highlighting the study objectives and key features of the relevant digital concept. The time spent on this part of the interview was usually around 15 min.
Participants had been selected for their degree of familiarity with aRMMs (see screening criteria in the ESM), and this was confirmed during the early parts of the interview—screenshots of paper aRMM materials were shown to participants to ensure that their understanding of what constitutes an aRMM was aligned with the interviewer’s. It was therefore not considered necessary to include a detailed overview of paper aRMMs as part of the introduction. To reduce the likelihood of an overly positive presentation biasing participants towards the digital concept, only one slide of the presentation listed the potential benefits of digital technology for aRMMs, with fact-based wording that explained how digital technology offers easy updates of content, whilst integrating with a modern lifestyle and offering the possibility of a more interactive/engaging user experience through the inclusion of rich media. It was explained to the respondents that the Adobe XD demonstration was a digital concept for discussion and not a finalised website or app, and distinctions were made between the digital concept and a possible future website or mobile phone app.
Following the overview PowerPoint presentation, participants were able to use their own computer via a remote link to interact with the Adobe XD demonstration of the digital concept while answering the study questions displayed on screen in the Qualtrics survey tool (see ESM for details of survey questions). The time required for this part of the interview varied by participant but was typically around 20 mins. Healthcare professionals viewed both the website-base concept for HCPs as well as the patient app-based concept; patients viewed the patient app-based concept only.
Survey Questions
Questions were asked on the preferred format for a digital aRMM, ease of use, usefulness, and comparison with paper aRMMs and other paper-based educational materials (see ESM for full details of survey questions).
Data Capture
Participants gave their consent for recordings to be made of the discussions and all interviews were recorded to assist with the analysis. Participants were informed that there may be a need to report adverse events, and all interviewers were trained using the AbbVie’s adverse event identification and reporting procedures. Interviews were conducted in English, Czech, or German languages.
Data Analysis
Data for the analysis were exported from Qualtrics into Microsoft Excel. The data in the transcription templates (German language interviews) were manually added to the Qualtrics export to create a single full data set of all responses. Data were analyzed using Microsoft Excel.
Results
Demographics of Survey Respondents
30 HCPs and 20 largely English-speaking patients participated in the study from six countries (Fig. 6). Of the HCPs, 19 were rheumatologists and 11 were dermatologists. Sixteen patients had rheumatologic (rheumatoid arthritis, psoriatic arthritis, ankylosing spondylitis) conditions and four had atopic dermatitis. The number of patients seen by each HCP in a typical week and the conditions reported by patients are shown in Figs. 7 and 8, respectively.
Fig. 6.
Distribution of survey participants. HCPs healthcare professionals
Fig. 7.
Patients seen in an average week by specialty type
Fig. 8.
Conditions reported by patients
The generalizability of the results is limited by a number of demographic factors. The tool was presented in the English language only, and we therefore recruited participants who were comfortable interacting in English, with a few exceptions (see Sect. 2). Additionally, Adept Field Solutions, Ltd. was engaged to recruit participants. Their network of HCPs and patients constitutes individuals who are comfortable with discussing their conditions/areas of practice and who are conversant with field-based research and online/remote discussions. Thus, the opinions of the participants may not reflect the general HCP and patient populations for rheumatological and dermatological conditions, and indeed the selection of patients and HCPs with these conditions or a professional alignment with these conditions also limits the generalizability. A larger study that draws participants from a variety of sources, including potentially social media, with a broader range of conditions/specialties would improve the generalizability of the results, as could presenting material for discussion in local languages.
Comparison of a Digital Tool with Paper-Based Additional Risk Minimization
Most HCPs indicated that they would use the digital concept, with 87% stating they would either use the digital concept instead of or alongside paper versions of the educational aRMM (see Fig. 9). Only 13% (a rheumatologist and a dermatologist both from Switzerland, a dermatologist from Canada and a rheumatologist from Germany) would prefer paper aRMMs, stating that not everyone has Internet access and that paper materials can be easier to use as no technical ability is required. Looking at this figure by specialty, 18% of dermatologists and 10% of rheumatologists stated a preference for paper only.
Fig. 9.
Healthcare professional (HCP) and patient responses about use of paper, digital, or both types of additional risk minimization measures
We asked HCPs whether they would recommend the digital concept/aRMM for all their patients, or just particular sub-groups of patients. Only 23% of HCPs would recommend the patient digital concept to all patients, mentioning that traditional paper-based aRMM could be better for older or less tech-savvy patients and for patients who prefer to have a physical object to take away from their consultation.
All patients in the study said they would use the digital concept, either instead of or alongside paper aRMMs, with none preferring paper-based versions. Half of the patients in the study would recommend the digital concept/aRMM to others with their condition, regardless of age, with one respondent commenting that “even the elderly are using computers—most don't want paper”. The possible discrepancy between HCP opinions of patient preferences of digital vs paper and the reality of patient preferences regarding these formats could be an area for further study.
Preferred Format for Educational Digital Concept aRMMs
In total, 83% of HCPs preferred the digital concept to be based on a website, with only 17% preferring a mobile app-based format. Patients overwhelmingly preferred the digital concept to be based on an app, with just 5% preferring a website. Our assumption is that HCPs prefer websites as they tend to have ready access to a desktop machine in their places of work; however, this could be a topic for further exploration with a larger sample.
Usefulness of Digital Additional Risk Minimization Tool
Of the HCPs participating in the study, 70% felt they would benefit from the digital concept/aRMM with 17% unsure and 13% stating that they would not benefit, as they had access to the information from other sources or may not recall how to find the aRMM website.
Usefulness of the Features Included in the Educational Digital Concepts
The features of the digital concepts are described in the Methodology section. Healthcare professionals rated prescribing guidance, resources, and interactive risk messaging as the most useful features of the HCP digital concept (Fig. 10). Patients rated frequently asked questions and interactive risk messaging as the most useful features of the patient digital concept (Fig. 11). The knowledge check/clinical scenario features were not rated highly by either HCPs or patients.
Fig. 10.
Usefulness of the features in the healthcare professional (HCP) digital concept as rated by the HCPs. FAQs frequently asked questions
Fig. 11.
Usefulness of the features in the patient digital concept as rated by the patients. FAQs frequently asked questions, PAC Patient Alert Card
Suggestions for Improvement to the Concept
Healthcare professionals and patients identified several improvements to the concept tool that could be compatible with current aRMM practice and suggested other options that would be beyond the remit of current aRMMs (see Tables 1 and 2).
Table 1.
Suggested improvements to the site/app aligned with current additional risk minimization material practice
| Identified by HCPs | Identified by patients | |
|---|---|---|
| Other useful resources, e.g., travel certificates, fact sheets, checklists | Yes | Yes |
| Information on drug–drug interactions, contraindications | Yes | Yes |
| Link with confirmation of receipt of educational materials | Yes | No |
| Calendar entries, reminders | No | Yes |
Table 2.
Suggested improvements to the site/app beyond the remit of current additional risk minimization practice
| Identified by HCPs | Identified by patients | |
|---|---|---|
| Condition assessment tool, risk scoring | Yes | No |
| Information on product benefits | No | Yes |
| Relevant information on diet, lifestyle, complementary therapies | No | Yes |
| Integration with labs/healthcare systems | No | Yes |
Discussion
Traditional paper-based educational aRMMs have several limitations, including the possibility of loss or damage, difficulties with version control, cost associated with printing and distribution, and reduced opportunities for interactive learning experiences.
In accordance with the current regulations relating to aRMMs [2], a variety of media (e.g., paper, audio, video, web, in-person training) can be used to distribute educational aRMMs, to ensure that access is not limited by disability or Internet connectivity. Given the high level of uptake of digital technologies across demographic groups [5] and the limitations of paper-based educational programs, pharmaceutical companies have begun to explore digital methods for communication of important risk minimization messages. To date, these approaches have generally been relatively simple, focusing on re-stating the information on the paper materials and on the provision of PDF versions of paper materials on websites or smart devices, offering an easier route to maintaining and acquiring up-to-date materials, but not exploiting the full potential of digital technology. There are further benefits to be realised from going beyond this and adopting a more comprehensive digital-based approach, including improved opportunities for collecting evidence on usage through page dwell times and on understanding through built-in knowledge checks. There is also greater scope, beyond making information interactive and more engaging, such as through personalization by sex, age, and existing medical conditions and the addition of rich media.
The authors therefore see an opportunity for pharmaceutical companies to explore more sophisticated digital approaches to the design and implementation of educational aRMMs. This qualitative survey study has shown that the majority of HCPs and patients in our small sample were open to using digital formats of educational aRMMs, either alongside or instead of paper. A combined approach, with both digital and paper versions being available, is likely to be the most realistic way to introduce digital aRMMs.
Both HCPs and patients voiced interest in a digital aRMM linked to other helpful resources, with content beyond the targeted safety concerns required for an aRMM tool, demonstrating that these stakeholders do not recognize the boundaries on the content of aRMMs outlined by regulators. An important next step is therefore opening a dialogue between pharmaceutical companies and regulators on possible future directions for digital aRMMs, using information gathered from this type of study with end users (HCPs, patients) to enable user-centric design, within the framework of aRMM regulations.
Healthcare professionals and patients rated the clinical scenarios and knowledge check sections of the digital concept lower than other features. These features may however be of interest to pharmaceutical companies and regulators as they may help to provide effectiveness evaluation measures, which for example, the European Medicines Agency requires to assess whether the objectives of an aRMM are being met [1]. Pharmaceutical companies have struggled to gather data for such an effectiveness evaluation via traditional means such as surveys, as response rates are generally low [7–9]. Therefore, it may be worthwhile considering how to make the knowledge check features more attractive and engaging to users—further user testing would help with the creation of a user-centric design.
Some differences were observed between the opinions of rheumatologists and dermatologists—18% of dermatologists, as compared with 10% of rheumatologists, would prefer a paper-only approach to aRMM implementation. Our data also showed that the dermatologists’ weekly patient volumes were all at the higher end of the indicated range. Thus, the dermatologists may have felt that having ‘another app or website’ to refer to would be burdensome. We therefore suggest that a tailored approach to each HCP specialty may be required when selecting the most appropriate methods for delivering aRMMs, based on various factors including healthcare/clinical practice settings, workload constraints, and patient populations.
In addition to the small sample size, the broader applicability of the results of this exploratory study is limited by the selection of mostly English-speaking participants, the focus on specific therapeutic areas, the qualitative nature of the survey, and the participants being recruited from the network of a fieldwork agency, and thus not being representative of the wider HCP and patient populations. As the interviews took place remotely because of the COVID-19 pandemic, we recruited individuals who would be comfortable with using e-mail and Zoom, another factor limiting the broader application of the findings.
Furthermore, the concept of an aRMM is based on European Union guidance and that the work was carried out in six countries only. Whilst many countries do follow GVP guidelines there may be differences in approach or acceptance globally. In the USA, for example, the concept of additional risk minimization may be considered in the Risk Evaluation and Mitigation Strategy [10]. As this study was not conducted in the USA, these results may not be applicable to the US population and expectations for digital risk minimization by the US Food and Drug Administration may be different to that of the European Medicines Agency.
Nevertheless, when taken as an indication of HCP and patient opinions of digital communications, the learnings from this exploratory study are potentially applicable beyond the participating countries and even beyond the boundaries of aRMMs. They show that regardless of country, both HCPs and patients are interested in having important healthcare information provided digitally. Additional findings provide suggestions for designing digital materials. To test this hypothesis further, a larger study would be required, with materials presented in local languages and with participants drawn from a more diverse range of sources, for example, potentially recruited from social media and representing a broader range of conditions and therapeutic areas. Key themes to focus on more closely in any subsequent work could include the possible discrepancy between HCPs opinions of patient need for paper and patient preferences for digital vs paper, and the preference of HCPs and patients for websites and apps, respectively.
Conclusions
Given the importance of patient safety and the increasing use of digital technologies to share and access information, pharmaceutical companies and regulators have an opportunity to explore digital design and delivery of education aRMMs. This study demonstrates interest from HCP and patient stakeholders in digital approaches to educational aRMMs and can therefore act as an input into these discussions. Both the format and content of any digital tool would need to be tailored to local regulations, the objective of the educational tool, and clinical situations.
Supplementary Information
Below is the link to the electronic supplementary material.
Acknowledgements
AbbVie and the authors thank the patients and healthcare professionals who participated in the research project. Adept Field Solutions Ltd. was sub-contracted by Axian to recruit research participants. Medical writing assistance was provided by Axian and was funded by AbbVie. MCW and MP would like to acknowledge Steven Wells for his contribution to the study.
Declarations
Funding
AbbVie Inc. funded this study and participated in the study design, research, analysis, data collection, interpretation of data, review, and approval of the publication. All authors had access to relevant data and participated in the drafting, review, and approval of this publication. No honoraria or payments were made for authorship.
Conflict of interest
Barbara Da Silva-Tillmann, Hetal Doshi, Fabio Lievano, and Cheryl Renz are full-time employees of AbbVie, and own AbbVie stock.
Ethics approval
Not required.
Consent to Participate
All patients and healthcare providers provided written informed consent to participate in the study.
Consent for Publication
Not applicable.
Availability of Data and Material
The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.
Code Availability
Not applicable.
Authors’ Contributions
All authors contributed to the study concept/design, data acquisition, analysis, and interpretation, as well as writing and critical review of the manuscript. All authors provided final approval of the submitted version.
References
- 1.EMA. Guideline on good pharmacovigilance practices (GVP). Module V: risk management systems (Rev 2) EMA/838713/2011 Rev 2*. 2017. https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-good-pharmacovigilance-practices-module-v-risk-management-systems-rev-2_en.pdf. Accessed 31 Aug 2021.
- 2.EMA. Guideline on good pharmacovigilance practices (GVP). Module XVI: risk minimisation measures: selection of tools and effectiveness indicators (Rev 2) EMA/204715/2012 Rev 2*. 2017. https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-good-pharmacovigilance-practices-module-xvi-risk-minimisation-measures-selection-tools_en-3.pdf. Accessed 31 Aug 2021.
- 3.Collins J, Bonneh-Barkay D. Considerations for successful risk-minimisation strategies in the EU. Pharm Med. 2016;30(5):257–261. doi: 10.1007/s40290-016-0161-7. [DOI] [Google Scholar]
- 4.EMA. Guideline on good pharmacovigilance practices (GVP). Module XVI Addendum I: educational materials EMA/61341/2015. 2015. https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/guideline-good-pharmacovigilance-practices-gvp-module-xvi-addendum-i-educational-materials_en.pdf. Accessed 31 Aug 2021.
- 5.Office for National Statistics. Internet access: households and individuals, Great Britain. 2020. https://www.ons.gov.uk/peoplepopulationandcommunity/householdcharacteristics/homeinternetandsocialmediausage/bulletins/internetaccesshouseholdsandindividuals/2020. Accessed 28 Jan 2021.
- 6.BHBIA. Legal and ethical guidelines for healthcare market research. 2020. https://www.bhbia.org.uk/guidelines-and-legislation/legal-and-ethical-guidelines. Accessed 31 Aug 2021.
- 7.Banerjee AK, Zomerdijk IM, Wooder S, Ingate S, Mayall SJ. Post-approval evaluation of effectiveness of risk minimisation: methods, challenges and interpretation. Drug Saf. 2014;37(1):33–42. doi: 10.1007/s40264-013-0126-7. [DOI] [PubMed] [Google Scholar]
- 8.Landsberg W, Al-Dakkak I, Coppin-Renz A, Geis U, Peters-Strickland T, van Heumen E, et al. Effectiveness evaluation of additional risk minimization measures for adolescent use of aripiprazole in the European Union: results from a post-authorization safety study. Drug Saf. 2018;41(8):797–806. doi: 10.1007/s40264-018-0662-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Artime E, Qizilbash N, Garrido-Estepa M, Vora P, Soriano-Gabarró M, Asiimwe A, et al. Are risk minimization measures for approved drugs in Europe effective? A systematic review. Expert Opin Drug Saf. 2019;18(5):443–454. doi: 10.1080/14740338.2019.1612875. [DOI] [PubMed] [Google Scholar]
- 10.US FDA. Risk evaluation and mitigation strategies: modifications and revisions. Guidance for industry revision 1. 2019. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/risk-evaluation-and-mitigation-strategies-modifications-and-revisions-guidance-industry. Accessed 30 Aug 2021.
- 11.Australian Government. Pharmacovigilance responsibilities of medicine sponsors: Australian recommendations and requirements. 2021. https://www.tga.gov.au/sites/default/files/190214_pharmacovigilance-responsibilities-medicine-sponsors.pdf. Accessed 10 Nov 2021.
- 12.Health Canada. Good Pharmacovigilance Practices (GVP) guidelines (GUI-0102). 2013. https://www.canada.ca/en/health-canada/services/drugs-health-products/compliance-enforcement/good-manufacturing-practices/guidance-documents/pharmacovigilance-guidelines-0102.html. Accessed 10 Nov 2021.
- 13.Swissmedic. Good Clinical Practice (GCP) and Good Pharmacovigilance Practices (GVP) inspections. 2020. https://www.swissmedic.ch/swissmedic/en/home/humanarzneimittel/clinical-trials/clinical-trials-on-medicinal-products/gcp-gvp-inspections.html. Accessed 10 Nov 2021.
- 14.Duran Alvarez H. Good pharmacovigilance practices and RMP: improvement to the strategy of implementation for the new pharmacovigilance standard in Mexico. J Formul Sci Bioavail 2019;3:66. https://www.hilarispublisher.com/proceedings/good-pharmacovigilance-practices-and-rmp-improvement-to-the-strategy-of-implementation-for-the-new-pharmacovigilance-standard-in-m-34390.html [Abstract submitted to Joint Event on 12th International Conference and Exhibition on Pharmacovigilance and Drug Safety and 22nd International Conference and Exhibition on Pharmaceutical Formulations and 21st Euro-Global Summit on Toxicology, 4-6 July, 2019; Valencia, Spain. Accessed 10 Nov 2021].
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.