Abstract
Introduction
Pharmacovigilance, the monitoring of drug safety after marketing approval, highly depends on the adequate reporting of adverse drug reactions (ADRs). To improve pharmacovigilance awareness and future ADR reporting among medical students, we developed and evaluated a student-run pharmacovigilance programme.
Methods
In this project, teams of medical students (first- to fifth-year) assessed real ADR reports, as submitted to the national pharmacovigilance centre. After assessment of causality, including identification of a potential pharmacological explanation for the ADR, the students wrote a personalized feedback letter to the reporter, as well as a summary for the European Medicines Agency (EMA) and World Health Organization (WHO) pharmacovigilance databases. This student assessment was then verified and evaluated by staff from The Netherlands Pharmacovigilance Centre Lareb (Lareb), using an e-questionnaire. Student attitudes, intentions, skills, and knowledge of ADR reporting were evaluated using the e-questionnaire, before and after participation in the programme.
Results
From May 2014 to January 2015, a total of 43 students assessed 100 different ADR reports selected by Lareb staff (n = 3). Student assessments were rated as useful (93%), scientifically substantiated (90%), accurate (92%), and complete (92%), and, on average, did not cost Lareb staff extra time. Medical students were positive about ADR reporting, and their awareness of ADR reporting increased significantly following participation in the programme (p < 0.05). After participation in the programme, the students intended to report serious ADRs in their future practice, and their knowledge of pharmacovigilance and ADR reporting showed they had a high overall level of pharmacological understanding.
Conclusion
The student-run pharmacovigilance programme is a win–win venture. It offers students a valuable ‘pharmacovigilance experience’, creates awareness in future doctors, and has the potential to increase pharmacovigilance skills and knowledge.
Electronic supplementary material
The online version of this article (doi:10.1007/s40264-016-0502-1) contains supplementary material, which is available to authorized users.
Key Points
| The student-run pharmacovigilance programme has mutual benefits for students and pharmacovigilance. |
| Undergraduate medical students can make useful, scientifically substantiated, accurate, and complete assessments of adverse drug reaction (ADR) reports. |
| Participating students were positive about ADR reporting, their awareness of ADR reporting increased, and they would likely report ADRs in the future. |
Introduction
Annually, millions of patients experience an adverse drug reaction (ADR) and, with the increasing use of medicinal drugs, the number of ADRs is also increasing [1]. ADRs can range from minor harm to full anaphylaxis, and even death, and may cause hospital admission, patient burden, and additional costs [1–4]. Although drug registration protocols require sound (pre)clinical testing of the safety and ADRs of new drugs, relatively little is known about these aspects in real-life circumstances prior to the drugs being given marketing approval [1, 3]. The monitoring/surveillance of ADRs after marketing approval (pharmacovigilance) is essential for identifying previously undetected, uncommon, or serious ADRs, and for improving understanding of drug risk profiles and medication safety [5, 6].
Pharmacovigilance centres play a major role in the postmarketing monitoring of drug safety, which, in many countries, is based on spontaneous (or voluntary) reporting [7]. Clinical observations, from both patients and healthcare professionals, serve as a starting point for reporting suspected ADRs. Most reported suspected ADRs are reported by health professionals, but also by patients [7], which means that health professionals should have sufficient knowledge, adequate abilities, and a positive attitude to evaluating and reporting possible ADRs encountered in daily practice. They are encouraged, and in some countries legally obliged, to report serious and unknown ADRs to the competent authority [8, 9]. Although ADR reporting is a professional responsibility, the rate of underreporting is high and this hinders optimal ADR monitoring [7, 10, 11]. Previous studies have identified multiple factors as underlying the low level of ADR reporting: indifference, lack of motivation, lack of knowledge, negative attitudes, misconceptions, and difficulty in accessing forms [11].
While medical and pharmacy students recognize the importance of ADR reporting and express the intention to report ADRs [12, 13], they are insufficiently prepared to handle ADRs and have inadequate pharmacovigilance skills and knowledge [12–14]. This may hamper optimal patient care and the safe use of drugs. Thus, there is a need to raise awareness, knowledge, and skills in recognizing, managing and reporting ADRs. While several interventions have proven effective for practising health professionals [3, 15], only a few interventions focus on future health professionals, such as medical or pharmacy students [15–17]. Most of these interventions have a theoretical basis (lectures), whereas students have indicated that they prefer active forms of learning [12–14]. Among trainee general practitioners, a practice-based method led to more and better documented ADR reports than a lecture-based approach [18]. Such exposure and practice are known to be necessary to master clinical skills: ‘practice makes perfect’ [19].
A practice-based and innovative approach for medical students could be the learner-centred student-run clinic (LC-SRC), which is based on the conceptual framework of ‘learning by doing’ [20]. In the LC-SRC, medical students get the opportunity and responsibility to contribute to a real clinical task, such as a consultation with a patient [21]. In this way, they practice clinical skills (such as prescribing) in a real context-based situation as early as possible in their medical education [22]. To meet students’ wishes for an active learning approach, we postulated that a pharmacovigilance project within the existing LC-SRC would facilitate the development of students’ pharmacovigilance attitudes, knowledge and skills in recognizing, managing, and reporting ADRs in real-life clinical practice.
The aim of this project was to increase the pharmacovigilance awareness and skills of medical students so that they would recognize and be able to manage and report ADRs in their future practice. The primary objectives were to analyse (1) the feasibility of the LC-SRC pharmacovigilance programme, and (2) the educational value of such a programme in terms of students’ pharmacovigilance skills and knowledge.
Methods
This prospective cohort study involved the Pharmacotherapy Section, Department of Internal Medicine, VU University Medical Center (VUmc), Amsterdam, The Netherlands, and The Netherlands Pharmacovigilance Centre Lareb (Lareb). Lareb is responsible for the collection and analysis of ADRs to medicines and vaccines, and for education on medication safety in The Netherlands. The pharmacovigilance programme was set up in April 2014 as an initiative within the LC-SRC of the VUmc. The overall aim of the LC-SRC is to improve undergraduate pharmacotherapy education [22].
Setting
From May 2014 to January 2015, the LC-SRC received three anonymized ADR reports from Lareb on a weekly basis. The reports had been selected (by Lareb staff) for their suitability regarding adequate documentation, relevance, and potential underlying pharmacological mechanism. This selection was in accordance with the educational aim of the project. The LC-SRC project was coordinated by students with experience in the LC-SRC who volunteered to take on a coordinating role. These coordinators added information, including a students’ manual, step-by-step assessment form, and additional database information. Furthermore, they gave guidance and made weekly appointments with the student teams (three to six participants) to provide feedback (see Fig. 1). Student teams had 6–10 days to assess the causality of the ADR, study the potential pharmacological mechanism, write a personalized feedback letter to the ADR reporter, and write a summary for the pharmacovigilance databases of the European Medicines Agency (EMA) and World Health Organization (WHO). Students were allowed to use any resource (e.g. summary of product characteristics (SmPC), Uptodate®, Micromedex®, which are regularly used by the Lareb assessors). After a final evaluation, the student coordinator mailed the final ADR assessment to an assessor at Lareb, where the assessment and concept feedback letter were reviewed. Th eLareb assessor also provided feedback on the assessment and sent the final feedback letter to students to optimize learning.
Fig. 1.
Handling of ADR reports. 1 Health professional or patient reports ADR to The Netherlands Pharmacovigilance Centre Lareb; 2 Lareb staff send ADR reports to the student coordinator of the pharmacovigilance programme; 3 Student coordinator sends report to student teams who assess the ADR report; 4 Student coordinator supports and gives feedback on ADR assessment; 5 Students send concept assessment to student coordinator; 6 Student coordinator sends final assessment to Lareb; 7 Lareb staff verify and score ADR assessments, submit report, and send feedback letter to the health professional or patient. ADR adverse drug reaction
Population
First- to fifth-year medical students of the VUmc School of Medical Sciences who participated in the LC-SRC (n = ±80) were eligible for this extracurricular pharmacovigilance programme.
Evaluation Instruments
The student-run pharmacovigilance programme was considered feasible if (1) the quality of the student assessments was acceptable; (2) the time it took to supervise was reasonable; and (3) the project had positive effects on students (which was assessed together with educational value). The feasibility of the LC-SRC pharmacovigilance programme and its educational value were evaluated using three different e-questionnaires: students completed one questionnaire before participation in the programme and another after participation, while Lareb staff completed one questionnaire after verification of each ADR assessment. The three questionnaires are available in Electronic Supplementary Material 1.
Lareb Supervisor E-questionnaire
Lareb staff were asked to fill in a short (2 min, six questions) e-questionnaire after they verified each ADR assessment. Assessments were rated for completeness, substantiation, inaccuracies, and time it took to verify the student ADR assessment, compared with the time it would take the staff member to deal with the ADR report themselves. Lastly, the final assessment was graded (scores 1–10, minimum–maximum). This questionnaire was composed of closed and multiple-choice questions (5-point Likert scales) and there was room for feedback.
Student Pre-Participation E-Questionnaire
Students completed a short (2–3 min, six questions) e-questionnaire to assess their familiarity with pharmacovigilance and ADR reporting prior to taking part in the programme. Besides providing information about their characteristics (student number, sex, study year), students also answered an open question about how they would manage a suspected ADR. Closed questions were asked relating to their awareness of ADRs and ADR reporting.
Student Post-Participation E-Questionnaire
After participation in the programme, students completed a more detailed (10 min, 15 questions) e-questionnaire to assess their role and progress in ADR management and their opinion about their pharmacovigilance training and the student-run pharmacovigilance programme. The questionnaire also focused on student attitudes and intentions regarding ADR reporting. These multiple-choice questions (7-point Likert scale) were based on the Dutch national pharmacovigilance study [12] and the studies of Gavaza et al. [13, 23, 24]. As in the national study [12], open-ended and dichotomous questions were used to investigate students’ knowledge and skills regarding basic pharmacovigilance and ADR reporting. Additional open-ended questions relating to pharmacovigilance knowledge were added: “What is the meaning of the black triangle on the packaging of medications?”; “Which resources can you use to see if an ADR is known?”; “What does a positive de- or re-challenge mean?”; and “Which patient-related factors could play a role in the development of an ADR?”
Data Analysis
Data were analysed using SPSS Statistics 22 (IBM Corporation, Armonk, NY, USA). Descriptive statistics were computed for the student and supervisor populations, assessment rating, and student outcomes. Student outcomes were analysed based on the levels of Kirkpatrick’s hierarchy [25] and were divided into three groups: intentions/attitudes, knowledge and skills of ADR reporting. Student open-ended questions were analysed using content/thematic analysis, and student responses on the questions in both the pre- and post-questionnaire (where, why, and what to report, and what they would do if they encountered an ADR) were analysed using a generalized estimating equation (GEE) analysis to test if participation in the programme improved pharmacovigilance skills and knowledge. The GEE analysis has several advantages compared with a repeated measures t test (e.g. homogeneity of variance is not necessary and no loss of information when parts of the longitudinal data are missing). Results of changes in responses (pre–post) are displayed in absolute differences (%). A one-way analysis of variance (ANOVA) was used to compare the mean intention scores for reporting in three different situations (serious, unknown, and all ADRs). A significance level with an α of 5% was considered statistically significant (p < 0.05) for all analyses.
Ethical Aspects
All ADR reporters agreed to the Lareb privacy statement (http://www.lareb.nl/Footer/Privacy), and the ADR reports were anonymously forwarded to students by Lareb staff. The Institutional Review Board (IRB) of the VUmc reviewed the research protocol and concluded that the study did not fall under the scope of the Dutch Medical Research Involving Human Subjects Act (WMO) [reference 15348). Participation in the pharmacovigilance programme was voluntary, and students, coordinators, and Lareb supervisors did not receive any form of credit, payment, or incentive. All students gave informed consent to participate in this study and for using the e-questionnaires for scientific purposes.
Results
In total, 100 ADR reports were assessed by 43 medical students working in teams. The students assessed 1–10 (mean 2.9, standard deviation [SD] 2.7) ADR reports. Eighty-seven reports originated from healthcare providers and 13 originated from patients. In total, 115 drugs were mentioned; antidepressants (16%) and antibiotics (11%) were the main drug groups reported. Sixty-two percent of the suspect drugs were non-essential drugs, defined as not being included in the 19th WHO essential medicine list [26]. The ADR reports mentioned 148 different symptoms; neurological (13%) and psychological (12%) symptoms were the most commonly reported events.
All student assessments of the ADR reports were remotely supervised and evaluated by Lareb staff (n = 3) using an e-questionnaire (see Table 1). They rated the assessments as being useful/very useful in 93% of cases (mean 4.58, SD 0.65), scientifically substantiated in 90% of cases (mean 4.49, SD 0.73), and complete (as in ‘not lacking important information’) in 92% of cases; 92% of the reports did not contain inaccuracies. The overall assessment was scored 8.29 (SD 1.15) out of 10 (maximum). The Lareb staff indicated that the student assessments saved time in 33% of cases, were time neutral in 56% of cases, and cost them extra time in 11% of cases.
Table 1.
Rating of student assessments by Lareb staff
| Assessment rating | N | Mean (SD) | Fully disagree | – | Neutral | – | Fully agree |
|---|---|---|---|---|---|---|---|
| Useful assessment | 100 | 4.58 (0.65) | – | 1 | 6 | 27 | 66 |
| Scientifically substantiated assessment | 100 | 4.49 (0.73) | – | 2 | 8 | 29 | 61 |
| Less time | – | Neutral | – | More time | |||
|---|---|---|---|---|---|---|---|
| Time spent verifying the student ADR assessment, compared with self-handling | 100 | 2.78 (0.77) | 3 | 30 | 56 | 8 | 3 |
| Low score (≤5) | Intermediate score (6–7) | High score (8–10) | |||||
|---|---|---|---|---|---|---|---|
| Total assessment rating | 100 | 8.29 (1.15) | 2 | 16 | 82 |
| Yes | No | ||||||
|---|---|---|---|---|---|---|---|
| Assessment was accurate | 100 | – | 92 | 8 | |||
| Assessment was complete | 100 | – | 92 | 8 |
SD standard deviation, ADR adverse drug reaction
Student Outcomes—Skills and Knowledge (Longitudinal Study)
A total of 29 (67.5%) participants completed the e-questionnaire before and after participation in the programme. The characteristics of study participants are available in Electronic Supplementary Material 2. Before participation in the programme, most students (89.7%) were aware of the reasons for ADR reporting, although only 62.1% of students knew where to report ADRs, and fewer (27.6%) knew what information was needed to fill in the ADR report. After participation in the pharmacovigilance programme, students were better informed as to where to report ADRs (+37.9%; p < 0.05) and what was needed for a qualitatively good report (+55.2%; p < 0.05). More students knew why ADRs should be reported to Lareb, however this improvement was statistically non-significant (+6.9%; p > 0.05) [see Fig. 2].
Fig. 2.
Student responses to where, what, and why report a suspected ADR pre- and post-participation in the student-run pharmacovigilance programme. ADR adverse drug reaction, n.s. non-statistically significant difference, *indicates statistically significant difference
How students responded to an ADR in one of their patients did not change significantly after participation in the programme. In Table 2, the specific actions students suggested to take before and after participation are displayed.
Table 2.
Student-acquired skills and knowledge (longitudinal)
| Participants indicating “I know where (in The Netherlands) to report an ADR” [% (n)] | Participants indicating “I know which essential information is needed for a qualitatively good ADR report” [% (n)] | Participants indicating “I know why ADRs should be reported to The Netherlands Pharmacovigilance Centre Lareb” [% (n)] | ||||
|---|---|---|---|---|---|---|
| Pre- | Post- | Pre- | Post- | Pre- | Post- | |
| Student responses to where, what, and why report a suspected ADR | ||||||
| First year (B1) | 42.9 (3/7) | 100 (2/2) | 0 (0/7) | 50 (1/2) | 100 (7/7) | 100 (2/2) |
| Second year (B2) | 33.3 (2/6) | 100 (7/7) | 33.3 (2/6) | 71.4 (5/7) | 66.7 (4/6) | 100 (7/7) |
| Third year (B3) | 78.6 (11/14) | 100 (12/12) | 28.6 (4/14) | 83.3 (10/12) | 92.9 (13/14) | 91.7 (11/12) |
| Fourth year (M1) | 100 (2/2) | 100 (7/7) | 100 (2/2) | 100 (7/7) | 100 (2/2) | 100 (7/7) |
| Fifth year (M2) | – | 100 (1/1) | – | 100 (1/1) | – | 100 (1/1) |
| Total | 62.1 (18/29) | 100 (29/29)a | 27.6 (8/29) | 82.8 (24/29)a | 89.7 (26/29) | 96.6 (28/29) |
| Pre [n = 29] (%) | Post [n = 29] (%) | p-value (95% CI) | |
|---|---|---|---|
| Student responses when encountering an ADR | |||
| Search for additional information | 62.1 | 76.7 | 0.115 (0.850–4.480) |
| Search for an alternative drug | 27.6 | 30.0 | 0.818 (0.259–2.909) |
| Discontinuing the suspected drug | 24.1 | 20.0 | 0.957 (0.258–3.278) |
| Altering (lower) dose of suspected drug | 17.2 | 10.0 | 0.224 (0.657–5.992) |
| Depends on severity/indication | 20.6 | 10.0 | 0.504 (0.167–1.522) |
| Report to pharmacovigilance centre | 31.0 | 43.3 | 0.317 (0.202–1.680) |
| Communication with patient | 3.4 | 6.7 | 0.548 (0.046–5.139) |
Upper part: Student skills and knowledge to adequately report a suspected ADR
Lower part: Student responses when encountering an ADR and GEE analysis outcome of statistical difference
ADR adverse drug reaction, GEE generalized estimating equation
aStatistical significance between pre- and post-participation
Student Outcomes (Post-Participation)
Intentions
After participation in the programme, all students (n = 29) intended to report serious (mean 6.38, SD 0.73) and unknown ADRs (mean 6.31, SD 0.81), but were less prepared (one-way ANOVA; p < 0.05) to report all ADRs encountered (mean 2.93, SD 1.22) to the competent authority. Six students (21%) had already reported at least one ADR to Lareb. Student intentions towards ADR reporting in different situations are displayed in Table 3.
Table 3.
Students’ intentions and attitudes to reporting ADRs
| N | Mean (SD) | Extremely unlikely (%) | Neither likely nor unlikely (%) | Extremely likely (%) | |||||
|---|---|---|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | |||
| Intentions and attitudes towards ADR reporting | |||||||||
| I intend to report serious ADRs that I encounter to the competent authority | 29 | 6.38 (0.73) | – | – | – | – | 4 (13.8) | 10 (34.5) | 15 (51.7) |
| I intend to report unknown ADRs that I encounter to the competent authority | 29 | 6.31 (0.81) | – | – | – | 1 (3.4) | 3 (10.3) | 11 (37.9) | 14 (48.3) |
| I intend to report all ADRs that I encounter to the competent authority | 29 | 2.93a (1.22) | 3 (10.3) | 8 (27.6) | 10 (34.5) | 5 (17.2) | 2 (6.9) | 1 (3.4) | – |
| How likely do you think the following outcomes will be if you report an ADR? | |||||||||
| Contributes to the safe use of medicines | 29 | 6.31 (0.66) | – | – | – | – | 3 (10.3) | 14 (48.3) | 12 (41.4) |
| Improves patient safety | 29 | 6.21 (0.77) | – | – | – | – | 6 (20.7) | 11 (37.9) | 12 (41.4) |
| Educates others about drug risks | 29 | 5.93 (0.92) | – | – | – | 1 (3.4) | 10 (34.5) | 8 (27.6) | 10 (34.5) |
| Personally beneficial | 29 | 4.14 (1.68) | 1 (3.4) | 6 (20.7) | 2 (6.9) | 8 (27.6) | 6 (20.7) | 3 (10.3) | 3 (10.3) |
| Time consuming to report | 29 | 4.00 (1.67) | 3 (10.3) | 2 (6.9) | 5 (17.2) | 9 (31.0) | 4 (13.8) | 4 (13.8) | 2 (6.9) |
| Disrupts the normal workflow | 29 | 3.83 (1.63) | 2 (6.9) | 7 (24.1) | 2 (6.9) | 5 (17.2) | 10 (34.5) | 2 (6.9) | 1 (3.4) |
| Increases risk of malpractice | 29 | 2.72 (1.33) | 2 (6.9) | 16 (55.2) | 4 (13.8) | 4 (13.8) | 2 (6.9) | – | 1 (3.4) |
| Breaks trust with patients | 29 | 2.14 (0.74) | 5 (17.2) | 16 (55.2) | 7 (24.1) | 1 (3.4) | – | – | – |
Upper part: Student intentions to report serious, unknown and all encountered ADRs to the competent authority
Lower part: Student behaviour beliefs towards reporting an ADR
ADR adverse drug reaction, SD standard deviation
aStatistical significant difference
Attitudes
Students had a high score for attitude relating to reporting ADRs after participation in the programme, and students rated ‘contributing to medication safety’ (mean 6.31, SD 0.66, 7-point Likert scale) as the main reason to report ADRs. ‘Improving patient safety’ (mean 6.21, SD 0.77) and ‘educating others about drug risks’ (mean 5.93, SD 0.92) were also important reasons. Students did not believe reporting ADRs could ‘break trust with patients’ (mean 2.14, SD 0.74) or ‘increase the risk of malpractice’ (mean 2.72, SD 1.33). Student attitudes towards ADR reporting in different situations are displayed in Table 3.
Knowledge and Skills
After they had participated in the programme, over three-quarters (82.8%) of students knew which items are necessary for a qualitatively good ADR report (scores for students’ skills and knowledge tests are available in Electronic Supplementary Material 2). Comedication (72.4%), a description of the reported ADR[s] (62.1%), and patient information (age and sex) and history (both 48.3%) were the most frequently mentioned essential items. Students were familiar with the resources they could consult if they encountered an ADR, and mentioned the SmPC (60.7%), Farmacotherapeutisch Kompas [Dutch independent medication information system for health professionals] (57.1%), and the website of Lareb (50%). Micromedex®, PubMed, and other resources (e.g. books and Up-to-date®) were mentioned less frequently (14.3, 28.6, and 7.1%, respectively). Students had a mean score for knowledge of general pharmacovigilance of 82.5% for dichotomous questions, uncorrected for guessing, and 53.5% for open questions. The items with the highest scores were the reporter’s identity and the necessity of reporting, even if all relevant information was not available. Incorrect answers were most often given to the questions relating to understanding of the term ‘pharmacovigilance’ and the explanation of a ‘de-challenge or re-challenge’. After participation in the programme, 75.9% of students knew that patients and/or medical students could report ADRs (even during their clerkships) (Electronic Supplementary Material 2).
Students’ Reflections on Participating
Overall, students valued the pharmacovigilance programme. They responded to having learned skills such as performing an ADR assessment (n = 10) and searching and assessing scientific literature (n = 9) in response to the open question relating to what they had learned by participating in the programme (see Table 4). Other subjects they reported to have learned included the importance of reporting ADRs (n = 7) and general pharmacological knowledge (n = 6). Students found assessing ADR reports educational (mean 4.33, SD 0.88), more instructive than fictive casuistry (mean 4.22, SD 0.70), and felt responsible for assessing the ADR reports (mean 4.22, SD 0.80). They did not consider that their current curriculum covered pharmacovigilance well (mean 2.70, SD 1.03) and thought that more pharmacovigilance education was needed (mean 3.96, SD 0.94). They thought that assessing ADR reports should be included in their curriculum (mean 3.93, SD 0.78).
Table 4.
Quotations/statements by participating students regarding what they learned in response to the open question “What have you learned by participating in the student-run pharmacovigilance programme? [See Electronic Supplementary Material 1]
| Theme | Quotations |
|---|---|
| Intentions and attitudes | |
| Importance of reporting (n = 7) | “That a lot is learned about medication by reporting ADRs” “The importance of reporting ADRs and assessing them properly” |
| Pharmacovigilant attitude (n = 2) | “To better look at the medications patients are using. Many new patient complaints could be better explained by adverse drug reactions instead of a new diagnosis” |
| Knowledge | |
| Pharmacological knowledge (n = 6) | “The existence of dangerous interactions between certain drugs” “Additional pharmacological knowledge and knowledge regarding the specific mechanism of action in the ADR reports I assessed” |
| Knowledge regarding ADRs (n = 4) | “I learned about the physiology/mechanisms that underlie an adverse drug reaction” “I know more about the side effects of several drugs” |
| Getting to know Lareb (n = 1) | “The existence of the pharmacovigilance center Lareb” |
| Skills | |
| Performing an ADR assessment (n = 10) | “What happens if you have reported an ADR” “I have learned how I can assess the causality of a suspected adverse drug reaction” |
| Searching and assessing scientific literature (n = 9) | “Searching for evidence-based literature regarding an adverse drug reaction” “I also learned which sound information sources are available” |
| Reporting an ADR (n = 8) | “How and where to report an adverse drug reaction” |
| Writing a scientific substantiated feedback letter (n = 2) | “To write a medical feedback letter that is short and concise” “To write a clear pharmacological explanation of the ADR and to write a feedback letter to the reporting physician” |
Themes are sorted based on Kirkpatrick’s hierarchy and were divided into three groups: intentions/attitudes, knowledge, and skills
ADR adverse drug reaction
Discussion
This study shows that undergraduate medical students can make high-quality (useful, scientifically substantiated, accurate, and complete) assessments of ADR reports, without costing Lareb staff extra time. Moreover, the programme improved the pharmacovigilance skills and awareness of future health professionals and provided the opportunity to give instruction on basic and clinical pharmacovigilance skills and knowledge. The programme gives undergraduate medical students the unique opportunity to participate in real pharmacovigilance practice. Therefore, the feasibility criteria were met and the student-run pharmacovigilance programme appears to be a win–win venture for both Lareb and medical students.
It is surprising that the medical students provided such high-quality assessment because ADRs are perceived as a difficult subject in pharmacotherapy practice and education. There are no previous studies of students contributing to pharmacovigilance by assessing ADR reports, therefore we could not compare our findings with those of other studies. However, in an earlier study, undergraduate students were able to solve difficult pharmacotherapy problems and performed at junior doctor level in an LC-SRC [21], which might be attributable to students responding well to the opportunity and responsibility of contributing to a real clinical task early in their medical education, showing greater intrinsic motivation and willingness to invest time and energy voluntarily [27, 28].
In general, supervising the student assessments, comprising selection of useful reports for the programme, and rephrasing feedback letters to professional communication cost little extra time compared with a full assessment by Lareb staff themselves. This is an essential finding because it helps to secure the future of this project and is an important condition for the win–win venture between students and Lareb. Only 11% of the ADR reports cost Lareb staff extra time, but this was not because the reports were of poor quality but because the extensive and in-depth reports prepared by the students necessitated Lareb staff taking extra time to check the additional referenced literature/resources. These reports were awarded high marks (data not shown).
The programme raised the pharmacovigilance awareness of future health professionals, an essential aspect of rational prescribing and medication safety. Nearly all participating students knew where, why, and what was needed for a qualitatively good report. Compared with earlier studies, students participating in our programme had a more positive attitude towards reporting serious and unknown ADRs and had higher intention scores than pharmacists, and pharmacy and medical students [12, 13, 23]. Potential negative aspects of ADR reporting, such as ‘disrupting the normal workflow’ and ‘time consuming to report’, were considered less likely. Students mentioned important items needed for a qualitatively good report more often, and had higher scores for basic pharmacovigilance knowledge than in an earlier study [12].
Most earlier interventions to raise awareness of ADR reporting predominantly targeted health professionals and were passive (reminders, lectures, etc.) instead of active educational interventions [15]. Furthermore, the main outcomes used consisted of the number of spontaneous reports, which is an intermediate outcome because it does not consider the additional value of the reports for pharmacovigilance (quality, novelty, etc.). Interventions for pharmacy students used lectures or other theoretical means and not contemporary educational interventions [16]. The current study was based on the conceptual framework of ‘learning by doing’ [20] and incorporated educational theory to improve pharmacovigilance teaching and practice, and measured relevant and direct outcomes.
The strength of this study lies in the unique collaboration with Lareb, whereby real and legitimate ADRs could be assessed and used for educational purposes while contributing to the monitoring of real ADRs. The use of previously published questionnaires [12, 13, 23, 24] on ADR reporting allowed us to compare the intentions, attitudes, knowledge, and ADR handling capability of future health professionals. The use of pre- and post-participation questionnaires enabled us to investigate educational values and to monitor student progress in this longitudinal study design.
The major limitations of this study are the relatively small heterogeneous sample size (43 students) and the response rate (67.5%) for the questionnaires, both of which limited study power. Furthermore, self-selection bias played a role since only students who had voluntarily participated in the LC-SRC were eligible to participate in the pharmacovigilance programme. Students who participated were probably more interested in the topic, having a greater interest in pharmacotherapy. Thus, we may have gained an overpositive impression of the general medical student population. The concept of the student-run pharmacovigilance programme and the presented results would be of interest to other universities and to other countries where pharmacovigilance centres play a similar role as in The Netherlands. As a WHO collaborating centre in pharmacovigilance education, Lareb plays an important role in developing, testing, distributing, and sharing innovative and successful educational methods [29].
Conclusions
Undergraduate medical students can make high-quality (useful, scientifically substantiated, accurate, and complete) assessments of ADR reports, and that making such assessments increases the pharmacovigilance awareness of students. Thus, a student-run pharmacovigilance programme is feasible and a win–win venture for Lareb and medical students. This study contributed to insight into the intentions, skills, and knowledge of pharmacovigilance and ADRs of undergraduate medical students by providing a unique opportunity to participate in real pharmacovigilance practice. This study also showed that students valued the extra attention paid to pharmacovigilance and would prefer to have more real-life practice in their medical curriculum. Further research is needed to determine the additional value of this novel approach, compared with, for instance, an ADR reporting assignment [30] or a lecture, on students’ pharmacovigilance skills. Future research should also focus on the long-term effects of innovative pharmacovigilance projects on ADR reporting.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Acknowledgements
The authors would like to thank Mrs. D.W.M. Pijnenburg (Pharmacist, The Netherlands Pharmacovigilance Centre Lareb) for her help in rating the student assessments; J.J. Sikkens, MSc, (Department of Internal Medicine, Pharmacotherapy Section, VUmc, Amsterdam, The Netherlands) for his help with the statistical analysis; Mrs. S. Groenland, Mrs. S. de Boer and Mrs. L. van Gastel (VUmc School of Medical Sciences, Amsterdam, The Netherlands; student-run pharmacovigilance programme and SRC coordinators); all previous contributors to the LC-SRC project in VUmc; and all students participating in the student-run pharmacovigilance programme.
Compliance with Ethical Standards
Funding
No sources of funding were used to assist in the preparation of this study.
Conflicts of interest
Tim Schutte, Jelle Tichelaar, Michael O. Reumerman, Rike van Eekeren, Leàn Rolfes, Eugène P. van Puijenbroek, Milan C. Richir, and Michiel A. van Agtmael have no conflicts of interest that are directly relevant to the content of this study.
Ethical Approval
See ethical aspects section in Methods.
References
- 1.Sultana J, Cutroneo P, Trifiro G. Clinical and economic burden of adverse drug reactions. J Pharmacol Pharmacother. 2013;4(Suppl 1):S73–S77. doi: 10.4103/0976-500X.120957. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Gyllensten H, Rehnberg C, Jonsson AK, Petzold M, Carlsten A, Andersson Sundell K. Cost of illness of patient-reported adverse drug events: a population-based cross-sectional survey. BMJ Open. 2013;3(6):pii e002574. [DOI] [PMC free article] [PubMed]
- 3.Molokhia M, Tanna S, Bell D. Improving reporting of adverse drug reactions: systematic review. Clin Epidemiol. 2009;1:75–92. doi: 10.2147/CLEP.S4775. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Pirmohamed M, James S, Meakin S, Green C, Scott AK, Walley TJ, et al. Adverse drug reactions as cause of admission to hospital: prospective analysis of 18,820 patients. BMJ. 2004;329(7456):15–19. doi: 10.1136/bmj.329.7456.15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Pirmohamed M, Breckenridge AM, Kitteringham NR, Park BK. Adverse drug reactions. BMJ. 1998;316(7140):1295–1298. doi: 10.1136/bmj.316.7140.1295. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Martin K, Begaud B, Latry P, Miremont-Salame G, Fourrier A, Moore N. Differences between clinical trials and postmarketing use. Br J Clin Pharmacol. 2004;57(1):86–92. doi: 10.1046/j.1365-2125.2003.01953.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Miguel A, Azevedo LF, Lopes F, Freitas A, Pereira AC. Methodologies for the detection of adverse drug reactions: comparison of hospital databases, chart review and spontaneous reporting. Pharmacoepidemiol Drug Saf. 2013;22(1):98–102. doi: 10.1002/pds.3348. [DOI] [PubMed] [Google Scholar]
- 8.Safety monitoring of medicinal products . Guidelines for setting up and running a pharmacovigilance centre. Uppsala: The Uppsala Monitoring Centre, WHO Collaborating Centre for International Drug Monitoring; 2011. [Google Scholar]
- 9.Backstrom M, Mjorndal T, Dahlqvist R. Under-reporting of serious adverse drug reactions in Sweden. Pharmacoepidemiol Drug Saf. 2004;13(7):483–487. doi: 10.1002/pds.962. [DOI] [PubMed] [Google Scholar]
- 10.Hazell L, Shakir SA. Under-reporting of adverse drug reactions: a systematic review. Drug Saf. 2006;29(5):385–396. doi: 10.2165/00002018-200629050-00003. [DOI] [PubMed] [Google Scholar]
- 11.Lopez-Gonzalez E, Herdeiro MT, Figueiras A. Determinants of under-reporting of adverse drug reactions: a systematic review. Drug Saf. 2009;32(1):19–31. doi: 10.2165/00002018-200932010-00002. [DOI] [PubMed] [Google Scholar]
- 12.Schutte T, Tichelaar J, Reumerman MO, van Eekeren R, Rissmann R, Kramers C, et al. Pharmacovigilance skills, knowledge and attitudes in our future doctors: a nationwide study in The Netherlands. Basic Clin Pharmacol Toxicol. doi:10.1111/bcpt.12712. (Epub 24 Nov 2016) [DOI] [PubMed]
- 13.Gavaza P, Bui B. Pharmacy students’ attitudes toward reporting serious adverse drug events. Am J Pharm Educ. 2012;76(10):194. doi: 10.5688/ajpe7610194. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Elkalmi RM, Hassali MA, Ibrahim MI, Widodo RT, Efan QM, Hadi MA. Pharmacy students’ knowledge and perceptions about pharmacovigilance in Malaysian public universities. Am J Pharm Educ. 2011;75(5):96. doi: 10.5688/ajpe75596. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Pagotto C, Varallo F, Mastroianni P. Impact of educational interventions on adverse drug events reporting. Int J Technol Assess Health Care. 2013;29(4):410–417. doi: 10.1017/S0266462313000457. [DOI] [PubMed] [Google Scholar]
- 16.Rosebraugh CJ, Tsong Y, Zhou F, Chen M, Mackey AC, Flowers C, et al. Improving the quality of adverse drug reaction reporting by 4th-year medical students. Pharmacoepidemiol Drug Saf. 2003;12(2):97–101. doi: 10.1002/pds.797. [DOI] [PubMed] [Google Scholar]
- 17.Durrieu G, Hurault C, Bongard V, Damase-Michel C, Montastruc JL. Perception of risk of adverse drug reactions by medical students: influence of a 1 year pharmacological course. Br J Clin Pharmacol. 2007;64(2):233–236. doi: 10.1111/j.1365-2125.2007.02882.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Gerritsen R, Faddegon H, Dijkers F, van Grootheest K, van Puijenbroek E. Effectiveness of pharmacovigilance training of general practitioners: a retrospective cohort study in the Netherlands comparing two methods. Drug Saf. 2011;34(9):755–762. doi: 10.2165/11592800-000000000-00000. [DOI] [PubMed] [Google Scholar]
- 19.Issenberg SB, McGaghie WC. Clinical skills training–practice makes perfect. Med Educ. 2002;36(3):210–211. doi: 10.1046/j.1365-2923.2002.01157.x. [DOI] [PubMed] [Google Scholar]
- 20.Schutte T, Tichelaar J, Dekker RS, van Agtmael MA, de Vries TP, Richir MC. Learning in student-run clinics: a systematic review. Med Educ. 2015;49(3):249–263. doi: 10.1111/medu.12625. [DOI] [PubMed] [Google Scholar]
- 21.Dekker RS, Schutte T, Tichelaar J, Thijs A, van Agtmael MA, de Vries TP, et al. A novel approach to teaching pharmacotherapeutics–feasibility of the learner-centered student-run clinic. Eur J Clin Pharmacol. 2015;71(11):1381–1387. doi: 10.1007/s00228-015-1916-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Schutte T, Tichelaar J, van Agtmael M. Learning to prescribe in a student-run clinic. Med Teach. 2016;38(4):425. doi: 10.3109/0142159X.2015.1072145. [DOI] [PubMed] [Google Scholar]
- 23.Gavaza P, Brown CM, Lawson KA, Rascati KL, Wilson JP, Steinhardt M. Influence of attitudes on pharmacists’ intention to report serious adverse drug events to the Food and Drug Administration. Br J Clin Pharmacol. 2011;72(1):143–152. doi: 10.1111/j.1365-2125.2011.03944.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Gavaza P, Brown CM, Lawson KA, Rascati KL, Wilson JP, Steinhardt M. Texas pharmacists’ knowledge of reporting serious adverse drug events to the Food and Drug Administration. J Am Pharm Assoc (2003). 2011;51(3):397–403. [DOI] [PubMed]
- 25.Kirkpatrick DI. Evaluation of training. In: Craig RL, Bittel, LR (eds). Training and Development Handbook. New York: McGraw-Hill; 1976.
- 26.WHO essential medicine list, 2015. Geneva: World Health Organization; 2015.
- 27.Kusurkar RA, Croiset G, Ten Cate TJ. Twelve tips to stimulate intrinsic motivation in students through autonomy-supportive classroom teaching derived from self-determination theory. Med Teach. 2011;33(12):978–982. doi: 10.3109/0142159X.2011.599896. [DOI] [PubMed] [Google Scholar]
- 28.Schutte T, Tichelaar J, Dekker RS, Thijs A, de Vries TPGM, Kusurkar RA, et al. Motivation and competence of participants in a learner-centered student-run clinic: an exploratory pilot study. BMC Med Educ. 2017 doi: 10.1186/s12909-017-0856-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.WHO Collaborating Centre for Pharmacovigilance in Education and Patient Reporting. http://www.who.int/medicines/areas/quality_safety/safety_efficacy/collab-centres-netherlands/, http://www.lareb.nl/whocc?lang=en-GB. Accessed 7 Jan 2017.
- 30.van Eekeren R, van der Horst P, Hut F, van Grootheest K. Leer studenten bijwerkingen herkennen. Medisch Contact. 2014;04:150–154. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.