Skip to main content
PLOS ONE logoLink to PLOS ONE
. 2020 Aug 13;15(8):e0235591. doi: 10.1371/journal.pone.0235591

Increasing adverse drug reaction reporting—How can we do better?

Miri Potlog Shchory 1,2,*,#, Lee H Goldstein 3, Lidia Arcavi 4, Renata Shihmanter 4, Matitiahu Berkovitch 5,, Amalia Levy 1,
Editor: Karen Cohen6
PMCID: PMC7425892  PMID: 32790671

Abstract

Adverse drug reactions (ADRs) are associated with morbidity and mortality worldwide. Although national systems for reporting ADRs exist there is a low reporting rate. The aim of the current study was to evaluate an intervention plan for improving ADRs reporting among medical professionals (physicians and nurses). A multicentre intervention study was conducted, in which one medical centre was randomly assigned to the intervention group and two medical centres to the control group. The study consisted of 3 phases: baseline data collection, intervention and follow-up of the reporting rate. The questionnaire that was filled in at base line and at the end of study, contained questions about personal/professional demographic variables, and statements regarding knowledge of and behaviour toward ADRs reporting. The intervention program consisted of posters, lectures, distant electronic learning and reminders. An increase in the number of ADRs reports was noted in the intervention group (74 times higher than in the control group) during the intervention period, which was gradually decreased with as the study progressed (adjusted O.R = 74.1, 95% CI = 21.11–260.1, p<0.001). The changes in the "knowledge related to behaviour” (p = 0.01) and in the "behaviour related to reporting" (p<0.001) score was significantly higher in the intervention group. Specialist physicians and nurses (p<0.001), fulfilling additional positions (p<0.001) and those working in other places (p = 0.05) demonstrated a high rate of report. Lectures were preferable as a method to encourage ADRs reporting. The most convenient reporting tools were telephone and online reporting. Thus, implementation and maintenance of a continuous intervention program, by a pharmacovigilance specialist staff member, will improve ADRs reporting rates.

Introduction

An integral part of drug therapy is the adverse reactions associated with the drug [1]. These reactions can cause personal injury, hospitalization overload and increase in health costs, thereby creating a heavy burden on national healthcare systems [25]. Studies around the world have demonstrated that 3–7% of all hospitalizations are a result of adverse drug reactions (ADRs) and 10–20% of inpatients suffer from drug-related adverse reactions [3, 68]. Serious ADRs were found to be the fourth to sixth causes of death in hospitalized patients in the US, leading to extended hospitalization and doubling of the cost of treating these patients [9,10]. Therefore, both healthcare teams and patients share a common goal of early detection and prevention of ADRs.

Pharmacovigilance is defined by the WHO as the science and activities relating to the detection, assessment, understanding and prevention of adverse reactions or any other drug-related problem. National systems for reporting drugs' adverse reactions can be found in almost every country [1012]. Spontaneous reporting of medical staff regarding the occurrence of adverse reactions is the major source for monitoring and investigating adverse reactions of marketed drugs. However, only 1 in 20 adverse reactions is actually detected and defined as a real side effect; this leads to the erroneous assumption that the incidence of adverse reactions is much lower than it actually is [1315]. Inadequate ADR reporting may lead to loss of clinical information that could prevent substantial damage to patients and consequently minimize health costs.

Hence, it is very important to encourage medical staff to report any definite or suspected ADR, as well as to establish and maintain an accurate database which can be used for analyzing and processing of accumulated data, drawing conclusions and providing further recommendations. This series of actions could optimize patients’ wellbeing and safety and improve the functioning of the healthcare system.

Israel has been an official member of the World Health Organization international program for monitoring drugs since 1973. In August 2012, the Israeli ministry of health (MOH) published guidelines for reporting adverse events and new safety information. This document specifies the type of information that the Marketing Authorization Holder (MAH) requires reporting and enables the MAH to appoint a pharmacist to serve as a qualified person responsible for matters related to the reports included in this standard operating procedure, according to the standard worldwide practice. Those regulations were update at February 2013 in order to clarify the work processes related to reporting ADRs and new safety information, to update the definitions of the SOP and to update the types of information requiring reporting by the MAH. The aim of additional update from May, 2013 was to adjust the SOP to the Pharmacists Regulations [16]. A new regulation was launched at October, 2014, in which a reporting system in medical institutions for both common and severe ADRs was established. In July 2019, a new portal for reporting adverse events to the Risk Management Department was established. The purpose of all these regulations was to raise the awareness of the healthcare teams to the importance of reporting ADRs [17]. Nevertheless, the reporting rate in Israel, as is worldwide, is still quite low. Schwartzberg el al., identified 16,409 of Individual Case Safety Reports (ICSRs) submitted to the MOH’s ADRs central database between September 2014 and August 2016. However, of these reports, only 5.5% were submitted by health care professionals from medical institutions, while 94.3% were submitted by pharmaceutical companies (MAH and importers) and only 0.2% of the reports have been submitted by patients and the general public [18].

The purpose of the present study was to establish and evaluate an intervention plan in order to improve the reporting norms of ADRs among medical professionals (physicians and nurses).

Materials and methods

Study design

The study was conducted over a period of 17 months (August 2012 to December 2013) and included the healthcare teams (physicians and nurses) of internal medicine divisions from three public medical centers in Israel: "A", "B" and "C", while each division served as a cluster. The medical centers selected for the study were public hospitals serving an urban and rural population of 0.5–1 million people each. In Israel, a high percentage of physicians and nurses are from the Commonwealth States and Russian is their first language. In order to make it easier for them to complete the questionnaire and to increase the response rate it was translated into Russian. Staff members who lacked sufficient knowledge of Hebrew or Russian to fill in the questionnaires were excluded from the study. Medical Center "A" was randomly selected to be the intervention group. Medical centers "B" and "C" were merged and served as the control group. The randomization among the centers was raffled by an external person who was not related to the study. ADRs reports were collected from the Israeli Ministry of Health computerized website for all three medical centers, and reports from medical center "A" (the intervention group) were also collected from the physical reporting binders available in the departments. The study was approved by the institutional review board of each medical center respectively. "A"—Ethics ("Helsinki") Committee; (protocol Number 180/10); "B"—Committee Helsinki; (protocol Number EMC-0107-10); "C"—Ethics ("Helsinki") Committee (protocol Number ver:1 KAP 1). In the introduction to the questionnaire a detailed explanation regarding the research and its rational was provided (See Supplement 1 in the S1 File). In this section the participant was required to give consent to be included in the research. Verbal consent was received during a meeting in which the research was presented to each participant. As part of the consent procedure the interviewer explained to each medical staff member (physicians and nurses) about the study. Potential participants were informed that taking part in the study was voluntary. Those who agreed to participate gave their oral consent. A record of all participants who provided oral consent was kept by the principal investigator. Every local IRB approved the use of verbal consent. Written consent was not obtained from the participants because the participants were staff members and not patients.

The study consisted of 3 phases: The first phase of baseline data collection lasted three months and included handing out a questionnaire to the healthcare teams. The questionnaire contained questions about personal and professional demographic data, and statements regarding the knowledge of and behavior toward ADRs reporting.

The second phase of the study was a 5-month intervention phase. The purpose of the intervention plan was to improve the staff reporting rate of ADRs. Site "A" was randomly selected as the medical center for intervention. The intervention program consisted of the following: a) posters for raising the awareness of the medical staff; these were placed at various locations throughout the departments, such as: physicians’/ nurses’ rooms, medication rooms, and dining areas; b) Forty-five minute lectures that were given during divisional meetings of physicians and nurses separately. The lectures included: definitions of ADRs and pharmacovigilance, an explanation about the importance of the issue, data from international studies in the field, information about the relationship between adverse drug events and morbidity/mortality rates, incidence and prevalence of ADRs during hospital admission, the costs of ADRs for the healthcare system and the patients, presenting the reasons of ADR under reporting and a description of the current practices in Israel and around the world; c) Program promotion. This included: visiting the departments and talking with the medical staff twice weekly, presenting the program in the medical center portal and homepage and sending text messages to the participants on their mobile phones every two weeks (a total of 8 text messages were sent); d) introduction of distant electronic learning into the medical center portal.

The third phase of the study lasted 9 months and included following up on the monthly reporting rates. ADRs were reported through the Ministry of Health computerized website (for both the intervention and the control group) or documented in binders available only in the departments of the intervention group. At the end of this phase the participants from both groups were asked to fill in the same questionnaire again.

The questionnaire (Supplement 1 in S1 File)

The questionnaire design was based on the causes for underreporting of ADRs among professionals, known as the “seven deadly sins” and on the combined theoretical model of the factors affecting the conditions for ADR reporting by healthcare professionals [17,1932].

Demographic data included profession and degree, date and place of birth, gender, country of professional training, years of experience, expertise, as well as additional roles and positions in other medical institutions.

"Knowledge related to behavior" was intended to explore the knowledge of the participants about the ADR reporting procedure and its importance. It was investigated using the following general question which dealt with identification of ADRs and the reasons for reporting/not reporting: "You may notice an irregular adverse reaction from drug treatment and not report it since:" Then the participants were prompted to choose one of 5 statements: "A. I know the adverse drug reaction has already been documented by the pharmaceutical company"; "B. I do not know that there is a center for reporting adverse drug reactions"; "C. I am not aware of the need for reporting adverse drug reactions"; "D. I don't know how to report adverse drug reactions"; and "E. Reporting one adverse drug reaction does not significantly contribute to the reporting mechanism". The answers were ranged on a scale from 1 point—no reason for not reporting to 10 points—good reason for not reporting. This means that a staff member who reports ADRs will receive a lower score.

The score of "behavior related to reporting" was constructed from two items that analyzed the patterns of reporting to the National Center of pharmacovigilance and to pharmaceutical companies. The statements were: "A. I spoke with pharmaceutical companies about the possibility of adverse drug reactions with their drugs"; and "B. Have you ever reported adverse drug reactions to a national reporting center?". The average of this score ranged from 1 point—non reporting behavior—through 10 points—reporting behavior. The higher rate of reporting achieved a higher score.

The main hypothesis: after the intervention program there would be more ADRs reporting among medical professionals (physicians and nurses) in the intervention group compared to the control group and to ADRs reporting base line.

Data analysis

Data analysis was carried out using the SPSS 21 software (PASW inc., USA). The statistical analysis was conducted according to the phases of the study. Means and standard deviation were used for continuous variables and examined by T or One-Way ANOVA/Mann Whitney tests based on the variables distribution. The score of "Knowledge related to behavior" and the score of "behavior related to reporting" didn’t distribute normally, thus we used a- parametric test (Mann Whitney). Percentages and numbers were used for categorical variables and were tested by chi square or Fisher’s exact tests as appropriate. Statistical significance was established as p≤0.05. The multi-variable models examined the independent effect of the intervention program on reporting. The difference (change) was viewed as the dependent variable, and factors forecasting change were examined through multi-variable linear regression models. Identifying independent predicting factors of reporting adverse reactions was done through multi-variable logistic regression models.

Quantitative variables, such as the differences between groups, comparison between physicians and nurses and between medical centers, were analyzed by chi square or Fisher’s exact tests. The differences and changes between the various parameters (knowledge and reporting patterns) were calculated by comparing the data collected after the intervention phase with the baseline data (change). The differences in knowledge and reporting among the study groups were compared separately by T tests with independent samples or through One-Way ANOVA. The building strategy for multivariable models was forced all the independent variables to one block. Both statistical and clinical justifications were considered. The models included all the variables that were found to be significant (p<0.05) in the univariate data analysis, and covariates that were important to controlling for, as a baseline characteristic, according to the research questions. All the independent variables that were included in the analyses were presented in the results of the multivariable models. The multivariate models examined the effect of the intervention program on knowledge and reporting. The difference (change) was defined as the dependent variable and factors forecasting change were examined through linear regression. The adjusted factors of reporting adverse reactions (with 95% confidence intervals, 95% CI) were performed through multivariate logistic regression models. The independent effect of every measure index of the questionnaire was adjusted to demographic and professional variables.

Results

433 (81.5%) medical staff members, physicians and nurses, completed the questionnaire twice, before and after the intervention. 47.8% of the participants were from the "A" medical center, 28.4% from "B" and 23.8% from "C". Distribution by gender was 69.1% females and 30.9% males. 73% were nurses and 27% physicians. No selection bias was found between the staff members completed the questionnaire the first time and those completed it twice. No differences in personal or professional variables were found between the intervention group ("A" medical center) and control group ("B" and "C" medical centers), except for the ratio between physicians and nurses and the subjects country of origin and average age."

During the research period, 336 ADRs were reported, of which 288 (85.7%) were reported in Medical Center “A”, with 285 ADRs from the reporting binders and 3 ADRs from the Ministry of Health’s computerized website. The ADRs reports from the control groups comprised 10 reports (3%) in center “B” and 38 reports (11.3%) in center C; these were reported to the Health Ministry’s computerized website. The reports were checked and there were no duplicate reports by the staff members on both reporting channels in the intervention group.

The number of ADRs reports in the intervention vs. the control groups during the study period is presented in Fig 1.

Fig 1. Number of ADR reports in the intervention vs. the control groups during the study period.

Fig 1

A rapid and substantial increase in the number of ADR reports was noted in the intervention group during the 5-month intervention period, which gradually decreased toward the end of the study.

A rapid and substantial increase in the number of ADR reports was noted in the intervention group during the 5-month intervention period, which gradually decreased toward the end of the study. The reporting in the intervention group went nearly back to baseline and was even lower than in the control group. On the other hand, there was almost no change in the number of ADR reports in the control group during the entire study duration. After the intervention period the reporting rate in the intervention group reverted to almost baseline and was lower than the control group, similarly to the trend that was observed in the baseline.

Comparison of the rate of ADRs reports received from physicians vs. nurses indicated that in both groups a substantial increase in the number of ADR reports was observed during the intervention period, which gradually decreased toward the end of the study.

Comparison of the score of "knowledge related to behavior" showed that before any intervention the mean score of the control group was significantly lower than that observed in the intervention group (3.84±2.20 and 4.37±2.27 respectively, p = 0.02), demonstrating that the control group was more aware of ADRs reporting. Nevertheless, the changes in the "knowledge related to behavior” score was significantly higher in the intervention than in the control group (a change of -0.69±2.58 in the intervention group vs. -0.11±2.19 in the control group, P = 0.01)) Table 1).

Table 1. Comparison of the changes in the mean score of "knowledge related to behavior" of the intervention group vs. the control group.

The changes in the "knowledge related to behavior” score was significantly higher in the intervention than in the control group.

Score before the intervention Score after the intervention Differnce in score changes
Mean ±SD (n) Mean ± SD (n) Mean ±SD(n)
Intervention group 4.37±2.27 (206) 3.67±2.16 (205) - 0.69±2.58 (205)
Control group 3.84±2.20 (220) 3.73±2.14 (225) - 0.11±2.19 (221)
P value 0.02 0.79 0.01

When the score of "behavior related to reporting" was compared between the intervention and control groups upon intervention, a statistically significant increase in the score "behavior related to reporting" was observed in the intervention group, with a mean change of 0.65±2.22 (2.21±1.88 before intervention and 2.37±2.87 after intervention, P<0.001). No significant difference in the score of "behavior related to reporting" was noted in the control group (Table 2).

Table 2. Comparison of the changes in the mean score of "behavior related to reporting" of the intervention group vs. the control group.

Score before the intervention Score after the intervention Differnce in scores changes
Mean ± DS (n) Mean ± DS (n) Mean ± DS (n)
Intervention group 2.21±1.88 (207) 2.87±2.37 (207) 0.65±2.22 (207)
Control group 2.54±2.13 (226) 2.48±2.12 (226) - 0.06±2.16 (226)
p value 0.09 0.79 0.001

Comparison of patterns in "behavior related to reporting" was conducted according to various demographic and professional-related variables. The results revealed that the nurses demonstrated less changes in "behavior related to reporting" than physicians (P = 0.003). A significant positive correlation was found between the numbers of patients treated per day by the medical staff (nurses and physicians) and "behavior related to reporting", i.e. as the number of patients per caregiver increased, the change in "behavior related to reporting" score was higher (P = 0.02). In addition, the results revealed that the more aware the caregiver is of the fact that the patients are consuming more than one medication per day, a larger change in "behavior related to reporting" score is observed (P = 0.02, r = 0.13) (Table 3).

Table 3. Comparison of patterns in "behavior related to reporting" according to various demographic- and professional-related variables of the intervention group vs. the control group.

Nurses demonstrated less changes in "behavior related to reporting" than physicians A significant positive correlation was found between the numbers of patients treated per day by the medical staff (nurses and physicians) and "behavior related to reporting", as well as between the awareness of the caregiver that the patients are consuming more than one medication per day, and the change in "behavior related to reporting" score.

Variable Number Change in "behavior related to reporting" (Mean±SD) P-value
Total No
Gender Female 299 0.20±2.04 P = 0.27
433 Male 134 0.46±2.57
Profession Nurse 316 0.06±1.91 P = 0.003
433 Physician 117 0.89±2.79
Country of origin Israel 200 0.11±2.37 P = 0.25
421 Eastern Europe 184 0.49±1.96
  Other 37 0.38±2.52
Country of professional training        
  Israel 291 0.18±2.07 P = 0.31
422 Eastern Europe 89 2.48±2.24
  Other 42 0.62±3.05
Specialty Yes 144 0.42±2.79 P = 0.40
433 No 289 0.21±1.93
Fulfilling other positions Yes 91 0.38±2.89 P = 0.51
392 No 301 0.17±1.91
Working in other places Yes 47 0.70±3.01 P = 0.29
423 No 376 0.22±2.11
Age   404 r = 0.04 P = 0.40
404        
Age ≤35 143 0.17±2.21 P = 0.70
404 36–45 131 0.32±2.23
  ≥46 130 0.40±2.41
Years of seniority in the profession   392 r = 0.04 P = 0.49
392        
Years of seniority in the profession <7 110 0.17±2.21 P = 0.48
392 7–20 172 0.18±2.13
  >21 110 0.50±2.38
Years of seniority in the profession in Israel   213 r = 0.06 P = 0.42
213        
Years of seniority in the profession in Israel ≤20 113 0.45±1.76 P = 0.72
213 >21 96 0.35±2.21
Years of Seniority in the internal division   414 r = 0.03 p = 0.57
414        
Years of Seniority in the internal division ≤3 137 0.16±1.97 P = 0.31
414 3.1–13 140 2.55±2.22
  ≥13.1 137 0.23±2.43
Number of patients treated per day   382 r = 0.12 P = 0.02
382        
Number of patients treated per day ≤12 195 0.06±2.11 P = 0.02
382 >13 187 1.56±2.34
No. of drugs distributed/examined per day   319 r = -0.04 P = 0.50
319        
No. of drugs distributed/examined per day ≤50 190 2.42±0.41 P = 0.71
319 >51 129 2.09±0.31
% of Patients taking more than one drug per day   380 r = 0.13 P = 0.02
380        
Patients taking more than one <90% 82 0.41±2.40 P = 0.77
drug per day >90% 298 0.38±2.34
380      

The demographic and the professional variables and reporting/non reporting behavior were later examined within the intervention group. As seen in Table 4, physicians reported more than nurses (56.9% vs. 36.5%, p = 0.009). Specialists, both nurses and physicians, reported more than non-specialists (60.6% vs. 32.6%, p<0.001). Those (both physicians and nurses) fulfilling additional positions and those working in other places beside the hospital demonstrated high rates of reports (66.7% vs. 33.6%, p<0.001 and 60.0% vs. 39.8%, p = 0.05, respectively) (Table 4).

Table 4. Comparison of the demographic and the professional variables and reporting/non-reporting behavior in the intervention group vs. the control group.

Physicians reported more than nurses; specialists, reported more than non-specialists; those fulfilling additional positions and those working in other places beside the hospital demonstrated high rates of reports.

Variable (Total No.)   Did not report Reported Total p-value
% (n) % (n)
Gender (207) Female 60.8 (96) 39.2 (67) 158 0.23
Male 51.0 (25) 49.0 (24) 49
Profession (207) Nurse 63.5 (99) 36.5 (57) 156 0.009
Physician 43.1 (22) 56.9 (29) 51
Country of origin (202) Israel 47.9 (34) 52.1 (37) 71 0.12
Eastern Europe 63.4 (71) 36.6 (41) 112
Other 64.3 (11) 42.1 (8) 19
Country of Professional Training (200) Israel 55.9 (71) 44.1 (56) 127 0.83
Eastern Europe 60.8 (31) 39.2 (20) 51
Other 59.1 (13) 40.9 (9) 22
Specialty (207) Yes 39.4 (26) 60.6 (40) 66 <0.001
No 67.4 (95) 32.6 (46) 141
Fulfilling other positions (194) Yes 33.3 (15) 66.7 (30) 45 <0.001
No 66.4 (99) 33.6 (50) 149
Working in other places (201) Yes 40.0 (10) 60.0 (15) 25 0.05
No 60.2 (106) 39.8 (70) 176
Age (Mean ±SD) 57.13±7.21 42.87±9.34 103/82 0.92
Age (185) ≤35 48.8 (21) 51.2 (22) 43 0.08
36–45 66.7 (44) 33.3 (22) 66
≥46 50.0 (38) 50.0 (38) 76
Years of seniority in the profession (Mean ±SD) 16.52±10.06 17.14±10.21 103/83 0.68
Years of seniority in the profession (186) <7 54.3 (19) 45.7 (16) 35 0.68
7–20 58.6 (51) 41.4 (36) 87
>21 51.6 (33) 48.4 (31) 64
Years of seniority in the profession in Israel (Mean ±SD) 19.64±9.05 21.4±10.83 81/47 0.32
Years of seniority in the profession in Israel (128) ≤20 64.8 (46) 35.2 (25) 71 0.69
>21 61.4 (35) 38.6 (22) 57
Years of Seniority in the internal division (Mean ±SD) 11.26±9.49 13.36±10.11 114/84 0.32
Years of Seniority in the internal division (198) ≤3 65.4 (34) 34.6 (18) 52  
3.1–13 60.3 (41) 39.7 (27) 68 0.19
≥13.1 50.0 (39) 50.0 (39) 78  
No. of patients treated per day (Mean ±SD) 15.92±9.22 16.34±11.19 104/79 0.78
No. of patients treated per day (183) ≤12 54.5 (54) 45.5 (45) 99 0.50
≥13 59.5 (50) 40.5 (34) 84
No. of drugs distributed/examined per day (Mean ±SD) 53.72±43.90 65.04±54.71 91/69 0.15
No. of drugs distributed/examined per day (160) ≤50 59.1 (55) 40.9 (38) 93 0.50
>51 53.7 (36) 46.3 (31) 67
% of Patients taking more than one drug per day (Mean ±SD) 89.44±19.70 87.19±24.35 102/77 0.56
Patients taking more than one drug per day (179) <90 58.3 (21) 41.3 (15) 36 0.86
>90 56.6 (81) 43.4 (62) 143

Further analysis of the previous data to physicians and nurses revealed that the demographic and professional variables among the physicians did not have any effect on the percentage of ADRs reports. However, among the nurses, specialty (56.1% vs 29.6%, p = 0.02) and fulfilling additional positions (63.3% vs. 28.8%, p<0.0001) indeed increased reporting rates, while no difference was observed with regard to working in other places besides the hospital (37.5% vs. 36.6%, p = 0.96).

The intervention plan had a strong, independent and statistically significant effect on "behavior related to reporting" (p = 0.008). In addition, profession and number of patients treated per day by the caregiver also had a significant effect on the "behavior related to reporting" (p = 0.01 and p = 0.02, respectively) (Table 5).

Table 5. The effect of the intervention program and demographic characteristics on "behavior related to reporting".

The intervention plan had a strong, independent and statistically significant effect on "behavior related to reporting". Profession and number of patients treated per day by the caregiver also had a significant effect on the "behavior related to reporting".

Variable β p value
Intervention (Yes / No) 0.13 0.008
Professional (physician / nurse) 0.14 0.01
Number of patients treated per day 0.12 0.02
Gender (Male / Female) 0.009 0.87

In order to examine the independent effect of the intervention plan on reporting (yes / no), a logistic model was constructed. We found that the intervention plan had a strong, independent, statistically significant effect on the staffs’ actual ADRs reporting. After standardization for specialist, expertise, holding managerial positions and those who work in other places, subjects in the intervention group reported 74 times higher than their counterparts in the control group (O.R = 74.1, 95% CI = 21.11–260.1, p<0.001) (Table 6).

Table 6. The independent effect (adjusted Odd Ratios and 95% CI) of the intervention program on ADR reports.

A logistic model revealed that the intervention plan had a strong, independent, statistically significant effect on the staffs’ actual ADRs reporting.

Variable O.R CI 95% p value
Intervention (Yes / No) 74.1 21.11–260.1 0.001>
Fulfilling other roles (Yes / No) 3.16 1.45–6.92 0.004
Specialty (Yes / No) 2.32 1.14–4.70 0.02
Professional (physician / nurse) 2.70 1.31–5.57 0.007

Education and lectures were preferable, while payment for reporting was the least desirable method for encouraging medical staff to report side effects. The most convenient reporting tools were found to be the telephone and an internet site.

Discussion

The purpose of this study was to establish and evaluate an intervention plan for increasing ADR reporting rate among physicians and nurses. This study demonstrates that the rate of ADR reporting increased significantly during the intervention period, and declined gradually thereafter. However, almost no change in the numbers of reports was observed in the control group during the entire duration of the study. The trend presented in the data that the rate of reports decreased over time after the intervention program was discontinued implies that continued intervention may be required to maintain the high rate of reports. Interestingly, the reporting rate in the control group was higher than in the intervention group at base line, though it was not statistically significant. The intervention plan increased the reporting rate and the differences between the control and the intervention rose and were statistically significant throughout the intervention period. However, as distance from the intervention period increased, the amount of reports decreased eventually reverting to the trend that was observed at the baseline. The impact of the intervention waned with time. An example to this trend was demonstrated also in an educational intervention program to improve physicians' reporting of ADRs. In this study the reporting rate in the intervention group increased during the intervention, while it gradually decreased through 13 months of follow-up [9].

The compliance rate of the participants in the present study for filling in both questionnaires was rather high (81.5%). A lower rate of compliance was reported in similar studies, in which 22.8% (Biagi 2013) or 47% (Passier 2009) of the physicians answered questionnaires regarding ADRs reporting [33, 34]. Another study from Venezuela (Garciani 2011) reported higher compliance rate of 65.4% among physicians and pharmacists and 60% among nurses [35]. Interestingly, a much lower compliance rate was found in studies in which the questionnaires were sent to the participants by e-mail or regular mail [36]. The relatively high rate of compliance in the current study may be associated with the constant presence of the investigator at the medical centers and the personal contact with the study participants.

According to the results of the current study, physicians reported ADRs more than nurses. In addition, the change in the “Behavior related to reporting” score in the intervention group was higher among physicians compared to the nurses. The changes in the “Behavior related to reporting” is also demonstrated by the increase in the rate of ADRs in the intervention group compared the control group. This finding is consistent with other studies [1, 3739]. In a study that took place in Korea, spontaneous reports of ADRs by e-mail were 13% among nurses vs. 53% reports by physicians. Some studies demonstrated that nurses mostly tend to report an ADR to a physician. Hanafi et al. have shown that 89% of the nurses who participated in the study said that they reported the ADR to the physician [40]. The Hajebi’s study found that 56% of the nurses that come across a drug-related side effect reported to the department physician, 26% to the head nurse and 13% to a pharmacist [41]. The availability and accessibility of physicians to nurses who work in hospitals probably encouraged the reporting to physicians. This tendency could explain the difference in ADR reporting rates between nurses and physicians. Contrarily, in a research that was conducted in an Israeli public hospital where the medical staff was encouraged to report ADRs to the clinical pharmacology unit, the nurses were found to have reported more ADRs than the physicians [42]. In the present study, despite the fact that the nurses reported less than the physicians, their rate of reports peaked more quickly and decreased more slowly than that of the physicians.

We also found that professionals who fulfill additional positions in the department or in other health institutions (such as: community health services or treating senior citizens at nursing homes) demonstrated higher rates of ADR reporting. In addition, the results of the present study showed that specialists reported more ADRs than non-specialists and that the rate of ADR reporting was associated with the number of patients treated per day by the caregiver. A contrary observation was reported by a study in Ireland, which investigated the rate of ADR reporting among 118 hospital-based physicians. This study found a higher rate of ADR reporting among general physicians than among surgeons [43].

The present study demonstrated that the preferred method for increasing the rate of ADR reporting was lectures and education A study that compared telephone-interview intervention with a workshop intervention showed that the latter increased ADR reporting rate by a fourfold on average compared to the control group over 20 months post-intervention. However, no significant difference vs. the control group in ADR reporting was found in the telephone-interview intervention [44]. Other studies have shown that improved communication with fellow physicians and involvement of pharmacists might be the best ways to improve ADR reporting [21, 34]. Regular newsletter on current awareness in drug safety, information on new ADRs, and international drug safety information were also identified as tools or methods that may motivate ADR reporting in a study conducted by Santosh et al. among 450 healthcare professionals working at Regional Pharmacovigilance Centers in Nepal [21].

Our research shows that the preferred means of reporting were telephone or website. Other studies also report these methods as preferential. A study of 500 nurses from a teaching hospital in Teheran showed that among the 10% who reported an ADR, the majority of the nurses preferred using the telephone [45]. Among physicians in India and the Netherlands, most of the ADRs were reported using the computerized system [34, 46].

Payment for reporting was found to be the least favored method to encourage ADR reporting in the current study. A survey of 91 practice nurses, health visitors, school nurses and general physicians conducted by Pulford et al., has shown that payment for ADR reporting was indeed the least acceptable out of 14 other options of gratuity [47].

Conclusions

The results of this study indicate that training and educating medical practitioners and providing them with relevant knowledge regarding ADR reporting is essential. Due to the observation that the reporting rate decreased with time upon the finalization of the intervention period, it seems that maintaining a program to encourage reporting, is necessary. Regular implementation of such a program in the healthcare system will increase the awareness of the medical staff and improve reporting rates. Maintaining the intervention program could be carried out by nomination of a pharmacovigilance specialist trustee to administer a routine intervention program. This expert could be a physician, a nurse or a pharmacist. Visits, personal discussions, posters, lectures about the importance of ADRs reporting and how to carry it out and sending text messages to the medical staff on their mobile phones with reminders and relevant information should be used in order to continuously raise their awareness reporting about ADRs. In addition, the personal contact of the medical staff with the trustee will encourage their commitment to report about ADRs. This will probably improve monitoring of medication use, decrease morbidity/mortality rates and hospitalization duration.

Limitations

The study was carried out in internal divisions of only 3 public medical centers, and therefore may not have an external validation in other hospitals.

The study included only physicians and nurses, therefore the results cannot be applied to additional medical professionals.

There were some differences in the basic characteristics between the clusters (hospitals) which may have affected the quality of the intervention to a certain extent. However, after adjusting for the demographic variables, we can assume that the results of the study are indeed due to the effect of the intervention.

We cannot assume from this study that the effect of the intervention would improve safety of medicine use in the long term and reduce health costs.

Supporting information

S1 Data

(SAV)

S1 File

(DOC)

S2 File

(DOC)

Data Availability

All data files are available from the ICPSR database - project ID-openicpsr-119582.

Funding Statement

The author(s) received no specific funding for this work.

References

  • 1.Segev S, Yahalom Z. The role of the pharmaceutical industry in disseminating pharmacovigilance practice in developing countries food and drug. Law Journal. 2008; 63: 701–11. [PubMed] [Google Scholar]
  • 2.WHO, Geneva. Safety of Medicines: A guide to detecting and reporting adverse drug reactions. 2002; URL: http://www.who.int/medicuned/en/d/jh2992e/
  • 3.Van der Linden CM, Jansen PA, van Marum RJ, Grouls RJ, Korsten EH, Egberts AC. Recurrence of adverse drug reactions following inappropriate re- prescription: Better documentation, availability of information and monitoring are needed. Drug Safety. 2010; 33: 535–8. 10.2165/11532350-000000000-00000 [DOI] [PubMed] [Google Scholar]
  • 4.Atkinson AJ, Abernethy DR, Daniels CE, FASHP RP, Robert L. Dedrick RL, et al. Principles of Clinical pharmacology. second edition (2007) Chapter 25 Clinical analysis of adverse drug reaction 389–402.
  • 5.Clavenna A, Bonati M. Adverse drug reaction in childhood: a review of prospective studies and safety alerts. Arch Dis Child. 2009; 94: 724–8. 10.1136/adc.2008.154377 [DOI] [PubMed] [Google Scholar]
  • 6.Edwards IR, Aronson JK. Adverse drug reactions: Definitions, diagnosis, and management. The Lancet. 2000; 356: 1255–9. [DOI] [PubMed] [Google Scholar]
  • 7.Avery AJ, Anderson C, Bond CM, et al. Evaluation of patient Reporting of adverse drug reactions to the UK‘Yellow Card Scheme’: literature review, descriptive and qualitative analyses, and questionnaire surveys. Health Technology Assessment. 2011; 15: 1–234. [DOI] [PubMed] [Google Scholar]
  • 8.Lindquist M. The need for definitions in pharmacovigilance. Drug Safety. 2007; 30: 825–830. 10.2165/00002018-200730100-00001 [DOI] [PubMed] [Google Scholar]
  • 9.Figueiras A, Herdeiro MT, Polonia J, Gestal-Otero JJ. An educational intervention to improve physician reporting of adverse drug reactions. A cluster-randomized controlled trial. JAMA. 2006; 296: 1086–93. 10.1001/jama.296.9.1086 [DOI] [PubMed] [Google Scholar]
  • 10.Rabinovitz H, Berkovitch M, Golik A, Shani S. Adverse drug reactions definitions and terminology. Harefuah. 2001; 140: 1181–6. [PubMed] [Google Scholar]
  • 11.Ferandopulle R.B.M, Weerasuriya K. What can consumer adverse drug reaction reporting add to existing health professional-based systems? Focus on the developing world. Drug Safety. 2003; 26: 219–25. 10.2165/00002018-200326040-00002 [DOI] [PubMed] [Google Scholar]
  • 12.Roughead EE, Gilbert AL, Primrose JG, Sansom LN. Drug-related hospital admissions: A review of Australian studies published 1988–1996. Med J Aust. 1998; 168: 405–408. [DOI] [PubMed] [Google Scholar]
  • 13.Cobaugh DJ, Krenzekok EP. Adverse drug reactions and therapeutic errors in older adults: A hazard factor anakysis of poison center data. Am J Health Syst Pharm. 2006; 63: 2228–34. 10.2146/ajhp050280 [DOI] [PubMed] [Google Scholar]
  • 14.Lugardon S, Desboenf K, Fernet P, Montastruc JL, Lapeyre-Mestre M. Using a capture- recapture method to assess the frequency of adverse drug reactions in a French university hospital. Br J Clin Pharmacol. 2006; 62: 225–31. 10.1111/j.1365-2125.2006.02633.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Lazarou J, Pomeranz BH, Corey PN. Incidence of adverse drug reactions in hospitalized patients: A meta-analysis of prospective studies. JAMA. 1998; 279: 1200–5. 10.1001/jama.279.15.1200 [DOI] [PubMed] [Google Scholar]
  • 16.https://www.health.gov.il/hozer/DR_6E.pdf
  • 17.Senst BL, Achusim LE, Genest RP, Cosentino LA, Ford CC, Little JA, et al. Practical approach to determining costs and frequency of adverse drug events in a health care network. Am J Health-Syst Pharm. 2001; 58: 1126–32. 10.1093/ajhp/58.12.1126 [DOI] [PubMed] [Google Scholar]
  • 18.Schwartzberg E, Berkovitch M, Dil Nahlieli D, Nathan J, Gorelik E. Pharmacovigilance in Israel—tools, processes, and actions. Isr J Health Policy Res. 2017;6: 29 10.1186/s13584-017-0154-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Pankaj G. Udupa A. Adverse drug reaction reporting and pharmacovigilance: Knowledge, attitudes and perceptions amongst resident doctors. J Pharm Sci & Res. 2011; 3: 1064–9. [Google Scholar]
  • 20.Lopez-Gonzalez E, Herdeiro MT, Figueiras A. Determinants of under- reporting of adverse drug reactions: A systematic review. Drug Safety. 2009; 32: 19–31. 10.2165/00002018-200932010-00002 [DOI] [PubMed] [Google Scholar]
  • 21.Santosh KC, Tragulpiankit P, Gorsanan S, Edwards IR. Attitudes among healthcare professionals to the reporting of adverse drug reactions in Nepal. BMC Pharmacol Toxicol. 2013; 14–16. 10.1186/2050-6511-14-14 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Inman WHW. Assessment of drug safety problems In: Gent M, Shigmatsu I, editors. Epidemiological issues in reported drug-induced illnesses. Honolulu (ON): McMaster University Library Press, 1976:17–24. [Google Scholar]
  • 23.Inman WHW, Weber JCT. The United Kingdom In Inman WHW, editor. Monitoring for drug safety. 2nd ed Lancaster: MTP Press Ltd, 1986: 13–47. [Google Scholar]
  • 24.Inman WHW. Attitudes to adverse drug-reaction reporting. Br J Clin Pharmacol. 1996; 41: 433–5. [PubMed] [Google Scholar]
  • 25.Dos Santos Pernas SI, Herdeiro MT, da Cruz e Silva OAB, Figueiras A. Attitudes of Portuguese health professionals toward adverse drug reaction reporting. Int J Clin Pharm. 2012; 34: 693–8. 10.1007/s11096-012-9675-6 [DOI] [PubMed] [Google Scholar]
  • 26.Islam AF. Improving drug utilization: What about the KAP gap? Tropical Doctor. 1993; 23: 89–90. 10.1177/004947559302300219 [DOI] [PubMed] [Google Scholar]
  • 27.Hong YP, Kwon DW, Kim SJ, Chang SC, Kang MK, Lee EP, et al. Survey of knowledge, attitudes and practices for tuberculosis among general practitioners. Tuber Lung Dis. 1995; 76(5): 431–435. 10.1016/0962-8479(95)90010-1 [DOI] [PubMed] [Google Scholar]
  • 28.Maslow A.H. A dynamic theory of human motivation. Psychological Review. 1943; 50: 370–96. [Google Scholar]
  • 29.Slotnick HB. How doctors learn: The role of clinical problems across the medical school-to-practice continuum. Acad Med. 1996; 71: 28–34. 10.1097/00001888-199601000-00014 [DOI] [PubMed] [Google Scholar]
  • 30.Herdeiro MT, Polonia J, Gestal-Otero JJ, Figueiras A. Factors that influence spontaneous reporting of adverse drug reactions: a model centralized in the medical professional. J Eval Clin Pract. 2004; 10: 483–9. 10.1111/j.1365-2753.2003.00456.x [DOI] [PubMed] [Google Scholar]
  • 31.Irujo M, Beitia G, Bes-Rastrollo M, Figueiras A, Herbabdez-Diaz S, Lasheras B. Factors that influence under-reporting of suspected adverse drug reactions among community pharmacists in a Spanish region. Drug Safety. 2007; 30: 1073–82. 10.2165/00002018-200730110-00006 [DOI] [PubMed] [Google Scholar]
  • 32.Bates DW. Drugs and adverse drug reaction. How worried should we be? JAMA. 1998; 279: 1216–67. 10.1001/jama.279.15.1216 [DOI] [PubMed] [Google Scholar]
  • 33.Biagi C, Montanaro N, Buccellato E, Roberto G, Vaccheri A, Motola D. Underreporting in pharmacovigilance: An intervention for Italian GPs (Emilia–Romagna region). Eur J Clin Pharmacol. 2013; 69: 237–44. 10.1007/s00228-012-1321-7 [DOI] [PubMed] [Google Scholar]
  • 34.Passier A, Napel M, Grootheest K, Puijenbroek E. Reporting of adverse drug reactions by general practitioners: A questionnaire- based study in Netherlands. Drug Safety. 2009; 32: 851–8. 10.2165/11314490-000000000-00000 [DOI] [PubMed] [Google Scholar]
  • 35.Pérez García M, Figueras A. T he lack of knowledge about the voluntary reporting system of adverse drug reactions as a major cause of underreporting: direct survey among health professionals. Pharmacoepidemiol Drug Safety 2011; 20: 1295–302. [DOI] [PubMed] [Google Scholar]
  • 36.Oosterhuis I, van Hunsel FP, van Puijenbroek EP. Expectations for feedback in adverse drug reporting by healthcare professionals in the Netherlands. Drug Safety 2012; 35: 221–32. 10.2165/11594910-000000000-00000 [DOI] [PubMed] [Google Scholar]
  • 37.Guidelines for reporting side effects to medication by a registry (June 1998) Ministry of Health.
  • 38.Vora MB, Paliwal NP, Doshi VG, Barvaliya MJ, Tripathi CB. Knowledge of adverse drug reactions and pharmacovigilance activity among the undergraduate medical students of Gujarat. Int J Pharm Sci Res. 2012; 3: 1511–15. [Google Scholar]
  • 39.Lee JH, Park KH, Moon HJ, Lee YW, Park JW, Hong CS. Spontaneous reporting of adverse drug reactions through electronic submission from regional society healthcare professionals in Korea. Yonsei Med. J 2012; 53: 1022–27. 10.3349/ymj.2012.53.5.1022 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Hanafi S, Torkamandi H, Hayatshahi A, Gholami K, Shahmirzadi NA, Javadi MR. An educational intervention to improve nurses’ knowledge, attitude, and practice toward reporting of adverse drug reactions. Iran J Nurs. Midwifery Res 2014; 19: 101–6. [PMC free article] [PubMed] [Google Scholar]
  • 41.Hajebi G, Mortazavi SA, Salamzadeh J, Zian A. A survey of knowledge, attitude and practice of nurses towards pharamacovigilance in Taleqani hospital. Iran J Pharm Res. 2010; 9: 199–206. [PMC free article] [PubMed] [Google Scholar]
  • 42.Goldstein LH, Berlin M, Saliba W, Elias M, Berkovitc M. Founding an Adverse Drug Reaction (ADR) Network: A method for improving doctors founding an adverse drug Spontaneous ADR reporting in a general hospital. J Clin Pharmacol. 2013; 53:1220–5. 10.1002/jcph.149 [DOI] [PubMed] [Google Scholar]
  • 43.McGettigan P, Golden J, Conroy RM, Arthur N, Feely J. Reporting of adverse drug reactions by hospital doctors and the response to intervention. Br J Clin Pharmacol. 1997; 44:98–100. 10.1046/j.1365-2125.1997.00616.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Herdeiro MT, Ribeiro-Vaz I, Ferreira M, Polónia J, Falcão A, Figueiras A. Workshop- and telephone-based interventions to improve adverse drug reaction reporting a cluster- randomized trial in Portugal. Drug Safety 2012; 35: 655–65. 10.1007/BF03261962 [DOI] [PubMed] [Google Scholar]
  • 45.Hanafi S, Torkamandi H, Hayatshahi A, Gholami K, Shahmirzadi NA, Javadi MR. Knowledge, attitudes and practice of nurse regarding adverse drug reaction reporting. Iran J Nurs Midwifery Res. 2012; 17:21–5. [PMC free article] [PubMed] [Google Scholar]
  • 46.Palanisamy S, Kottur SG, Kumaran A, Rajasekaran A. Knowledge assessment in adverse drug reactions and reporting. Arch Pharm Prac. 2013; 4: 104–19. [Google Scholar]
  • 47.Pulford A, Malcolm W. Knowledge and attitudes to reporting adverse drug reactions. British J Nurs. 2010; 19: 899–904. [DOI] [PubMed] [Google Scholar]

Decision Letter 0

Karen Cohen

28 Sep 2019

PONE-D-19-24600

Increasing Adverse Drug Reaction Reporting –

How can we do better?

PLOS ONE

Dear Dr. Shchory Potlog,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

We would appreciate receiving your revised manuscript by Nov 12 2019 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Karen Cohen

Academic Editor

PLOS ONE

Journal Requirements:

1. When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for including your ethics statement: The study was approved by the institutional review board of each medical center respectively.

Please amend your current ethics statement to include the full name of the ethics committee/institutional review board(s) that approved your specific study.

Once you have amended this/these statement(s) in the Methods section of the manuscript, please add the same text to the “Ethics Statement” field of the submission form (via “Edit Submission”).

For additional information about PLOS ONE ethical requirements for human subjects research, please refer to http://journals.plos.org/plosone/s/submission-guidelines#loc-human-subjects-research.

3. Please include additional information regarding the survey or questionnaire used in the study and ensure that you have provided sufficient details that others could replicate the analyses. For instance, if you developed a questionnaire as part of this study and it is not under a copyright more restrictive than CC-BY, please include a copy, in both the original language and English, as Supporting Information.

4. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified (1) whether consent was informed and (2) what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If consent was implied due to the completion and returning of the questionnaire, please state this in the Methods section.

4. Please include your tables as part of your main manuscript and remove the individual files. Please note that supplementary tables (should remain/ be uploaded) as separate "supporting information" files

5. Please amend the manuscript submission data (via Edit Submission) to include authors Lee H. Goldstein, Lidia Arcavi, Renata Shihmanter, Matitiahu Berkovitch and Amalia Levy.

6. Please amend either the abstract on the online submission form (via Edit Submission) or the abstract in the manuscript so that they are identical.

7.  We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

Reviewer #2: No

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Dear Colleagues

Thank you for the opportunity to review and comment on your reserach. I am aware of the amount of work done and my comments aim at helping you improve your manuscript.

The first thing that suprises me is that your research dates from 2013 and 6 years have elapsed since then, some statements you make (see below) might be out of date.

The references used in the introduction are old/very old:

81-84: "...new legislation was recently launched...2014" or "the reporting rate in Israel...is still very low", this with a reference from 1999. While underreporting might still be a problem, I would be very surprised that nothing has changed in the past 20 yrs.Please update with actual recent figures.

Methods:

133-142: the answers to the question with "because" do not make sense in the context of the questions, while valid as stand alone. Please clarify.

To the reader it is not clear why the focus was on Hebrew (obvious) and Russian speakers (much less obvious). Please explain.

143-148: you mention two statements analyzed, but B is actually a question. Please clarify. Also, behaviour change cannot be measured by subjective statements. In this case behaviour change can be measured by the number of reports submitted before and after the intervention.

The statistical analysis of the results of the questionnaires is very detailed but unfortunately does not lead to conlusions on how to improve future interventions.

Results:

You write that the reporting in the intervention group was higher than in the control and that the effect diminished over time. What you do not mention is that according to Fig 1, the reporting in the intervention group went nearly back to baseline and was even lower than in the control group. While the single values might not reach statistical significance, the trend is still impressive and must be mentioned and discussed - not doing so misleads the reader. Please rectify (179-183).

Discussion:

see comment above

Conclusion:

see above and please explain more in detail how interventions can be improved based on your data and their analysis.

Language: the manuscript would benefit from a revision by a native English speaker.

Reviewer #2: Methods section: description (grade) is not very clear. Scoring and description seem to contradict. Should be rephrased.

Inconsistent/non standardized/interchangeable use of the terms side effects, adverse effects, adverse reactions.

Pharmacovigilance is described as a pharmacy practice. I am not aware of any such categorization.

How were duplicate reports identified/handled (since for Group A, post intervention reports came both from the facility and the MoH website.

Interventions such s these are known to have a temporary positive impact on reporting. What is more important is to discuss how to sustain this. Also, will such an intervention lead to quality improvements?

Authors have explained that there is no data repository. And that primary data could be provided as email attachments.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2020 Aug 13;15(8):e0235591. doi: 10.1371/journal.pone.0235591.r003

Author response to Decision Letter 0


16 Dec 2019

Reviewer #1

• "Thank you for the opportunity to review and comment on your research. I am aware of the amount of work done and my comments aim at helping you improve your manuscript. The first thing that surprises me is that your research dates from 2013 and 6 years have elapsed since then, some statements you make (see below) might be out of date.

The references used in the introduction are old/very old: 81-84: "...new legislation was recently launched...2014" or "the reporting rate in Israel...is still very low", this with a reference from 1999. While underreporting might still be a problem, I would be very surprised that nothing has changed in the past 20 yrs. Please update with actual recent figures."

Answer: Newer references were used in the revised manuscript, as recommended. This information was added to the "Introduction" (Page 5, Line 91) of the revised version and it now appears as follow:

In August 2012, the Israeli ministry of health (MOH) published guidelines for reporting adverse events and new safety information. This document specifies the type of information that the Marketing Authorization Holder (MAH) requires reporting and enables the MAH to appoint a pharmacist to serve as a qualified person responsible for matters related to the reports included in this standard operating procedure, according to the standard worldwide practice. Those regulations were update at February 2013 in order to clarify the work processes related to reporting ADRs and new safety information, to update the definitions of the SOP and to update the types of information requiring reporting by the MAH. The aim of additional update from May, 2013 was to adjust the SOP to the Pharmacists Regulations (16). A new regulation was launched at October, 2014, in which a reporting system in medical institutions for both common and severe ADRs was established. In July 2019, a new portal for reporting adverse events to the Risk Management Department was established.

Data regarding the low reporting rate in Israel was updated - Page 5, line 107, ref.18:

Schwartzberg el al., identified 16,409 of Individual Case Safety Reports (ICSRs) submitted to the MOH’s ADRs central database between September 2014 and August 2016. However, of these reports, only 5.5% were submitted by health care professionals from medical institutions, while 94.3% were submitted by pharmaceutical companies (MAH and importers) and only 0.2% of the reports have been submitted by patients and the general public [18].

• "Methods:

133-142: the answers to the question with "because" do not make sense in the context of the questions, while valid as stand alone. Please clarify."

Answer: This question refers to the reasons for not reporting after identifying adverse reaction effects of drug therapy. It contains 5 possible causes for not reporting. The word “because” has been replaced by the word “since”- Page 9, line 182.

• "To the reader it is not clear why the focus was on Hebrew (obvious) and Russian speakers (much less obvious). Please explain."

Answer: In Israel, a high percentage of physicians and nurses are from the Commonwealth States and Russian is their first language. In order to make it easier for them to complete the questionnaire and to increase the response rate it was translated into Russian.

This explanation was added in the revised version, as suggested (Page 7, line 123).

• "143-148: you mention two statements analyzed, but B is actually a question. Please clarify."

Answer: The word “statements” has been changed to “items” – Page 11, line 191.

• "Also, behavior change cannot be measured by subjective statements. In this case behavior change can be measured by the number of reports submitted before and after the intervention."

Answer: Indeed, as stated by the reviewer, behavior change cannot be measured by subjective statements. However, it could be measured by the number of reports submitted before and after the intervention. This is presented in the data of Figure 1, where a rapid and substantial increase in the number of ADR reports was noted in the intervention group during the intervention period while almost no change in the number of ADRs reports was observed in the control group. This information is presented in the "Discussion" of the revised manuscript, as recommended, and it now appears: (Page 19, line 346)

The trend presented in the data that the rate of reports decreased over time after the intervention program was discontinued implies that continued intervention may be required to maintain the high rate of reports. Interestingly, the reporting rate in the control group was higher than in the intervention group at base line, though it was not statistically significant. The intervention plan increased the reporting rate and the differences between the control and the intervention rose and were statistically significant throughout the intervention period. However, as distance from the intervention period increased, the amount of reports decreased eventually reverting to the trend that was observed at the baseline.

• "The statistical analysis of the results of the questionnaires is very detailed but unfortunately does not lead to conclusions on how to improve future interventions."

Answer: According to the statistical analyzed data, the increase in the rate of ADRs in the intervention group was noted during the 5-month intervention period. Nevertheless, it was decreased toward the end of the study. Therefore, it was suggested that the intervention program, as well as training and educating the medical practitioners should be maintained. This was addressed to the conclusion (page 22, line 424)

Maintaining the intervention program could be carried out by nomination of a pharmacovigilance specialist trustee to administer a routine intervention program. This expert could be a physician, a nurse or a pharmacist. Visits, personal discussions, posters, lectures about the importance of ADRs reporting and how to carry it out and sending text messages to the medical staff on their mobile phones with reminders and relevant information should be used in order to continuously raise their awareness reporting about ADRs. In addition, the personal contact of the medical staff with the trustee will encourage their commitment to report about ADRs.

and Abstract (page 2, line 51).

Thus, implementation and maintenance of a continuous intervention program, by a pharmacovigilance specialist staff member, will improve ADRs reporting rates.

• "Results: You write that the reporting in the intervention group was higher than in the control and that the effect diminished over time. What you do not mention is that according to Fig 1, the reporting in the intervention group went nearly back to baseline and was even lower than in the control group."

Answer: Indeed as the reviewer commented, although the reporting in the intervention group was lower than that of the control group, it was not statically significant. The following sentence was added to the revised version, as suggested and it appears:

After the intervention period the reporting rate in the intervention group reverted to almost baseline and was lower than the control group, similarly to the trend that was observed in the baseline.

(Page 12, line 244)

• "While the single values might not reach statistical significance, the trend is still impressive and must be mentioned and discussed - not doing so misleads the reader. Please rectify (179-183)."

Answer: This information was added to the "Discussion" of the revised manuscript page 19, line 346.

The trend presented in the data that the rate of reports decreased over time after the intervention program was discontinued implies that continued intervention may be required to maintain the high rate of reports. Interestingly, the reporting rate in the control group was higher than in the intervention group at base line, though it was not statistically significant. The intervention plan increased the reporting rate and the differences between the control and the intervention rose and were statistically significant throughout the intervention period. However, as distance from the intervention period increased, the amount of reports decreased eventually reverting to the trend that was observed at the baseline.

• "Discussion:

see comment above"

Answer: The discussion section was amended according to the comment – page 20, line 346

• "Conclusion:

see above and please explain more in detail how interventions can be improved based on your data and their analysis."

Answer: The conclusion section was amended according to the comment (page 22, line 424):

Maintaining the intervention program could be carried out by nomination of a pharmacovigilance specialist trustee to administer a routine intervention program. This expert could be a physician, a nurse or a pharmacist. Visits, personal discussions, posters, lectures about the importance of ADRs reporting and how to carry it out and sending text messages to the medical staff on their mobile phones with reminders and relevant information should be used in order to continuously raise their awareness reporting about ADRs. In addition, the personal contact of the medical staff with the trustee will encourage their commitment to report about ADRs.

• "Language: the manuscript would benefit from a revision by a native English speaker."

Answer: The manuscript was reviewed by a professional native English speaker.

Reviewer #2:

• "Methods section: description (grade) is not very clear. Scoring and description seem to contradict. Should be rephrased."

Answer: Thank you for this very correct note. The word “grade” has been replaced by the word “score” all over the text and in the tables.

• "Inconsistent/non standardized/interchangeable use of the terms side effects, adverse effects, adverse reactions."

Answer: The terms side effects and adverse effects were changes to "adverse reactions" all over the text.

• "Pharmacovigilance is described as a pharmacy practice. I am not aware of any such categorization."

Answer: Thank you for the remark. The definition of Pharmacovigilance by the WHO was introduced into the text of the revised version, as suggested– (Page 4, line 74):

Pharmacovigilance is defined by the WHO as the science and activities relating to the detection, assessment, understanding and prevention of adverse reactions or any other drug-related problem

• "How were duplicate reports identified/handled (since for Group A, post intervention reports came both from the facility and the MOH website."

Answer: The reports were checked and there were no duplicate reports by staff members on both reporting channels in the intervention group. This information was added to the revised manuscript, as recommended (Page 11 Line 228)

The reports were checked and there were no duplicate reports by the staff members on both reporting channels in the intervention group.

• "Interventions such as these are known to have a temporary positive impact on reporting. What is more important is to discuss how to sustain this. Also, will such an intervention lead to quality improvements?"

Answer: This issue was already discussed –page 22, line 424

• "Authors have explained that there is no data repository. And that primary data could be provided as email attachments."

Answer: All needed data is presented.

Attachment

Submitted filename: Answeres plos one 26 10 19 .docx

Decision Letter 1

Karen Cohen

2 Mar 2020

PONE-D-19-24600R1

Increasing Adverse Drug Reaction Reporting - How can we do better?

PLOS ONE

Dear Dr. Shchory Potlog,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================

 In particular two of the reviewers of the revised manuscript  raised a number of questions regarding  the statistical analysis, as well as some queries regarding study design. Please can you address the issues they have raised?

==============================

We would appreciate receiving your revised manuscript by Apr 16 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Karen Cohen

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #3: (No Response)

Reviewer #4: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: (No Response)

Reviewer #3: Partly

Reviewer #4: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: (No Response)

Reviewer #3: No

Reviewer #4: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: (No Response)

Reviewer #3: Yes

Reviewer #4: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: (No Response)

Reviewer #3: Yes

Reviewer #4: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

Reviewer #3: From the data presented and the statistical input that was implemented the investigators have concluded that. an increase in the number of ADR reports was noted in the intervention group during the intervention period, the changes in the "knowledge related to behavior” and in the "behavior related to reporting" score was significantly higher in the intervention group. Also specialist physicians and nurses, fulfilling additional positions and those working in other places demonstrated a high rate of report.

The data analysis approach is fairly routine and appears to be sufficient. Also the results are striking. However, the study lacks a rigorous study design justification. Three institutions were randomized, but the unit of analysis is the individual. There is no mention or investigation of the intraclass correlation in this type of setting and how that could impact on the varied analyses and the impact on the type I error. There is no central hypothesis. Other statistical issues include:

1. The multiple comparison testing issue is totally ignored, especially in Tables 3 and 4.

2. Tables 5 and 6 would suffice for a descriptive analysis. However, where is the institution effect in these tables?

3. There is insufficient tabular description of the number of cases , specialties, etc. within each of the institutions.

4. One of the key points mentioned by the investigators is “Education and training of medical staff in good pharmacovigilance practice is necessary for adequate ADR reporting and for the reduction of ADR rates and lower the costs in the healthcare system”. There is no analysis or meaningful discussion of costs considerations.

5. Reporting tools were conveniently telephone or online, but there is no validation of responses.

The investigators would do well to have this manuscript thoroughly edited by a statistician.

Reviewer #4: I really appreciate reading your work. I feel that the results are rather interesting. The employed methods are sound and the data are rich of info. I have just minor comments.

1. T-tests, as well as any other tests, must satisfy some assumptions. These assumptions are overlooked in the analysis. Please, provide evidence that the assumptions behind the t-test are fulfilled, to avoid misleading conclusions. Non-parametric tests could be used instead.

2. The use of regressions is sound. Nevertheless, to be confident on the obtained results, the authors must check for the Gauss-Markov assumptions. Inference on regression parameters is valid if and only if those assumptions are fulfilled. Moreover, I am wondering if multicollinearity may be an issue. Please, extend the regression model to include interactions. In general, the analysis of residuals may be included. It would be very useful to evaluate the reliability of the results.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #3: No

Reviewer #4: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2020 Aug 13;15(8):e0235591. doi: 10.1371/journal.pone.0235591.r005

Author response to Decision Letter 1


2 Apr 2020

To: Karen Cohen

Academic Editor

PLOS ONE

Re: PONE-D-19-24600R1 “Increasing Adverse Drug Reaction Reporting – How can we do better?”

Dear Editor,

Thank you for your letter of March 2, 2020, and for the attached reviews. We have found your comments and the reviewers' comments most helpful and have addressed them in our revised version.

As for the comments:

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #3: (No Response)

Reviewer #4: (No Response)

________________________________________

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: (No Response)

Reviewer #3: Partly

Reviewer #4: Yes

________________________________________

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: (No Response)

Reviewer #3: No

Reviewer #4: Yes

________________________________________

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: (No Response)

Reviewer #3: Yes

Reviewer #4: Yes

________________________________________

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: (No Response)

Reviewer #3: Yes

Reviewer #4: Yes________________________________________

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

Reviewer #3: From the data presented and the statistical input that was implemented the investigators have concluded that. an increase in the number of ADR reports was noted in the intervention group during the intervention period, the changes in the "knowledge related to behavior” and in the "behavior related to reporting" score was significantly higher in the intervention group. Also specialist physicians and nurses, fulfilling additional positions and those working in other places demonstrated a high rate of report.

The data analysis approach is fairly routine and appears to be sufficient. Also the results are striking. However, the study lacks a rigorous study design justification. Three institutions were randomized, but the unit of analysis is the individual. There is no mention or investigation of the intraclass correlation in this type of setting and how that could impact on the varied analyses and the impact on the type I error. There is no central hypothesis.

Answer:

The research groups were compared in terms of background characteristics and were found to be similar in most variables, so there is no reason to assume there is an Intraclass correlation. The main hypothesis: after the intervention program there would be more ADRs reporting among medical professionals (physicians and nurses) in the intervention group compared to control group and to ADRs reporting base line. This was added to the methods section (page 10, line 204)

Other statistical issues include:

1. The multiple comparison testing issue is totally ignored, especially in Tables 3 and 4.

Answer:

The variables compared in Tables 3 and 4 include two groups, thus there is no need to correct for multiple comparisons. The p value of all the variables with 3 groups were found to be not significant so there is no need to correct for multiple comparison.

2. Tables 5 and 6 would suffice for a descriptive analysis. However, where is the institution effect in these tables?

Answer:

Indeed, beyond the intervention there could be also the influence of the medical institution itself. However, there were no significant differences between the participants from the various medical centers with regard to the basic demographic characteristics.

This information was added to the results section page 11 line 236: "433 (81.5%) medical staff members, physicians and nurses, completed the questionnaire twice, before and after the intervention. 47.8% of the participants were from the "A" medical center, 28.4% from "B" and 23.8% from "C". Distribution by gender was 69.1% females and 30.9% males. 73% were nurses and 27% physicians. No selection bias was found between the staff members completed the questionnaire the first time and those who completed it twice. No differences in personal or professional variables were found between the intervention group ("A" medical center) and the control group ("B" and "C" medical centers), except for the ratio between physicians and nurses and the subjects country of origin and average age."

3. There is insufficient tabular description of the number of cases, specialties, etc. within each of the institutions.

Answer:

This data was added to the results section of the revised manuscript, page 11 line 236.

4. One of the key points mentioned by the investigators is “Education and training of medical staff in good pharmacovigilance practice is necessary for adequate ADR reporting and for the reduction of ADR rates and lower the costs in the healthcare system”. There is no analysis or meaningful discussion of costs considerations.

Answer:

Thanks for the comment. Since reduction of costs due to ADRs reporting was not the scope of this study, we removed this section from the revised manuscript.

5. Reporting tools were conveniently telephone or online, but there is no validation of responses.

Answer:

This was one of the questions that was asked in the questionnaire. "ADRs were reported through the Ministry of Health computerized website (for both the intervention and the control group) or documented in binders available only in the departments of the intervention group." (Page 9, line 171). Therefore, it seems that there is no need to validate it.

The investigators would do well to have this manuscript thoroughly edited by a statistician.

Answer:

A statistician reviewed the revised manuscript, as suggested.

Reviewer #4: I really appreciate reading your work. I feel that the results are rather interesting. The employed methods are sound and the data are rich of info. I have just minor comments.

1. T-tests, as well as any other tests, must satisfy some assumptions. These assumptions are overlooked in the analysis. Please, provide evidence that the assumptions behind the t-test are fulfilled, to avoid misleading conclusions. Non-parametric tests could be used instead.

Answer:

Thank you for your remark. In the "Methods" section, there is a section of data analysis in page 10, line 209: "Means and standard deviation were used for continuous variables and examined by T or One-Way ANOVA/Mann Whitney tests based on the variables distribution." We added that: "The score of "Knowledge related to behavior" and the score of "behavior related to reporting" didn’t distribute normally, thus we used a- parametric test (Mann Whitney). Page 10 line 2011.

2. The use of regressions is sound. Nevertheless, to be confident on the obtained results, the authors must check for the Gauss-Markov assumptions. Inference on regression parameters is valid if and only if those assumptions are fulfilled. Moreover, I am wondering if multicollinearity may be an issue. Please, extend the regression model to include interactions. In general, the analysis of residuals may be included. It would be very useful to evaluate the reliability of the results.

Answer:

Thank you for your Suggestion. We have checked again for Gauss-Markov assumptions and the Inference on regression parameters was found valid and there was no multicollinearity and interactions between the independent variables.

________________________________________7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #3: No

Reviewer #4: No

Sincerely,

Miri Potlog shchory

Attachment

Submitted filename: Response to Reviewers.docx

Decision Letter 2

Karen Cohen

12 May 2020

PONE-D-19-24600R2

Increasing Adverse Drug Reaction Reporting - How can we do better?

PLOS ONE

Dear Dr. Shchory Potlog,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================

It is essential that the following points be addressed:

Please can you provide some information as to how the 3 sites were allocated to either "intervention" or "control"

The reviewer's comment re site-specific factors has not been addressed. There may be differences between the sites that contribute to the differences seen, that are not captured in the variables presented in the manuscript. Please address this comment more comprehensively, and include this in the limitations

 Please include description of the model building strategy for multivariable models. How were the included variables chosen for the multivariable models presented? Are there any other variables included in these models? If yes please specify.

The discussion and conclusion does not adequately address the fact that the impact of the intervention waned with time. It is therefore by no means certain that this intervention would improve safety of medicine use in the long term. There is little basis for reaching the conclusion that this intervention would lead to "overall reduced health costs"- suggest remove this statement from the conclusion.

Please add a paragraph on the limitations of this study. This should include the small number of sites included.

==============================

We would appreciate receiving your revised manuscript by  30 May 2020. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Karen Cohen

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2020 Aug 13;15(8):e0235591. doi: 10.1371/journal.pone.0235591.r007

Author response to Decision Letter 2


3 Jun 2020

To: Karen Cohen

Academic Editor PLOS ONE

Re: PONE-D-19-24600R2

Increasing Adverse Drug Reaction Reporting - How can we do better?

PLOS ONE

Dear Editor,

Thank you for your letter of May 12, 2020, and for the comments. We have found your comments most helpful and have addressed them in our revised version.

As for the comments:

"It is essential that the following points be addressed:

- Please can you provide some information as to how the 3 sites were allocated to either "intervention" or "control" The reviewer's comment re site-specific factors has not been addressed. There may be differences between the sites that contribute to the differences seen, that are not captured in the variables presented in the manuscript. Please address this comment more comprehensively, and include this in the limitations"

Answer:

Three medical centers have participated in the study (Center "A", "B" and "C"). "The randomization among the centers was raffled by an external person who was not related to the study." (Page 7, lines 131-132). "Medical center "A" was randomly selected to be the intervention group. Medical centers "B" and "C" were merged and served as the control group" (Page 7, lines 129-131). "The medical centers selected for the study were public hospitals serving an urban and rural population of 0.5-1 million people each." (Page 7, lines 123-124). The participants were from the: "internal medicine divisions from three public medical centers in Israel: "A", "B" and "C", while each division served as a cluster." (Page 7, lines 121-123). This information is presented in the revised manuscript, as suggested.

- "Please include description of the model building strategy for multivariable models. How were the included variables chosen for the multivariable models presented? Are there any other variables included in these models? If yes please specify."

Answer:

The following paragraph was added to the method section (page 11 lines 231-237): "The building strategy for multivariable models was forced all the independent variables to one block. Both statistical and clinical justifications were considered. The models included all the variables that were found to be significant (p<0.05) in the univariate data analysis, and covariates that were important to controlling for, as a baseline characteristics, according to the research questions. All the independent variables that were included in the analyses were presented in the results of the multivariable models."

_____________________________________________________________________

- "The discussion and conclusion does not adequately address the fact that the impact of the intervention waned with time. It is therefore by no means certain that this intervention would improve safety of medicine use in the long term. There is little basis for reaching the conclusion that this intervention would lead to "overall reduced health costs"- suggest remove this statement from the conclusion.

Please add a paragraph on the limitations of this study. This should include the small number of sites included."

Answer:

It is mentioned in the discussion section that: "The trend presented in the data that the rate of reports decreased over time after the intervention program was discontinued implies that continued intervention may be required to maintain the high rate of reports." (Page 20 lines 377-380) and also that: "However, as distance from the intervention period increased, the amount of reports decreased eventually reverting to the trend that was observed at the baseline. (Page 21 lines 384-386)

We also added that: "The impact of the intervention waned with time. An example to this trend was shown in an educational intervention program to improve physicians' reporting of ADRs. In this study the reporting rate in the intervention group increased during the intervention, while it gradually decreased through 13 months of follow-up. [9]." (Page 21 lines 386-390)

"Due to the observation that the reporting rate decreased with time upon the finalization of the intervention period it seems that….." This was added to the conclusions section (Page 24, lines 458-460)

We agree that: "It is therefore by no means certain that this intervention would improve safety of medicine use in the long term and also reduce health costs". As suggested, this statement was removed from the conclusion.

We added a paragraph on the limitations of the study, as suggested. (Pages 24-25 Limitations

The study was carried out in internal divisions of only 3 public medical centers, and therefore may not have an external validation in other hospitals.

The study included only physicians and nurses, therefore the results cannot be applied to additional medical professionals.

There were some differences in the basic characteristics between the clusters (hospitals) which may have affected the quality of the intervention to a certain extent. However, after adjusting for the demographic variables, we can assume that the results of the study are indeed due to the effect of the intervention.

We cannot assume from this study that the effect of the intervention would improve safety of medicine use in the long term and reduce health costs.

Attachment

Submitted filename: Response to Reviewers.docx

Decision Letter 3

Karen Cohen

19 Jun 2020

Increasing Adverse Drug Reaction Reporting - How can we do better?

PONE-D-19-24600R3

Dear Dr. Shchory Potlog,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Karen Cohen

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Acceptance letter

Karen Cohen

27 Jul 2020

PONE-D-19-24600R3

Increasing Adverse Drug Reaction Reporting - How can we do better?

Dear Dr. Potlog Shchory:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Karen Cohen

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 Data

    (SAV)

    S1 File

    (DOC)

    S2 File

    (DOC)

    Attachment

    Submitted filename: Table 6.doc

    Attachment

    Submitted filename: Answeres plos one 26 10 19 .docx

    Attachment

    Submitted filename: Response to Reviewers.docx

    Attachment

    Submitted filename: Response to Reviewers.docx

    Data Availability Statement

    All data files are available from the ICPSR database - project ID-openicpsr-119582.


    Articles from PLoS ONE are provided here courtesy of PLOS

    RESOURCES