Abstract
AIM:
This study aimed to assess the effectiveness of a recognised antimicrobial resistance (AMR) online module on knowledge and perception among dental students, using a randomised controlled trial study design.
METHODS:
Dental students (n = 64, aged 21-25 years) in clinical years agreed to participate in this triple-blinded, parallel, randomised controlled trial. There were 34 students in the study group and 30 students in the control group. The study group participated in an online course covering information about AMR, while students in the control group received another online course about microorganisms in dentistry. Both groups were assessed three times using online questionnaires: before the intervention (T1), after the intervention (T2), and two months later (T3). Each one of T1, T2 and T3 had 22 questions. The questions were repeated each time in T1, T2, and T3 asking about AMR but with different question format, to avoid the possibility of students to memorise the answers.
RESULTS:
The mean (m) of correct answers for all students on T1 was 12.56, with standard deviation (SD) of 3.2. On T2, m = 14.03 and SD = 3.85, and on T3, m = 14.36 and SD = 3.71. Scores ranged from 0 to 22. The participants in the study and control groups showed significant score improvements from T1 to T2, immediately after the intervention, but there was no significant difference between T2 and T3. The study group students’ scores did not improve significantly from T1 to T3, in contrast to the control group students’ scores. More importantly, there was no significant difference in improvement from T1 to T2 when comparing the study and control groups.
CONCLUSION:
Online courses might not be reliable learning methods for ensuring the optimal levels of AMR knowledge that are needed by dental practitioners.
Keywords: Antimicrobial resistance, Randomized Controlled Trial, Online module, Dental
Introduction
The World Health Organization (WHO) recognises antimicrobial resistance (AMR) as one of the greatest threats to human health [1]. According to the National Health Service (NHS) in England, antimicrobial agents are used as a treatment for bacterial infections that are characterised by widespread infection, long healing times, or having serious complications [2]. Antimicrobial drugs are classified into six main families: penicillins, cephalosporins, aminoglycosides, tetracyclines, macrolides, and fluoroquinolones [2]. In dentistry, several antimicrobial agents, including amoxicillin, metronidazole, and doxycycline, are used after dental extractions and treatment of dental abscesses, periapical discharges, and infections [3]. These antimicrobial drugs have many side effects, such as diarrhoea, nausea, vomiting, abdominal pain, loss of appetite, bloating, and indigestion [2]. In some cases, the use of antimicrobial drugs may cause mild to moderate allergies [2].
Nowadays, the NHS and health organisations all over the world are attempting to decrease the prescribing of antimicrobials [2], especially for less serious conditions, because overuse of antimicrobial drugs (antibiotics) leads to decreasing effectiveness [2], or AMR. The WHO defines AMR as the potential for microorganisms to gain resistance to antimicrobial agents [4] when misuse and overuse of antibiotics make more sensitive bacteria die and allow non-sensitive bacteria to proliferate [5]. By 2050, according to the United Kingdom public sector information website, 10 million deaths will occur each year globally at the cost of $100 trillion for the global economy due to the failure to acknowledge and handle the threat from AMR [6].
Multiple systematic reviews and studies highlighted that dentist in an increasing trend to prescribe unnecessarily antibiotic [7], [8], [9]. A cross-sectional study conducted in Saudi Arabia found overall levels of AMR awareness are good among dentists in the Western region; however, awareness of antibiotic prescribing guidelines was insufficient [10]. Another national-scale study concluded that levels of knowledge about AMR are inadequate among dentists in Saudi Arabia on a national level [11]. Similarly, other studies in other countries such as Italy, Poland and Yemen indicating insufficient knowledge about antibiotic prescribing practices and AMR [12], [13], [14].
Some randomized controlled trials (RCTs) have been conducted among general medical practitioners, health care professionals, and the general public to assess the effects of an educational program on attitudes about antimicrobial drugs, and they have found promising results in terms of improvements in knowledge about prescribing of antibiotics [15], [16], [17], [18]. Similarly, a few interventional studies have been conducted among dental students and dental practitioners aimed at improving knowledge and attitudes about prescribing antimicrobial agents [19], [20]. One such study, conducted among dental students in three European countries, used an online interventional module and found it was a helpful and effective tool for teaching about prescribing practices for antimicrobial agents [19]. This study noted that online intervention modules are promising due to their low cost in comparison to other interventions and because they can be widely used among dental students and doctors without the necessity to physically attend classes. However, this was a preliminary study with a small sample size (n = 39), and the design lacked a control group. Another study in Germany was aimed at optimising antimicrobial prescribing behaviour among general dentists by using an interventional seminar and videotapes [20]. However, this was a methodological paper without documented results.
Two auditing studies were conducted examining interventions to improve antimicrobial prescribing practices among dentists [21], [22]. One was an audit and feedback interventional study on antimicrobial prescribing in Scotland using the Reducing Antibiotic Prescribing in Dentistry (RAPiD) program with a face-to-face educational presentation [21]. The other was an audit intervention study in England [22]. Both studies found a significant decrease in antimicrobial prescribing after auditing.
Nevertheless, more interventional studies are needed to assess improvement in AMR knowledge, attitudes, and prescribing behaviours among dental students and dentists. It is also important to assess the effectiveness of online programs, given that they are generally more convenient and cost-effective. No interventional studies have been conducted to date in Saudi Arabia that was aimed at improving dental students’ or practising dentists’ knowledge of AMR, despite documented low levels of such knowledge. Thus, this study aimed to evaluate the effectiveness of an AMR online module for improving knowledge about AMR among dental students in Saudi Arabia, using a randomised controlled trial study design. This interventional article aimed to assess the ability of online course as a convenient and low-cost solution to increase the level of knowledge and awareness about AMR in a dental practice. Such findings might be useful to validate the effectiveness of another similar online course in some universities, and might be a potential method to be applied by other health organisations.
Material and Methods
Design and participants
A parallel randomised controlled trial design (1:1) that was tripling blinded and used an active reference group was selected for this study. The participants were randomly allocated to either a study group or a reference group. The participants in each group received different content materials; however, both courses were online (to ensure a similar way to deliver the content material). The content material of the reference group considered to be placebo because the student received the content material with no intention to improve AMR knowledge. The intervention (in the study group) aimed to investigate the effects of an online course on levels of knowledge among dental students in Saudi Arabia. Male and female students in their fourth, fifth, or sixth academic year at the Dental Faculty of Umm Al-Qura University, Makkah, Saudi Arabia, were invited to participate in the study. The inclusion criteria were: 1. Student at Umm Al-Qura University; 2. The dental student in the fourth, fifth, or sixth year; and 3. Approve and submit an online informed consent to participate in the study.
It should be mentioned that the bachelor of dentistry in Saudi Arabia is seven years program, starting with three non-clinical years and followed by three clinical years and intern. Also, usually students admitted to dental school immediately after graduation from high school at the age of 18 to 19.
Recruiting and randomisation
The study invitation link was disseminated to class leaders of each academic year (both male and female) along with consent forms in January 2018. Students who agreed to participate filled out an online form and were enrolled by a third party, to ensure randomisation concealment and a confirmed number of participants. After an enrollment list was created, students were randomly assigned to either the study or control group using Excel software or a random number generator. The students with odd and even numbers were allocated to either the study or the control group by a third party. The third party gave each group a special code that was revealed after data analysis finished. This was important to ensure that all participants, examiners and statistician were not aware of the students’ allocation groups until data analysis was finished. Thus, the study was triple blinded.
The intervention and reference
None of the students was aware if s/he is in the study or control group to ensure participant blindness. Students in both groups received a link via mobile phone text sent by a third party to ensure examiner blindness. All the students went through a similar process when they clicked on the link, as shown in the participant’s flow chart (Figure 1) and the data collection process (Figure 2).
Figure 1.
Flow of participants through the randomised controlled trial
Figure 2.
Data collection process
First, they reviewed the consent form and signed it by clicking “next.” Second, they filled out a baseline questionnaire (T1). Third, they entered an interventional online course or a reference group who had placebo online course, according to their group allocation. Fourth, immediately following completion of the online course, students answered another questionnaire (T2). These steps were done in February 2018. Fifth, about two months after the completion of the online courses (April 2018), students received another follow-up questionnaire to measure their knowledge retention (T3). In addition to the time spent taking the intervention course or the placebo course, students spent approximately 20 min to answer each of the T1 and T2 questionnaires. They spent approximately 7 min to answer the T3 questionnaire.
The study group’s interventional course consisted of 25 slides with written material in English, and participants were given an option to listen to an accompanying audio recording of the written material. The interventional video was approximately 9 min and was aimed at improving participants’ knowledge about AMR and the proper use of antimicrobial agents in dentistry. The content of the intervention was derived from different sources [4], [5], [6], [23], [24], [25] and was validated by a panel of eight dental consultants from a content point of view.
The content was divided into seven sections: the definition of AMR, introduction, types of actions of antibiotics, how AMR develops, causes of AMR, the dentist’s role in reducing AMR, and proper prescribing of antibiotics by dentists. The link for the study group course can be accessed from here: https://goo.gl/forms/7RA079LPB5H6YE033.
The reference group was given an online course about microorganisms in dentistry (with no mention of anything related to AMR), consisting of 34 slides and written material in English. This video was about 5 min, and the content was derived from a previous resource [23]. This course was divided into eight sections: definition of normal flora, types of normal flora, common habitats of human microbial flora, changes of oral flora with lifestyle and oral habits, benefits of resident oral flora, disadvantages of resident oral flora, and factors modulating the growth of bacteria in the oral cavity. For ethical reasons, after completion of the T3 questionnaire, all students in the reference group received a link for the main interventional course to improve AMR knowledge and prescribing behaviour. The link for the control group course can be accessed from here: https://goo.gl/forms/2lMWNuG4WBuZSyEk2
Both courses’ links were sent separately and individually to participants in avoid intervention contamination. Both courses were previously evaluated in two pilot studies using 12 students to ensure the validity of the intervention and the questionnaire. Also, the courses were adjusted concerning technical problems, content, structure, and course usability.
The process of recruitment of participants and the flow of the study are shown in Figure 2.
The questionnaires
Students in both groups received three questionnaires: baseline before the study (T1), immediately after the intervention/placebo (T2), and a follow-up two months later (T3). The T1 and T3 questionnaires were identical. The T2 questionnaire contained the same questions as in T1 and T2 but with minor differences in wording to eliminate the chances of participants remembering the questions. All three questionnaires (T1, T2, and T3) consisted of two sections and had a total of 28 questions. Section one was composed of 22 questions regarding knowledge of AMR that were either in the form of multiple-choice questions with only one correct answer or questions where participants needed to select all the correct answers from a list. Section two was composed of six demographic questions asking about an academic year, gender, marital status, financial status, GPA, and phone number. The phone numbers were used for follow-up at T3 and to give out randomly picked incentives; then the numbers were deleted from all documentation after data entry, making the data anonymous.
Statistical analysis
After data entry, study group and control group participant data were coded by a third party to ensure statistician blindness. SPSS was used for data analysis. Descriptive statistics were used to generate frequency tables, means, and standard deviations. Chi-square, t-test, and paired t-test were used to compare the study and control group results.
Ethical approval and incentives
This study received prior ethical approval from Umm Al-Qura University, Faculty of Dentistry, with the number 77-17 and was registered with the ISRCTN registry (ISRCTN13442659). All the students submitted an electronic consent form before participating in the study. As an incentive, students who completed all three questionnaires were entered into a random drawing for a chance to win one of six 200 Saudi Riyal coupons to a well-known bookstore in Saudi Arabia, three for male participants and three for female participants.
Results
All the students in the 4th, 5th and 6th years were invited (200 students). Only 64 completed the T1 questionnaire, took the online (study or control) course, and answered the T2 questionnaire, yielding a 32% response rate. There were 34 participants in the study group and 30 participants in the control group. All the participants answered the T1 and T2 questionnaires, with no dropouts; however, there was a dropout rate of 23.43% at T3, as shown in Figure 1. Participants’ demographic data are displayed in Table 1.
Table 1.
Participant demographic data (n = 64)
| Variable | Total, n (%) | Study Group, n (%) | Control Group, n (%) | p-value | |
|---|---|---|---|---|---|
| Gender | Male | 27 (42.18) | 10 (29.40) | 17 (56.70) | 0.042 |
| Female | 37 (57.80) | 24 (70.60) | 13 (43.30) | ||
| Academic year | 4th year | 36 (56.25) | 21 (61.80) | 15 (50.00) | 0.702 |
| 5th year | 6 (9.30) | 3 (8.80) | 3 (10.00) | ||
| 6th year | 22 (34.30) | 10 (29.40) | 12 (40.00) | ||
| Marital status | Single | 61 (95.30) | 31 (91.20) | 30 (100.00) | 0.241 |
| Married | 3 (4.60) | 3 (8.80) | 0 (0.00) | ||
| Family annual income | < 5,000 | 6 (9.30) | 4 (11.80) | 2 (6.70) | 0.149 |
| 5,000–15,000 | 28 (43.70) | 11 (32.40) | 17 (56.70) | ||
| > 15,000 | 30 (46.80) | 19 (55.90) | 11 (36.70) | ||
| GPA | A | 31 (48.40) | 17 (50.00) | 14 (46.70) | 0.617 |
| B | 27 (42.18) | 15 (44.10) | 12 (40.00) | ||
| C | 6 (9.30) | 2 (5.90) | 4 (13.30) |
Comparing the demographic variables between the study and control groups using Chi-square and Fisher’s exact test showed that there were no significant differences except for gender, where females were more significantly represented in the study group than in the control group (p = 0.042).
The mean (m) of correct answers for all students on T1 was 12.56, with SD of 3.2. On T2, m = 14.03 and SD = 3.85, and on T3, m = 14.36 and SD = 3.71. The maximum possible score was 22, and the minimum was zero.
In the study group, a paired t-test for repeated measures showed that there was a significant improvement in knowledge scores from T1 (m = 13.5, SD = 3.06) to T2 (m = 14.94, SD = 4.46); (p = 0.040). However, the scores from T2 to T3 (m = 15.75, SD = 3.47) and from T1 to T3 were not significantly different.
In the control group, a paired t-test for repeated measures showed that there was a significant improvement in knowledge scores from T1 (m = 11.5, SD = 3) to T2 (m = 13.0, SD = 2.74); (p = 0.002). Besides, there was a significant improvement from T1 to T3 (m = 13.04, SD = 3.49); (p < 0.018). However, the score from T2 to T3 was not significantly different.
To compare the differences in knowledge scores between the study group and the control group, we calculated the differences from T1 to T2, T2 to T3, and T1 to T3, as displayed in Table 2. Using t-test, the score differences of the study and control groups were compared, and there were no significant differences, as shown in Table 2.
Table 2.
Differences in knowledge scores on T1, T2, and T3
| Study group | Control group | ||
|---|---|---|---|
| Knowledge | Mean (SD) | Mean (SD) | p-value |
| Difference between T1 and T2 | 1.441 (3.94) | 1.500 (2.37) | 0.942 |
| Difference between T2 and T3 | 0.290 (2.83) | 0.280 (2.44) | 0.988 |
| Difference between T1 and T3 | −1.125 (3.74) | −1.320 (2.59) | 0.834 |
Figure 3 shows the trajectory of the participants’ knowledge scores for both groups. It should be noted that the paired t-test and t-test was conducted as a parametric test after meeting test assumptions including normal distribution of the variables used in these tests.
Figure 3.
Knowledge scores on T1, T2, and T3 in study and control groups
Discussion
Our results suggested that there was no significant improvement in the knowledge levels of participants in the study group when compared to the control group, despite both groups showing significant improvement immediately following the intervention. Our results seem to be different from those in previous studies [19], [21], [22], which showed significant improvements in study groups after interventions. However, these previous studies did not have active placebo controls for comparison [19], [21], [22]. Our results may be more accurate due to the active control group showing that the improvement found in the study group was no different from the improvement that occurred in the control group. Nevertheless, it is hard to compare our study with the mentioned previous study in micro-level because there are many differences, bearing in mind that these studies are the only interventional studies in this research area according to our best knowledge.
A number of possibilities could explain the insignificant improvement after the intervention in the study group as compared to the control group. First, the online module might not be the ideal learning method for students because, low response to online course as mentioned in a previous article [26], not taking the course seriously, students might skip slides just to finish, which has been observed in other online courses among dental students by this study’s authors. Second, the interventional course might have contained a high volume of content to be absorbed by the students, and some students might have had some technical problems [27], [28]. Third, we had a small sample size, which could have affected the ability to detect differences between the groups.
It is interesting to note that students showed an increase in scores from T1 to T2 in both groups, which were not surprising for the study group, where students had received pertinent information in their online course, but it could be a little surprising for the control group. However, this may be explained by the possibility of students in the control group searching the internet for correct answers to the questions on the T2 questionnaire, especially if they were unsure about their answers on T1.
This has important implications for mandatory online courses in the health sector in general, especially for organisations that make these online courses a prerequisite for moving forward into other job responsibilities. Online courses tend to be more feasible to ensure that participants achieve a course outcome [29] especially for the organization, and in fact, sometimes the online systems issue auto-generated certificates of completion after participants pass a course exam. Our results indicate that these mandatory online courses might be inappropriate for accomplishing student or health practitioner goals. The students in our study seemed to take a careless attitude toward the online courses, which was reflected in the results not showing the expected improvement for the study group. While the students in the pilot studies described the study group course content as straightforward and simple, that could be because the students in the pilots were personally invited to take the course. This might have made the students in the pilots approach the course more seriously and answer the questionnaires with care, while the students in the main study had no relationship with the research team and so behaved more naturally. Regardless, the main study participants represented the desired population.
Further, some online courses allow participants to take a final exam more than once until the exam is passed; this can lead to participants skipping slides and searching for answers online, or taking other actions, to pass the final exam and get their certificate [30], [31]. But this violates the main reasons for taking such courses. We believe that better specifications could make these online courses more effective. For example, courses could calculate the time spent on each slide to detect if participants are skipping quickly to the end of the course. However, such recommendations or suggestions cannot be made without further research, investigation and innovative ideas to determine the best approaches. It should be noted that the main aim to use the placebo in the reference course in the control group is to minimise the Hawthorne effect [32] generated to the participants after answering the baseline questionnaire, as we assumed that some students would search for the correct using the internet.
In addition to future similar studies needing to use larger sample sizes, the intervention should be reviewed in terms of technical difficulties and user experiences to create a better course design. It is suggested that future interventions include a required follow-up exam that can be taken only once within a certain time period and which participants must pass with a predetermined score to receive a certificate of completion.
This study had some strengths, particularly the use of an RCT design with an active reference group and triple blindness. This is also the first study of its kind in Saudi Arabia using an online module covering this topic and examining the feasibility of online courses, given their minimal time commitments and low costs. On the other hand, this study’s limitations include a small sample size, which makes it hard to generalise the study finding, short follow-up period, and a sample was taken from single-centre, which cast more doubts about generalizability for the study result. It is recommended to have a further study with a larger sample size and use multicenter for more generalizable results.
In conclusion, according to our results that showed there is no significant change in the students’ AMR knowledge score after the intervention, online courses might not be an appropriate learning method for the goal of improving AMR knowledge levels because participants may not take the module seriously. Further investigations and interventional studies regarding AMR are needed to assess different modalities for improving dental students’ knowledge and practices around AMR. Future studies might assess online modules with the addition of technical features to increase the levels of responsibility required so that participants are prompted to take these courses with care and attention.
Acknowledgments
We want to thank Dr Ahmad Darderi, Dr Majed Almalki, and Dr Abdualarhman Yossef for reviewing the intervention course content. We also want to thank Dr Abdullah Alharthi, Dr Ahmad Zidan, and Dr Noura Alhomaidhi for helping in data collection and ensuring blindness.
Footnotes
Funding: This research did not receive any financial support
Competing Interests: The authors have declared that no competing interests exist
References
- 1.WHO. The evolving threat of antimicrobial resistance: options for action. Geneva: World Health Organization; 2012. [Google Scholar]
- 2.NHS. Antibiotics: National Health Service. 2016. cited November 13, 2018. Available from: https://www.nhs.uk/conditions/antibiotics/
- 3.Gowri S, Mehta D, Kannan S. Antibiotic use in dentistry: A cross-sectional survey from a developing country. Journal of Orofacial Sciences. 2015;7(2):90. https://doi.org/10.4103/0975-8844.164310. [Google Scholar]
- 4.WHO. Antimicrobial resistance: Antimicrobial resistance in the Region. 2018. cited November 13, 2018 Available from: http://www.emro.who.int/health-topics/drug-resistance/regional-situation.html .
- 5.CDC. Antibiotic Resistance Questions and Answers USA: Centers for Disease Control and Prevention. 2017. cited November 13, 2018 Available from: https://www.cdc.gov/antibiotic-use/community/about/antibiotic-resistance-faqs.html .
- 6.Gov.uk. Health matters: preventing infections and reducing antimicrobial resistance: United Kingdom public sector information website: Public Health, England. 2017. cited November 13, 2018. Available from: https://www.gov.uk/government/publications/health-matters-preventing-infections-and-reducing-amr/health-matters-preventing-infections-and-reducing-antimicrobial-resistance .
- 7.Cummins J, McCarthy M, Esterman A, Lee A, Kavre A. Knowledge and compliance of dentists' and dental students' with respect to relevant guidelines for prescribing antibiotic prophylaxis for the prevention of infective endocarditis: A systematic review. Journal of Evidence Based Dental Practice. 2019 doi: 10.1016/j.jebdp.2019.01.007. https://doi.org/10.1016/j.jebdp.2019.01.007. [DOI] [PubMed] [Google Scholar]
- 8.Klinge A, Khalil D, Klinge B, Lund B, Naimi-Akbar A, Tranaeus S, et al. Prophylactic antibiotics for staged bone augmentation in implant dentistry. Acta Odontologica Scandinavica. 2019:1–10. doi: 10.1080/00016357.2019.1656819. https://doi.org/10.1080/00016357.2019.1656819 PMid:31483177. [DOI] [PubMed] [Google Scholar]
- 9.Aidasani B, Solankis M, Khetarpal S, Ravi SP. Antibiotics: their use and misuse in paediatric dentistry. A systematic review. European Journal of Paediatric Dentistry. 2019;20(2):133–8. doi: 10.23804/ejpd.2019.20.02.10. [DOI] [PubMed] [Google Scholar]
- 10.Al-Harthi SE, Khan LM, Abed HH, Alkreathy HM, Ali AS. Appraisal of antimicrobial prescribing practices of governmental and non-governmental dentists for hospitals in the western region of Saudi Arabia. Saudi Medical Journal. 2013;34(12):1262–9. [PubMed] [Google Scholar]
- 11.Halboub E, Alzaili A, Quadri M, Al-Haroni M, Al-Obaida M, Al-Hebshi N. Antibiotic Prescription Knowledge of Dentists in Kingdom of Saudi Arabia: An Online, Country-wide Survey. The Journal of Contemporary Dental Practice. 2016;17(3):198–204. doi: 10.5005/jp-journals-10024-1827. https://doi.org/10.5005/jp-journals-10024-1827 PMid:27207198. [DOI] [PubMed] [Google Scholar]
- 12.Al-Haroni M, Skaug N. Knowledge of prescribing antimicrobials among Yemeni general dentists. Acta Odontologica Scandinavica. 2006;64(5):274–80. doi: 10.1080/00016350600672829. https://doi.org/10.1080/00016350600672829 PMid:16945892. [DOI] [PubMed] [Google Scholar]
- 13.Struzycka I, Mazinska B, Bachanek T, Boltacz-Rzepkowska E, Drozdzik A, Kaczmarek U, et al. Knowledge of antibiotics and antimicrobial resistance amongst final year dental students of Polish medical schools-A cross?sectional study. European Journal of Dental Education. 2019;23(3):295–303. doi: 10.1111/eje.12430. https://doi.org/10.1111/eje.12430 PMid:30729642. [DOI] [PubMed] [Google Scholar]
- 14.Salvadori M, Audino E, Venturi G, Garo M, Salgarello S. Antibiotic prescribing for endodontic infections: a survey of dental students in Italy. International Endodontic Journal. 2019;2(9):1388–96. doi: 10.1111/iej.13126. https://doi.org/10.1111/iej.13126 PMid:30982994. [DOI] [PubMed] [Google Scholar]
- 15.Altiner A, Brockmann S, Sielk M, Wilm S, Wegscheider K, Abholz H-H. Reducing antibiotic prescriptions for acute cough by motivating GPs to change their attitudes to communication and empowering patients: a cluster-randomized intervention study. Journal of Antimicrobial Chemotherapy. 2007;60(3):638–44. doi: 10.1093/jac/dkm254. https://doi.org/10.1093/jac/dkm254 PMid:17626023. [DOI] [PubMed] [Google Scholar]
- 16.Foucault C, Brouqui P. How to fight antimicrobial resistance. FEMS Immunology & Medical Microbiology. 2006;49(2):173–83. doi: 10.1111/j.1574-695X.2006.00172.x. https://doi.org/10.1111/j.1574-695X.2006.00172.x PMid:17181560. [DOI] [PubMed] [Google Scholar]
- 17.Finch RG, Metlay JP, Davey PG, Baker LJ. Educational interventions to improve antibiotic use in the community: report from the International Forum on Antibiotic Resistance (IFAR) colloquium, 2002. The Lancet Infectious Diseases. 2004;4(1):44–53. doi: 10.1016/s1473-3099(03)00860-0. https://doi.org/10.1016/S1473-3099(03)00860-0. [DOI] [PubMed] [Google Scholar]
- 18.Milos V, Jakobsson U, Westerlund T, Melander E, Mölstad S, Midlöv P. Theory-based interventions to reduce prescription of antibiotics-a randomized controlled trial in Sweden. Family Practice. 2013;30(6):634–40. doi: 10.1093/fampra/cmt043. https://doi.org/10.1093/fampra/cmt043 PMid:23960104. [DOI] [PubMed] [Google Scholar]
- 19.Berr L, Donaldson N, Hatzipanagos S, Paganelli C, Reynolds P. The impact on dental students' knowledge in three European countries through an online module on antibiotic prescribing: a preliminary study. Bulletin du Groupement International pour la Recherche Scientifique en Stomatologie et Odontologie. 2013;51(3):25–6. [PubMed] [Google Scholar]
- 20.Löffler C, Böhmer F, Hornung A, Lang H, Burmeister U, Podbielski A, et al. Dental care resistance prevention and antibiotic prescribing modification-the cluster-randomised controlled DREAM trial. Implementation Science. 2014;9(1):27. doi: 10.1186/1748-5908-9-27. https://doi.org/10.1186/1748-5908-9-27 PMid:24559212 PMCid:PMC3936853. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Elouafkaoui P, Young L, Newlands R, Duncan EM, Elders A, Clarkson JE, et al. An audit and feedback intervention for reducing antibiotic prescribing in general dental practice: The RAPiD cluster randomised controlled trial. PLoS Medicine. 2016;13(8):e1002115. doi: 10.1371/journal.pmed.1002115. https://doi.org/10.1371/journal.pmed.1002115 PMid:27575599 PMCid:PMC5004857. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Palmer N, Dailey Y, Martin M. Pharmacology: Can audit improve antibiotic prescribing in general dental practice? British dental journal. 2001;191(5):253–5. doi: 10.1038/sj.bdj.4801156a. https://doi.org/10.1038/sj.bdj.4801156 PMid:11575760. [DOI] [PubMed] [Google Scholar]
- 23.Greenwood D, Slack RC, Barer MR, Irving WL. Medical microbiology e-book: a guide to microbial infections: pathogenesis, immunity, laboratory diagnosis and control. With student consult online access: Elsevier Health Sciences. 2012 [Google Scholar]
- 24.Wilson W, Taubert KA, Gewitz M, Lockhart PB, Baddour LM, Levison M, et al. Prevention of infective endocarditis: guidelines from the American heart association: a guideline from the American heart association rheumatic fever, endocarditis, and Kawasaki disease committee, council on cardiovascular disease in the young, and the council on clinical cardiology, council on cardiovascular surgery and anesthesia, and the quality of care and outcomes research interdisciplinary working group. Circulation. 2007;116(15):1736–54. doi: 10.1161/CIRCULATIONAHA.106.183095. https://doi.org/10.1161/CIRCULATIONAHA.106.183095 PMid:17446442. [DOI] [PubMed] [Google Scholar]
- 25.Palmer N, Longman L, Randall C, Pankhurst C. Antimicrobial prescribing for general dental practitioners. Faculty of general dental practitioners (FGDP), UK. 2012 [Google Scholar]
- 26.Morrison R. A comparison of online versus traditional student end?of?course critiques in resident courses. Assessment & Evaluation in Higher Education. 2011;36(6):627–41. https://doi.org/10.1080/02602931003632399. [Google Scholar]
- 27.Vu P, Cao V, Vu L, Cepero J. Factors driving learner success in online professional development. The International Review of Research in Open and Distributed Learning. 2014;15:3. https://doi.org/10.19173/irrodl.v15i3.1714. [Google Scholar]
- 28.Lee SJ, Srinivasan S, Trail T, Lewis D, Lopez S. Examining the relationship among student perception of support, course satisfaction, and learning outcomes in online learning. The Internet and Higher Education. 2011;14(3):158–63. https://doi.org/10.1016/j.iheduc.2011.04.001. [Google Scholar]
- 29.Liyanagunawardena TR, Williams SA. Massive open online courses on health and medicine. Journal of Medical Internet Research. 2014;16(8):e191. doi: 10.2196/jmir.3439. https://doi.org/10.2196/jmir.3439 PMid:25123952 PMCid:PMC4155756. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Van Rooij SW, Lemp LK. Positioning e-learning graduate certificate programs: Niche marketing in higher education. Services Marketing Quarterly. 2010;31(3):296–319. https://doi.org/10.1080/15332969.2010.486691. [Google Scholar]
- 31.Lewis KO, Cidon MJ, Seto TL, Chen H, Mahan JD. Leveraging e-learning in medical education. Current Problems in Pediatric and Adolescent Health Care. 2014;44(6):150–63. doi: 10.1016/j.cppeds.2014.01.004. https://doi.org/10.1016/j.cppeds.2014.01.004 PMid:24981664. [DOI] [PubMed] [Google Scholar]
- 32.Sedgwick P, Greenwood N. Understanding the Hawthorne effect. BMJ. 2015;351:h4672. doi: 10.1136/bmj.h4672. https://doi.org/10.1136/bmj.h4672 PMid:26341898. [DOI] [PubMed] [Google Scholar]