Abstract
Microbiology rounds are an integral part of infectious disease consultation service. During microbiology rounds, we highlight microbiology principles using vignettes. We created case-based, interactive, microbiology online modules similar to the vignettes presented during microbiology rounds. Since internal medicine residents rotating on our infectious disease elective have limited time to participate in rounds and learn microbiology, our objective was to evaluate the use of the microbiology online modules by internal medicine residents. We asked residents to complete 10 of 25 online modules during their infectious disease elective. We evaluated which modules they chose and the change in their knowledge level. Forty-six internal medicine residents completed assessments given before and after accessing the modules with an average of 11/20 (range, 6 to 19) and 16/20 (range, 9 to 20) correct questions, respectively (average improvement, 5 questions; P = 0.0001). The modules accessed by more than 30 residents included those related to Clostridium difficile, anaerobes, Candida spp., Streptococcus pneumoniae, influenza, Mycobacterium tuberculosis, and Neisseria meningitidis. We demonstrated improved microbiology knowledge after completion of the online modules. This improvement may not be solely attributed to completing the online modules, as fellows and faculty may have provided additional microbiology education during the rotation.
INTRODUCTION
In today's residency training environment, duty hour restrictions and competing educational conferences make it difficult to include activities that should be incorporated into elective rotations. Specifically, when internal medicine residents take an elective in infectious diseases, learning or reviewing microbiology concepts is paramount to being able to interpret laboratory results adequately and provide optimal patient care. However, in many academic centers, microbiology rounds have been discontinued or are only available at off-site locations (1). Thus, internal medicine residents rotating in infectious diseases are frequently not able to experience the clinical microbiology laboratory and review microbiology concepts related to clinical care.
Microbiology rounds at Emory University Hospital occur daily and are attended by the Infectious Disease consult teams (attendings, fellows, and students), pharmacists, pathology residents rotating in clinical microbiology, and the clinical pathology faculty. Because microbiology rounds occur at the same time as required core internal medicine lectures, residents rotating on the infectious disease consult service cannot participate. Microbiology rounds last approximately 45 to 50 min and begin with a 15-min discussion of a focused topic or pathogen presented as a microbiology vignette. The vignettes consist of a clinical case with photographs of agar plates, different stains, and biochemical assays pertinent to the organism, and a series of five to eight open-ended questions regarding microbiology principles (2). Following the vignette, participants go to each bench in the microbiology laboratory to address clinical questions for the inpatient infectious diseases consult services. More than 50 of these microbiology vignettes have been converted into online interactive modules that are available on a webpage. The online modules were originally designed to reinforce concepts presented in the vignettes during rounds. We decided to evaluate the effectiveness of the online modules for teaching clinical microbiology to internal medicine residents in the infectious disease elective who could not attend microbiology rounds due to interference with core conferences.
MATERIALS AND METHODS
Creation of online modules.
The online modules contain more questions (10 to 15) than those present in the vignettes presented at microbiology rounds. The questions and feedback content were added to better simulate the structure of rounds, and the feedback content of each question provides information on general microbiology principles in addition to the correct answer to the question. The multiple-choice questions and feedback underwent a review process by at least one microbiologist and infectious diseases clinician. The microbiology vignettes with multiple-choice questions and feedback were entered into the Articulate software (Articulate Global Inc., New York, NY). A webpage was created to house all of the published online modules: http://www.path.emory.edu/Vignettes/ (username and password of MicroV). In addition to giving immediate feedback after each question is answered, this software allows for review of the questions answered at the end of the module, and a certificate can be printed or saved as a pdf file.
Selection of online modules and quizzes.
Twenty-five online modules were selected as being most pertinent to internal medicine residents by the clinicians involved in this study. Topics included common and unusual bacteria, anaerobes, mycobacteria, fungi, viruses, and parasites. At the beginning of the infectious disease elective, internal medicine residents were asked to answer a 20-question knowledge quiz. Correct answers to the quiz were not provided to the residents. Then, they were given a list of the selected modules and asked to choose 10 out of the 25 modules from the microbiology vignette webpage. As they accessed the modules at some point in time during the month-long elective, they were asked to print the certificate of the completed chosen modules. Residents were encouraged, but not required, to explore any of the other online modules. At the end of the month, a second 20-question knowledge quiz was administered. Both quizzes contained questions from the online modules. Although the questions from the second quiz were different from those present in the first quiz as we did not want to “teach for the test,” 18 were from the same online modules, allowing assessment of improvement on the same topic. As can be noted in Table 1, which presents the type of organism and question selected for the quizzes, the questions assessed general microbiology principles taught by multiple modules.
TABLE 1.
Type of organism and question in each knowledge quiz
| Type of organism or question | No. of questions |
|
|---|---|---|
| First quiz | Second quiz | |
| Type of organism | ||
| Gram-negative bacteria | 5 | 4 |
| Gram-positive bacteria | 2 | 1 |
| Anaerobes | 4 | 4 |
| Mycobacteria | 1 | 1 |
| Fungi | 4 | 5 |
| Parasites | 3 | 3 |
| Viruses | 1 | 2 |
| Type of question | ||
| Growth patterns | 6 | 7 |
| Biochemical tests | 3 | 2 |
| Pathogenesis | 3 | 1 |
| Test sensitivity | 3 | 3 |
| Susceptibility testing | 2 | 2 |
| Staining characteristics | 2 | 4 |
| Specimen collection | 1 | 1 |
Statistical analysis.
We evaluated which modules were most frequently chosen by the residents and if they had accessed other modules not included in the list. We evaluated the number of correct answers in the two knowledge quizzes and recorded the improvement obtained after they had completed the online modules. To assess significance, a paired t test was performed using the online calculator http://www.graphpad.com/quickcalcs/ttest2/. In addition, we performed a chi-square test with the Yates correction to assess significance in improvement for questions that were on the same topic.
RESULTS
During the academic year 2013 to 2014, 46 internal medicine residents completed both knowledge assessments. The average score on the first quiz was 11 correct questions out of 20 (range, 6 to 19), and on the second quiz, the average score was 16 out of 20 (range, 9 to 20) with an average improvement of 5 (range, −4 to 12) (P = 0.0001). In the first quiz, 23 (50%) residents answered 12 or more questions correctly. In the second quiz, all but one resident (98%) answered 12 or more questions correctly. Figure 1 compares the number of correct answers by the residents before and after accessing and completing the online modules.
FIG 1.
Comparison of correct answers by residents before and after accessing the online modules. The number of correct answers is shown on the x axis.
The residents were asked to access and complete 10 out of the 25 online modules, and 36 (77%) did so. There were 4 (9%) residents that worked on 7 to 9 modules, 5 (11%) that worked on 11 to 14 modules, and one resident did not present printouts of the certificates but indicated he had worked on all modules. Figure 2 presents the frequency in which the online modules were accessed and completed by the residents. It should be noted that seven modules were accessed by more than 30 residents. These modules included those related to Clostridium difficile, anaerobes, Candida spp., Streptococcus pneumoniae, influenza, Mycobacterium tuberculosis, and Neisseria meningitidis. There were some modules that were not in the subgroup of 25 selected by faculty but were accessed and completed by residents. These modules included anthrax (3 residents), group A streptococci (3 residents), polymicrobial infection (2 residents), hepatitis (1 resident), toxoplasmosis (1 resident), and Yersinia enterocolitica (1 resident).
FIG 2.
Numbers of modules on different microbiological topics accessed and resolved by residents. The number of modules is shown on the x axis.
Of the 18 modules for which there were questions in both the pre- and postassessment (same topic), statistically significant improvement was noted for 10. Table 2 presents the organisms, type of question for both quizzes, and the number of residents with correct answers in both the first and second quizzes of these 10 topics. Improvement ranged from having 36 more residents answering the second question in the pair correctly to having 11 more residents with a correct answer in the second quiz. Four of the paired topic questions were related to the online modules accessed by more than 30 residents (S. pneumoniae, N. meningitidis, influenza, and M. tuberculosis).
TABLE 2.
Organism and type of questions in 10 online modulesa
| Disease or organism | Quiz before accessing online modules |
Quiz after accessing online modules |
No. of residents that improved | P value | ||
|---|---|---|---|---|---|---|
| Type of question | No. of residents with correct answer (%) | Type of question | No. of residents with correct answer (%) | |||
| Meningococcemiab | Pathogenesis | 7 (15) | Specimen collection | 43 (93) | 36 | 0.0001 |
| Histoplasmosis | Test sensitivity | 10 (22) | Growth pattern | 45 (98) | 35 | 0.0001 |
| Bacteroides | Susceptibility testing | 18 (39) | Growth pattern | 46 (100) | 28 | 0.0001 |
| Malaria | Staining characteristics | 4 (9) | Staining characteristics | 29 (63) | 25 | 0.0001 |
| Mucormycosis | Growth pattern | 7 (15) | Growth pattern | 30 (65) | 23 | 0.0001 |
| Tuberculosisb | Test sensitivity | 19 (41) | Growth pattern | 35 (76) | 16 | 0.0013 |
| Proteus | Growth pattern | 32 (69) | Biochemical test | 46 (100) | 14 | 0.0001 |
| Influenzab | Specimen collection | 30 (65) | Test sensitivity | 43 (93) | 13 | 0.0015 |
| Pneumococcusb | Biochemical test | 29 (63) | Biochemical test | 41 (89) | 12 | 0.0063 |
| Serratia | Growth pattern | 32 (69) | Growth pattern | 43 (93) | 11 | 0.006 |
Organism and type of questions in the 10 online modules that had questions in both assessments and for which statistically significant improvement was observed.
Online modules accessed by more than 30 residents.
DISCUSSION
In this study, we demonstrate that internal medicine residents' knowledge of microbiology improved after completing 10 online modules during their infectious diseases consult elective. Online education is advantageous to teachers, as it provides standardized content and the ability to track a learner's activities (3). Advantages for learners include ease of access (time of day, different locations) and the ability to individualize the pace of instruction. In circumstances where residents rotating in infectious diseases cannot attend microbiology rounds, this online interactive activity allowed residents to explore microbiology concepts that complement the clinical learning that is occurring while seeing patients on the wards. The choice of modules selected by the residents likely indicates what they are most frequently seeing as they rotate through the different hospitals in our system. It is not surprising that C. difficile, anaerobes, and Candida spp. were chosen frequently, as these are commonly considered diagnoses in our tertiary care medical center. Similarly, M. tuberculosis is a frequent diagnosis at Grady Memorial Hospital, the public county hospital where our residents rotate. S. pneumoniae and influenza virus are pathogens commonly encountered by residents. The interest in N. meningitidis is less clear, as this organism is not frequently seen here, and we can only attribute this to resident interest in the topic.
In this exercise, some of the residents were curious about other modules that were not part of the subgroup selected by the faculty and accessed them independently from the assignment. The webpage contains a large variety of modules that span the different disciplines that are included in microbiology (bacteriology, mycology, parasitology, and virology). They include frequent and infrequent pathogens that physicians interested in infectious diseases may encounter. Although this evaluation did not include infectious disease fellows, we are aware that these trainees as well as infectious disease faculty have accessed the modules to prepare for board examinations and recertification.
As the participating residents were on an infectious disease rotation, the improvement in microbiology knowledge may not be solely attributed to having completed the online modules. It is likely that faculty or fellows mentioned different microbiology concepts while they were seeing patients. In some instances, they may have discussed or shared the vignette that was presented during microbiology rounds, reinforcing concepts present in the online vignettes. Having a combined approach that includes online learning together with face-to-face discussion is viewed as a valuable approach to teaching (4).
The program used to create the online modules allows the use of blank slides in which a case or situation can be presented along with high-resolution images, and it offers the option to include various types of questions (multiple choice, link two columns, true/false, fill in the blank, etc.) that have immediate feedback with educational content. These features allowed us to build each case in stages such that information can be given in a manner similar to that which occurs in a clinical situation. The clinical history and physical examination results are presented first, questions are asked about which tests to order, the test results are given to the learner who is then asked more sequenced questions regarding the microbiology, pathology, epidemiology, and treatment of the case. This manner of building the cases allows for simulation of clinical situations (5) and tries to emulate microbiology rounds by including questions from multiple disciplines. It takes between 10 to 20 min to complete a module. The interactive nature of the program engages the learner as they cannot progress through the module unless they answer the question. This likely surpasses the effectiveness of passive learning which is frequently seen on websites that post lectures or reading material online. In addition, many learners like to be challenged with a variety of question formats (multiple choice, true/false, jeopardy, and others) to practice for board examinations and certifications (6). Last, the feedback contained general microbiology principles that are repeated through the online modules, such as growth patterns, biochemical tests, and different staining techniques, which allowed us to assess the improvement in knowledge and retention of microbiology principles even though the questions in both quizzes were different.
As presented in this study, the online microbiology modules served as a substitute for microbiology rounds. However, the online cases do not substitute for the important interaction between physicians and the laboratory. Nowadays, many physicians do not know how to use the laboratory and the expertise of the people who provide them with results, as widespread adoption of electronic medical records has allowed physicians to obtain laboratory results from a computer interface without ever speaking to the laboratory personnel. This has created a gap between clinical laboratories and physicians, and in many situations, there is very little understanding of laboratory principles, test availability, changes in report format, reference ranges, and other concepts.
The evaluation of the online microbiology modules was not planned as an academic exercise, and sources of bias should be acknowledged. The use of online modules was triggered when obligatory core internal medicine conferences were scheduled at the same time as microbiology rounds. The online modules were already available, and the decision was made to substitute online modules for microbiology rounds as deficits in clinical microbiology knowledge have been documented (7). Because all this occurred within 1 month, we were left with no time to collect data from a control group of residents who had taken the infectious disease elective and could do the pretest and posttest without completing the online modules. In addition, the proverbial ethical decision of holding back a potentially effective educational intervention from a control group determined that we would not seek one (8). Currently, all residents taking the infectious disease elective are required to do the quizzes and online modules. Although not having used the same questions for both quizzes can be viewed as a limitation, we did not want to “teach for the test.” The questions selected for the quizzes were taken from the online modules and assessed, for the most part, basic microbiology principles, allowing us to test for comprehension rather than recall.
In summary, online learning modules such as those evaluated in this study can be helpful for review of microbiology and laboratory concepts. We demonstrated that there was an increase in knowledge after completing the online modules. The ultimate goal is to integrate microbiology knowledge with the growing clinical expertise of resident learners so that they can make better practice decisions and provide more effective care of patients.
REFERENCES
- 1.Fekete T. 2013. Who killed lab rounds? Pharos Alpha Omega Alpha Honor Med Soc 76:32–34. http://alphaomegaalpha.org/pharos/PDFs/2013-4-Fekete.pdf. [PubMed] [Google Scholar]
- 2.Spicer J, Kraft C, Burd E, Armstrong W, Guarner J. 2014. The value of case-based teaching vignettes in clinical microbiology rounds. Am J Clin Pathol 141:318–322. doi: 10.1309/AJCPW71HRNSSBYPO. [DOI] [PubMed] [Google Scholar]
- 3.Maxwell S, Mucklow J. 2012. e-Learning initiatives to support prescribing. Br J Clin Pharmacol 74:621–631. doi: 10.1111/j.1365-2125.2012.04300.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Goodwin L. 2002. Web-based informatics education: lessons learned from five years in the trenches. Proc AMIA Symp 2002:300–304. [PMC free article] [PubMed] [Google Scholar]
- 5.Ciraj A, Vinod P, Ramnarayan K. 2010. Enhancing active learning in microbiology through case based learning: experiences from an Indian medical school. Indian J Pathol Microbiol 53:729–733. doi: 10.4103/0377-4929.72058. [DOI] [PubMed] [Google Scholar]
- 6.Blewett E, Kisamore J. 2009. Evaluation of an interactive, case-based review session in teaching medical microbiology. BMC Med Educ 9:56–65. doi: 10.1186/1472-6920-9-56. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Guarner J, Armstrong W, Satola S, Mehta A, Jerris R, Hilinski J, Burd E, Kraft C, del Rio C. 2013. Development, implementation, and evaluation of a 4th year medical school elective course in clinical microbiology using case-based vignettes. J Med Microbiol 62:1098–1110. doi: 10.1099/jmm.0.056481-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Sullivan G. 2011. Getting off the “gold standard”: randomized controlled trials and education research. J Grad Med Educ 3:285–289. doi: 10.4300/JGME-D-11-00147.1. [DOI] [PMC free article] [PubMed] [Google Scholar]