Abstract
Background
The acquisition of new knowledge is a primary goal of residency training. Retrieving and retaining influential primary and secondary medical literature can be challenging for house officers. We set out to investigate the effect of a Universal Serial Bus (USB) drive loaded with landmark scientific articles on housestaff education in a pilot study.
Methods
We created a USB syllabus that contains 187 primary scientific research articles. The electronic syllabus had links to the full-text articles and was organized using an html webpage with a table of contents according to medical subspecialties. We performed a prospective cohort study of 53 house officers in the internal medicine residency program who received the USB syllabus. We evaluated the impact of the USB syllabus on resident education with surveys at the beginning and conclusion of the nine-month study period.
Results
All 50 respondents (100%) reported to have used the USB syllabus. The self-reported number of original articles read each month was higher at the end of the nine-month study period compared to baseline. Housestaff rated original articles as being a more valuable educational resource after the intervention.
Conclusions
An electronic syllabus with landmark scientific articles placed on a USB drive was widely utilized by housestaff, increased the self-reported reading of original scientific articles and seemed to have positively influenced residents' attitude toward original medical literature.
Keywords: medical education, electronic syllabus, USB drive
The acquisition of new knowledge is a primary goal of residency training. House officers are expected to be familiar with the landmark studies that have shaped clinical practice or guidelines and stay abreast of new discoveries that are published in the medical literature. Retrieving and retaining influential primary and secondary medical literature can be challenging for residents. Resources like ‘UpToDate’ (1) and eMedicine (2) are valuable in synthesizing data into readily digestible bites; however, in using these materials trainees do not have to critically appraise publications or extract the clinical messages. It is believed that habits formed early in one's career can shape our actions and promote lifelong learning (3).
Individual physician-teachers often give relevant medical literature to medical trainees to read and to keep. However, this process is not standardized and the exposure of trainees to scientific research articles is variable. Medical trainees believe that a fundamental role of attending physicians is to share and teach them about scientific articles that are ‘need-to-knows’ (4). Analysis of evaluations completed by medical trainees show that they desire and value articles that are shared with them by others in clinical settings (5).
In an attempt to standardize this process, house officers at the Johns Hopkins Bayview Medical Center (JHBMC) Internal Medicine residency program were provided with a USB drive loaded with landmark scientific articles related to the subspecialties of internal medicine. We hypothesized that a USB syllabus would encourage trainees to read the influential primary and secondary medical literature and stimulate house officers to read more primary medical literature. We posited that the electronic format on a USB drive with a structured yet editable format would facilitate their keeping track of their reading and would make the process more efficient. This manuscript describes the result of this pilot effort at our institution.
Methods
Study design
The effect of the USB syllabus on resident education was assessed in a prospective observational cohort pilot study. This study was conducted during the 2006–2007 academic year at the JHBMC. The Institutional Review Board approved the study and informed consent was obtained from study participants.
Study population
The residency program in Internal Medicine at the JHBMC is an Accreditation Council for Graduate Medical Education (ACGME) accredited university-based training program that had 53 house officers during the academic year 2006–2007 (Table 1). The stated goal of the residency program is ‘to develop healers and leaders in medicine.’ All 53 residents of the JHBMC internal medicine residency program received a 256 MB USB drive with a hyperlinked electronic syllabus.
Table 1.
Characteristics of the study population (n = 53)
| Age in years, mean (SD) | 29 (2.5) |
| Female gender, n (%) | 30 (57) |
| Level of training, n (%) | |
| PGY-1 | 25 (48) |
| PGY-2 | 14 (26) |
| PGY-3 | 14 (26) |
USB syllabus
The USB syllabus contained links to 187 primary scientific research articles. The links to the full-text articles were organized on the USB drive using an html webpage with a table of contents with 14 headings according to medical subspecialties (e.g., Cardiology, Endocrinology, etc.).
The headings of the table of content were hyperlinked to specific heading locations on the html webpage, where each medical subspecialty was then divided into 2–12 context-related subheadings (e.g., Cardiology had the subheadings atrial fibrillation, congestive heart failure, etc.). The titles of the individual articles were organized under the subheadings on the html webpage and provided the hyperlink to the full-text article in PDF format in the medical school's digital library. The individual user could download the full-text article in PDF format using a structured folder system on the USB drive, resembling the headings and subheadings of the html webpage, to organize the downloaded primary and secondary scientific research articles. To comply with copyright regulations, the full-text PDF documents could not be stored a priori on the USB syllabus. Once the article was downloaded and stored in the pre-organized folder system, it could be used on any computer (e.g., at home) and its use was not dependent on internet access.
Article selection for inclusion in the USB syllabus focused on landmark scientific articles. One author (M.E.) surveyed local experts, senior faculty members in the Department of Medicine at the Johns Hopkins University School of Medicine. The experts were asked to recommend the most influential and commonly cited primary and secondary research articles related to their field of specific expertise.
The USB drives were usable at every USB port of any computer at JHBMC and the syllabus could be expanded and edited by the individual learner. The structured folder system was constructed in a way to allow for both expansion of the literature collection and for keeping track of reading progress. The residents retained the USB syllabus as their personal property.
Trainees were also encouraged to save other important information related to the acquisition of clinical competence on the ‘my portfolio’ section of the USB including: (a) interesting patients seen, (b) presentations given, and (c) procedures performed. The USB drives were individualized and were password protected.
Data collection and evaluation
Housestaff filled out a brief baseline survey before receiving the USB drive in September 2006. The baseline survey included questions pertaining to the following areas: (a) four demographic questions, (b) two questions about their reading of primary medical literature, and (c) seven questions related to the use of computers, USB drives, and use of educational resources. Most questions had Likert scale response options.
At the conclusion of the study period in May 2007, an end-of-study survey was completed. The end-of-study survey primarily asked participants about (a) their reading of primary medical literature, (b) their use of and experiences with the USB syllabus, and (c) suggestions about ways to improve the USB syllabus.
Data analysis
For each variable, we examined frequency of responses to look for irregularities in the distribution of responses. For the continuous variables, distributions and descriptive statistics were examined for evidence of skewness, outliers, and non-normality to ensure the appropriate use of parametric statistical tests. Comparisons of the data from before and after the ‘USB syllabus intervention’ were examined using t-tests and the Wilcoxon signed-rank test. Data were analyzed using STATA 8.0 (STATA Corp., College Station, TX, USA).
Results
Forty-five of the 53 (85%) house officers at JHBMC completed the baseline survey. The majority of internal medicine house officers at JHBMC are planning a career in academic medicine (80%), most as clinical investigators (69%) (Table 2).
Table 2.
Baseline characteristics of the 45 respondents
| Age in years, mean (SD) | 28 (2.4) |
| Female gender, n (%) | 25 (56) |
| Level of training, n (%) | |
| PGY-1 | 20 (44) |
| PGY-2 | 14 (31) |
| PGY-3 | 11 (24) |
| Plan for career in academia, n (%) | 36 (80) |
| Plan for career as an investigator/researcher, n (%) | 31(69) |
| Current USB usage, n (%) | 40 (89) |
| ‘Strongly Agree’ that computers will become increasingly valuable in medical education, n (%) | 43 (95) |
| ‘Agree’ or ‘Strongly Agree’ with the statement: ‘I love computers’, n (%) | 33 (73) |
| Educational value of resources, mean (SD) | |
| Original articles | 3.6 (0.8) |
| Review articles | 4.4 (0.8) |
| Electronic resources such as ‘UpToDate’ | 4.7 (0.6) |
| Textbooks | 3.1 (1.2) |
| (Five-point Likert scale with 1 = useless and 5 = excellent) | |
Almost all house officers (89%) had been using USB drives, and most (95%) affirmed that computer-based learning is valuable in medical education. With respect to the perceived ‘educational value’ of resources at their disposal, at baseline (prior to the intervention) house officers rated electronic resources like ‘UpToDate’ as most valuable (mean 4.7, SD ±0.8, on a five-point Likert scale with 1 = useless and 5 = excellent). Original articles were rated as ‘fair–good’ (mean 3.6, SD ±0.8) and textbooks were rated the least favorable (mean 3.1, SD ±1.2).
Characterization of usage and impact of the USB syllabus
At the conclusion of the nine-month study period, 50 of 53 (94%) house officers completed the end of study survey. All 50 respondents (100%) reported using the USB syllabus. Two of the 50 house officers (4%) lost their USB syllabus during the study period and both were given a replacement.
In characterizing their experience, 45 of 50 house officers (90%) strongly agreed with the following: ‘I liked the idea that I was given the USB syllabus.’ When asked, if their time using the USB syllabus was well spent, 80% agreed or strongly agreed. Forty-nine of 50 house officers (98%) disagreed or strongly disagreed that there were technical difficulties using the USB syllabus.
Most house officers (86%) agreed or strongly agreed that the USB syllabus met their educational needs. Similarly, 90% agreed or strongly agreed that the USB syllabus stimulated them to read more primary literature. When asked whether the USB syllabus helped them to take better care of patients, 88% agreed or strongly agreed.
As 45 residents had filled out the baseline survey and 50 residents the post-intervention survey, we were able to compare pre- and post-surveys for 45 residents. Self-reported original articles read by housestaff increased from 3.4 per month at baseline to 4.5 per month by the end of the nine-month study period, p < 0.005 (Table 3).
Table 3.
Number of original articles read (n=45)
| Level of training (n) | Pre-mean (SD) | Post-mean (SD) | p-Value |
|---|---|---|---|
| Total (45) | 3.4 (2.6) | 4.5 (3.0) | <0.005 |
| PGY-1 (20) | 2.0 (1.2) | 3.2 (1.8) | 0.01 |
| PGY-2 (14) | 4.2 (2.1) | 5.3 (2.4) | 0.18 |
| PGY-3 (11) | 5.1 (3.0) | 5.9 (3.1) | 0.33 |
Note: Number of original articles read by trainee per month according to level of training. The comparison between baseline survey and post-intervention survey was available for 45 residents.
Sixteen of the 45 (45%) increased their self-reported reading of original medical papers by more than three articles per month. Post-graduate year 1 (PGY-1) level trainees showed the greatest self-reported increase in original articles read per month, from 2.0 to 3.2 per month, p = 0.01. Housestaff rated original articles as being more valuable to them for the acquisition of knowledge after being exposed to this intervention as compared to baseline (p < 0.005) (Fig. 1).
Fig. 1.
Rating of the value of specific educational resources (n=45).
Rating of the value of specific educational resources with respect to the trainees' acquisition of knowledge at baseline and at the end of the study period (mean values). The comparison between baseline survey and post-intervention survey was available for 45 residents.
Review articles also were rated higher (p = 0.02), whereas the perceived value of other educational resources (electronic media like ‘UpToDate’ (p = 0.09) and textbooks (p=0.18)) did not change significantly.
Selected comments to an open-ended question are presented below. All comments were positive and spoke to the usefulness and merit of the USB syllabus.
This is an awesome resource!
Super creative idea. I don't think I can do without this tool, now that I have it. I find myself adding to it.
Excellent! Often referred to it to find useful articles on common topics for my own benefit and for teaching medical students.
I loved having a portable resource for all of my rotations – I wish someone had thought of this sooner!
The introduction of the USB Syllabus was a fantastic idea that should definitely be perpetuated.
Discussion
In this pilot study of internal medicine house officers who were given a USB syllabus with pertinent research articles, it appears as if the intervention may have stimulated trainees to read more original articles. In addition to reading more landmark studies that had been chosen by senior clinicians because of their clinical relevance, house officers developed a newfound appreciation of the educational value of original research articles.
All respondents were using the USB syllabus making it, percentage-wise, a highly accepted electronic medical education resource, comparing favorably to the use of electronic educational resources reported in the literature. Kabrhel and colleagues reported on an online collection of emergency medicine literature for their trainees and showed that one month after its installation, 34% of residents were actively using the web-based electronic syllabus (6). Using a similar approach, Tannery et al. reported on an educational website developed as an adjunct education resource for a community and ambulatory medicine clerkship that included links to influential full-text journal articles and web resources (7). They found that 69% of the students were using this electronic resource regularly. In contrast to these web-based educational interventions, the USB drive has several advantages: (a) favorite articles can be saved and read even without internet and server access, (b) the collection can be expanded and edited by the individual learner, and (c) other clinically relevant materials and information can be saved on it.
The cohort of house officers in our study was comfortable with and frequently used electronic resources for their learning needs. This is in line with the reports by others and speaks to the technological sophistication of contemporary trainees (8, 9). Lai studied reading habits in primary care internal medicine residents and noted that electronic materials were the most commonly used educational media: 98% reported using resources like ‘UpToDate’ regularly (8).
Resources like ‘UpToDate’ or eMedicine, while wonderfully valuable and superb for finding answers or guidance efficiently, do not involve critical appraisal and do not challenge house officers to extract the relevant clinical messages. Studies have shown that trainees primarily go to these authoritative resources, rather than original research articles, to answer their clinical questions (8, 10) and that time constraint is the primary obstacle related to evidence-based practice (11, 12). The USB syllabus described herein may have affected the concerns about efficiency, possibly because of its organization and the effort that went into article selection. If these factors were responsible for the residents' increased reading of the primary medical literature, the syllabus has realized its primary goal.
The USB syllabus was organized by specialty around common clinical questions in an easy to use format. The prepared format also provided trainees with an information management framework that allowed them to systematically keep track of their reading and learning. The house officers in our study felt that their time using the USB syllabus was well spent. Developing effective reading habits during residency may very well continue throughout these physicians' professional lives. The USB syllabus has the potential to have a positive influence on their approach to lifelong learning and information management (3).
Limitations
Several limitations of this study should be considered. First, this study relied exclusively on self-report and we were unable to actually observe the trainees' reading practices and patterns. Second, our study design was observational and thus the findings in this paper represent associations rather than causality. As all residents received the USB syllabus, we cannot test the educational effect against a control group. While some comments to the investigators during the study period make us believe that the increased reading of original research articles was because of the USB syllabus, we cannot be certain that other stimuli were not also influential. It is not possible to differentiate the effect of nine months of clinical training (study period) on reading habits of the housestaff from the educational impact of the USB syllabus. The curricula of the PGY-1, PGY-2, and PGY-3 year in the residency program at JHBMC differ significantly (PGY – Post-graduate year). The PGY-1 year in general has the longest working hours and least amount of elective time. In the PGY-2 and more so in the PGY-3 year the elective time increases and average working hours decrease. This difference in available time to read literature could explain why the post-intervention PGY-1 group was found to read fewer articles than the pre-intervention PGY-2 group (the baseline reading assessment was performed three months after the start of the academic year). Third, reading more original medical literature by itself may not necessarily translate into practicing evidence-based medicine. Windish and colleagues have found that some residents lack knowledge in biostatistics that may be needed to interpret published clinical research (13). Although residents felt that the USB syllabus helped them in taking care of their patients, this study design did not allow us to test such hypotheses and we do not know if this resource translated into improved patient care and patient outcomes. Fourth, it is unclear how the USB syllabus would compare to other electronic resources, e.g., web-based only formats. Fifth, this pilot study was performed at an academic medical center, where the majority of residents reported planning a scientific career and the majority of graduates pursue academic careers. Thus, the external validity and generalizibility of our findings to other residency programs remains unclear. Finally, a limitation of the USB syllabus is that a structured mechanism to update the most recent best evidence is not included. However, one could envision a system of a web-based syllabus that includes regular updates combined with a personalized USB syllabus handed to house officers at the beginning of their training to capitalize on the benefit of each technology.
Conclusions
In conclusion, this USB drive-based electronic syllabus seems to have had a positive effect on resident education. Importantly, this intervention eliminated the haphazard way in which trainees are exposed to the medical education. All three authors can recall points in their training when they were thrilled that they had been given a particular article by an attending physician and even felt somewhat covetous when seeing colleagues being given articles by role models. Additional research may help to more accurately quantify the effect of a USB syllabus on residents' reading practices and patient care, or how this intervention might influence the acquisition of the six ACGME core competencies. Creative solutions using computers and technology will allow us to more effectively educate the physicians of tomorrow in innovative ways.
Acknowledgements
The authors are indebted to Dr. Hellmann, Dr. Ziegelstein, Dr. Wolfe, and Dr. Christmas for providing the USB drives and to Dr. Rastegar, Dr. Lane, Dr. Wung, Dr. Murthy, and Ms. Cheri Smith for their involvement in this project. Dr. Wright is an Arnold P. Gold Foundation Associate Professor of Medicine. Dr. Wright receives support as a Miller-Coulson Family Scholars through the Johns Hopkins Center for Innovative Medicine.
Conflict of interest and funding
The authors have not received any funding or benefits from industry to conduct this study.
References
- 1.UpToDate. 2009. Available from: http://www.uptodate.com/home/index.html [cited 15 September 2009].
- 2.eMedicine. 2009. Available from: http://emedicine.medscape.com [cited 15 September 2009].
- 3.Hojat M, Veloski J, Nasca TJ, Erdmann JB, Gonnella JS. Assessing physicians' orientation toward lifelong learning. J Gen Intern Med. 2006;21:931–6. doi: 10.1111/j.1525-1497.2006.00500.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Toedter LJ, Thompson LL, Rohatgi C. Training surgeons to do evidence-based surgery: a collaborative approach. J Am Coll Surg. 2004;199:293–9. doi: 10.1016/j.jamcollsurg.2004.04.006. [DOI] [PubMed] [Google Scholar]
- 5.Epling J, Smucny J, Patil A, Tudiver F. Teaching evidence-based medicine skills through a residency-developed guideline. Fam Med. 2002;34:646–8. [PubMed] [Google Scholar]
- 6.Kabrhel C, Liu S, Takayesu JK, Thomas SH. Creation of an online collection of emergency medicine literature. Acad Emerg Med. 2005;12:173–5. doi: 10.1197/j.aem.2004.09.018. [DOI] [PubMed] [Google Scholar]
- 7.Tannery NH, Foust JE, Gregg AL, Hartman LM, Kuller AB, Worona P, et al. Use of web-based library resources by medical students in community and ambulatory settings. J Med Libr Assoc. 2002;90:305–9. [PMC free article] [PubMed] [Google Scholar]
- 8.Lai CJ, Aagaard E, Brandenburg S, Nadkarni M, Wei HG, Baron R. Brief report: multiprogram evaluation of reading habits of primary care internal medicine residents on ambulatory rotations. J Gen Intern Med. 2006;21:486–9. doi: 10.1111/j.1525-1497.2006.00432.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Jao CS, Brint SU, Hier DB. Making the neurology clerkship more effective: can e-textbook facilitate learning? Neurol Res. 2005;27:762–7. doi: 10.1179/016164105X35639. [DOI] [PubMed] [Google Scholar]
- 10.McCord G, Smucker WD, Selius BA, Hannan S, Davidson E, Schrop SL, et al. Answering questions at the point of care: do residents practice EBM or manage information sources? Acad Med. 2007;82:298–303. doi: 10.1097/ACM.0b013e3180307fed. [DOI] [PubMed] [Google Scholar]
- 11.Bhandari M, Montori V, Devereaux PJ, Dosanjh S, Sprague S, Guyatt GH. Challenges to the practice of evidence-based medicine during residents' surgical training: a qualitative study using grounded theory. Acad Med. 2003;78:1183–90. doi: 10.1097/00001888-200311000-00022. [DOI] [PubMed] [Google Scholar]
- 12.Green ML, Ruff TR. Why do residents fail to answer their clinical questions? A qualitative study of barriers to practicing evidence-based medicine. Acad Med. 2005;80:176–82. doi: 10.1097/00001888-200502000-00016. [DOI] [PubMed] [Google Scholar]
- 13.Windish DM, Huot SJ, Green ML. Medicine residents' understanding of the biostatistics and results in the medical literature. JAMA. 2007;298:1010–22. doi: 10.1001/jama.298.9.1010. [DOI] [PubMed] [Google Scholar]