Materials employed by medical students preparing for subject examinations: supporting collection development

INTRODUCTION

Medical students in the United States are tested extensively throughout their four years of medical school, including both regular course examinations and standardized tests. The first standardized examination encountered by US medical students is the United States Medical Licensing Examination (USMLE) step one, produced by the National Board of Medical Educators (NBME), taken upon completion of the second year of training. Passing this examination is a significant milestone in the medical school curriculum, as most schools require students to pass before promotion to the third year [1].

The NBME produces not only the USMLE step exams but also produces standardized “subject examinations” [2]. In the United States, the American Association of Medical Colleges' (AAMC) Curriculum Management Information Tool (CurrMIT) database reports that 85% of US medical schools require their students to take subject examinations and pass them as a requirement to complete core clerkships [1]. At the University of Illinois College of Medicine located in Urbana-Champaign, students are required to pass subject examinations at the end of each core clerkship, occurring during the third and fourth years of medical school: family medicine, internal medicine, obstetrics and gynecology, surgery, psychiatry, and pediatrics. The subject examinations also serve in part to prepare students for the capstone examination to graduation and the granting of the medical doctor (MD) degree, the USMLE step two.

Standardized examinations do not end with the granting of the MD. Resident physicians undergo a third and final board examination, the USMLE step three, after completion of the first year of residency training. Passing this examination in many states will earn resident physicians their permanent medical license [1]. Even after residency, passing standardized examinations continues to be a responsibility of practicing physicians. Many US hospitals only issue privileges to “board certified” (The American Board of Medical Specialties) physicians. Internists, for example, must pass the American Board of Internal Medicine on a recurring basis to maintain board certification status. Thus, test preparation retains its importance throughout a physician's career.

TEST PREPARATION STRATEGIES

In a 1998 survey of students who took the step one exam, Thadani found that 98% had used commercial review books [3], suggesting the relevance of these materials for students. The popularity of these books is also reflected by the number and variety that are currently available. Some book series include Blueprints, High-Yield, Underground Clinical Vignettes, Board Review Series (BRS), National Medical Series (NMS), and PreTest. Zhang investigated factors affecting performance on the USMLE step one exam, including participation in commercial test preparation courses versus independent study. This research indicated that, among those students who did not enroll in the commercial test preparation course, most felt that independent study was an efficient use of their time [4]. The study also found that “personal learning habits and advice from other students were the two most important factors” affecting students' decisions on how to prepare for the exam [4]. This type of peer advice occurs at the University of Illinois College of Medicine with the yearly distribution of student-to-student advice regarding the USMLE step one examination process. The advice is given to students in a hard copy format and is available to students on a secure web page.

Students can and do use review materials throughout all four years of medical school. Many of the basic science courses in medical school curricula utilize standardized exams such as the basic science subject exam, which is administered more than 30,000 times annually [5]. Standardized exams are also used in the clinical years to ensure students progress from basic science to clinical science in preparation for graduate medical education. Many of the review resources are designed and heavily used for the specific subject areas of clinical rotations. Given the importance and extensive use of these exams in progressing through the curriculum, access to review and study materials is of utmost importance to medical students.

At the Library of the Health Sciences at the University of Illinois, librarians and staff have observed that exam review materials are among the most highly requested resources by medical students, although this subject is not covered in the library literature and the medical education literature has very little coverage.

The purpose of the current study was to document the use of exam review materials by students during their clinical rotations in the third and fourth years of medical school. This information will enhance the ability of college of medicine faculty to advise students by providing them with information about the usefulness and popularity of various study aids during clinical rotations. The results of this study may also assist librarians in collection development decisions and in recommendations to students during reference transactions. This study was composed of two segments:

1. third- and fourth-year students were surveyed regarding their exam preparation choices

2. usage patterns were documented for library-owned exam review materials

METHODS

Survey

The University of Illinois College of Medicine has four campuses. The subjects of this study were those students enrolled in the clinical program of the Urbana campus (approximately 100 students).

The authors developed a very simple instrument that was distributed over the period of eighteen months (July 2004–December 2005) to students in each clinical rotation immediately before beginning each subject examination. Institutional review board approval was obtained for this survey, and students were asked to voluntarily offer the information requested in the brief questionnaire. Students were given the survey each time they sat for a subject examination and may have been given this survey multiple times over the eighteen-month period of the research. The questions in the survey were open-ended to provide students the ability to give the researchers as much information as they were willing. Students were asked to comment on the following:

• What subject examination are you taking?

• What resource(s) did you use to prepare?

• What worked for you in preparation for the subject examination?

• What didn't work for you in your preparation?

• Any other tips you'd like to give future students?

Circulation records

The library actively collects materials in the area of exam review and preparation to meet the needs of the medical students. Since 2002, the library's collection policy has been to purchase at least two copies of each exam review book, placing one copy in the book stacks, which are available to all users and can be checked out for two weeks. Additional copies of books are placed on open reserve. The open reserve collection is limited to medical students, and the circulation period is shorter (1 week).

Circulation statistics were extracted from the online catalog system for exam review books published in the years 1995 to 2006. Books in the selected call number ranges were reviewed from both the open reserve and book stacks collections. Exam review books are typically classified in the 18.2 area (Educational Materials) under the National Library of the Medicine classification [6]. For example, in the case of pediatrics, books in the WS 18.2 range of the collection were reviewed; for surgery, books in the WO 18.2 range were reviewed.

Additional searches based on Medical Subject Headings (MeSH) were performed in the library's online catalog to create a representative sampling of exam review materials in each of the core clerkship areas, with the exception of family practice. At the time of this study, family practice was not included as students typically used information from all the other clinical areas—including medicine, pediatrics, psychiatry, surgery, and obstetrics/gynecology—to prepare for this examination. The NBME has changed this examination recently and made it more specific to family practice, so the need for student preparation using materials specific to family practice will likely be seen in future studies in this area.

Next, the circulation record for each exam review book was examined to note the total number of historical charges (the number of times a book was checked out and/or renewed) and browses (in-house uses). Circulation data were collected during January and February 2007. The data pertain to charges from January 2002 to the time of data collection (January– February 2007). The charges were tallied by subject area and by book series.

RESULTS

Ninety-five students (43%) sitting for subject examinations responded, out of a possible 222. By discipline, the responses were: 12 students responded to the family medicine preparation survey; 13 students responded to the internal medicine preparation survey; 22 students responded to the obstetrics and gynecology preparation survey; 12 students responded to the pediatric preparation survey; 18 responded to the psychiatry preparation survey; and 18 responded to the surgery preparation survey (Table 1).

Table 1 Survey results: reported use of materials by students

The students reported using a wide variety of resources and the popularity of the publishers' series varied by discipline. These data were also confirmed by library circulation results (Tables 2 and 3 online).

DISCUSSION

It is important to understand the multiple and high-stake assessment methods that take place during and after medical school to provide appropriate resources at the most useful time during the curriculum. The results of this study confirm that exam review materials are highly valued and used by medical students. Both the results of the survey and the library data collection showed that medical students used a variety of resources to prepare for the subject examinations. Students reported in questionnaires that review materials, in addition to the actual clerkship experiences, were essential in preparing for subject examinations.

It is important to note that the extensive use of the survey instrument may have become tiresome to the subjects of this study, given that some students may have answered the same questions at different subject examinations over an eighteen-month period. Therefore, their answers might have become shorter over time. Those surveys answered near the end of the project may not contain the same depth of information as those answered in the beginning of the research period.

Overall, the Appleton & Lange, High Yield, and Underground Clinical Vignettes were the most popular series, followed closely by PreTest, according to the circulation data, although these results varied by clerkship rotation. The results should be considered with caution, as the number of books in the collection and corresponding publication years varied. For example, a book published in 2000 would likely show more use than one published in 2006. Also, while the circulation data sampling technique captured the majority of review books in the collection in the scope of the study (1995–2006), students might also have been using other library materials not reflected in the current usage data sample. However, the results clearly showed that a variety of exam review materials were used by the students.

The library data do not account for books that students purchased themselves or library books that students may share with each other. It is also important to note that this study was limited to commercial publications. Some schools may develop their own unique resources for exam review that are not available commercially. Also, students may utilize commercial coaching courses, particularly for USMLE step one, two, and three examinations. However, it is not clear if commercial courses are superior to independent study. Werner and Bull found that students taking these three-to-four-week courses performed similarly to those who prepared for USMLE step one on their own [7], while Thadani found “little or no evidence of achievement of higher scores as a consequence of using commercially prepared material” versus other methods, such as student-to-student produced study materials and study materials gleaned from previous coursework [3]. These previous studies along with the results from the current study provide evidence supporting the popularity and usefulness of exam review books among medical students.

CONCLUSION

In light of the wide variety of learning styles exhibited by students, it comes as no surprise that multiple resources in a variety of formats would be employed in preparing for these high-stakes examinations. Those resources with the greatest usage may vary by clerkship area and by availability of materials.

Given the popularity of exam review materials, medical libraries supporting medical school curricula may want to provide students with access to a wide array of sources in this area to meet the needs of students with varying learning styles. Being aware of the schedule of exams, both the step exams and the subject exams, as well as the consequences of these tests is important information for medical librarians. Libraries can do much, not only in providing these materials, but in promoting them to students at the times when they are most needed.

As exam review preparation is vital to medical students but the literature is scant in this area, future reports might include studies of online exam review materials, other techniques students use to prepare, and techniques or resources that were not successful for students.