A two-year experience teaching computer literacy to first-year medical students using skill-based cohorts

Abstract

Because it is widely accepted that providing information online will play a major role in both the teaching and practice of medicine in the near future, a short formal course of instruction in computer skills was proposed for the incoming class of students entering medical school at the State University of New York at Stony Brook. The syllabus was developed on the basis of a set of expected outcomes, which was accepted by the dean of medicine and the curriculum committee for classes beginning in the fall of 1997. Prior to their arrival, students were asked to complete a self-assessment survey designed to elucidate their initial skill base; the returned surveys showed students to have computer skills ranging from complete novice to that of a systems engineer. The classes were taught during the first three weeks of the semester to groups of students separated on the basis of their knowledge of and comfort with computers. Areas covered included computer basics, e-mail management, MEDLINE, and Internet search tools. Each student received seven hours of hands-on training followed by a test. The syllabus and emphasis of the classes were tailored to the initial skill base but the final test was given at the same level to all students. Student participation, test scores, and course evaluations indicated that this noncredit program was successful in achieving an acceptable level of comfort in using a computer for almost all of the student body.

INTRODUCTION

The 1984 report of the Association of American Medical Colleges (AAMC), “Physicians for the Twenty-first Century” [1], recommended that “a general professional education should prepare medical students to learn throughout their professional lives rather than simply master current information and techniques.” This conclusion was based largely on anticipated advances in biomedical knowledge and the increasing pace of introduction of new technology into medical practice. The resulting paradigm shift to lifelong learning has meant that information management and retrieval skills have become indispensable. The computer is the primary tool with which this change can be effected. Electronic mail, computer-based testing, computer-aided instruction, and digital textbooks and images, along with valuable resources on the Internet, are now essential learning and coping tools, even for first-year students.

Although the importance of computer literacy in today's rapidly changing environment cannot be denied, it has been reported that medical education has been slow to incorporate such skills into the curriculum [2, 3]. Some of the constraints cited include deficiency of funds, scarcity of space, absence of faculty who believe that a computer course is important in the education of medical students, and lack of a long-range plan to incorporate computers into medical student education [4]. Nevertheless, as Florance et al. write, “because computers are a widely deployed and increasingly important medical technology, learning to use them to manage knowledge effectively is a core competency for modern medical practice” [5]. Medical students who have not acquired basic computer information technology skills by the third year of undergraduate training (referring in this case to the British curriculum) are unlikely to do so in the final hospital-based years [6].

Clearly, medical students need to acquire computer and information management skills at the beginning of their medical education, an assertion supported by a recent two-year survey at the University of Illinois College of Medicine at Rockford [7]. Although the 1984 AAMC report [8] recommends that informational sciences and computer technology be incorporated into the curriculum, a search of the MEDLINE and Library Literature databases reveals that little has been written regarding formal computer instruction programs for first-year medical students. A 1996 article by Hannigan and Edwards records such a program at the Texas A & M College of Medicine, although, as an elective, only ten students initially enrolled [9]. Other universities are known to include computer literacy instruction, although the data exists mostly in anecdotal form through e-mail discussion lists and other informal channels. This paper describes the development and implementation of a formal and intensive course of introduction to the use of computers as a tool for managing information.

BACKGROUND

At the Health Sciences Center of the State University of New York at Stony Brook (SUSB), an informal program in medical informatics had provided some support for the small scale use of computers as an adjunct to the medical school curriculum and a low level of support with computer applications for students. Initial assistance to first-year medical students primarily focused on a one-time demonstration of available electronic resources. Although this demonstration might have been sufficient for students possessing advanced computer skills, other students reported that they derived little benefit. In general, students deficient in the use of computers were forced to learn what they could in a piecemeal manner from whatever resource was available (other students, individual faculty, librarians, etc.). This approach did not address consistency or quality of computer instruction for all students.

Although there was a general appreciation that a more intensive and formal introduction to computer technology was needed, the absence of facilities precluded significant progress on this issue. By the spring of 1997, however, the advent of a reengineered health sciences library, including the creation of an education services department and a newly constructed electronic classroom, provided an infrastructure suited to hands-on computer instruction. This situation, combined with the formal establishment of a Department of Medical Informatics and cooperation of the school administration and curriculum committee, led to a willingness to create a formal course. Liaison Committee on Medical Education (LCME) suggestions for curricular reform and the widespread, vocalized dissatisfaction of the medical student body were additional catalysts for changing the way computer and information management training was to be provided to first-year medical students.

PROGRAM DESIGN AND CURRICULUM

Finding time for several extra hours of instruction in the medical school curriculum was not easy. However, with support from the dean of the School of Medicine and the associate dean for academic affairs, the solution was fairly simple. Basic Life Support (BLS), a required course, accommodated only a portion of the class at one time, thus allowing remaining students to be incorporated into the computer literacy sessions.

The course content was derived by combining the components of a basic computer literacy course and an introduction to finding medical information on the Internet as well as the campus intranet. Several months prior to the arrival of the class of 2001, the dean of medicine and a curriculum subcommittee met and the course was introduced with a set of concrete goals that encompassed understanding basic concepts as well as the ability to perform specific tasks (Table 1). The theme throughout was learning to use the power of the modern multitasking operating system and the graphical user interface to store, combine, and manipulate data derived from multiple sources.

Table 1.

Expected outcome of computer literacy course

The curriculum for the two sessions, with a general progression from beginning to advanced, is shown in Table 2. The second session culminated with a test designed to bring together the various elements of the course in a simulated real world situation. The test itself was sent to an electronic mailing list set up for the class—thus requiring students to have subscribed successfully to the list. The general requirements of the test were as follows:

Table 2.

Computer literacy syllabus

▪ Find an English-language review article published within the last five years. Note: a different topic was used for each class, chosen in each case with restricted criteria that would lead to a unique citation (requires use of MEDLINE).

▪ Determine whether the library owns the source title (requires use of the online catalog).

▪ Locate biographical data on the lead author (requires searching the Internet).

▪ Annotate the MEDLINE citation with the biographical data, indicate whether the library owns the journal, and format the document (requires bringing data into a word processor).

▪ E-mail the document to the instructors as an attachment (requires attaching files to e-mail).

▪ In the body of the e-mail indicate thoughts on the test and the sessions in general.

This curriculum, as developed, addresses the first of three computer-training approaches postulated by Koschman, learning about computers. It also creates a foundation for the optimal approach of learning with computers. This structure requires students to use computers on a daily basis to support their curricular activities [10].

PROGRAM IMPLEMENTATION

Based on casual observation of previous years' students, the entering class was expected to demonstrate computer skills ranging from complete novice to expert. There seemed to be a consensus that teaching a hands-on class to such a mixed group was almost impossible and because the schedule dictated the sessions be taught to three sections, a survey was used to generate the data needed to separate the neophytes from the more accomplished (Appendix). The self-assessment survey used a simple scale for the students' own feelings about their level of mastery and a more factual tally of specific skills or points of knowledge. The survey was mailed to all prospective students, the majority of whom returned it by mail; most of the remaining students handed in their completed surveys at the initial orientation session. Survey results allowed respondents to be assigned to one of three groups: novice, intermediate, or advanced. Although some students might have been assigned to a group inappropriately, the survey was sufficiently detailed to ensure that computer-literate students and novices were not combined, which was adequate for the classes. With the cooperation of the Office of Academic Affairs, the scheduling of the computer classes and BLS classes was coordinated using this skill grouping.

The classes were taught in the electronic classroom of the library's Barry S. Coller Learning Center. This facility contained thirty-one personal computers, including the instructor's workstation. Students were able to follow and view the instructor's screen via display monitors situated between workstations. Content for each class was delivered through a combination of didactic, demonstration, and hands-on instruction. Aside from the lead instructor, there was always one, if not two, additional faculty members available to aid students.

The initial medical school orientation included an introduction to the course, course directors (informatics and library faculty), and other key staff, and informed students of their group assignment. Additionally, students were told that while session attendance was optional they would be expected to take the test, which they would receive by e-mail, at the same time as the rest of their group. Only one student reported that she was able to figure out how to use all the software without instruction and took the test without attending the classes.

The division of students into skill-based groups obviated the problem of combining students with diverse skills, but presented the challenge of adapting the syllabus to three skill levels. With the advanced group, for areas where proficiency had already been indicated via the survey, most of session one was spent on an overview of the particulars of the campus network, mail servers, and classroom environment. With this knowledge, some members of the class would be able to serve as a resource for others with less computer acumen. Similarly, advanced Web search engine techniques were emphasized instead of instruction on the basics of the browser software. The content in session two was taught identically to each group. For both sessions, the goal was to ensure that a minimal level of competence for the areas as described in the expected outcomes had been achieved for all students.

One change made between the first and second year of this course was to reverse the order of the groups and teach the advanced students first. The rationale was that novice students would be less likely to think the computer session was not important enough to interfere with the preparation for their first test in anatomy or cells. Also, students who found a session too advanced could easily repeat it. Another modification was required by the introduction of the School of Medicine's new curriculum database, which students used to keep demographic information current, access their grades, and enter course evaluations.

Somewhat surprisingly, the students took longer than expected to complete the exam, with many of them taking up to an hour. Two factors seemed to account for this situation. For many, this test was their first time using MEDLINE in earnest, and, even though the test came soon after instruction, they were still inexperienced in constructing appropriate search strategies. They also found that conducting Internet searches for specific information (the biographical details of the author of the citation) was quite different from just browsing the Web. Another possible factor might have been the assumption by some students that the exam would be an easy exercise, so that they were not always attentive to the instruction. For example, some were observed playing with their e-mail during instruction on MEDLINE. In any case, students were given as much time as needed to finish the exam and the instructors were available to clarify any questions.

The original intention was to score the tests as simply completed or not. However, when the tests were received, there was enough variation in the results that a subjective score could be applied. The first year this course was taught, a scale of 0 (failure) to 5 (excellence) was awarded. This scale was adjusted to 0 to 6 for the second year to reflect the effort of those exceeding general expectations.

OUTCOMES ASSESSMENT

The most objective measure of effectiveness for the seven-hour course was the distribution of scores obtained in the graded tests. Of the 101 students in the class of 2001, only seven failed to complete the class and take the test, three of whom were advanced in their knowledge of computers and likely felt they had no need of this exercise. The most important finding was that sixty-eight of the students received a score of 5 or 4 and only eleven received a score of 2 or less. At least one of the 0 scores was awarded to a student in the advanced group who failed to attach his answer to the e-mailed response. The one student who took the test without coming to class achieved a score of 5. There were 103 entering students in the class of 2002, whose overall performance was better than their predecessors. From this group, 89% received a score of 4, 5, or 6, and there were no failures. The overall distribution of these scores is shown in Figure 1. In general, students' success rate correlated positively with entering skill level as measured by the survey. Additionally, the data showed that the great majority of students achieved a satisfactory level of skill, indicating that at least some of the initial diversity was ameliorated.

Figure 1.

Distribution of scores awarded for the test

The comments from students ran the gamut from those who felt they needed more instruction (“I do not feel as if seven hours of instruction is enough for me due to my lack of computer knowledge.”) to those who would rather have been supplied with a few hand-outs and not have been given a class at all (“My suggestions? Hand out a book of tips and tricks for the important programs, and make a computer class optional for those who feel they need it.”). Most comments, however, indicated an appreciation that the school had scheduled time for these classes.

The students were also requested to complete a one-page formal evaluation at the end of the course, which used a Likert scale (strongly agree, agree, undecided, disagree, strongly disagree) and addressed seven areas. In the two years this course was taught, 173 of 204 possible evaluations were returned, and the quantitative results from year one to year two were insignificantly different. Cumulative totals indicating either strongly agree or agree for the areas evaluated included 90% stated that they had improved their knowledge and skill level; 66% felt that content was delivered at the appropriate level; 84% believed the presentation of material was effective; and 66% would recommend this course to others.

Collectively, these results show the course to have been both well received by students and worthwhile from an educational viewpoint. They allowed faculty to be confident that the majority of the students began their first year able to utilize the information resources available to them.

PROGRAM EVALUATION AND FUTURE MODIFICATIONS

In the fall of 1998, this minicourse was reviewed for the first time by the School of Medicine's curriculum committee. The overall assessment was laudatory but did include three recommendations for future consideration:

▪ move some of the course to orientation week, thereby avoiding the proximity of the computer literacy to the first exams in anatomy or cells

▪ look for, or develop, online tutorials or self-study modules for those students who wish to learn on their own

▪ in grouping students by ability above the novice level, use an instrument to test their sophistication so students will have a more objective idea of their own capabilities and their areas of need

Although the course as described was successful, it cannot be repeated without adjustment. Besides incorporating curriculum committee suggestions, continued changes in the use of information technology in the curriculum and in school administration will necessitate changes in the syllabus. For example, the acquisition of a new CD-ROM server and software, such as Netter's Clinical Atlas and Doctor's Dilemma, have created a new class of computer-based resources that need introduction. Fortunately, the increasing familiarity of the incoming students with the basics of computing will make time available for these new tasks.

While this course is optional and will remain so, maintaining its formal inclusion in the students' first-year medical school schedule is important. This scheduling is in accord with studies that show time must be set aside for students to learn information management skills [11, 12]. It also aligns with what 50% of graduate medical students themselves state, that the amount of time devoted to instruction in the use of computers is inadequate [13].

Notwithstanding the overall success of the class, there were a small number of students who still seemed to have difficulties at the end of the course. Partly to provide a resource for those students, a few of their more advanced colleagues were recruited to act as a consulting team. This team fielded many “how-to” questions from classmates in the course of the year and, toward the end of the year, offered a “refresher class” as a device to identify those students still having difficulties. No student from either incoming class came forward stating the need for such remediation. By the end of their first year, all students presumably felt that their skills were at least sufficient for them to handle the courses that required heavy computer use. Most significantly, these courses included Medicine in Contemporary Society that employed a Web-based forum and various pathology courses that used Web-based image collections.

For the future, the possibility of broadening the role of formal computer instruction for students beyond their initial year in medical school must be considered. As students enter clinical rotations, their information needs become increasingly sophisticated. This situation will require an expanded use of computer skills and knowledge for graduates to meet the realities of a modern medical practice with its new emphasis on using evidence-based medicine in clinical decision making, improving patient education, and monitoring quality of care [14]. Areas where competence is necessary at this stage of medical education include citation management software, databases in general, and presentation software. Incorporating these topics into the third and fourth years would capitalize on what Rankin calls the “teachable moment,” which invariably occurs with curriculum integration [15].

Finally, the needs of other groups in the medical school should be considered. Most notably, these groups include interns and residents, who are recruited from elsewhere, and, like the students, have a range of computer skills when they arrive. Individual departments at SUSB have made ad hoc arrangements for similar classes, but there is no formal school-wide or hospital-wide program because time constraints on these groups are formidable.

CONCLUSION

Medical schools are in the process of preparing students for a future practice that is increasingly dependent on access to and integration of information available primarily online. Medicine has always been an “information-intensive” occupation, and the penetration of information technology into practice and education is generally welcomed. Nonetheless, institutional provision of opportunities and support for medical students to acquire the necessary technical skills has been comparatively slow and inconsistent. In addition to the constraints described in the Introduction, schools may also assume that entering medical students are necessarily proficient in the use of computers. SUSB's experience is that this case has not uniformly been true. Thus, for the immediate future, it is incumbent upon the school to provide instruction ensuring all students begin their studies with an adequate skill level and a sufficient core of knowledge that will enable them to access the institution's computer-based resources and develop the habits of continuing acquisition of new knowledge.

Implementing a formal computer literacy course with stated objectives and measurable outcomes for first-year medical students is one way to instill a minimal level of competence and to target those students who are in need of further instruction. Some educators believe that if training of this kind is not offered, the demand for curriculum reform will, in any case, be student driven [16]. For today's incoming medical students, the computer as an instrument will be just as indispensable as the microscope was for their counterparts a decade ago. How well students learn to use this instrument will profoundly influence their effectiveness as tomorrow's practitioners.

APPENDIX

University Hospital and Medical Center, Stony Brook, School of Medicine Computer literacy survey for incoming students (sent to each member of the incoming medical school class)

University Hospital and Medical Center, Stony Brook, School of Medicine Computer literacy survey for incoming students (sent to each member of the incoming medical school class) Please take a moment to complete this survey and mail it back to us before you come to school. It will help us place you in the appropriate group for orientation classes in computer skills.

Please take a moment to complete this survey and mail it back to us before you come to school. It will help us place you in the appropriate group for orientation classes in computer skills.