Information literacy skills of occupational therapy graduates: a survey of learning outcomes

Abstract

Objectives: The purpose of this study is to assess whether recent graduates of the Ohio State University's Occupational Therapy division are applying information-seeking skills they learned as undergraduates, and to seek their advice on ways to improve information-literacy instruction for current and future occupational therapy students.

Method: A survey was sent to a sample of graduates from 1995–2000. The results were entered into an SPSS database, and descriptive and inferential results were calculated to determine the information-seeking patterns of these recent graduates.

Results: A majority of the occupational therapy graduates who responded to the survey prefer to use information resources that are readily available to them, such as advice from their colleagues or supervisors (79%) and the Internet (69%), rather than the evidence available in the journal literature. Twenty-six percent (26%) of the graduates have searched MEDLINE or CINAHL at least once since they graduated. Formal library instruction sessions were considered useful by 42% of the graduates, and 22% of the graduates found informal contacts with librarians to be useful.

Conclusions: Librarians and occupational therapy faculty must intensify their efforts to convey the importance of applying research information to patient care and inform students of ways to access this information after they graduate. In addition to teaching searching skills for MEDLINE and CINAHL, they must provide instruction on how to assess the quality of information they find on the Internet. Other findings suggest that occupational therapy practitioners need access to information systems in the clinical setting that synthesize the research in a way that is readily applicable to patient-care issues.

INTRODUCTION

Higher education programs are increasingly being called upon to demonstrate student achievement using outcomes assessment. As an integral part of college and university academic services, academic libraries are expected to document success in teaching information literacy principles and skills [1, 2].

Since the 1980s, the faculty librarians of the Ohio State University (OSU) John A. Prior Health Sciences Library (HSL) have cooperated closely with the faculty of the OSU Division of Occupational Therapy (OT) to integrate information literacy into the undergraduate curriculum to meet the requirements in the American Council for Occupational Therapy Education (ACOTE) Standards for an Accredited Educational Program:

Standard 7.0 The Use of Research. The ability to read, understand, and be conversant with current research that has implications for practice and impact on the provision of OT services shall facilitate development of the performance criteria listed below. The student will:7.2 Know when and how to find and use informational resources including appropriate literature within and outside occupational therapy. This includes using both national and international resources [3].

The joint OT/library program also fulfills the mission of the HSL's Education Services, “to instruct library users in the skills necessary to locate and utilize information—skills that are essential for independent problem solving and lifelong learning” [4]. This program as it existed in the 1980s was described in a 1989 paper published in the American Journal of Occupational Therapy [5].

Library instruction in occupational therapy.

In the 1990s, as networked access to MEDLINE and CINAHL became available to HSL users, the library instruction offered to OT students was updated to emphasize acquiring skills in effective online-database searching instead of using printed indexes. The learning modules were revised to make clear to the students that the skills being taught were not only to help them succeed in completing requirements in a particular class, but also to help them become lifelong learners, able to access, evaluate, and use information after graduation. Information about the importance of acquiring information literacy was incorporated into the OT library instruction program.

Every year, approximately 60 juniors are admitted into the OSU OT program. In their first year, students are required to take OT 540, “Alternative Theoretical Constructs of Occupational Therapy Practice and Professional Issues.” While taking this course, students receive an introduction to services provided by the health sciences library. They also receive instruction and hands-on practice in basic CINAHL and MEDLINE searching skills. Each student is then expected to use these skills to find ten articles pertinent to an OT topic of his/her choice, summarize each article individually, write up to three paragraphs synthesizing the information, and produce a bibliography.

In the senior year, students are required to take OT 670, “Introduction to Alternative Research Methodologies Used in Occupational Therapy.” During the library session of this course, students receive instruction in advanced searching techniques for CINAHL and MEDLINE and perform an in-class exercise that helps them to understand indexing practices. They are also introduced to the principles of information literacy so that they understand that their course instructor and the librarian believe it is important for them to apply the skills they are learning after they graduate. After the library classroom instruction and a session of hands-on practice in a computer lab, each student is expected to do an extensive literature search and produce a research report on a type of disability, an area of human function, or an intervention approach, emphasizing clinical measurement or intervention efficacy.

As stated above, one desired outcome of these efforts is that graduates will gain the information-seeking skills required to find reliable information for patient care, professional activities, and lifelong learning purposes when they become practicing occupational therapists.

OUTCOMES ASSESSMENT

Library professional associations have been encouraging academic librarians to engage in outcomes assessment of user-education programs [6–8]. Although many reports of such projects have appeared in the library literature during the last two decades, most are focused on input and output measures of information literacy for students who have not yet graduated. Reports describing where graduates are finding information, what types of information they value, and how they are applying it in professional situations are few.

Two exceptions were found in the literature review for this paper. In 1991, Kennedy reported on a survey of the graduates of Eastern Pentecostal Bible College [9]. Five questions on library skills were included in this survey (which was sent out by the institution), asking whether libraries were available to and used by the graduates (49% used local libraries), whether library techniques taught by the faculty were useful to them (53% said “yes”), whether they put the techniques to practical use in their ministry (75% said “yes”), and whether these skills were useful in developing a personal library (59% said “yes”). Although the author pointed out that the terms used in the questionnaire were not well defined, he still felt that bibliographic training at the college had had some influence on graduates' information seeking and use.

A study conducted by Fox and colleagues compared the information-literacy skills of nursing school graduates before and after library instruction sessions were initiated into the curriculum [10]. They noted that graduates who completed the library instruction program in nursing school read a wider selection of professional journals and participated earlier in scholarly activities than did graduates who had not received library instruction. Forty-five percent (45%) of the practicing nurses who had received library instruction before graduation were involved in writing articles, presenting papers at conferences, or participating in research projects or quality-assurance studies. Only 10% of those who had graduated before library instruction was initiated were involved in any kind of writing projects, and many of them were job-related activities, such as writing care standards or job descriptions.

It must be noted that no statement of certainty regarding the effectiveness of library instruction can be made in the above two studies. Kells states:

Data cannot be used to demonstrate proof [of the effectiveness of instruction] because, in the absence of controlled conditions, an institution or program conducting educational or economic impact studies will have difficulty distinguishing the impact of the institution or program in question from the effect of the input variables and the external or other intervening variables. It is not possible to prove that your program alone caused the particular outcome that you may be fortunate enough to measure accurately [11].

As stated in the 1989 article on the OSU HSL library instruction program for OT students, the classroom faculty saw circumstantial evidence of the efficacy of the library sessions in their students' successful completion of term paper assignments compared to before library instruction was initiated, as well as positive comments from students themselves on evaluations at the conclusions of the courses [12]. Based on students' successful completion of similar course assignments during the 1990s, we have reason to believe that more recent efforts to teach information-seeking skills have been equally successful, though no formal studies have been undertaken. Therefore, we assume that students graduating from OSU's OT program understand the basics of effective library research. However, do they use these skills when they begin to work as occupational therapists?

Busy occupational therapists face the same pressures and time restrictions as do other health professionals who provide direct patient care. An article reviewing the literature on health care providers' information seeking behavior shows that they prefer colleagues, supervisors, and the Internet as information sources. Other common sources of information were professional meetings and continuing education courses, information in personal libraries, and journal subscriptions [13]. An article discussing fifty-nine obstacles to clinicians' use of evidence-based medicine principles points out that “practicing doctors do not have time to search multiple sites or scroll through long text… They need to pick the right resource the first time, the information in that resource needs to be readily found, and all the information must be there” [14]. The authors recommend the development of clinical information systems designed to overcome the identified obstacles.

A recent study showed that occupational therapists experience similar barriers to incorporating evidence-based information into their patient-care plans, citing lack of time to access research information, high continuing education costs, weak research analysis skills, and a perception that research conclusions may not be easily translated into useful treatment plans [15]. It was noted in this study that occupational therapists who have entered the profession more recently have better attitudes toward finding and using research information than do those who have been in the profession longer. The authors suggest that those who have been in professional education recently are more willing and better equipped to use research information.

PURPOSE

The overall goal of the survey was to gain further understanding of the “real world” with which graduates are dealing. How do they find information and how do these techniques relate to the skills they learned in their undergraduate years? Besides teaching access and use of the research journal literature, are there additional ways we can help future graduates find quality information for their professional needs? As Rabine and Cardwell point out, “Student assessment is important to libraries … because it has the potential to truly improve the quality of instruction” [16].

We constructed the survey in the appendix to answer the following questions:

• Why have the OT graduates sought information since they graduated?

• Where do the OT graduates seek the information, and what information formats do they find most helpful?

• Have the OT graduates used MEDLINE, CINAHL, or other bibliographic databases since they graduated? How successful were they at finding needed information in these databases?

• Do the OT graduates believe that they have been successful at analyzing and applying the information they found to their patient care issues or other needs?

• Do the OT graduates feel that their university experience provided sufficient instruction in lifelong learning skills? How can the OT Division and the HSL help students to develop these skills more effectively?

METHODOLOGY

The survey was sent to individuals who had graduated from 1995 to 2000. The OT department supplied address labels for the last known addresses of these graduates, totaling 341 individuals. In order to create a manageable, but representative, dataset, the simple random-sampling method and random number table in Powell's book [17] were used to select a sample of 223 graduates.

Along with the surveys (which were numbered to permit a follow-up), the mailing included postage-paid envelopes, cover letters explaining the project, and coupons that respondents could return to the authors to enter a drawing for a free registration to an OSU OT conference in May 2002 that featured a nationally-recognized OT expert as the primary speaker. This was done in order to give the survey recipients an incentive to respond. Since the coupons required respondents to write down their names, current addresses, phone numbers, and email addresses if available, they were assured in the cover letter of the confidentiality of their responses, and the coupons were immediately separated from the surveys when returned to the authors.

The first surveys were sent in late January 2002, and the follow-up was conducted in early March 2002. Forty-one (41) of the surveys were returned as undeliverable; therefore we assume that 182 graduates received the mailing. After both the original mailing and the follow-up, we received 85 responses, for a response rate of 46.7% based on the 182 individuals who presumably were contacted.

The respondents' answers were entered into an SPSS database, and both descriptive and inferential results were computed. In many questions, graduates were permitted to respond with more than one answer; therefore answers to these questions totaled more than 85 responses. We did not ask graduates to rank their answers, but simply asked them to choose the most important ways that they find and use information.

RESULTS

The graduation years of the respondents are listed in Table 1. Except for a response from a single individual who graduated in 1995, a minimum of 11 and a maximum of 20 individuals from each year responded to the survey.

Table 1 Years of respondents' graduation

The next two questions on the survey were demographic, intended to determine types of employment. Of the 85 respondents, 79 (93%) reported working in a patient-care facility or home-health service, or as occupational therapists in elementary or secondary schools. Of the remaining 6 respondents (7%), 2 worked in health administration, 2 were in the health care industry, and 2 were not working in OT or any related field.

Graduates' information searching.

Why have the OT graduates sought information since they graduated?

As shown in Figure 1, 81 (95%) of the respondents reported seeking information for patient-care purposes, an unsurprising result given the employment demographics. In addition, 50 (59%) reported the need to gather information for in-service presentations, another common activity in patient-care settings. Scholarly purposes were reported by a minority of the respondents: 4 (5%) needed information for grant applications; 5 (6%) had undertaken research projects; 19 (22%) had made presentations at professional meetings; and 5 (6%) had sought information for graduate school projects. Eight (9%) of the respondents reported other needs for information, mostly related to teaching or engaging in business activities.

Figure 1.

Purpose for finding information

Where do the OT graduates seek information, and what information formats do they find most helpful?

Most of the graduates sought information that is easily accessed in patient-care settings or from home: 67 (79%) used the advice of colleagues and supervisors; 65 (77%) preferred continuing education courses; and 59 (69%) used the Internet (see Figure 2). In addition, 52 (61%) used their personal libraries or journal subscriptions. Only 12 respondents (15%) had used an academic health sciences library since graduation; 9 (11%) had used hospital libraries; and 7 (8%) had used public libraries. Of the 4 (5%) who answered “other” on this question, 2 had used academic libraries not specializing in health sciences, and the other 2 consult information they had kept from their educational program at OSU.

Figure 2.

Where occupational therapy graduates find information

Figure 3 provides details about the information formats preferred by graduates. Sixty-two (73%) of the respondents found the information obtained from colleagues most helpful, and 63 (74%) valued information they have learned in continuing education sessions or presentations at professional meetings. However, only 43 of the respondents (51%) reported preferring information obtained from the Internet, although 69% of the respondents had accessed the Internet to find information. Sizable minorities valued more traditional sources: 36 (42%) preferred information from books, and 27 (32%) preferred information found in journal articles.

Figure 3.

Most useful information formats

Have the OT graduates used MEDLINE, CINAHL, or other bibliographic databases since they graduated? How successful were they at finding needed information in these databases?

Twenty-two (26%) of the respondents have searched MEDLINE or CINAHL at least once since they graduated, and 21 of these (95%) found needed information by doing so. Given the pronounced preference for using colleagues and continuing education courses that the respondents to this survey expressed, and has been reported in many previous studies of health professionals' information-seeking practices, finding that 1 in 4 of the OT graduates have used high-quality bibliographic databases to seek professional information was a positive outcome. In addition, 10 respondents (12%) report using other online bibliographic databases since graduation; of these 9 felt that they found what they needed.

Do the OT graduates believe that they have been successful at analyzing and applying the information that they found to their patient care issues or other needs?

In order to assess this aspect of information literacy, we asked graduates to write a brief description of the most recent patient-care issue or research project for which they have sought information, so that they could relate their search to a specific need and their success at filling the need. The respondents reported searching for information on a wide range of subjects, such as many physical and mental disorders, assistive technologies, treatment options and their efficacy, clinical guidelines, health care law, information systems, and patient education resources. Of the 85 respondents, only 3 (4%) reported failure at analyzing and applying the information they found; 34 (40%) felt they had been very successful, and 45 (53%) were ambivalent (“just okay”). Two respondents (3%) did not answer this question.

RECOMMENDATIONS FOR IMPROVING INSTRUCTION

Do the OT graduates feel that their university experience provided sufficient instruction in lifelong learning skills? How can the OT Division and the HSL help students to develop these skills more effectively?.

Figure 4 shows the value that respondents placed on various aspects of their university experience, including library instruction. Networking with other students was cited by 53 (62%) of the respondents as one of the most useful ways to develop lifelong learning skills, and 45 respondents (53%) valued their formal classroom contacts with OT faculty. The formal library instruction sessions were considered useful by 36 (42%) of the respondents, and informal contacts with librarians were mentioned by 19 (22%) of the graduates.

Figure 4.

Ohio State University experience helpful for information skills

When asked if their total university experience had prepared them to handle lifelong learning needs, 57 respondents (67%) said yes, and 18 (21%) said no. (A few respondents checked both “Yes” and “No,” and 2 did not respond to the question.)

We asked the graduates to explain their responses, and also solicited their advice on ways to improve the program. Fourteen comments were complimentary in nature, such as:

“The skills learned in our research class regarding use of CINAHL and MEDLINE are extremely useful. I have a good understanding of how to find information I need for practice.”

“OSU provided (at the very least) a knowledgebase and means by which to find and research needed information.”

“My undergrad [experience] gave me opportunities to learn to use the databases themselves. My graduate experience is helping me learn to really apply the information.”

There were 16 respondents who commented on ways to improve the library instruction program. They suggested that we teach how to search the Internet effectively, how to assess the quality of information found on the Internet, and to emphasize some Websites that are especially useful to occupational therapists (such as the American Occupational Therapy Association's Website). Some wanted information on how to access MEDLINE and other research databases from non-university settings, such as rural areas. Several graduates recommended that we provide more experiences in finding and applying research information.

“[I suggest that you offer] more experience and direction using databases. I, personally, needed a smaller group situation for that instruction. I still feel intimidated when researching online—I waste entirely too much time searching for what I need.”

“Maybe reiterate several times to students to become familiar with this method because it will be extremely useful in their professional career. That may have helped me to keep the information more handy.”

“I had minimal computer experience, and would have benefited from increased exposure and use through a class that was taught by the medical librarians. (They were wonderful, but very busy!)”

Associated variables.

We examined more closely some factors that may contribute to the OT graduates' valuing the research literature and considering libraries to be their most important information sources. Using the SPSS software, we computed cross-tabulations between preference for evidence-based information sources and value of the information-literacy instruction that the respondents received as undergraduates. Tests of significance used the chi-square test or the Fisher's exact test (when expected cell frequencies were small). A p-value of 0.05 or lower was considered significant. Reported p-values are from chi-square unless otherwise stated.

There was a statistically significant relationship between those respondents reporting the use of MEDLINE and CINAHL since graduation and those who preferred journal articles (Χ² = 7.11, df = 1, P = .008). Likewise, those graduates who have used other databases, such as PsycINFO, ERIC, and Periodical Abstracts, were also likely to value journal articles (Χ² = 7.64, df = 1, P = .006).

Those respondents who have used either academic health sciences libraries or hospital libraries since graduation showed a preference for journal articles (Χ² = 7.87, df = 1, P = .005 for academic health sciences libraries; Χ² = 9.83, df = 1, P = .002 for hospital libraries). However, those graduates who used public libraries as one of their preferred sources of information did not show a preference for journal articles (Χ² = .036, df = 1, P = .85).

Those graduates who reported using professional libraries since graduation also reported a preference for using MEDLINE, CINAHL and other databases to access information. Those who have used health sciences college libraries also reported using MEDLINE and CINAHL (Χ² = 12.12, df = 1, P = .001) and other databases (Χ² = 6.26, df = 1, P = .012); those who have used hospital libraries have also searched MEDLINE and CINAHL (Χ² = 8.73, df = 1, P = .003) and other databases (Χ² = 4.51, df = 1, P = .034).

Given the above, it might be assumed that these correlations are a result of formal library instruction, since the importance and usefulness of databases, journal articles, and professional libraries were emphasized during the library sessions. However, there was no statistically significant correlation between those graduates who reported valuing library instruction and those who valued journal articles (Χ² = 2.83, df = 1, P = .093) or between those who received library instruction and those who have used MEDLINE or CINAHL (Χ² = 3.4, df = 1, P = .065). Similarly, there was no correlation between the graduates who reported valuing formal library instruction and those who have used health sciences college libraries (Χ² = .335, df = 1, P = .563) or hospital libraries (Χ² = 2.44, df = 1, P = .118).

Interestingly, however, those who reported a preference for using hospital libraries also felt that informal contacts with librarians (e.g., at the reference desk) during their undergraduate years were valuable (Χ² = 6.39, df = 1, P = .011). Similarly, those who valued the informal contacts with librarians also reported having used MEDLINE and CINAHL since graduation (Χ² = 5.89, df = 1, P = .015).

Relationships between those reporting the use of MEDLINE and CINAHL and those who have undertaken research projects (Fisher's exact test P = .107) or presentations at professional meetings (Χ² = 1.53, df = 1, P = .216) were not significant. Similarly, relationships between those who have used other databases and those who have undertaken research projects (Fisher's exact test P = .103) or presentations (Χ² = .036, df = 1, P = .849) were not significant. There was also no significant relationship between those who reported valuing formal library instruction and those who have prepared presentations (Χ² = 2.58, df = 1, P = .108) or those who have undertaken research projects (Fisher's exact test P = .646).

Follow-up to survey.

After assessing the preliminary results of the survey, as the librarian, I followed up with the respondents who had submitted address coupons with their surveys. Along with a letter thanking them for their cooperation, I sent a page of information on how to access PubMed, Loansome Doc, and MEDLINEplus via the Internet, and suggested that they check with their nearest college or public library to find out what types of high-quality information could be accessed either online or in person. (For instance, the Columbus Metropolitan Library [CML] makes CINAHL available to in-library users, and the Gale Group's Health and Wellness Resource Center is available via the Internet to any Ohio resident who obtains a CML library card.) They were informed of the OSU HSL's Ask-A-Librarian email reference service, and they were encouraged to contact the reference department whenever they needed suggestions on finding information for professional purposes. Respondents were also given the URL for the Cochrane Collaboration, which reviews evidence-based information that may be of use to occupational therapists. Finally, suggestions on how to assess the quality of information they find on the Internet were included on the information page.

DISCUSSION AND CONCLUSION

Although responses to the survey suggested that some graduates believe they have benefited from formal library instruction and are using the information resources that were emphasized during this instruction, many former students joining the health care workforce find themselves confronted with “real world” factors that provide little time for them to search for and make optimal use of the best evidence from the research literature. Those who valued library instruction may now be in positions that preclude doing research in the ways in which they were trained during their undergraduate years. Perhaps because their information needs are immediate and schedules do not permit time for library searches, many therapists turn to the most easily accessible sources of information. To be sure, instruction and counsel from experienced fellow OT professionals can be valuable, but as the librarian emphasized to them in their senior year, often it is likely to be based on personal observations rather than objective evidence.

Although these results could be examined further in other research projects, we are led to the conclusion that library instruction, although important for students' successful completion of their undergraduate programs, is not as influential in graduates' information-seeking perceptions and behaviors as the work environments in which they find themselves. The interesting relationship between those graduates who valued informal contacts with librarians and those who are using high-quality information resources deserves more study because it suggests that one-on-one reference services and other contacts with librarians outside the classroom setting influence graduates' information-seeking behaviors.

All OT professional education programs in the United States will become graduate programs after 2006; after that time no bachelors' degrees in occupational therapy will be awarded. Many OT departments are undergoing transformations to conform to ACOTE accreditation requirements. The results of this survey will be useful as librarians and OT faculty plan the information-literacy portion of the master's degree curriculum. At OSU, we will continue instructing the graduate students in skills needed to search CINAHL and MEDLINE, because these databases index high-quality research reports found in peer-reviewed journals. In addition to orienting the graduate students to library services and how to search the databases, however, we need to include information on searching and evaluating information on the Internet and to emphasize the importance of OTs' continuing to use high-quality databases after they leave OSU to find research information relevant to their practice. The curriculum will need to include more suggestions on how to access this information conveniently in non-university settings. These curricular enhancements may take the form of enhanced course-integrated instruction, or they may become a separate course on information literacy and research competencies offered to graduate students.

The results of this survey also emphasize the conditions under which health professionals involved in patient care must operate. Although resources such as MEDLINE and CINAHL are valuable, the research information they contain is not perceived by many occupational therapists as easily translated into practical patient-care techniques [18], nor are there widely available clinical information services for occupational therapists that translate research results into a form that can be immediately applied in a fast-paced patient-care environment [19]. Collaboration among health sciences librarians, information scientists, and professional organizations such as AOTA to develop such clinical information systems could enhance the ways that OT practitioners prefer to access and apply information. Increasing practitioners' efficiency in information searching and immediate access to the most relevant information appear to be critical to the success of health care professionals in obtaining research-based information applicable to their clinical problems and practice needs.

APPENDIX

Occupational therapy graduates survey