Online journals' impact on the citation patterns of medical faculty

Abstract

Purpose: The purpose was to determine the impact of online journals on the citation patterns of medical faculty. This study looked at whether researchers were more likely to limit the resources they consulted and cited to those journals available online rather than those only in print.

Setting: Faculty publications from the college of medicine at a large urban university were examined for this study. The faculty publications from a regional medical college of the same university were also examined in the study. The number of online journals available for faculty, staff, and students at this institution has increased from an initial core of 15 online journals in 1998 to over 11,000 online journals in 2004.

Methodology: Searches by author affiliation were performed in the Web of Science to find all articles written by faculty members in the college of medicine at the selected institution. Searches were conducted for the following years: 1993, 1996, 1999, and 2002. Cited references from each faculty-authored article were recorded, and the corresponding cited journals were coded into four categories based on their availability at the institution in this study: print only, print and online, online only, and not owned. Results were analyzed using SPSS.

Results: The number of journals cited per year continued to increase from 1993 to 2002. The results did not indicate that researchers were more likely to cite online journals or were less likely to cite journals only in print. At the regional location where the number of print-only journals was minimal, use of the print-only journals did decrease in 2002, although not significantly.

Conclusion/Discussion: It is possible that electronic access to information (i.e., online databases) has had a positive impact on the number of articles faculty will cite. Results of this study suggest, at this point, that faculty are still accessing the print-only collection, at least for research purposes, and are therefore not sacrificing quality for convenience.

INTRODUCTION

Studies have documented that the introduction of online journals has impacted the use of the library and, more specifically, the use of the print journal collection. Beginning with the introduction of full text on CD-ROM, Bane and Pessah and Venturella noted a decline in the use of the print collection [1, 2]. This trend continued with the availability of remote access to full-text journals online [3–5]. Results of a study by De Groote and Dorsch at an academic health sciences library showed print journal usage decreased significantly following the introduction of online journals, regardless of whether a journal was available only in print or both online and in print [6]. De Groote and Dorsch observed this pattern even for journals on the Abridged Index Medicus (AIM) list, considered a set of core medical journals, and suggested that the users at the institution they studied have been compromising quality for convenience when selecting journal articles to read and use. In other words, they suggested that users were using the online collection because it was convenient to access without entering the library. Users were using the print collection less often, even when the journals were high quality, thus choosing convenience over quality.

Changes in faculty use patterns of the library have also been observed. Curtis, Weller, and Hurd—who examined faculty in the health sciences, sciences, and technology—reported faculty preferred accessing electronic databases from their offices rather than going to the library [7]. One study by Brennen, Hurd, Blecic, and Weller examined early adopters of the online journals [8] and found that research practices have changed as a result of online databases and online journals. Online databases allow more timely literature searching, provide quick access to a greater number of databases and online journals from remote locations, and require fewer trips to the library than print collections. Another study by De Groote and Dorsch examined faculty use of online resources. In this study, 18% of the faculty reported never entering the library to access online resources. In the same study, 85% of the faculty did report accessing online databases for research purposes [9].

Faculty's journal reading habits have also been examined. De Groote and Dorsch observed 79% of faculty printed out articles from online journals at their office or home and 54% read articles in their personal journals, although 61% photocopied print journals from the library [9]. Smith noted acceptance of online journals by faculty [10]. In a 2003 study examining the reading habits of faculty, Smith found that, at least once a week, 91% of faculty read at least one article from personal print subscriptions, 73% read library print subscriptions, and 67% read library online subscriptions. Personal online subscriptions were used by only 29% of faculty on a weekly basis. Seventy-four percent of faculty reported reading at least 1 online journal article every week. A 2002 study by Sathe, Grady, and Giuse, examining the effect of journal format on the research process, found that fellows, medical students, and residents preferred online journals, while faculty preferred to use print journals [11]. Sathe, Grady, and Giuse noted the reasons for using online journals versus print journals also varied. Reasons for patrons using the online journals included browsing journals (39%), checking references (41%), printing articles (58%), and reading articles (16%). The main reason for accessing journals in print included browsing journals (72%), checking references (22%), photocopying articles (36%), reading articles (20%), and reading tables of contents (32%). The above findings contrasted with a 1997 study by Curtis, Weller, and Hurd examining faculty reading habits prior to online journals [7]. Curtis, Weller, and Hurd reported 78% of the health sciences faculty relied on personal subscriptions as sources of journal articles and 85% went to the library to photocopy articles. These differences in numbers might reflect the rapid change in retrieving journal articles due to online access.

Other studies also offer more data that the availability of journals online are having an impact, such as online journals affecting the impact factor. “The journal impact factor is a measure of the frequency with which the ‘average article’ in a journal has been cited in a particular year” [12]. In a study examining the impact factor of biomedical scientific journals, Curti, Pistotti, Gabutti, and Klersy observed that journals had a significant increase in their impact factor when they had an online table of contents, online abstracts, and online full text [13]. Another study by Morse examined the use of the print collection compared to the use of the online collection. Although the overall use of the online collection was much greater than the overall use of the print collection, the percentage “of usage on the most popular titles was almost identical for the print and electronic lists. In both cases just 20% of the titles accounted for nearly 60% of total usage. Conversely, the bottom 40% of both ranked lists accounted for only 9% of total usage” [4].

Two important factors appear very clear. First, the use of the print collection is decreasing, even if the journal is not available online. Second, it appears that faculty use of the library is changing with the utilization of online resources. This study examines the impact of online journals on the citation patterns of medical faculty. Are researchers citing journals available online more than journals available only in print, since the introduction of online journals? The purpose of this study is to determine if research faculty are limiting their use of the library collection to those journals available online.

METHODOLOGY

This study compared citation patterns of medical faculty over time to determine if the advent of online journals affected their citation patterns in the articles that they published. The authors hypothesized that researchers were utilizing online journals and avoiding those journals only available in print, regardless of content or quality.

To test this hypothesis, a retrospective, longitudinal study was conducted. The selected study period was 1993 to 2002. The years 1993, 1996, and 1999 were examined to establish a pattern of journal citing prior to the introduction of online journals, and 2002 was examined to see if any change in citing journals occurred after the introduction of online journals.

Faculty publications from the college of medicine at a large university were examined for this study. The university used in this study was a large urban university with one urban medical campus and three remote medical campuses. Each of the medical college campuses had their own local health sciences library providing access to print journal collections. The test data for this study included faculty citations from the urban campus and one regional campus. Choosing these locations provided the opportunity to compare two locations that had an equal number of online journals but very different numbers in terms of the print journal collection. In 2002, the urban site library held more than 1,830 active print subscriptions, and the regional site library held 392 active print journal subscriptions. The number of online journals available for faculty, staff, and students at the institution increased from an initial core of 15 biomedical online journals in 1998 to over 3,000 journals by the end of 2000. Of the 3,000 online journals owned in 2000, more than half were related to health or biomedical sciences. Unrestricted campus and remote access to the online journals was available to all faculty, regardless of site.

To identify possible citation patterns, a set of faculty articles were identified and the cited references from these articles were examined. To identify faculty authors, searches by author affiliation were performed in the ISI Web of Science to find all articles written by faculty members in the college of medicine at the selected campuses. The Web of Science was used in this study because of its ability to access both the article citation and all of the references cited by the authors. Separate searches were performed for faculty authors at the urban college of medicine and the regional college of medicine. Searches were conducted for the following years: 1993, 1996, 1999, and 2002. Inclusion criteria for the test data are noted below.

• Only articles where the first or second author was affiliated with the college of medicine at the study institution were included. First authors are generally considered the primary author. Studies of authors have found that first authors contribute more than their coauthors to their publications [14]. It was assumed that the primary authors would access information primarily through their own institutions, including use of print journals, online journals, and interlibrary loan (ILL) services.

• Only those cited references with publication dates in the previous six years were retained for each year studied. This criterion was based on an earlier study by Maxfield, DiCarlo, and CiCarlo [15] that found that approximately 90% of all use of a journal issue has occurred within 5 years of the publication date. An additional year was added to the 5 years to account for the lag time between actual publication date, use of a journal issue, and time of subsequent citation. This procedure allowed for a more equitable comparison between years. For each year where original authored articles were selected (1993, 1996, 1999, 2002), the cited references for those articles would span an equal number of years. Without this limitation, an article published in 2002 would have an additional 9 years from which an article could be cited when compared to an article from 1993.

Cited references from each faculty-authored article were recorded for each of the selected years. The number of times a journal was cited in 1993, 1996, 1999, and 2002 was entered into a spreadsheet. Cited journals were coded separately for the urban and regional campuses, as each location held different print subscriptions. Cited journals were separated into four categories based on their availability at the institution in this study:

1. “Print only” included those journals that were available only in print format from 1993 to 2002. A decrease in the use of these journals in 2002 might indicate the impact of online journals (i.e., users did not access the print journals, because other journals were more conveniently accessible online).

2. “Online only” included those journals that became available online between 1999 and 2000 at the University of Illinois at Chicago (UIC), were still available online in 2002, and were not available in print from 1993 to 2002 at the specific campus locations. These journals would primarily have been obtained through ILL prior to 1999. An increase in the use of these journals might indicate users were more likely to cite a journal they might not have used as often before, because it was readily available online.

3. “Online and print” included journals that were available in print from 1993 to 2002 at the specific campus location, became available online at UIC between 1999 to 2000, and were still available online in 2002.

4. “Not owned” included those journals that were not available in print or online during the study period at the specific campus locations. It was assumed that the majority of these articles would have been obtained through ILL, but they could also have been obtained from personal journal collections or colleagues.

The majority of the journals that were made available online in 1999 to 2000 included retrospective access to 1995 to 1996. Given this access, in 2002, authors would have had approximately six years of online issues that could be cited.

This study focuses on the impact that the online journals, obtained in 1999 and 2000 at this institution, may have had on the citation patterns of faculty. A time lag exists between the publication of an article and the time the article is cited in a published manuscript. For example, if an article was published in 2000, it is reasonable to assume that it could not normally be cited until 2001. If the journals that were obtained online in 2001 were examined, there is a possibility that not enough turn-around time would have occurred between the availability of the journal articles and any possible citing in 2002. Therefore, certain journals have been excluded from this study to avoid confounding the results.

• A journal was excluded from the analysis (specific to location) if the journal title became available in an online format at UIC after 2000.

• A journal was also excluded if its status changed during the study period: for example, if a journal ceased publication during the study period or the journal began publication after the start of the study period.

RESULTS

Table 1 depicts the number of journal articles published by first and second authors from each campus during each year examined in this study. The number of publications on the urban campus decreased over the years, while the number of publications at the regional campus increased.

Table 1 Total number of journal articles published by college of medicine faculty (first or second author) at each campus by year

Urban campus

A total of 1,496 journals were cited by faculty at the urban campus. The total and mean number of times journal articles were cited each year by faculty at this location is presented in Table 2. An examination of these data showed that for all 5 categories (print-only journals, online-only journals, print-and-online journals, not-owned journals, and excluded journals), the number of cited journal articles increased from 1993 to 2002. Of the cited journals, 231 journals were available only in print, 119 journals were available only in online format at the institution, 394 journals were available in both print and online, and 311 cited journals were not available at the urban campus. The remaining 450 journals cited by urban faculty were excluded from statistical analysis, because they did not meet the study criteria. These numbers remained constant for all years of the study.

Table 2 Number of times journals were cited each year by journal status on the urban campus

A repeated measures analysis of variance (ANOVA) test was performed to examine the effects of online journals on the citing patterns of the urban faculty. The analysis did not find a statistically significant interaction between the years versus the print status. In other words, the findings suggest that, in 2002, journals in print were not less likely to be cited and journals available online were not more likely to be cited, when compared to the previous three years studied. There was a statistically significant difference (F = 3.953, P < 0.008) in the use of the journals for all formats between years examined in this study (1993, 1996, 1999, 2002). Post hoc tests were run to identify between what years the statistically significant differences in the number of journals articles cited occurred. These post hoc tests revealed a significant increase in the journals articles cited in 2002 compared to 1993, 1996, and 1999 (P < 0.05). Although a decrease in citations was noted between 1996 and 1999, it was not significant.

Regional campus

A total of 593 journals were cited by faculty at this campus. The total and mean number of times journal articles were cited by the regional faculty is presented in Table 3. An examination of this table demonstrated that overall, the number of journal articles cited increased from 1993 to 2002 in all of the journal categories. Of the cited journals, 34 journals were available in print only, 159 journals were available only in online format, 75 journals were available in both print and online, and 210 cited journals were not available at the regional campus. The remaining 114 journals cited by regional faculty were excluded from statistical analysis, because they did not meet the study criteria.

Table 3 Number of times journals were cited each year by journal status on the regional campus

The ANOVA test was performed to examine the effects of online journals on the citing patterns of faculty at this location. The analysis did not find a statistically significant interaction between the years versus the print status. There was a statistically significant difference in the use of the journals for all formats between years (F = 10.448, P < 0.000). Post hoc tests revealed a statistically significant difference between 1993 and 1996, 1999, and 2002 (P < 0.05). A nonsignificant decrease was noted in the number of citations between 1996 and 1999. Although not a significant difference, between 1999 and 2002, the use of the print-only journals decreased, while the online-only and online-and-print journals increased in use.

DISCUSSION

It was expected that, once online journals had been introduced, the use of those journals available only in print would decrease while the use of the online journals would increase. This expectation however did not prove to be the case. The number of citations from journals available only online in 2002 increased, but the number of citations from print-only journals also increased.

It is possible that not enough time has passed since the introduction of online journals to show a change in the use of journals for research by medical faculty. For example, some faculty may not readily embrace the availability of online journals and continue to travel to the library for their information needs. On the urban campus, a number of journals remained in print-only status. Faculty might have thought that enough journals remaining in print made it worth a trip to the library. Another possibility is that more turn-around time is required once an article has been published before it shows up as being cited in another publication.

A surprising outcome from this study was the finding that, in general, the number of articles cited increased from 1993 to 2002. It might be that the increased online accessibility to databases such as MEDLINE through Internet Grateful Med, PubMed, and Ovid increased the span of literature authors could review and access. Since 1999, licenses to the following major databases had been acquired at the institution in this study: Web of Science, Ovid MEDLINE, Ovid EBM Reviews, MD Consult, and multiple other databases. Databases such as Current Contents, the Cumulative Index to Nursing and Allied Health Literature (CINAHL), PsycINFO, PubMed, and Internet Grateful Med (IGM) had been available prior to 1999. The idea that the increase in citations might be due to the increased availability of online databases was supported by Brennen, Hurd, Blecic, and Weller, who noted faculty believed online databases allowed more timely literature searching by providing quick access to a greater number of articles [8]. An increase in and improvement of end-user searching might also have improved access to relevant articles.

Despite the fact that in 2002 both campuses had increased access to online journals, both also demonstrated an increase in the number of journals cited by faculty that were not available either online or in print. It is assumed that in most cases these articles would have been obtained through ILL.

At both locations, the number of articles cited in 1999 appeared to have decreased. An examination of the number of articles produced by faculty at each location (Table 1) did not reveal a decrease in faculty publications from 1996 to 1999. Although overall the number of publications increased at the regional campus, the number did not increase between 1996 and 1999. On the urban campus, the number of articles written actually decreased. This decrease would suggest that the number of articles produced at each location in the studied years did not explain this pattern. Because the numbers were small and not significant, the change might be attributed to normal fluctuations in publication patterns.

The results of this study reflected the use of the online and print collections as they were related to faculty research. This study did not reflect use of journals for other purposes such as patient care or keeping current in the field. Although studies have demonstrated that use of print journal collections is decreasing, exceptions do exist. In a study by Blecic and Robinson examining a journal exchange program, many users preferred examining a journal in print even if it was available online [16]. Another study by Blecic found that “many clinical review titles tend to have a low local citation rate but high in-house use and circulation rates, suggesting that these are being used for educational and clinical purposes but not for research” [17]. As this study did not show a significant decrease in the use of print-only journals for research, this study would suggest that continued access to print journals, especially those available only in print, would be an important part of a collection to maintain for research purposes.

To avoid confounding issues, a number of journals were excluded from the study as noted in the methodology section. Although these restrictions made the methodology sounder by providing comparable test data that was equivalent across the time period, some journals that faculty highly used were excluded.

While the average number of articles written over the time period did not increase, the average number of citations per article did increase. However, during the time period of the study, there was no significant difference in the use of the journal in the print-only, online-only, and print-and-online categories. Therefore, it did not appear, with regard to journals available through the library, that researchers were relying more heavily on online journals than on print journals.

CONCLUSIONS AND FUTURE RESEARCH

Results of this study suggest that, at this point, faculty are not sacrificing quality for convenience. It can be expected that in the future, online journals will have more time to present an impact as a result of increased user acceptance and use. It is possible that those patterns identified by studies in the late 1990s are also changing. Also, more journals continue to become available online.

The potential for information loss exists if individuals do not access resources available both in print and online. Because of the decreased use of the print journals at one location, a valid concern exists that access to valuable research may be lost if the observed trend continues. It would be interesting to know what patterns exist in other colleges such as nursing, pharmacy, and dentistry. The ever-changing world of academic libraries is in a period of great flux as more journals and databases become available online and end users continue to become more sophisticated searchers. This time of flux provides great opportunities to study changing patterns of research.