Bibliometric study of grey literature in core veterinary medical journals

Abstract

Objectives: Grey literature has been perceived by many as belonging to the primary sources of information and has become an accepted method of nonconventional communication in the sciences and medicine. Since little is known about the use and nature of grey literature in veterinary medicine, a systematic study was done to analyze and characterize the bibliographic citations appearing in twelve core veterinary journals.

Methods: Citations from 2,159 articles published in twelve core veterinary journals in 2000 were analyzed to determine the portion of citations from grey literature. Those citations were further analyzed and categorized according to the type of publication.

Results: Citation analysis yielded 55,823 citations, of which 3,564 (6.38%) were considered to be grey literature. Four veterinary specialties, internal medicine, pathology, theriogenology, and microbiology, accounted for 70% of the total number of articles. Three small-animal clinical practice journals cited about 2.5–3% grey literature, less than half that of journals with basic research orientations, where results ranged from almost 6% to approximately 10% grey literature. Nearly 90% of the grey literature appeared as conferences, government publications, and corporate organization literature.

Conclusions: The results corroborate other reported research that the incidence of grey literature is lower in medicine and biology than in some other fields, such as aeronautics and agriculture. As in other fields, use of the Internet and the Web has greatly expanded the communication process among veterinary professionals. The appearance of closed community email forums and specialized discussion groups within the veterinary profession is an example of what could become a new kind of grey literature.

INTRODUCTION

Today there are vast amounts of easily accessible information relating to animal health available to veterinarians and others. Some users of this information may not be aware of the existence of material that is not readily accessible to them. Annual reports from veterinary medical colleges, locally sponsored university extension seminars, and laboratory procedures used by the National Veterinary Services Laboratory are examples of material that is directed toward internal users only and is not typically published by mainstream sources or cited in indexes. With the introduction and success of new information technologies such as the Internet and desktop publishing software in the past decade, the character of literature with a “difficult” bibliographic identity is changing. In the past, such literature, often dubbed “grey” or “fugitive” has been perceived by many as belonging to the primary sources of information. As such, it has been viewed as a vital tool in the overall search process upon which effective decisions for research or treatments can be made. According to Auger, “Over the years, grey literature has come to constitute a section of publications ranking in importance with journals, books, serials and specifications” [1]. Among the reasons cited for this are its greater speed and flexibility of dissemination. In veterinary medicine demand by the public for up-to-date, knowledgeable treatment of family pets and matters of human and veterinary public health are two good examples that dictate veterinarians' need for the rapid dissemination of information that is provided by grey literature.

The debate over exactly what constitutes grey, or fugitive, literature resulted in a consensus definition at the third International Conference on Grey Literature held in Luxembourg in 1997. The consensus definition stated that “Grey literature is that which is produced by government, academies, business, and industries, both in print and electronic formats, but which is not controlled by commercial publishing interests and where publishing is not the primary activity of the organization” [2]. The Online Dictionary of Library and Information Science defines a commercial publisher as “a publisher in the business of producing and selling books and/or other publications for profit as opposed to a university press or the publishing arm of a scholarly society, professional organization, or other non-profit organization that operates on a cost-recovery basis” [3].

The elusiveness of grey literature, both bibliographically and in terms of physical acquisition, can be attributed to the types of materials produced by noncommercial entities such as educational centers and universities, research institutes, laboratories, professional societies, government and quasi-government organizations [4–6]. Some of this material is intended for low distribution to a limited readership, never reaching dispersion through normal trade channels. Grey literature may emanate as conference proceedings, newsletters, theses/dissertations, trade literature and house journals, government publications, working papers, committee reports, and more. Auger discussed in detail the amorphous nature of this literature, some of which becomes “open and permanent over time, whatever the intentions of the originator may have been” [7]. For some of the materials in these categories, adequate indexing and availability does exist. For example, papers of major conference proceedings are indexed along with journal articles and other literature in a variety of bibliographic databases. UMI abstracts most of the dissertations issued in the United States and makes them readily available for individual and institutional purchase. MarciveWeb DOCS, the free-access Web version of the MARCIVE database, cites documents issued by the U.S. Government Printing Office, indicates item availability in area libraries and other libraries nationwide, and, in some cases, provides a direct link to the full text of the actual document.

Williams and Croft [8] used the end-user survey of health scientists described by Alberani and colleagues [9] to speculate that “the level of grey literature in veterinary medicine is definitely above 16%,” but no systematic review has been undertaken to characterize the nature and extent of this material in the scholarly literature of the profession. Print (and increasingly online) journals are a dynamic and popular medium that is an accepted method of dispersing scholarly information. A survey by Pelzer and Leysen showed that the most common information resource used by veterinary practitioners was veterinary journals [10]. Bloom, Morris, and others assert that journals will continue to be a preferred venue for professional communication and be of value to the researcher and practitioner for some time to come [11, 12]. The objective of the present study was an empirical evaluation of the actual use of grey literature in an important component of veterinary medicine, its core journals, through analysis and characterization of bibliographic citations appearing with the articles.

METHODS

Citation analysis, an accepted method of assessing actual use of resources, was used to determine the incidence of grey literature appearing in the bibliographies of articles published in twelve core veterinary journals. Several criteria were used to select a representation of titles that could be considered core journals in veterinary medicine: (1) that the journal be held widely; (2) that the journal be of importance to veterinary professionals, as shown by the impact factor compiled by the ISI Journal Citation Reports (JCR)—Science Edition [13]; and (3) that the journal be indexed widely. Six agricultural, medical, and interdisciplinary bibliographic indexes were chosen as a measure of the amount of exposure of these journals (Appendix). These indexes were chosen for their depth of coverage of veterinary topics and because they are well known and widely available.

To serve the Iowa State University College of Veterinary Medicine, the library subscribes to an extensive collection of veterinary journals. In the year 2000, all of these titles were searched in the Online Computer Library Center (OCLC) bibliographic database for the number of institutional holdings. Print titles with over 100 holdings were noted. According to Houston's application of bibliometrics to establish veterinary science primary literature, “the number of citations to particular journals can give an indication of the importance of the journals to a field [14]. Therefore, under the category Veterinary Sciences, the JCR-impact factors for veterinary medical journals were scrutinized and considered alongside OCLC holdings. Finally, veterinary medical journal titles that merited further interest based on their OCLC holdings or JCR-impact factor were checked for extent of indexing.

Most of the journals selected as core journals in this study met all three of the criteria named in Table 1. The two exceptions were Veterinary Microbiology, which, unlike the other titles, is held by somewhat fewer than 100 libraries, and Veterinary Medicine (Edwardsville, Kan.), which had a low impact factor compared to the others. The latter title was included in this study because it is known to be a popular journal among practicing clinical veterinarians and is held by over 200 institutions in OCLC. Eleven of the twelve journals selected were in the top one-third of JCR ranking by impact factor in the veterinary sciences category, with seven of them being among the highest ranked. In JCR‘s list of 129 veterinary science journals, only 22 were listed with impact factors of more than 1.0. Six of the journals included in this study are among those twenty-two, while one more was listed with an impact factor of 0.941. The varying information needs of basic researchers versus those of clinical practitioners could influence the use of grey literature; therefore, it was important to identify representative journals from both areas.

Table 1 Core journals in veterinary medicine

The selection criteria identified three journals with a basic research orientation in a subject area, three journals that covered small-animal clinical practice, and six journals that included a mix of subjects and both basic research and clinical medicine (Table 2). The journals were, coincidentally, divided evenly between those that were published by veterinary associations (six), and those from commercial publishers (six).

Table 2 Core veterinary journal statistics for grey literature

Journal issues published during the year 2000 were analyzed for the number of articles that appeared in each issue, for the number of total citations appearing in those articles, and for the number of citations that were considered to be grey literature. In an attempt to characterize the topical nature of the citations, each article was assigned a veterinary specialty emphasis according to the listing of the American Veterinary Medical Association's (AVMA) recognized specialty boards in the 2000 AVMA directory. This listing includes anesthesiology, behavior, dentistry, dermatology, emergency/critical care, internal medicine, laboratory animal medicine, microbiology, nutrition, ophthalmology, pathology, pharmacology, preventative medicine, radiology, surgery, theriogenology, toxicology, and zoological medicine.

Citation analysis can reveal other useful information, such as the relative use of different categories of documents. Thus, grey literature citations were analyzed further by dividing them into types of publications. The categories chosen for particular relevance to veterinary medicine were conference proceedings, government publications, newsletters, corporate organization literature, foreign theses and dissertations, and U.S. master's degree theses. Miscellaneous and undetermined categories were added for citations falling outside of the preceding list. Conference and corporate organization literature issued by commercial publishers was not considered to be grey literature. A conference held under the auspices of an organization (corporate or government) was counted in the conference category rather than in the organization category. When the nature of the issuing entity or publisher was not immediately obvious, an investigation was carried out using the Web, OCLC, or local resources to verify the source. The same investigation was used for newsletters issued by organizations. The miscellaneous category included any remaining citations that could be characterized as grey literature, but which did not fit into any of the defined types (patents, personal communications, drug package instruction inserts, etc.). The last category, undetermined, was used only when all efforts failed to reveal the character of the citation. Examples of this are articles that were submitted where no evidence of publication could be found or citations that could not be verified in the selected bibliographic indexes, OCLC, or other reference sources.

The sum of these methods was intended to show empirically the extent to which grey literature appeared in the core veterinary journal literature, in which specialties it was appearing, and types that appeared. The viewpoint of the authors was decidedly North American-centric, meaning that those materials that were determined to be difficult to obtain in the United States would likely be called grey literature. Professional judgment, based on many years of providing bibliographic control and reference service, entered into every decision.

RESULTS

The twelve core veterinary journals published in the year 2000 were selected for citation analysis. The journal issues yielded a total of 2,159 articles for study. Within the bibliographies of these articles, a total of 55,823 citations were found. As expected, the journals that emphasized research over clinical practice dominated in both number of articles and number of citations. The journals that focused primarily on case studies, reporting treatments for clinical use, showed not only fewer citations but also a lower percentage of grey literature. Of the 55,823 citations found, 3,564 (6.38%) were considered to be references to grey literature.

Internal medicine was the most-represented veterinary specialty, followed by substantial representation from pathology, theriogenology, and microbiology. This group of four specialties accounted for 70% of the total number of articles with citations that were analyzed for grey literature in this study. The three small-animal clinical practice journals (Journal of the American Animal Hospital Association; Veterinary Clinics of North America, Small Animal Practice; Journal of Small Animal Practice) cited from 2.56–2.99% grey literature, about one-third that of the journals with a basic research orientation (Theriogenology, Veterinary Microbiology, Veterinary Parasitology), which ranged from 5.5% to 9.69%. Those journals that covered a mix of subjects and reported both basic and clinical medicine showed a range of grey literature citations from about 3% to about 10%, with the British journal Veterinary Record topping the list (Table 2).

A breakdown of grey literature by type of publication can be found in Table 3. Nearly 90% of the incidences of grey literature appeared as conferences, government publications, and corporate organization literature. In veterinary medicine as in many other fields, conferences play an important role, and they alone accounted for 50.1% of the grey literature citations in the core veterinary journals. These were comprised of both topical and organization events, with many being held serially over the years and many others appearing to be acquired only once.

Table 3 Number of grey literature citations by type

Government publications accounted for another 18.2% of the grey literature citations. A wide variety of government publications were encountered in this study, in no small part a result of the international nature of the authorship found in several of the journals. There are many non-U.S. publications included in the citation count at federal and national; state, provincial, and territorial (etc.); and local-government levels from the United Kingdom, Europe, Africa, Australia, Japan, and other areas of the world. From a North American perspective, the majority of these documents would have to be considered grey literature in the true sense. Others, such as USDA and CDC publications, for example, are readily identifiable and available in libraries and through United States Government Printing Office channels, or in some cases, directly from the government agencies that authored them. Furthermore, free-access, full-text versions of many documents related to veterinary medicine can now be accessed directly from a variety of government Websites, such as those of the Center for Veterinary Medicine [15] and Veterinary Services [16].

A select group of government agencies and corporate organizations encountered within the citations of the articles from the core group of veterinary journals did not meet the definition of a commercial publisher, but the literature issued by these entities, as noted above, is well-known for easy accessibility in the veterinary profession. To remain consistent with the definition of grey literature described earlier, these government agencies and corporate organizations were included in the citation count as grey literature. Table 4 presents a break-out of the government agencies and corporate organizations whose publications were considered, for the most part, to be mainstream journals in the field of veterinary medicine. These entities have services supporting document publication and dissemination. Among this group is CAB International, with a Website that touts their publishing division as “one of the world's foremost publishers of databases, books, journals, CD-ROMs and Internet producers in the applied life sciences” [17]. It is notable that the incidence of “not really grey literature in our estimation” accounted for an average of 21% of the total citations for government and corporate organizations as a group.

Table 4 Number of grey literature citations considered to be “mainstream”

Corporate organization literature comprised 20.1% of the citations counted as grey literature. The materials issued by this group are frequently intended for internal users, with no external distribution planned through commercial means. Veterinary and medical societies, pharmaceutical companies, animal science and zoological groups, research institutes, and equipment or software manufacturers accounted for many of the citations in this category.

At 7.3%, and 1.2%, respectively, theses or dissertations and newsletters showed relatively minor appearances in the core veterinary journals. Dissertations issued by academic institutions in the United States and published by UMI, a commercial publisher, were excluded from this count, while master's degree theses issued in the United States were included only because of the selective coverage by UMI. Citations such as patents, drug package-instruction inserts, personal communications, personal Websites, online databanks, software packages not available commercially, and unpublished data were counted as miscellaneous items and represented only 1.7% of the citations counted. Another fifty citations, due to the incomplete nature of the citation, were simply unable to be assigned to a category at all.

Sorting the serial literature from the desired categories proved to be a rigorous exercise and brought into question the practice of excluding all journals from grey literature, though the majority of scientific journals, even if not commercially controlled, are indexed and readily accessible. Articles from some of the foreign-language journal titles encountered in this study would surely be as difficult to obtain as would some grey literature, if they could be acquired at all. Citations were encountered from journals that were held by only a few OCLC libraries. Some, such as Boletin Medico del Hospital Infantil de Mexico, were not listed in OCLC at all.

DISCUSSION

For the first time, using a numerical approach, the use of grey literature was determined in a core set of journals that serve the veterinary profession. The overall incidence of citations that were considered to be grey literature averaged 6.38% in the group of journals. Veterinary journals were chosen as the target group for study because they are a popular medium for disseminating information within the profession. A conservative, rather than broad, approach was taken that did not go beyond traditional published literature in that the journals selected were determined to be of importance to the veterinary community. This approach may not reflect the totality of grey literature in veterinary medicine, and it could account for the fact that the quantity is less than that proposed by Williams and Croft. However, Auger states that the fields of medicine and biology experience less grey literature than do some other fields, such as aerospace or agriculture, because of less government involvement in veterinary research and also an emphasis on “open” literature. Even when government bodies are involved, he notes that the research reports in medicine and biology are often published in a commercial manner [18]. On the other hand, an argument can be made against excluding some (primarily foreign) serial publications from consideration as grey literature, as was done in this study. Some of the citations that were encountered listed serial titles that would be very difficult, if not impossible, to obtain in the United States; for example, the serials Bulletin des G.T.V. Dossiers Techniques Vétérinaires, Les Dossiers de l'Élevage, O & D V: Obiettivi e Documenti Veterinari are held by only a few libraries, which may or may not have complete holdings and which may or may not be willing to lend the desired items.

By type, the preponderance of grey literature occurred as conference proceedings (50.1%), corporate organization literature (20.1%), and government publications (18.2%). Aina states, “unlike in most other fields where journal format constitutes the principal medium for reporting research findings, the opposite is the case with GL [grey literature] where proceedings or conferences provide the medium for reporting GL” [19]. Though conference proceedings have been notoriously difficult to identify and acquire in the past, they may no longer be the shade of “grey” that they once were. According to Williams and Croft [20], adequate indexing for conferences within the veterinary profession exists with the CAB Abstracts and Veterinary Conference Proceedings databases; in the latter database, in some cases a direct link is now provided to a conference Website that displays abstracts or even the full text of conference papers [21]. An increasing number of Websites provide access to current and past proceedings for programs, abstracts, full papers, or, in some cases, entire proceedings; for example, the United States Animal Health Association has provided free full-text online access to its proceedings since 1997. Additional indexing sources for veterinary conferences include the AGRICOLA and AGRIS databases. Furthermore, a recent study by Pelzer and Pine showed that in a sampling of serial issues analyzed as monographs, nearly 70% of the conferences that were encountered in those issues were indexed in the database ProceedingsFirst [22]. In the case of major conferences that are serial in nature, bibliographic and physical access to conference proceedings is readily available through the issuing organization or from commercial publishers. Truly grey conferences remaining will include those that never publish proceedings or that only provide abstracts or selected papers to conference attendees and never make their way to an open trade channel.

As has occurred in other fields of study, use of the Web has greatly expanded the communication process among veterinary professionals [23–25], spurring the organization of resources and decreasing the “grey” qualities of the Web as a means of information dissemination. Swartz states, “In addition to researching existing printed texts and reference materials, veterinarians can take advantage of Internet sites that offer extensive information resources” [26]. Examples of this are the online searchable resources offered by the Veterinary Information Network (VIN), the AVMA's Network of Animal Health (NOAH), and, as noted above, government Websites, in which free, full-text documents are readily available. Similar circumstances exist for conferences and journals. Three of the journals reviewed in this study, Veterinary Parasitology, Veterinary Microbiology, and Theriogenology, are available as e-journals (Elsevier's ScienceDirect) from the Iowa State University Library Web page. Newsletters, which accounted for only 1.2% of citations in this study, may benefit from increased exposure via availability at major sites or by keyword searching on the Internet. Healthy Animals [27], for instance, is a quarterly publication of the Agricultural Research Service with “news and expert resources on the health and well-being of agricultural animals and fish” [28]. Other good examples of free-access newsletters with substantial content on the Web are the FDA Veterinarian [29] and the JAVMA News, a newsletter-like feature of the AVMA Website. The latter resource presents articles and news items from the News section of issues of the semimonthly Journal of the American Veterinary Medical Association. The JAVMA News section includes an archive searchable by issue from January 1, 2000, to present. These news items and topical, informative articles are freely accessible at the AVMA Website to professionals or anyone who is interested in veterinary and animal health issues [30].

The availability of state experiment-station literature in the United States is another example of a type of grey literature that has been affected by technology. The bibliographic identity and accessibility of many extension publications have been vastly improved through mounting of these materials on university Websites. Technological advances have made it possible for veterinary medical colleges, such as Iowa State University's, to collaborate with University Extension in offering information to veterinarians through distance continuing education. Telemedicine, email lists, and personal email may also be added to the burgeoning means of Internet communications.

The caveat for a portion of the Internet-related applications that have vastly improved the isolation of individual veterinary practitioners from information resources is that the chances are great that some of the new-found collaborative communication may not survive in a permanent manner, nor even be accessible to anyone but professionally qualified subscribers. The community email forums and specialized discussion groups offered by VETPLUS-L, the AVMA, and VIN are closed lists and are very popular for the great speed and flexibility with which they disseminate the type of information that characterizes grey literature. This may be likened somewhat to a telephone conversation, where a record of the conversation taking place is available to telephone company personnel, but to no one else without first getting permission. While these lists currently maintain archives, how extensively will this continue without the guidance of information professionals? What is the consequence of professional discussions that are available only to qualified subscribers? This is unprecedented “capturing” of professional quality communication without traditional mainstream dissemination to the world at large if list members are satisfied that the e-archive is sufficient for their (and the world's) historical needs.

CONCLUSION

This bibliometric study of core veterinary journals corroborates Auger's statement that the incidence of grey literature is lower in medicine and biology than in some other fields. As a primary source of information, references to grey literature were low in the core journals selected for study, indicating that mostly conventional channels of publication (books and journals) were used by veterinary researchers and clinicians for reporting the results of their work. Based on the results of this study, which examined a popular medium for dispersing scholarly information in the profession, on the whole, management of grey literature in veterinary medicine from the viewpoint of the library practitioner seems not to be a major issue. It should be recognized, however, that the appearance of grey literature citations in a library's bibliographic resources does not necessarily translate to easy accessibility or procurement of these materials.

The shifting of information resources to the Internet is likely to reduce the incidence of grey literature in veterinary medicine even further because the Internet is now accessible to nearly everyone. However, the appearance of community email forums and specialty discussion groups on the Internet is an example of a new way in which veterinary professionals share information that could create a new kind of grey literature.

APPENDIX