Trends in health sciences library and information science research: an analysis of research publications in the Bulletin of the Medical Library Association and Journal of the Medical Library Association from 1991 to 2007

Abstract

Objective:

This study analyzed trends in research activity as represented in the published research in the leading peer-reviewed professional journal for health sciences librarianship.

Methodology:

Research articles were identified from the Bulletin of the Medical Library Association and Journal of the Medical Library Association (1991–2007). Using content analysis and bibliometric techniques, data were collected for each article on the (1) subject, (2) research method, (3) analytical technique used, (4) number of authors, (5) number of citations, (6) first author affiliation, and (7) funding source. The results were compared to a previous study, covering the period 1966 to 1990, to identify changes over time.

Results:

Of the 930 articles examined, 474 (51%) were identified as research articles. Survey (n = 174, 37.1%) was the most common methodology employed, quantitative descriptive statistics (n = 298, 63.5%) the most used analytical technique, and applied topics (n = 332, 70%) the most common type of subject studied. The majority of first authors were associated with an academic health sciences library (n = 264, 55.7%). Only 27.4% (n = 130) of studies identified a funding source.

Conclusion:

This study's findings demonstrate that progress is being made in health sciences librarianship research. There is, however, room for improvement in terms of research methodologies used, proportion of applied versus theoretical research, and elimination of barriers to conducting research for practicing librarians.

Highlights

• Between 1991 and 2007, the percentage of papers published in the Bulletin of the Medical Library Association (BMLA) and Journal of the Medical Library Association (JMLA) that report research findings increased.

• A growing variety of research methods and techniques are being employed in articles published in JMLA.

• The subject of research articles is shifting from a focus on the physical operations of the library to studies of the behavior, attitudes, and opinions of those using the library.

Implications

• Applied research and descriptive analytical techniques continue to be the most commonly used; a need for more theoretical studies exists.

• There has been progress in encouraging research activities among health sciences librarians, but lack of funding remains a barrier to research.

INTRODUCTION

In its 2007 The Research Imperative: The Research Policy Statement of the Medical Library Association, the Medical Library Association (MLA) defined research as “the foundation of the profession” and “a key ingredient for professional growth” [1]. This policy statement challenged the health information profession to develop a culture where “applying and creating research-based evidence are the norm rather than the exception” [1]. This was not MLA's first call to action for a research agenda for health sciences librarians. The 1987 Strategic Plan of the association stated in goal III that “MLA is dedicated to improving health through professional excellence and leadership in research in health information science” [2]. Later that year in her Janet Doe Lecture, Erika Love, FMLA, declared that “strengthening our research component today is a matter of political survival” [3]. In 1995, MLA published its first research policy statement, with an action plan “to improve the research environment for its members” in the areas of education, research support, funding, dissemination, recognition, and measurement [4]. The 2007 Research Imperative documented significant research milestones that have been achieved since 1995 [5] and affirmed MLA's commitment to “continue to annually assess the progress made toward enhancing its research knowledge and skills of its members and the achievement of its research agenda” [1].

Does recently published research in health sciences librarianship indicate progress in the profession's research endeavors? How have research methodologies and subject areas of interest changed over time? This study explores these questions by examining recent trends in research published in the Journal of the Medical Library Association (JMLA), the association's flagship publication and the leading peer-reviewed, professional journal for health sciences librarianship, and by comparing the results to those previously reported by Dimitroff for the period 1966 to 1990 [6].

LITERATURE REVIEW

Few articles in the published literature in library and information science specifically address the state of research in health sciences librarianship. Burdick et al. surveyed members of the Midcontinental Chapter of MLA in 1988 and found that time, money, and insecurity about research skills were the constraints most frequently mentioned by respondents [7]. In a 1992 study, Dimitroff performed a thorough content analysis of articles published between 1966 and 1990 in the Bulletin of the Medical Library Association (BMLA) [6]. She identified 363 research articles out of a possible 1,218 published articles (29.8%), a figure consistent with earlier studies by Peritz [8], Feehan [9], and Nour [10] that analyzed the general library literature. Haiqi reported a lower percentage of research articles, a range of 7%–19%, in an analysis of articles published 1990 through 1992 in the BMLA and 2 other medical library periodicals [11].

Several other content analyses of the library research literature were carried out in the 1990s; none of these included journals in the field of health sciences librarianship [12–14]. Koufogiannakis et al. conducted a large-scale content analysis of articles in 91 library and information science journals published in 2001 [15]. The BMLA was among the top 10 journals in terms of the number of research articles. The authors classified 30.3% of the articles in the 91 journals as research articles; however, because they were also testing a taxonomy they had developed, they found it difficult to compare their results with those of previous studies due to the variations in subject categories and research methods. In a 2002 study, Powell et al. surveyed members of 4 major professional library associations, including MLA, about their involvement in research [16]. Results showed that higher percentages of MLA and American Society for Information Science and Technology (ASIST) members had performed research. The BMLA was the most frequently mentioned journal in which respondents had published. Eldredge offered a helpful inventory of research methods used in librarianship and informatics, and many of his cited examples were drawn from BMLA and JMLA articles [17].

The JMLA editorial team from Vanderbilt University, whose term ended in 2008, published an editor's column with their review of the progress of research in the JMLA from 2002 through 2007 [18]. They identified on average 58% of the articles as research articles, a substantially higher number than that found in previous studies. This increase was across the tenure of several editors [19].

A majority of published studies addressing research in librarianship examine the general library and information science literature. Research analyzing publications in the health sciences literature is limited. The current study seeks to expand understanding of that literature by focusing on research published in the BMLA and the JMLA, the flagship journal of health sciences librarianship. By resuming the analysis of health sciences librarians' research publication patterns where Dimitroff ended in 1990 and replicating her methodology, the study also provides an opportunity to analyze publication trends in health sciences librarianship over more than forty years [6].

METHODOLOGY

This study used the same methods employed by Dimitroff [6]. Content analysis, a “research technique for making replicable and valid inferences from texts to the context of their use” [20], was the primary method used to systematically analyze concepts and themes, thereby providing a “method of quantitatively assessing subject interest and methodologies over time” [6]. Bibliometrics were then applied as a quantitative method to describe patterns of publication.

Source of research articles

Beginning where Dimitroff ended, the full text of articles from quarterly issues of the BMLA from January 1991 (volume 79, number 1) to October 2001 (volume 89, number 4) and from its succeeding title, JMLA, from January 2002 (volume 90, number 1) to October 2007 (volume 95, number 4 ) were retrieved from PubMed Central.†

Identification of research articles

The full text of each article was reviewed jointly by the 4 authors to identify research articles as defined by Peritz: an “inquiry which is carried out, at least to some degree, by a systematic method with the purpose of eliciting some new facts, concepts, or ideas” [8]. The analysis excluded letters, obituaries, essays, editorials, reviews, and commentary. Of the potential 930 articles examined, 474 met the selection criteria.

Analysis of research articles

The research articles (n = 474) were reviewed jointly by the 4 authors, and consensus was reached through discussion. Using the content analysis guidelines established in previously published articles [6,9], each article categorized as research was examined to identify the research method employed (Appendix A) and the subject studied (Appendix B). Just as Dimitroff wanted to track research on library automation, given its importance during the 25 years covered in her study, the authors wanted to track research on the Internet in libraries [6]. Thus each research article was additionally examined to determine if the focus was the Internet. Popular library conference topics such as consumer health, evidence-based medicine, and outreach were also tracked to see if they were being reported on in the research literature.

Analytical techniques were categorized as quantitative descriptive, quantitative inferential, nonquantitative descriptive, and nonquantitative inferential. Collected article bibliometrics included: year of publication, institutional affiliation of first author, funding sources, and number of authors, pages, and citations.

Statistical methods

The bibliographic and bibliometric information for each article was entered into a tracking database in Microsoft Access. To compare the results of this study to Dimitroff's, differences in proportions between the two studies were tested using a two-tailed Z test. This tests the hypothesis that the proportions with a characteristic are approximately equal, in other words, do not differ by more than one would expect by chance. Because many such tests were performed, the probability of finding significance when there really was not a difference was increased, so the P-values were adjusted to compensate for this by using a Sidak adjustment [21,22]. For outcomes that were measured on an ordinal scale (i.e., rank ordered data), differences between the studies were evaluated using the Mann-Whitney U test, which is usually interpreted as a test of the hypothesis that the two studies had equivalent medians [23].

RESULTS

Research articles

Of the 930 articles published in the BMLA and JMLA during the examined period (1991–2007), 474 (51.0%) were identified as research articles.

Subject

Descriptive statistics for research articles identified are listed in Table 1. Applied topics represented 70.0% of the published research articles. In comparison, professional concerns constituted 13.1% of articles, related fields 7.4%, theoretical topics 6.1%, and broad, general subjects 3.4%.

Table 1.

Descriptive statistics for Bulletin of the Medical Library Association (BMLA) and Journal of the Medical Library Association (JMLA) research articles, 1991–2007

Specific subject classification identified a wide range of research topics (Table 2), though the 3 most studied areas—library users (23.5%), materials or collections (18.6%), and public services (11.7%)—accounted for more than half (53.8%) of all research undertaken. The percentage of research articles focusing on users and public services was significantly greater (P = 0.0001) than that found by Dimitroff [6].

Table 2.

BMLA/JMLA research articles 1991–2007: research topic by specific subject area

Several evolving subject areas, those areas receiving attention in professional discussions and conferences during the time period of the articles examined in this study, were tracked. A number of studies examined issues related to use of the Internet (9.9%), possibly indicating a shift in the foundation of library systems today. Fewer focused on consumer health (1.9%), evidence-based medicine (1.7%), or outreach (2.7%).

Institutional affiliation

While the number of unique first authors was broad (n = 350), the majority of first authors of published research were individuals working in health sciences libraries (55.7%). Persons unaffiliated with any library and library school faculty constituted the other main source of authorship (13.9% and 13.5%, respectively). Those working in other libraries, government libraries specifically; hospital librarians; and society librarians represented authorship of the remaining studies (16.9%) (Table 1).

Research method

While use of the same types of research methodologies was observed between the time periods of this study and that of Dimitroff's, the overall general distribution of chosen method was significantly different between the periods (P<0.0001). Survey, however, remained the most frequent research methodology employed in health sciences library research, accounting for 37.1% of all research articles (Table 3). Bibliometrics (15.6%), observation and description (14.5%), and experimental design (13.7%) were the other common methods used.

Table 3.

BMLA/JMLA research articles 1991–2007: use of research methodology

Analytical technique

Quantitative descriptive statistics were used for analysis in 298 (63.5%) of the research articles. Quantitative inferential analysis was used in 114 articles (24.3%). Nonquantitative descriptive analysis was used in 59 articles (12.6%), and 3 articles (0.6%) utilized nonquantitative inferential techniques. These results showed a continued prevalence of descriptive techniques, though an increase in the use of inferential analysis was also evident compared to previous findings. Only 2.7% of research articles identified by Dimitroff used inferential techniques [6].

Funding

From 1991–2007, only 27.4% of research articles identified funding sources. The main source of financial support was also similar to previous findings. Government agencies provided the greatest support (13.9%), followed by associations (6.3%), the author's own institution (4.4%), and other sources (2.7%) (Table 1).

Bibliometric characteristics

Several bibliometric characteristics were examined, including the total number of authors, total number of pages, and total number of citations (Table 1). The total number of authors per article averaged 2.2, a significant increase (P = 0.0019) from Dimitroff's finding of 1.85 authors per article for the period 1966–1990 [6].

Authors cited between 2 and 39 sources per article (mean = 19.1). This represented a significant difference (P<0.0001) from that reported by Dimitroff (mean = 9.23) [6]. Further, 25 research articles (5.27%) cited 4 or fewer sources per article. This compares to 32.8% reported by Dimitroff [6]. Also, research articles written between 1991–2007 have significantly fewer (P = 0.0055) pages than articles written between 1966 and 1990 (mean = 7.0, SD 3.2) [6].

DISCUSSION

The purpose of this study was to analyze trends in research activity as represented in the published research in the leading peer-reviewed professional journal for health sciences librarianship. Comparing the findings with Dimitroff's for the years 1966 to 1990 allows one to observe how the profession has matured in terms of what types of research is being undertaken, what methodologies are employed, what subjects are studied, and how well newly published research builds on that already existing, thus creating a stronger body of evidence for health sciences librarianship [24].

Several observed variables supported the idea that research in health sciences librarianship is becoming more robust. Dimitroff identified 29.8% of articles published in BMLA from 1966–1990 as research articles, compared to the 51.0% found in this study [6]. The findings also confirmed a trend toward more published research in library and information science literature in general compared to previous studies [6,18]. Although the number of pages per article was significantly fewer than Dimitroff found, this could be a result of editorial policies or page layout design [6].

The average number of cited outside sources increased, demonstrating greater attention to the goal of linking studies together and producing stronger research. Finally, while the majority of first authors worked in academic health sciences libraries, a continuation of the pattern found during the years 1966–1990, the number of unique first authors was broad (n = 350) [6]. This may reflect greater involvement in research by members of the profession.

As with previous studies of publications in health sciences librarianship, as well as in the general library and information science literature, applied research topics continued to be the most common [6]. This might be a reflection of both the nature of the profession and the type of settings in which many health sciences librarians work. Fifty-six percent of respondents to the 2007 MLA Membership Survey reported working in a setting with five or fewer workers [25]. Such environments might not prove conducive to conducting research, with the day-to-day responsibilities of operating a library taking precedence. Further, studies that examine subjects related to these daily operations are likely to have greater value to librarians, given these circumstances. More so than theoretical questions of information organization or structure, studies reporting aspects related to public services, materials and collections, or library administration are likely to be both easier for practicing librarians to carry out and deemed more relevant to their work.

Some of the negative consequences of an over-emphasis on applied research have been previously reported [6,26]. Applied research often takes the form of action research, a type aimed at identifying specific problems in a specific setting and thus proposing particular solutions or strategies for addressing often unique situations.

Defined by its context to such a large degree, it is consequently characterized by a lack of external validity (whereby results are applicable to a variety of settings) and low reliability (or capacity to be replicated with accuracy and consistency). Episodic by nature, action research cannot easily be built into, and integrated with, previous studies, and this limits its ability to form part of a continuing and coherent whole. In-house research, in particular, tends to be non-cumulative and descriptive reporting with subsequent limited relevance beyond its original setting, and this fragmentation reduces the capacity of research to illuminate widely-applicable trends [27].

Creating a desired body of evidence that addresses professional concerns is continually hampered by this observed tendency in published health sciences library research [24]. The ongoing inclination to both research and publish articles that produce limited generalized findings makes it impossible to detect larger trends and draw larger conclusions regarding the important aspects of the profession.

Funding for health sciences library research remains either limited or nonexistent, hindering the ability of librarians in the field to devote the time and effort required to conduct studies. The number of articles identifying a funding source is comparable to previous findings [6]. The problem may be one of underreporting by authors, though the consistency in the findings of this study in comparison to Dimitroff's lead one to conclude that a lack of financial support for research is indicated. In its Research Imperative, MLA calls on the National Library of Medicine (NLM), a chief source of monetary support for projects and research in health sciences librarianship, to “provide funding opportunities for a full range of applied research and outcome studies” [1]. Findings of this study support the fact that NLM and other government funding sources (n = 66, 50.7% of funded research articles) heed this call. Interestingly, however, in acting to fund so much applied research and outcome studies, NLM is perhaps inadvertently impeding the development of the kind of research that ultimately leads to cumulative studies and thus the practice of evidence-based librarianship, “a process for integrating the best available, scientifically-generated evidence into making important decisions” [28].

The most frequent types of research methodologies employed remain the same as previously published findings [6,9,17]. Studies utilizing surveys, collection of bibliometric data, and observational techniques are still the most common. These 3 methods were utilized in 67.2% of all research articles reviewed in this study, continuing the trend observed by Dimitroff (75.5%). However, a more varied number and type of methods were observed including the use of experimental designs, focus groups, and use of multiple research methods in a study. Perhaps Dimitroff's observation of the need “to broaden the scope of our research efforts in terms of both subjects addressed and methods used” is bearing out [6]. As Eldredge also observes, health sciences librarians are utilizing a broader array of research methodologies and analytical techniques today [17].

Subjects examined in the research appear to demonstrate a shift in the interest and emphasis of the profession from the physical library itself (i.e., systems used, daily operations) to a greater focus on the library user and public services. For example, Dimitroff noted a steady increase in the percentage of articles addressing automation [6]. In contrast, only a minimal number (1.9%) of research articles identified in this study addressed this topic.

While studies related to library materials and collections continue to make up a sizeable percentage of the published literature, those examining how patrons use library resources are increasingly more prevalent. Almost a full quarter of the research published from 1991–2007 addressed some aspect of public services, compared to only 3.5% published during the prior 25 years [6]. Such a change is indicative of patrons' greater accessibility to online resources (e.g., journals, books, and databases), the competition for the library from popular web-based search tools, and the growing expectation among information consumers to be able to get the information they want, when and where they want it. Health sciences library research, more and more, reflects the attempt by members of the profession to answer questions related to libraries and the services they provide in such an environment.

LIMITATIONS

Articles examined in this study were drawn from one health sciences library journal. The JMLA has many competitors today, perhaps more than during the time period of Dimitroff's study. Research subjects and methods not found in this study may be reported in other journals. A similar study including some of these other journals would be valuable.

The four researchers jointly discussed how to categorize each article. This choice of method limits inter-rater reliability. However, the overall similarity of the findings in this study to Dimitroff's indicates that reliability is satisfactory.

CONCLUSION

The findings of this study demonstrate that progress is being made in health sciences librarianship research. More research articles are being published, a greater variety of research methods are being employed, and covered subjects are expanding. Still, there is room for improvement. Surveys remain the most frequent methodology used. Is this because it is always the most appropriate, or is it simply what librarians are most comfortable using? Could surveys be paired with other methods, such as experimental design, to strengthen research findings?

Applied research is problematic. Often the results of such studies are not generally applicable to different settings and/or not easily replicated, trends cannot be easily identified, and the profession's goal of creating a more synthesized body of evidence is unattainable. There are opportunities to build on existing research that would result in findings that are more applicable to the larger library community. Fuller states, “science progresses to the extent that it is cumulative and builds upon what has gone before. We must, likewise, ensure that our own work is solidly based on prior work and that we learn from what has gone on before us” [29].

The subjects of research articles are shifting from the physical operations of the library to the characteristics and information needs of those using the library. This emphasis on library users and related public services will likely continue, affording opportunities for further studies in this area. Other emerging topics in health sciences librarianship (consumer health, evidence-based medicine, outreach) have also yet to be explored as research projects as defined by Peritz [8]. Further exploration of these topics and future research topics will help build the body of evidence needed to run health sciences libraries more effectively and efficiently. These new areas of study could also possibly lead to new sources of funding, addressing the issue of the limited funds currently available for research. Further, collaborative research, as suggested by Humphreys, can help overcome some of the barriers to conducting research (time, skills, funding) [30].

In 1987, Love declared that library research was “a critical survival factor” for the profession and a means to develop the body of knowledge [3]. The challenge of the MLA Research Imperative to build a culture that both utilizes and creates research still remains; however, the findings of this study show continued movement in this direction.

APPENDIX A

Research methods

Bibliometrics

The measurement of interrelated aspects of writing, publication, and usage, including citation analysis.

Content analysis

A procedure designed to facilitate the objective analysis of the appearance of words, phrases, concepts, themes, characters, or even sentences and paragraphs contained in printed or audiovisual materials.

Delphi method

Designed for use in refining judgmental data collected from a panel of selected experts. Delphi is a systematic approach to the generation of consensus opinions among a group of carefully selected and anonymous respondents.

Experimental

Studies in which investigators specify exactly or control the conditions that will prevail in the investigation. This category includes both field experiments and those in artificially created environments.

Historical research

The collection, verification, and analysis of historical information.

Observation and description

Directed surveillance of an object or subject of an investigation including the recording of observed data. Case studies and systems analysis fall in this category. Survey research, because of its high occurrence rate, has been placed in a separate category.

Operations research

The application of scientific method to management operations to provide a quantitative basis for decision making. This method involves problem formulation, methodology design, data gathering, and model development.

Secondary analysis

Studies that reanalyze published data from other sources.

Survey research

Research based on data measured directly through interviews or questionnaires.

Multiple

Research employing two or more of the methods listed above.

Other

Any research method not falling into one of the other ten categories.

APPENDIX B

Subject classification scheme

1. General

Used for studies that provide a broad overview of library science or its foundations. Articles that dealt with a specific subject were placed in categories 2–4.

1.1. History of libraries or librarianship

1.2. Libraries and society

1.3. International librarianship

2. Professional concerns

Librarianship as a profession, including such concerns as status, salaries, and education.

2.1. Organizations

2.2. Education for librarianship

2.3. Status

2.4. Ethics

2.5. Other

3. Theoretical

For articles that examine or attempt to formulate theories or principles that can provide a theoretical basis for library and information science. Application of theories from other disciplines to library and information science is included here.

3.1. General

3.2. Communication theory

3.3. Information science theory

3.4. Structure of knowledge or information: includes use of information in different situations or disciplines, knowledge structure of disciplines

3.5. Organization of knowledge or information: includes the creation or analysis of intellectual systems for the classification or arrangement of knowledge

3.6. Dissemination or retrieval of information: includes the study of information transfer and of user interactions with systems

4. Applied

Studies of information science or librarianship in practical situations.

4.1. Administration and management

4.2. Public services: the direct provision of services, including reference and bibliographic instruction

4.3. Technical services: includes acquisitions and cataloging

4.4. Systems: systems used in or among libraries

4.5. Materials or collections: includes materials selection, collection development, and preservation

4.6. Buildings: includes physical characteristics of buildings and their furnishings

4.7. Cooperation or networks: all types of cooperative agreements between libraries including interlibrary loan (ILL)

4.8. Library users: the behavior, attitudes, and opinions of library users or nonusers

4.9. Other

5. Related fields

Any research not directly on libraries, library science, or information science.

5.1. Publishing: concerned with production

5.2. Bookselling: concerned with marketing

5.3. Other