Skip to main content
PMC home page
Journal of the Medical Library Association : JMLA logoLink to Journal of the Medical Library Association : JMLA
. 2012 Apr;100(2):135–138. doi: 10.3163/1536-5050.100.2.013

Does Bradford's Law of Scattering predict the size of the literature in Cochrane Reviews?

Charlotte E Nash-Stewart 1, Lisa M Kruesi 2,✉, Chris B Del Mar 3
PMCID: PMC3324807  PMID: 22514511

INTRODUCTION

Bradford's Law of Scattering is a law of diminishing returns and scattering. Bradford formulated the law in 1948 and claimed that for a given subject area “there are a few very productive periodicals, a larger number of more moderate producers, and a still larger number of constantly diminishing productivity” [1]. For any single issue, or subject area, the top third (Zone 1 or core) represents the journals that are the most frequently cited in the literature of that subject and that are, therefore, likely to be of highest interest to researchers in the discipline. The middle third (Zone 2) includes the journals that have had an average amount of citations, and the bottom third (Zone 3 or tail) comprises the long tail of journals that are seldom cited and regarded as of marginal importance to the subject [2]. Researchers have defined a subject area in lexical, semantic, and subject scattering terms [3], and some argue that problems in defining “subject” may not matter, provided it is applied consistently [4].

Bradford's law predicts that the number of journals in the second and third zones will be n and n2 times larger than the first zone respectively [5, 6], and therefore, it should be possible to predict the total number of journals containing papers on a subject once the number in the core and middle zone of journals is known. Once the total number of journals is known, it should be possible to predict how much relevant information is missing from an incomplete search. Given the time-consuming and extensive effort required to identify sources on a subject for a systematic review, accurately predicting the size (and quality, if possible) of the literature from Bradford's law would be useful for such studies.

Empirical testing of Bradford's law requires a complete and large bibliography, a well-defined subject, and a limited time frame [5, 7]. Bradford's law has been applied successfully to measure the literature of many subjects, such as nursing [1], science [8], crystallography [9], and occupational therapy [2]. In addition, many librarians will be familiar with Eugene Garfield's Science Citation Index, which is based on Bradford's law [10]. One analysis of all randomized controlled trials (RCTs) in the MEDLINE database found that the journal distribution varied from the standard Bradford's law [11]. However, there have not been any studies on the usefulness of Bradford's law to predict the size of literature in systematic reviews. Other methods to estimate the total number of articles when searching for systematic reviews, such as the Horizon Estimate, have been applied with varying success [12].

This study arose from work with the Acute Respiratory Infections Group, one of the fifty-two entities making up the Cochrane Collaboration. Cochrane Reviews are systematic literature reviews aiming at high quality and, therefore, explicitly striving for completeness. The validity of Bradford's law for systematic reviews has not been addressed in the literature. Accordingly, this study examined whether Bradford's law was valid for the Cochrane Review–identified literature on acute otitis media and pneumonia, conditions that are reported in a wide variety of clinical and health journals [13].

METHODS

In late 2006, a search in titles, abstracts, and keywords for the term, “acute otitis media,” and the term “pneumonia,” were undertaken in the Cochrane Library. The two conditions were chosen because they are the subject of many Cochrane Reviews and have a large literature. RCTs were extracted from the references in the chosen Cochrane Reviews that addressed only treatment of acute otitis media or pneumonia. Reviews and protocols concerning prevention rather than treatment were not selected for the study. Clinical subject experts were consulted to confirm the suitability of the treatment reviews for the study. Extracted RCTs were grouped by subject, marked by whether they were included or excluded by the Cochrane Review, and duplicates were removed. The included status was used as proxy for high-quality RCTs. These meet strict eligibility criteria to be included in Cochrane Reviews and are excluded if they fail to meet the criteria. For this study, RCTs were marked as included when at least one Cochrane Review article included it, even if it was excluded in other Cochrane Reviews. The purpose of categorizing included and excluded status was to identify whether higher-quality studies were more likely to be found in Zone 1 journals. RCTs were then sorted by journal to generate a cumulative distribution for the subject. Journals that had changed names or merged were listed under the most current journal title available from the MEDLINE journals database [14].

The following analyses were undertaken for each of the two subjects: (1) the journal distribution was analyzed (according to the number of RCTs each contained) to test Bradford's law; (2) the Zone 1 journals were identified; and (3) the correlation between inclusion status and publication in a Zone 1 journal was determined.

It was assumed that the Cochrane Reviews missed few references.

RESULTS

The search in the Cochrane Library on the subject acute otitis media found 22 Cochrane Reviews, 5 of which met inclusion criteria [15–19], yielding 135 articles from 73 journals. For pneumonia, the search in the Cochrane Library found 62 reviews, of which 9 met the inclusion criteria, yielding 251 articles from 110 journals [20–28].

Bradford's Law of Scattering

The journals publishing articles on each topic were divided into 3 regions, with approximately equal numbers of articles, detailed in Table 1 (online only). When journals were tied in terms of articles published, they were ranked according to convention “C1” in Heine (1998) [4]. This convention gives journals that produce the same number of articles an arbitrary and sequential rank from the previously ranked journal, rather than assigning all the journals the same rank. The 3 zones were determined by closest fit to the 1:n:n2 Bradford model, while keeping all journals that produced the same number of papers in the same zone. As a result, the articles were not divided into 3 equal parts. The core journals, Zone 1, were defined as those providing approximately one-third of all the RCTs. For acute otitis media, the core accounted for 41 of 135 papers (30%) from 5 journals, and for pneumonia, 69 (27%) of 251 papers from 6 journals. The middle journals, Zone 2, contained 15 journals (41 articles, 30%) for acute otitis media and 21 journals (87 articles, 35%) for pneumonia.

Journal rank (on a log scale) is plotted against cumulative articles in Figure 1 and gives similar appearances for both conditions. The figure does mirror broadly the shape of a Bradford-Zipf plot, albeit with wider horizontal axis to illustrate the zone information.

Figure 1.

Figure 1

Open in a new tab

Journal rank and cumulative number of articles for Cochrane Reviews of acute otitis media and pneumonia

This represents a larger tail than predicted by Bradford's law (53 versus 45 predicted Zone 3 journals for acute otitis media and 83 versus 73 predicted Zone 3 journals for pneumonia) and is statistically significant for pneumonia (χ2 = 5.09; P = 0.024) but not otitis media (χ2 = 3.68; P = 0.055 and pneumonia).

Quality and location of studies

For acute otitis media, 29 of the 41 articles in 5 journals (71%) from Zone 1 were included in Cochrane Reviews (proxy for high quality). This compares with 33 of 94 noncore (Zone 2 or 3) articles in 68 journals (35%) being included in Cochrane Reviews (χ2 = 14.6, P = 0.00013). Overall, only 29 of the 62 included articles (47%) were found in the 5 journals from Zone 1 and 14 (23%) were from the journals in Zone 3.

For pneumonia, 30 of 69 articles in 6 journals (43%) from Zone 1 were included in Cochrane Reviews. This compared with 50 of 182 noncore articles in 104 journals (27%) being included in Cochrane Reviews (χ2 = 4.96, P = 0.026). Only 30 of the 80 included articles (38%) were found in the 6 journals from Zone 1. This number was exceeded by the 33 included articles (41%) from Zone 3.

Overall, 86% of all RCTs and 92% of included studies came from MEDLINE-indexed journals.

DISCUSSION

Bradford's law predicts that finding the core literature addressing a topic will enable an estimate of the total literature, but this has not been tested empirically for systematic reviews. The Cochrane Reviews examined did not conform sufficiently to Bradford's law to allow the total number of RCTs on a subject to be predicted from just the number found in core journals. Although the journal distribution for both conditions broadly matches Bradford's law, uncertainty in the tail size creates difficulties in prediction based on only a set of core journals. This suggests little use in estimating literature size for subjects from RCTs found in core journals alone (e.g., when planning a review for a given subject) or in identifying how much information would be missed if the search was incomplete. Other tests of the law have also been disappointing, including Bradford's own and other empirical testing [29, 30]. This study's finding that Zone 3 size differed from that predicted by the classical Bradford model may reflect the highly interdisciplinary nature of acute respiratory infections (crossing the disciplines of primary care, paediatrics, and infectious diseases, as well as general internal medicine) [3] or uncertainties in classifying the “subject” [31].

MEDLINE indexes 94% of its medical literature [32]. Not all of the RCTs (86%) included in Cochrane Reviews that were identified in this study were indexed in MEDLINE, supporting the need for systematic reviewers to anticipate that perhaps 15% of available articles would be found in other places (e.g., databases such as Embase) and that, of these, about half could be valid for inclusion. The large proportion of high-quality RCTs that came from the large Zone 3 zone of journals in this study confirms previous opinions that searching the tail is necessary to capture relevant literature [32]. Finding studies in noncore journal publications is, therefore, necessary both for completeness and to avoid bias in the review.

CONCLUSION

In this case, Bradford's law was not useful for predicting the size of the literature on a subject from the number of articles appearing in core journals. However, the Zone 3 of the distribution in this study contained a large number of valid studies, essential to include in a complete systematic review. Further research is needed to enable accurate estimates of the anticipated literature size in systematic reviews of other Cochrane topics.

Electronic Content

Table 1. Journal of the Medical Library Association.
Click here for additional data file. (29.9KB, pdf)

Acknowledgments

The authors thank Professor David Goldgar, The University of Utah, for his help with the biostatistics.

Footnotes

EC

Supplemental Table 1 is available with the online version of this journal.

REFERENCES

  • 1.Bradford S.C, Egan M.E, Shera J.H. Documentation. 2nd ed. London, UK: Crossby Lockwood; 1953. [Google Scholar]
  • 2.Potter J. Mapping the literature of occupational therapy: an update. J Med Lib Assoc. 2010 Jul;98(3):235–42. doi: 10.3163/1536-5050.98.3.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Hjørland B, Nicolaisen J. Bradford's Law of Scattering: ambiguities in the concept of “subject.”. In: Crestani F, Ruthven I, editors. Context: nature, impact, and role: 5th International Conference on Conceptions of Library and Information Sciences. Springer; 2005. pp. 96–106. p. (Lecture Notes in Computer Science, v.3507.) [Google Scholar]
  • 4.Heine M. Bradford ranking conventions and their application to a growing literature. J Documentation. 1998 Jun;54(3):303–31. [Google Scholar]
  • 5.Fairthorne R. Empirical hyperbolic distributions (Bradford-Zipf-Mandelbrot) for bibliometric description and prediction. J Documentation. 2005;61(2):171–93. [Google Scholar]
  • 6.Garfield E. Bradford's law and related statistical patterns. Essays Inf Scientist. 1980 May;4:476–83. [Google Scholar]
  • 7.Naranan S. Bradford's law of bibliography of science: an interpretation. Nature. 1970 Aug 8;227(5258):631–2. doi: 10.1038/227631a0. [DOI] [PubMed] [Google Scholar]
  • 8.Brookes B.C. Bradford's law and the bibliography of science. Nature. 1969 Dec;224(5223):953–6. doi: 10.1038/224953a0. [DOI] [PubMed] [Google Scholar]
  • 9.Behrens H, Luksch P. A bibliometric study in crystallography. Acta Crystallogr B. 2006 Dec;62(6):993–1001. doi: 10.1107/S0108768106030278. [DOI] [PubMed] [Google Scholar]
  • 10.Bensman S.J. Garfield and the impact factor. Ann Rev Inf Sci Technol. 2007;41(1):93–155. [Google Scholar]
  • 11.Tsay M.Y, Yang Y.H. Bibliometric analysis of the literature of randomized controlled trials. J Med Lib Assoc. 2005 Oct;93(4):450–8. [PMC free article] [PubMed] [Google Scholar]
  • 12.Kastner M, Straus S.E, McKibbon K.A, Goldsmith C.H. The capture-mark-recapture technique can be used as a stopping rule when searching in systematic reviews. J Clin Epidemiol. 2009 Feb;62(2):149–57. doi: 10.1016/j.jclinepi.2008.06.001. [DOI] [PubMed] [Google Scholar]
  • 13.White H. “Bradfordizing”search output: how it would help online users. Online Inf Rev. 1993;5(1):47–54. [Google Scholar]
  • 14.National Library of Medicine. List of journals indexed for MEDLINE [Internet] The Library. 2010 [cited 17 Oct 2011]. < http://www.nlm.nih.gov/tsd/serials/lji.html>.
  • 15.Flynn C, Griffin G, Tudiver F. Decongestants and antihistamines for acute otitis media in children. Cochrane Database Syst Rev. 2001. p. CD001727. [DOI] [PubMed]
  • 16.Foxlee R, Johansson A, Wejfalk J, Dawkins J, Dooley L, Del Mar C. Topical analgesia for acute otitis media. Cochrane Database Syst Rev. 2006 Jul. p. CD005657. [DOI] [PMC free article] [PubMed]
  • 17.Glasziou P.P, Del Mar C.B, Sanders S.L, Hayem M. Antibiotics for acute otitis media in children. Cochrane Database Syst Rev. 2004. p. CD000219. [DOI] [PubMed]
  • 18.Kozyrskyj A, Klassen T.P, Moffatt M, Harvey K. Short-course antibiotics for acute otitis media. Cochrane Database Syst Rev. 2010 Sep. p. CD001095. [DOI] [PMC free article] [PubMed]
  • 19.Spurling G.K, Del Mar C.B, Dooley L, Foxlee R. Delayed antibiotics for symptoms and complications of respiratory infections. Cochrane Database Syst Rev. 2004 Oct. p. CD004417. [DOI] [PubMed]
  • 20.Bjerre L.M, Verheij T.J, Kochen M.M. Antibiotics for community acquired pneumonia in adult outpatients. Cochrane Database Syst Rev. 2004 Oct. p. CD002109. [DOI] [PubMed]
  • 21.Briel M, Bucher H.C, Boscacci R, Furrer H. Adjunctive corticosteroids for pneumocystis jiroveci pneumonia in patients with HIV-infection. Cochrane Database Syst Rev. 2006 Jan. p. CD006150. [DOI] [PubMed]
  • 22.Cheng A.C, Stephens D.P, Currie B.J. Granulocyte-colony stimulating factor (G-CSF) as an adjunct to antibiotics in the treatment of pneumonia in adults. Cochrane Database Syst Rev. 2007 Apr. p. CD004400. [DOI] [PubMed]
  • 23.Gavranich J.B, Chang A.B. Antibiotics for community acquired lower respiratory tract infections (LRTI) secondary to mycoplasma pneumoniae in children. Cochrane Database Syst Rev. 2005 Jul. p. CD004875. [DOI] [PubMed]
  • 24.Kabra S.K, Lodha R, Pandey R.M. Antibiotics for community acquired pneumonia in children. Cochrane Database Syst Rev. 2006 Jul. p. CD004874. [DOI] [PubMed]
  • 25.Ni J, Wei J, Wu T. Vitamin A for non-measles pneumonia in children. Cochrane Database Syst Rev. 2005 Jul. p. CD003700. [DOI] [PMC free article] [PubMed]
  • 26.Panpanich R, Lerttrakarnnon P, Laopaiboon M. Azithromycin for acute lower respiratory tract infections. Cochrane Database Syst Rev. 2004 Oct. p. CD001954. [DOI] [PubMed]
  • 27.Rojas M.X, Granados C. Oral antibiotics versus parenteral antibiotics for severe pneumonia in children. Cochrane Database Syst Rev. 2006 Apr. p. CD004979. [DOI] [PMC free article] [PubMed]
  • 28.Shefet D, Robenshtock E, Paul M, Leibovici L. Empiric antibiotic coverage of atypical pathogens for community acquired pneumonia in hospitalized adults. Cochrane Database Syst Rev. 2005 Apr. p. CD004418. [DOI] [PubMed]
  • 29.Bradford S.C. Sources of information on specific subjects (reprinted from Engineering an Illustrated Weekly Journal, vol 137, pg 85–86, 1934) J Info Sci. 1985 Apr;10(4):173–80. [Google Scholar]
  • 30.Drott M.C, Griffith B.C. An empirical examination of Bradford's law and the scattering of scientific literature. J Am Soc Inf Science. 1978 Sep;29(5):238–46. [Google Scholar]
  • 31.Nicolaisen J, Hjørland B. Practical potentials of Bradford's law: a critical examination of the received view. J Documentation. 2007;63(3):359–77. [Google Scholar]
  • 32.Royle P, Bain L, Waugh N. Systematic reviews of epidemiology in diabetes: finding the evidence. BMC Med Res Methodol. 2005 Jan;5(1):2. doi: 10.1186/1471-2288-5-2. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Table 1. Journal of the Medical Library Association.
Click here for additional data file. (29.9KB, pdf)

Articles from Journal of the Medical Library Association : JMLA are provided here courtesy of Medical Library Association

RESOURCES