Abstract
Introduction
The Journal of Bone and Joint Surgery (JBJS-Am) began publishing the level of evidence (LOE) for manuscripts in 2003. From 1975 to 2005 JBJS-Am saw a trend towards higher leveled studies. We aimed to demonstrate trends in the country of origin of manuscripts published in JBJS-Am, and hypothesized that not only were more publications coming from groups outside of North America, but that the studies originating within North America were of higher LOE.
Methods
All articles published in The Journal of Bone and Joint Surgery (American) in 1980, 1985, 1990, 1995, 2000, 2005, and 2010 were independently evaluated by two reviewers and graded based on country, LOE (using the JBJS-Am LOE guidelines), and study type. For articles published after 2003 we used the level and study type published within the manuscript.
Results
The proportion of publications from North America decreased in 2005 and 2010 when compared to the previous 20 years (p>.03), but the overall number of publications appeared stable. Overall, there was an increase in Level I (r>.74, p>.03), Level II (r>.79, p>.02), and Level III (r>.95, p<.001) evidence studies. There was a statistically significant decrease in North American Level IV studies (r>-.81, p>.01) and an increase in international Level IV studies (r>.70, p>.04). International groups have increased therapeutic (r>.86, p<.01) and diagnostic studies (r>.93, p<.001). In North America and internationally, prognostic studies have not changed. North American groups have increased economic and decision analysis research (r>.69, p>.04).
Conclusions
Over the past 30 years JBJS-Am has become more internationally diverse. International groups are publishing more therapeutic and diagnostic research than in the past, while North American groups have increased economic and decision analysis research. There has been a global effort towards higher leveled research.
Introduction
The Journal of Bone and Joint Surgery (JBJS-Am) began publishing the level of evidence (LOE) for manuscripts in 20031, an idea developed in the 1980s by Sackett2. JBJS-Am uses five tier-levels for each of four study content types: therapeutic, prognostic, diagnostic, and economic and decision analysis. LOE is derived from evidence-based medicine, a philosophy which promotes physicians and surgeons acting in the best interest of their patients and making recommendations founded in the best available data3. Publishing the study type and LOE with each manuscript helps the reader understand the quality of the research presented4,5. The JBJS-Am system is easily understandable; epidemiologically trained reviewers can use the system with high reliability and individuals without such training can use it proficiently4,5.
Previous studies have shown greater number of international publications found in major research journals6. In addition, journals such as JBJS-Am saw a trend towards higher leveled studies between 1975 and 20057. In light of seemingly fewer contributions from North America, we questioned if this would be associated with higher LOE studies, particularly since the system was implemented and published in 2003. To our knowledge, no study has previously investigated trends in the country of origin of manuscripts published in JBJS-Am. We hypothesized that a greater number of manuscripts have been published from groups outside of North America, and that the studies originating within North America were of higher LOE.
Materials and Methods
Study Design:
All articles published in The Journal of Bone and Joint Surgery (American) in the years 1980, 1985, 1990, 1995,2000, 2005, and 2010 were reviewed. Articles listed as Basic Science, Cadaver Studies, Editorials, Continuing Medical Education, Current Concept Reviews, and Instructional Course Lectures were excluded from analysis. The country of origin of each article was recorded. For data analysis, countries were grouped by continent with Australia and New Zealand classified together and multinational studies classified as collaborative projects. Papers published from Canada and the United States were grouped in the North America group with papers from other countries referred to as international. Each article was independently assigned a LOE and study type (therapeutic, prognostic, diagnostic, economic and decision analysis) by two of the authors with the JBJS-Am LOE guidelines (based on the standards of the Centre for Evidence-Based Medicine, Oxford, UK) in hand1,8. Each author was blinded to the other author's reviews. LOE and study type assignments were compared after all articles were reviewed and in situations in which level assignments differed, the two reviewers discussed each article until an agreement was reached. Articles for which agreement could not be reached were evaluated by a third reviewer and assignments were made based on the majority; if disagreement persisted, the article was excluded from analysis. For articles published after 2003 we used the LOE and study type published in the manuscript for data analysis.
Statistical Analysis:
Statistics were performed with SPSS (Statistical Packages for the Social Sciences) 18 (International Business Machines, Armonk, NY, USA). The total number of studies were summed by year and separated by LOE and then by geographic location. Pearson's correlation coefficient was calculated by comparing the year to the sum of the variable of interest (LOE and location). A separate analysis was carried out for each LOE category (Level I, II, III, and IV). Pearson's correlation coefficient was calculated with one-tailed statistics as we predicted increasing Level I-III data and decreasing Level IV data. Statistical significance was defined as p<0.05.
Results
Geographic Location and Number of Publications
A total of 1261 articles from the years 1980, 1985, 1990, 1995, 2000, 2005, and 2010 were reviewed. Of these, 821 articles met our inclusion criteria. Thirty-two countries including the United States produced the 821 articles.
The majority of articles, 618/821 (75%), were from North America (Table 1). In 1980, 91% of the publications in the JBJS-Am were from North America (83/91). In 2000, 2005, and 2010, there was a decline in the proportion of publications from North America (Figure 1); the number of publications remained relatively stable. On average, 84% (74-91%) of publications were from North America from 1980 to 2000; in 2005 and 2010, 65% and 57% of the articles were from North America, respectively (p=.03). International contributions increased in 2005 and 2010 with Europe and Asia producing more articles in these years (Table 1, Figure 1).
Table 1.
Number of articles published by year and grouped by geographic location.
| Year | Total | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| 1980 | 1985 | 1990 | 1995 | 2000 | 2005 | 2010 | |||
| Continent | North America | 83 | 91 | 102 | 83 | 65 | 112 | 82 | 618 |
| Europe | 3 | 8 | 10 | 12 | 11 | 29 | 34 | 107 | |
| Asia | 3 | 5 | 8 | 2 | 8 | 26 | 22 | 74 | |
| South America | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | |
| Africa | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | |
| Australia/New Zealand | 1 | 1 | 1 | 0 | 3 | 1 | 4 | 11 | |
| Collaborative | 0 | 1 | 0 | 0 | 1 | 4 | 3 | 9 | |
| Total | 91 | 106 | 121 | 97 | 88 | 173 | 145 | 821 |
Figure 1. Percentage of clinical publications by geographical location and stratified by year.
Geographic Location and Level of Evidence
There was greater than 99% agreement between the two different independent raters when comparing the LOE assigned to each article. Due to differing evaluations, six articles were reviewed by a third reviewer; a LOE was assigned to four articles based on majority agreement and two articles were discarded due to three different evaluations.
Overall, there were increases in Level I, II, and III evidence and decreases in Level IV evidence from 1980 to 2010 (Appendices 1-4). Level I evidence increased over the 30-year interval (r=.74, p =.03). When analyzed by geographic location, there was an increase in Level I studies from international sources (r=.84, p<.01), but no significant increase in Level I studies from North America (r=.57, p=.09) (Figure 2). International contributions were largely from Europe and Asia. There was a significant increase in the proportion of Level II (r=.79, p=.02) and Level III (r=.95, p<.001) evidence studies from 1980 to 2010 with significant increases in both Level II and III studies in North American (Level II: r=.75, p=.03; Level III: r=.84, p=.01) and international publications (Level II: r=.83, p<.01; Level III: r=.84, p=.01) (Figures 3 and Figure 4). However, there has been an overall trend towards fewer Level IV studies (r=-.48, p=.14). Yet, there has been a significant decrease in Level IV studies from North America (r=-.81, p=.01), whereas there has been an increase in international Level IV studies (r=.70, p=.04) (Figure 5).
Figure 2. Level I studies broken down by geographic location.
Figure 3. Level II studies broken down by geographical location.
Figure 4. Level III studies broken down by geographical location.
Figure 5. Level IV studies broken down by geographical location.
Geographic Location and Study Type
Of the 821 previously included articles, the reviewers reached consensus on study type for all but one article. Of the remaining 820 articles 590 were therapeutic, 64 were diagnostic, 161 were prognostic, and 5 were economic and decision analyses (Table 2). There has been an increase in international therapeutic (r=.86, p<.01) and diagnostic (r=.93, p<.001) publications. Neither North American nor international countries demonstrated changes in prognostic studies. All of the economic and decision analyses we evaluated were from North American groups and they have increased this type of research (r=.69, p=.04); no economic and decision analyses were published from 1980 to 1995.
Table 2.
Articles grouped by study type and year.
| Year | Total | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| 1980 | 1985 | 1990 | 1995 | 2000 | 2005 | 2010 | |||
| Study Type | Therapeutic | 64 | 82 | 82 | 70 | 68 | 134 | 90 | 590 |
| Diagnostic | 4 | 7 | 12 | 8 | 8 | 12 | 13 | 64 | |
| Prognostic | 23 | 16 | 27 | 19 | 11 | 24 | 41 | 161 | |
| Economic and Decision Analysis | 0 | 0 | 0 | 0 | 1 | 3 | 1 | 5 | |
| Total | 91 | 105 | 121 | 97 | 88 | 173 | 145 | 820 |
Discussion
To our knowledge, this is the first study to analyze publication trends in orthopaedic literature with regard to geographic location and LOE. The principle aim of this study was to determine changes in the number and proportion of studies contributed to JBJS-Am by international investigators. Our second goal was to detect changes in the LOE published over the past 30 years.
Our data demonstrates that there has been a significant increase in international contributions and that most of these publications are from Europe and Asia. Interestingly, while the proportion of North American contributions is decreasing, the number remains constant. In a world embracing simpler transfers of knowledge and sharing of ideas, this suggests that rather than shrinking the impact of North American groups in JBJS-Am, research globalization is broadening the research published.
We show that globally there have been more Level I,II, and III studies published and there has been a trend towards decreasing Level IV studies. There are cost and time savings by performing research in developing countries6, but since these changes were noted globally, they likely reflect a medical culture emphasizing evidence- based medicine in which investigators are executing more thorough research projects to promote the best possible patient outcomes3. It may also reflect JBJS-Am's position in the orthopaedic literature with a high impact factor, as it was shown that the percentage of Level I or II studies positively correlated with impact factor5. As Ombresky et al. noted, achieving a higher LOE could be achieved easily; by adding a control group, a case series could potentially be upgraded to a Level II or III cohort study5. However, we cannot comment on the LOE mix of JBJS-Am submissions versus publications.
These trends are also interesting given that the surgeon-patient encounter does not provide an ideal research setting. Treatment studies must be randomized to be Level I but a common obstacle to a randomized controlled trial (RCT) is patient preference for one procedure over another if they are perceived to be unequal in terms of benefit or side-effects. Other barriers include the commonality of the condition being treated surgically, lack of community equipoise, complicated methodologies, and lack of surgical familiarity with all studied treatment alternatives. Surgical RCTs were deemed possible in less than 40% of cases9.
In spite of these trends, and a statically significant decrease in Level IV studies from North America, Level IV studies are commonly published in JBJS-Am. With the breadth of orthopaedic knowledge growing at ever- increasing rates, these are often necessary to publicize new findings. A case series can be a well-designed, prospective, study with well-defined inclusion and exclusion criteria, treatment protocols, follow-up intervals, and validated outcomes measures5. It can publicize the rare consequences of procedures which cannot be accomplished with randomized and cohort studies10; it can also be used to present low prevalence diseases or clinical situations5. 2010 marked the first year (using our five-year cohorts) that Level IV studies were not the most prevalent study published. These results lead us to believe that there is a conscious effort to redesign research studies as well as publish higher quality articles.
There may be further explanation for our results. Institutional Review Boards in the US, and comparable committees in European countries, are essential for ensuring the ethical treatment of human subjects in medical research, but also present obstacles to performing clinical research6. Economic, educational, and social issues may facilitate research in developing countries6, and there are nuances between the rules guiding research within the United States and elsewhere11-13. It is possible that these factors may account for some of our findings.
The greatest strengths of the present study are its thoroughness and completeness. We believe that the publications included in the present study were accurately and reliably assessed for LOE. Studies evaluating reliability demonstrated that epidemiologic training is not necessary to accurately and reliably asses a manuscript's LOE4,5.
There are limitations to this study. First, we used manuscripts published in five-year cohorts. More specific trends could be obtained by using smaller time intervals. This study could also be conducted using additional orthopaedic journals to determine if these trends persist across the orthopaedic literature as a whole. It is also important to point out that while a higher LOE is a surrogate for higher quality research, it does not always correspond to higher quality research. Prior research has demonstrated substantial variability in the quality of even Level I studies14,15, and that the quality of Level I and Level II work is not statistically significantly different15. Further evaluating not only the LOE but the quality of research published would be useful.
The country of origin, LOE, and types of studies published in JBJS-Am have changed over the past 30 years. There have been increases in Level I, II, and III studies; an increase in therapeutic and prognostic studies from abroad; and an increase economic and decision analyses from North America. There are many reasons that may account for these changes, but the ease of information sharing and a motivation to produce higher quality research likely play major roles.
Conflict of interest statement
Each author certifies that he or she has no commercial associations (eg, consultancies, stock ownership, equity interest, patent/licensing arrangements, etc.) that might pose a conflict of interest in connection with the submitted article.
Appendix
Appendix 1.
Level I studies broken down by country and year.
| Year | Total | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| 1980 | 1985 | 1990 | 1995 | 2000 | 2005 | 2010 | |||
| Country | Australia | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 2 |
| Canada | 0 | 0 | 0 | 0 | 0 | 2 | 3 | 5 | |
| China | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 2 | |
| Collaborative | 0 | 0 | 0 | 0 | 1 | 2 | 2 | 5 | |
| Denmark | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | |
| Finland | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 2 | |
| France | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | |
| Germany | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 2 | |
| Greece | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | |
| Italy | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | |
| Netherlands | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 2 | |
| Norway | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | |
| South Korea | 0 | 0 | 0 | 0 | 0 | 2 | 1 | 3 | |
| Sweden | 0 | 0 | 0 | 2 | 0 | 1 | 0 | 3 | |
| Switzerland | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 2 | |
| Taiwan | 0 | 0 | 0 | 0 | 0 | 2 | 0 | 2 | |
| UK | 0 | 0 | 1 | 0 | 0 | 2 | 3 | 6 | |
| USA | 3 | 9 | 12 | 11 | 5 | 20 | 8 | 68 | |
| Total | 3 | 10 | 13 | 14 | 7 | 35 | 27 | 109 |
Appendix 2.
Level II studies broken down by country and year.
| Year | Total | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| 1980 | 1985 | 1990 | 1995 | 2000 | 2005 | 2010 | |||
| Country | Australia | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 1 |
| Austria | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | |
| Canada | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | |
| Collaborative | 0 | 1 | 0 | 0 | 0 | 1 | 2 | 4 | |
| Denmark | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 2 | |
| France | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | |
| India | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | |
| Italy | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | |
| Japan | 0 | 0 | 0 | 0 | 1 | 1 | 3 | 5 | |
| New Zealand | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | |
| Norway | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | |
| South Korea | 0 | 0 | 0 | 0 | 0 | 1 | 4 | 5 | |
| Switzerland | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 2 | |
| UK | 0 | 0 | 0 | 0 | 0 | 4 | 2 | 6 | |
| USA | 10 | 5 | 11 | 7 | 10 | 17 | 30 | 90 | |
| Total | 10 | 6 | 12 | 7 | 12 | 26 | 49 | 122 |
Appendix 3.
Level III studies broken down by country and year.
| Year | Total | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| 1980 | 1985 | 1990 | 1995 | 2000 | 2005 | 2010 | |||
| Country | Canada | 0 | 1 | 1 | 1 | 2 | 1 | 2 | 8 |
| China | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 2 | |
| Collaborative | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 2 | |
| Denmark | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | |
| Finland | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 2 | |
| France | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | |
| Germany | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 2 | |
| Italy | 0 | 2 | 0 | 0 | 0 | 0 | 0 | 2 | |
| Japan | 0 | 0 | 2 | 1 | 1 | 1 | 1 | 6 | |
| Netherlands | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | |
| Norway | 0 | 0 | 0 | 0 | 0 | 1 | 2 | 3 | |
| South Korea | 0 | 0 | 0 | 0 | 1 | 2 | 2 | 5 | |
| Sweden | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 2 | |
| Switzerland | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 2 | |
| Taiwan | 0 | 0 | 0 | 0 | 0 | 1 | 2 | 3 | |
| UK | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 2 | |
| USA | 9 | 11 | 15 | 12 | 13 | 19 | 15 | 94 | |
| Total | 10 | 14 | 19 | 16 | 20 | 28 | 31 | 138 |
Appendix 4.
Level IV studies broken down by country and year.
| Year | Total | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| 1980 | 1985 | 1990 | 1995 | 2000 | 2005 | 2010 | |||
| Country | Argentina | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 |
| Australia | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | |
| Austria | 0 | 0 | 0 | 0 | 3 | 0 | 1 | 4 | |
| Belgium | 0 | 0 | 0 | 0 | 0 | 2 | 0 | 2 | |
| Canada | 0 | 4 | 6 | 5 | 2 | 1 | 0 | 18 | |
| China | 2 | 1 | 0 | 0 | 0 | 1 | 0 | 4 | |
| Collaborative | 0 | 0 | 1 | 0 | 0 | 2 | 1 | 4 | |
| Czech Republic | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | |
| Denmark | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | |
| Estonia | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | |
| Finland | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 2 | |
| France | 0 | 1 | 2 | 0 | 0 | 3 | 1 | 7 | |
| Germany | 0 | 0 | 0 | 0 | 2 | 1 | 2 | 5 | |
| Greece | 1 | 0 | 0 | 1 | 0 | 0 | 1 | 3 | |
| India | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 2 | |
| Israel | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | |
| Italy | 0 | 1 | 1 | 0 | 0 | 1 | 0 | 3 | |
| Japan | 1 | 3 | 4 | 0 | 5 | 4 | 1 | 18 | |
| Netherlands | 0 | 0 | 2 | 2 | 1 | 1 | 0 | 6 | |
| New Zealand | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | |
| Saudi Arabia | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | |
| South Korea | 0 | 0 | 0 | 0 | 0 | 4 | 4 | 8 | |
| Spain | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | |
| Sweden | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 2 | |
| Switzerland | 0 | 2 | 0 | 2 | 4 | 3 | 1 | 12 | |
| Taiwan | 0 | 0 | 0 | 1 | 0 | 3 | 0 | 4 | |
| Tunisia | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | |
| Turkey | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | |
| UK | 0 | 1 | 0 | 0 | 0 | 3 | 2 | 6 | |
| USA | 61 | 61 | 56 | 47 | 32 | 52 | 22 | 331 | |
| Total | 68 | 76 | 77 | 60 | 49 | 84 | 38 | 452 |
References
- 1.Wright JG, Swiontkowski MF, Heckman JD. Introducing levels of evidence to the journal. J Bone Joint Surg Am. 2003;85-A(1):1–3. [PubMed] [Google Scholar]
- 2.Sackett DL. Rules of evidence and clinical recommendations on the use of antithrombotic agents. Chest. 1986;89(2 Suppl):2S–3S. [PubMed] [Google Scholar]
- 3.Stirrat GM. Ethics and evidence based surgery. J Med Ethics. 2004;30:160–165. doi: 10.1136/jme.2003.007054. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Bhandari M, Swiontkowski MF, Einhorn TA, Tornetta P, 3rd, Schemitsch EH, Leece P, Sprague S, Wright JG. Interobserver agreement in the application of levels of evidence to scientific papers in the American volume of the Journal of Bone and Joint Surgery. J Bone Joint Surg Am. 2004;86-A(8):1717–1720. doi: 10.2106/00004623-200408000-00016. [DOI] [PubMed] [Google Scholar]
- 5.Obremskey WT, Pappas N, Attallah-Wasif E, Tornetta P 3rd, Bhandari M. Level of evidence in orthopaedic journals. J Bone Joint Surg Am. 2005;87(12):2632–2638. doi: 10.2106/JBJS.E.00370. [DOI] [PubMed] [Google Scholar]
- 6.Glickman SW, McHutchison JG, Peterson ED, Cairns CB, Harrington RA, Califf RM, Schulman KA. Ethical and scientific implications of the globalization of clinical research. New Engl J Med. 2009;360(8):816–823. doi: 10.1056/NEJMsb0803929. [DOI] [PubMed] [Google Scholar]
- 7.Hanzlik S, Mahabir RC, Baynosa RC, Khiabani KT. Levels of evidence in research published in The Journal of Bone and Joint Surgery (American Volume) over the last thirty years. J Bone Joint Surg Am. 2009;91(2):425–428. doi: 10.2106/JBJS.H.00108. [DOI] [PubMed] [Google Scholar]
- 8.Wright JG. A practical guide to assigning levels of evidence. J Bone Joint Surg Am. 2007;89(5):1128–1130. doi: 10.2106/JBJS.F.01380. [DOI] [PubMed] [Google Scholar]
- 9.Solomon MJ, McLeod RS. Should we be performing more randomized controlled trials evaluating surgical operations? Surgery. 1995;118(3):459–467. doi: 10.1016/s0039-6060(05)80359-9. [DOI] [PubMed] [Google Scholar]
- 10.Tovey D, Bognolo G. Levels of evidence and the orthopaedic surgeon. J Bone Joint Surg Br. 2005;87(12):1591–1592. doi: 10.1302/0301-620X.87B12.16935. [DOI] [PubMed] [Google Scholar]
- 11.Angell M. The ethics of clinical research in the Third World. New Engl J Med. 1997;337(12):847–849. doi: 10.1056/NEJM199709183371209. [DOI] [PubMed] [Google Scholar]
- 12.Brody BA. Ethical issues in clinical trials in developing countries. Stat Med. 2002;21(19):2853–2858. doi: 10.1002/sim.1289. [DOI] [PubMed] [Google Scholar]
- 13.Lurie P, Wolfe SM. Unethical trials of interventions to reduce perinatal transmission of the human immunodeficiency virus in developing countries. New Engl J Med. 1997;337(12):853–856. doi: 10.1056/NEJM199709183371212. [DOI] [PubMed] [Google Scholar]
- 14.Bhandari M, Richards RR, Sprague S, Schemitsch EH. The quality of reporting of randomized trials in the Journal of Bone and Joint Surgery from 1988 through 2000. J Bone Joint Surg Am. 2002;84-A(3):388–396. doi: 10.2106/00004623-200203000-00009. [DOI] [PubMed] [Google Scholar]
- 15.Poolman RW, Struijs PA, Krips R, Sierevelt IN, Lutz KH, Bhandari M. Does a “Level I Evidence” rating imply high quality of reporting in orthopaedic randomised controlled trials? BMC Med Res Methodol. 2006;6:44. doi: 10.1186/1471-2288-6-44. [DOI] [PMC free article] [PubMed] [Google Scholar]