Scientific research output in orthopaedics from China and other top-ranking countries: a 10-year survey of the literature

Abstract

Objectives

Orthopaedics-related diseases and conditions are a significant burden worldwide. In this study, we aimed to compare the quantity and quality of research output in the field of orthopaedics from Mainland China (MC), USA, UK, Japan and Germany.

Setting

The USA, UK, Japan, Germany and MC.

Participants

We selected orthopaedics journals from the subject category ‘orthopedics’ from the Science Citation Index Expanded (SCIE).

Outcome measures

The number of publications, the number of publications in the surveyed publication types, impact factor (IF) and citations from the corresponding country from 2005 to 2014 were collected for quantity and quality comparisons.

Results

A total of 128 895 articles were published worldwide in orthopaedics-related journals from 2005 to 2014. The USA contributed the largest proportion (31 190 (24.20%)), followed by the UK (6703 (5.20%)), Japan (5718 (4.41%)), Germany (4701 (3.66%)) and MC (3389 (2.63%)). Publications from MC represented the fewest, but this quantity is rapidly increasing. The quantity of annual publications from MC has exceeded that of Germany since 2012. The USA plays a predominant role in all kinds of publication types under investigation in the study, except in the category of meta-analysis. MC was in the last place for cumulative IFs, and the average IF actually decreased from the beginning of the study. For total and average citations, MC still lags behind the other countries in the study.

Conclusions

The USA has occupied the dominant place in orthopaedics-related research for the last 10 years. Although MC has made great progress in the number of published works in the field of orthopaedics over the last 10 years, the quality of these publishing efforts needs further improvement.

Strengths and limitations of this study.

• Only journals listed in the Science Citation Index Expanded (SCIE) orthopedics category were identified. Some other orthopaedics-related journals not included in SCIE were not considered for this study.

• Some articles pertaining to orthopaedics that were published in some general journals were not included in our study.

• The country of origin of an article was classified according to the affiliation of the first author, but some of the articles were efforts of international collaboration.

• The top 10 high-impact journals were selected solely based on impact factor (IF).

Introduction

Orthopaedic diseases and conditions represent a significant burden worldwide. Globally, by 2013, two of the five leading causes of disability-adjusted life years were orthopaedics related.¹ In China, musculoskeletal disorders accounted for 25.8% of the causes for which adults live with disability.²

Over the last few decades, due to rapid improvement in the economy, Mainland China (MC)'s gross domestic spending on research and experimental development (R&D) has grown from US$78.7 billion in 2005 to US$317.8 billion in 2014, and now this spending is second only after the USA, according to estimates made by the Organisation for Economic Co-operation and Development (OECD).³ MC has made remarkable progress in the field of medicine. To a certain extent, the quantity and quality of scientific publications are measurable indexes of the research impact of an individual⁴ and to a larger extent, a nation. As revealed by the Chinese Institute of Scientific and Technical Information in the Statistical Data of Chinese S&T Papers 2013,⁵ the USA, MC, Germany, Japan and the UK were the top five countries with the most scientific articles published. Articles have compared publications between MC, Hong Kong and Taiwan in the field of orthopaedics.^{6 7} However, little is known about the situation with regard to publication of scientific articles specifically in the field of orthopaedics in MC compared with the other top-ranking countries for the period covering the last 10 years.

In this study, we aimed to compare the contributions of Mainland Chinese researchers with contributions from the other top five most published countries in the field of orthopaedics between the years 2005 and 2014, and to provide a more accurate measure to evaluate the development status of orthopaedics in MC.

Methods

We included 73 orthopaedics journals from the orthopedics category of the Science Citation Index Expanded (SCIE) designed by Thomson Reuters.⁸ The full list of the journals, including the International Standard Serial Number (ISSN), full title, abbreviated journal title and impact factors (IFs) from 2014 included in our study, is shown in the appendix (see electronic supplementary material 1). All of the 73 journals in the orthopedics category, which could be retrieved by PubMed and Web of Science, cover resources on surgery and medical appliances as a means to preserve or restore function or alleviate pain in the musculoskeletal system, particularly the bones and joints. A computerised bibliographic retrieval was conducted on 29 September 2015, and the articles published in the 73 journals from the USA, the UK, Japan, Germany and MC between 1 January 2005 and 31 December 2014 were identified. Because the name of the JBJS Br journal was changed to Bone Joint J in 2013, articles from the journals with these two names were pooled together for this study. The entire retrieval and data extraction process was conducted in duplicate by two independent researchers (YZ and QL). Differences of opinion were solved though discussion, until agreement was reached. The full search strategy was included in the electronic supplementary material 2. Articles that showed the first author’s affiliation (AD) with these five countries were considered as research outputs from the countries. The numbers of each specific type of article such as clinical trials, randomised controlled trials (RCTs), meta-analysis, reviews and case reports were also identified according to the publication types generated by PubMed.

Three methods were used to evaluate the quality of articles. First, the cumulative and average IF were calculated according to Journal Citation Reports (JCR) 2014 published by Thomson Reuters.⁹ Second, citation reports for the literature from each region were collected through Web of Science.¹⁰ Third, the number of articles published in the top 10 high-impact orthopaedics journals (based on IF) were counted and the 10 most published orthopaedics journals for each region (based on the number of publications) were also identified.

Statistical analysis

The non-parametric test for trend and time series analysis was performed using SPSS V.13.0 (SPSS, Chicago, Illinois, USA) to determine any significant change over the study period. The Kruskal-Wallis test was used to detect the differences among the five countries, and rank-sum test between two countries, if necessary. The test for significance was two-tailed and the value of p<0.05 was considered significant.

Results

Total amount and share of publications

A total of 128 895 articles were published worldwide between 2005 and 2014 in orthopaedics-related journals. Overall, the USA contributed the largest proportion (31 190 (24.20%)), followed by the UK (6703 (5.20%)), Japan (5718 (4.41%)), Germany (4701 (3.66%)) and China (3389 (2.63%)) (figure 1). Despite such large publication numbers, the share of publication numbers attributed to the USA has decreased for the last 10 years. However, the total number and share of publications from MC increased significantly from 2005 to 2014 (31 to 768, respectively, p<0.01, figure 1A and 0.43% to 4.26%, respectively, p<0.01, figure 1B). From 2012 onwards, the number of articles from MC has exceeded that of Germany.

Figure 1.

(A, B) The number (A) and share (B) of papers published in orthopaedics journals from the top five most published countries.

Publication types

The number of different article types, which include RCTs, clinical trials, reviews, case reports and meta-analysis published by each region, are shown in figure 2. The USA accounts for the largest share and highest quality in all types of articles except meta-analysis. In the last decade, MC has published the largest number of meta-analysis among all five countries. In addition, MC had published more RCTs and reviews than Japan for the last 10 years in total.

Figure 2.

The number of papers of each different publication type (including RCTs, clinical trials, reviews, case reports and meta-analysis) from different countries. RCT, randomised control trial.

Impact factors

Based on the Journal Citation Reports (JCR) 2014, we calculated the cumulative and average IF for each region in each year from 2005 to 2014. During the last decade, the total cumulative IFs were ranked in the following order: USA, Japan, UK, Germany and MC; MC was listed in the last place (p<0.01). According to the average IF calculation, the UK was lower than the USA (p<0.001), Germany (p<0.001), Japan (p=0.003) and MC (p=0.018) for the past 10 years. No statistical differences were detected between MC and the USA (p=0.119), Germany (p=0.055) and Japan (p=0.534). The details of cumulative and average IFs each year from each country are listed in table 1.

Table 1.

Cumulative and average IFs for articles from the five countries

Year	Cumulative IF					Average IF
	USA	UK	Japan	Germany	MC	USA	UK	Japan	Germany	MC
2005	5865.896	1056.832	1089.882	634.944	78.716	2.047	1.929	2.112	2.116	2.385
2006	6425.466	1045.266	1120.177	831.133	140.899	2.061	1.883	2.114	2.199	2.168
2007	6482.460	1195.260	1103.435	848.391	222.186	2.094	1.853	2.138	2.215	2.314
2008	6262.057	1094.861	1072.712	893.142	303.679	2.024	1.777	2.039	2.142	2.266
2009	7880.583	1135.516	1154.643	910.234	506.102	2.447	1.769	2.073	2.073	1.763
2010	6636.580	1035.284	1123.844	1084.724	561.488	2.126	1.681	2.025	2.135	1.707
2011	8127.619	1136.579	1178.769	1071.755	852.724	2.339	1.645	2.072	2.144	1.814
2012	7885.948	1311.986	1279.197	1089.660	997.298	2.087	1.676	2.508	2.137	1.885
2013	7716.064	1359.962	1308.423	1275.919	1279.832	2.387	1.828	2.035	2.137	1.888
2014	5048.107	1708.500	1637.344	1524.299	1555.280	2.305	1.980	2.057	2.278	2.025
Total	68 330.780	12 080.046	12 068.426	10 164.201	6498.204	2.191	1.802	2.111	2.162	1.917

IF, impact factor; MC, Mainland China.

Citation reports

As shown in table 2, the USA had the highest total citations and the UK had the highest average number of citations per article over the 10 years, while MC had the lowest total or average number of citations. However, citations to articles from MC grew rapidly from 2005 to 2011. By 2014, the gap between Japan, Germany and MC was quite narrow. In fact, MC has exceeded Japan in annual citations since 2011.

Table 2.

Total and average citations of articles from the five countries

Year	USA	UK	Japan	Germany	MC
2005	79 575	13 760	9455	9386	1348
2006	78 313	12 996	8943	11 592	1483
2007	71 782	13 202	7624	9287	1987
2008	62 347	13 924	6089	9619	2536
2009	54 476	10 722	5989	8886	3487
2010	45 068	9683	4892	8150	3275
2011	36 709	9954	4135	6303	4147
2012	27 476	7898	3380	4414	3683
2013	17 455	7447	2011	3234	2598
2014	8162	4913	1226	1405	1243
Total citations	481 363	104 499	53 744	72 276	25 787
Average citations	27.63	35.52	11.68	18.76	2.49

MC, Mainland China.

Top 10 high-impact orthopaedics journals

The top 10 high-impact journals were selected according to their IFs in 2014. Articles from each region published in these journals from 2005 to 2014 were counted. Again, the USA (8820 articles) had the biggest share, far more than the combination of the other four countries. In addition, 28.4% of the articles from the USA were published in the top 10 high-impact journals, while only 13.7% from MC were published in those journals (table 3). If journals that were closely related to physical therapy and with a small volume of publication (J Physiother and J Orthop Sports Phys Ther) were excluded and Spine (IF=2.297) and J Arthroplasty (IF=2.666), two highly influential journals, were included in the analysis, the percentage would have been 27.4% for MC, much higher than the previous result.

Table 3.

Articles in the top 10 high-impact orthopaedics journals from the five countries

Rank	Journal title	2014 IF	USA	UK	Japan	Germany	MC
1	J Bone Joint Surg Am	5.280	1729	98	126	102	33
2	Am J Sports Med	4.362	1424	54	154	155	33
3	Osteoarthritis Cartilage	4.165	600	186	119	93	64
4	J Physiother	3.708	107	8	1	0	0
5	Arthroscopy	3.206	902	41	165	136	70
6	Knee Surg Sports Traumatol Arthrosc	3.053	309	195	267	351	88
7	J Orthop Sports Phys Ther	3.011	345	19	16	10	0
8	J Orthop Res	2.986	546	59	200	174	88
9	Acta Orthop	2.771	51	65	45	80	11
10	Clin Orthop Relat Res	2.765	2789	134	182	129	75
Total			8802	859	1230	1275	462
Divided by total article volume of each country (%)			28.40	14.88	22.06	27.54	13.76

IF, impact factor; MC, Mainland China.

Most published orthopaedics journals

The details for the top 10 most published journals in each region are listed in table 4. Four of the top 10 most published orthopaedics journals in the USA (Clin Orthop Relat Res, J Bone Joint Surg Am, Am J Sports Med, Arthroscopy) were listed in the top 10 high-impact journals. Meanwhile, three journals in Germany (Knee Surg Sports Traumatol Arthrosc, J Orthop Res, Arthroscopy) and three journals in Japan (Knee Surg Sports Traumatol Arthrosc, J Orthop Res, Clin Orthop Relat Res), and one journal in MC (J Orthop Res), but none in the UK were ranked in the top 10 high-impact journals.

Table 4.

Top 10 most published orthopaedics journals in the five countries

	USA	N	UK	N	Japan	N	Germany	N	MC	N
1	CORR	2789	BJR	981	JOC	856	AOTS	555	Spine	390
2	Spine	2088	Injury	547	Spine	677	KSSTA	351	ESJ	318
3	JBJS	1729	JHS-E	271	ESJ	269	ESJ	335	Int Orthop	283
4	Orthopedics	1657	HIP INT	263	KSSTA	267	Int Orthop	294	Orthopade	238
5	AJSM	1424	Knee	260	JA	228	Injury	240	Orthopedics	176
6	JFAR	1420	ESJ	237	JHS-A	213	Spine	205	AOTS	175
7	SPINE J	1166	SR	234	AOTS	206	BMD	186	Injury	152
8	JHT	1086	BMD	226	JOR	200	JOR	174	JSDT	124
9	JAAOS	914	BJJ	216	CORR	182	AJSM	155	BMD	113
10	Arthroscopy	902	Int Orthop	208	JSDT	177	Arthroscopy	136	JOR	88

AJSM, Am J Sports Med, IF=4.362; AOTS, Arch Orthop Trauma Surg, IF=1.597; Arthroscopy, IF=3.206; BJJ, Bone Joint J, IF=1.961; BJR, Bone Joint Res, IF=1.64; BMD, BMC Musculoskelet Disord; IF=1.717; CORR, Clin Orthop Relat Res, IF=2.765; ESJ, Eur Spine J, IF=2.066; Hip Int, IF=0.756; Injury, IF=2.137; Int Orthop, IF=2.11; JA, J Arthroplasty, IF=2.666; JAAOS, J Am Acad Orthop Surg, IF=2.527; JBJS, J Bone Joint Surg Am, IF=5.28; JFAR, J Foot Ankle Res, IF=1.462; JHS-A, J Hand Surg Am, IF=1.667; JHS-E, J Hand Surg Eur Vol, IF=2.037; JHT, J Hand Ther, IF=2; JOC, J Orthop Sci, IF=0.941; JOR, J Orthop Res, IF=2.986; JSDT, J Spinal Disord Tech, IF=2.202; Knee, IF=1.936; KSSTA, Knee Surg Sports Traumatol Arthrosc, IF=3.053; Orthopade, IF=0.359; Orthopedics, IF=0.962; Spine, IF=2.297; SPINE J, IF=2.426; SR, Skeletal Radiol, IF=1.51.

IF, impact factor; MC, Mainland China.

Discussion

Our study compared the quantity and IF of scientific publications in the field of orthopaedics from MC with the USA, the UK, Japan and Germany. These five countries had the largest scientific output in the world and were also the top five areas with the highest GDP.

In 2005, only 31 articles in orthopaedics journals were from MC, and rapid growth in absolute number and share of publications was observed from 2005 to 2014. The number of articles from MC in 2014 reached almost 25 times the quantity of 2005 and that number has exceeded Germany and Japan since 2011 and is now quite close to that of the UK. An increase in R&D funding in addition to improved economic status has undoubtedly been the main reason for such progress in MC's output in scientific reports.

Well-designed, conducted and reported RCTs represent the gold standard in evaluating healthcare intervention.¹¹ It is noteworthy that the number of RCTs from China has exceeded that of Japan in the last decade, indicating that a greater quantity of original work is made available by China. Furthermore, our results also revealed that MC published more meta-analysis than the other four countries. In fact, a 10-fold increase in the number of orthopaedic systematic reviews and/or meta-analysis was observed in the past 10 years.¹² It is worth mentioning that though meta-analyses are secondary research, they provide one of the best tools for quality clinical evidence on very specific topics, and well-performed meta-analyses are the best evidence in the hierarchy of clinical evidence.

The IF for an academic journal is frequently used for measuring and comparing the influence of the journal. Journals with higher IF are generally considered to be more important and more influential.^{13 14} We took IF as an objective parameter evaluating the quality of publications from each region. However, it is possible that articles published in journals with low IF may be excellent work, and the opposite situation could also be the case. Thus, we further compared average number of citations of articles from each country. The data on cumulative and average IF are interesting in that it makes the point that while more publications came out of MC in recent years, this quantity did not change the average IF, which actually went down (table 1). The data on total and average citations are actually also quite telling, as it shows that although MC has greatly increased its number of publications, the average number of citations per article is extremely low compared with the other countries (table 2). The same result was found in publication status in the top 10 high-impact orthopaedics journals. All these data indicate that orthopaedics researchers from MC should be looking at improving the quality of their publications. Nevertheless, the very recent nature of the boom in publications from MC might also contribute to the low average number of citations from MC.

Some of the limitations with these articles should be addressed. First, we focused on publications only in the journals listed in the SCIE database ‘orthopedics’ category; there may be some good orthopaedics journals that were not included by SCIE. Second, some published articles in the journals included in our analysis may not be related closely to orthopaedics, while some articles pertaining to orthopaedics may have been published in some general journals that were not included in our study. Finally, we decided on the country of origin of a paper based on the affiliation of the first author, which was consistent with similar studies in other fields;^{15 16} however, some of the articles may be international collaborative efforts. Therefore, the contributions of other countries were ignored.

Conclusion

The number and share of scientific research articles from Chinese authors are increasing every year. These numbers are now comparable to the UK, Japan and Germany. However, the general quality of publications from MC is still in need of improvement. Of note and worth mentioning is the fact that articles from the UK, despite having the lowest average IF, have the highest average number of citations.

As the second largest economy in the world with a population of 1.3 billion, MC has great potential in the field of orthopaedics. However, there is still room for considerable improvement on the part of researchers in MC to achieve their potential. The world will benefit from even better performance in the field of orthopaedics research from China.