Worldwide orthopaedic research activity 2010-2014: Publication rates in the top 15 orthopaedic journals related to population size and gross domestic product

Abstract

AIM

To perform a bibliometric analysis of publications rates in orthopedics in the top 15 orthopaedic journals.

METHODS

Based on their 2015 impact factor, the fifteen highest ranked orthopaedic journals between January 2010 and December 2014 were used to establish the total number of publications; cumulative impact factor points (IF) per country were determined, and normalized to population size, GDP, and GDP/capita, comparison to the median country output and the global leader.

RESULTS

Twenty-three thousand and twenty-one orthopaedic articles were published, with 66 countries publishing. The United States had 8149 publications, followed by the United Kingdom (1644) and Japan (1467). The highest IF was achieved by the United States (24744), United Kingdom (4776), and Japan (4053). Normalized by population size Switzerland lead. Normalized by GDP, Croatia was the top achiever. Adjusting GDP/capita, for publications and IF, China, India, and the United States were the leaders. Adjusting for population size and GDP, 28 countries achieved numbers of publications to be considered at least equivalent with the median academic output. Adjusting GDP/capita only China and India reached the number of publications to be considered equivalent to the current global leader, the United States.

CONCLUSION

Five countries were responsible for 60% of the orthopaedic research output over this 5-year period. After correcting for GDP/capita, only 28 of 66 countries achieved a publication rate equivalent to the median country. The United States, United Kingdom, South Korea, Japan, and Germany were the top five countries for both publication totals and cumulative impact factor points.

Core tip: The total number of publications by a country is one of the best indicators of research output and productivity, and is an important aspect of clinical excellence. Our results demonstrate that the United States collectively published more articles and accumulated the highest number of impact factors during the study period, and confirms its overwhelming dominance of publications in the fifteen highest ranked journals in orthopaedics. However, after adjusting for population size, Switzerland was the most academically productive nation. Similarly, after adjusting the number of publications with respect to GDP, Croatia was the most productive, and “cost effective” country.

INTRODUCTION

The total number of publications by a country is one of the best indicators of research output and productivity[1], and is an important aspect of clinical excellence[2,3]. Prior bibliographic analyses of orthopaedic academic output have concentrated on the total number of publications per country over various periods ranging from five to ten years[4-6]. The United States, United Kingdom, Germany, Japan, and South Korea have all consistently ranked among the five most productive countries.

The availability of funding has been shown to result in higher publication output, favoring those countries with a larger population size and more powerful economies[6,7]. However, no prior bibliographic analysis of orthopaedic research and publications has accounted for population size or economic discrepancies. To adjust for these inconsistencies, the use of the gross domestic product (GDP) and gross domestic product per capita (GDP/capita) may provide a more meaningful result, and allow for a better comparison between countries[8]. Although the number of publications per capita is one simple way to minimize this inherent bias, it is not the only approach that can be used to determine how academically productive various nations have been. The reciprocal, population size per publication for example, is an equally valid metric that perhaps better expresses this relationship. This reciprocal approach has been employed instead in various iterations throughout this study, to more directly investigate how academically active each nation has been in the field of orthopaedics over the past five years.

Using the fifteen highest rated orthopaedic journals over a five year period, based on the 2015 impact factor, the purpose of this study was threefold: First, to investigate the number of publications and total impact factor from each country, and to then relate these variables to population size, GDP, and GDP per capita. Second, to determine the minimum number of publications required to be comparable to the country producing the median number of publications, when normalized for GDP per capita. Finally, to establish the number of publications that would be required from each country to be equivalent to the country having the highest research output, when normalized for GDP per capita.

MATERIALS AND METHODS

The 2015 Journal Citation report was accessed on the Web of Science (Thomson Reuters, New York, United States)[9], and the fifteen highest ranked journals based on their 2015 impact factor were selected from the category “orthopedics”. Journals were excluded from this list if they were not directly related to the field of orthopedics, or if their main purpose was to provide narrative review articles (Table 1). The abstracts of all articles published in these 15 journals between January 2010 and December 2014 were screened via the journals’ websites. Letters to the editor, editorials, editorial comments, historical articles, errata, proceeding papers, meeting abstracts, and notes were excluded. Only research articles (levels 1-4), systematic reviews, meta-analyses, non-solicited review articles, and case reports were included. The level of evidence was recorded for each published article; if the journal did not assign the level of evidence, the levels of evidence chart published by the Journal of Bone and Joint Surgery was used[10]. Each publication was assigned a country of origin defined by the location of the the authors’ principal institution, or defined by the country of origin of the corresponding author if the manuscript did not provide details about study location. Any discrepancies were resolved by agreement between the two senior authors. The total number of publications and the total number of impact factor points per country were collated.

Table 1.

Impact factors (2015 Journal Citation Reports - Thomson Reuters) and number of included publications from 2010-2014

	Journal	Impact points	Publications 2010-2014
1	Journal of Bone and Joint - American Volume	5.280	1833
2	American Journal of Sports Medicine	4.362	1561
3	The Bone and Joint Journal	3.309	1379
4	Arthroscopy – The Journal of Arthroscopic and Related Surgery	3.206	1072
5	Knee Surgery Sports Traumatology Arthroscopy	3.053	1747
6	Journal of Orthopaedic Research	2.986	1301
7	Acta Orthopaedica	2.771	565
8	Clinical Orthopaedics and Related Research	2.765	2027
9	Journal of Arthroplasty	2.666	1873
10	Spine Journal	2.426	1029
11	Spine	2.297	2848
12	Journal of Shoulder and Elbow Surgery	2.289	1324
13	Injury – International Journal of the Care of the Injured	2.137	1133
14	International Orthopaedics	2.110	1477
15	European Spine Journal	2.066	1852
	Total number of publications		23021

Excluded journals: Osteoarthritis Cartilage ( No. 3 - IF: 4.165); Journal of Physiotherapy (No.4 - IF: 3.708); Journal of Orthopaedic Sports Physiotherapy (No. 8 - IF: 3.011); Gait Posture (No. 12 - IF: 2.752); Journal of the American Academy of Orthopaedic Surgeons (No. 14 - IF: 2.527); Physical Therapy (No 15 - IF: 2.526); Clinical Journal Sports Medicine (No 19 - IF: 2.268)

GDP and GDP per capita were sourced from the World Bank website[11], and population size was extracted from the CIA World Factbook[12]. To describe the relationship between population size and the number of publications from a given nation, the population size of that country was divided by their total number of publications. The resulting value describes the population size per publication (PSPP) for that nation; in other words, the calculated value defines the population size per published article, allowing for a better and more direct comparison accounting for population size. Likewise, to define the population size per impact factor point (PSIP) from a given nation the population of that country was divided by their total impact factor points.

Extending this analysis, the gross domestic product was also divided by the total number of publications and impact factor points. These values provide an overview of the gross cost associated with producing a manuscript (GDPP), as well as the gross cost associated with producing one impact factor point (GDPI) for each country. Finally, to simultaneously adjust for population size and economic strength, the GDP per capita was divided by either the total number of publications or by cumulative impact factor points. These values then provide information regarding the gross cost per capita associated with producing a manuscript (GDPCP), or the gross cost per capita associated with producing one impact factor point (GDPCI) for each country.

The list for GDPCP was next ranked lowest to highest to identify the median country. This median country then served as the benchmark, and a correction coefficient was calculated that was normalized to this median country. In this way the number of publications of the median country could then be used to calculate the number of publications every country would need to produce to be considered equivalent to that median country. Dividing the GDPCP of each country by this normalizing coefficient, (NC_med) determined the number of publications that would be necessary for each country to produce to be considered equivalent to the median country. This provides an excellent measure, corrected for economic power (GDP/capita) and population size, of the expected academic output of different countries, normalized to the output of the median nation.

Finally, a very similar process was followed where a correction coefficient was determined that was instead normalized to the publication output of the current global leader in orthopaedic research. The most active country then served as the benchmark, and a coefficient was calculated that was normalized to the academic activity of that country (NC_top). This value was then used to calculate the number of publications every country would need to produce to be considered equivalent to the global leader. Dividing the GDPCP of each country by this NC_top thus determines the number of publications that would be necessary for each country to produce to be considered equivalent to the global leader. This provides an excellent measure, corrected for economic power (GDP/capita) and population size, of the expected academic output of different countries, normalized to the output of the leading nation.

RESULTS

A total of 23021 orthopaedic articles were published in the 15 highest ranked orthopaedic surgery journals during the study period, between January 2010 and December 2014 (Table 1). Table 2 demonstrates the top ten countries for each of the fifteen journals, in terms of number of publications. The United States was consistently the leading country in ten of the fifteen journals, and was also the most productive country with a total of 8149 publications; they were followed by the United Kingdom and Japan, having 1644 and 1467 publications, respectively. A total of 66 countries had published at least one article (Table 3) during the study period. Similar to the number of publications, the United States also accumulated the largest number of impact factor points (24744) followed by the United Kingdom (4776) and Japan (4053) (Table 3). Overall, the top five countries were the United States, United Kingdom, Japan, South Korea, and Germany, and these countries were together responsible for 60.4% of all publications, and 61.4% of all impact factor points.

Table 2.

Top 10 Number of publications per country for each of the 15 selected journals

Journal	1	2	3	4	5	6	7	8	9	10
JBJS-Am	USA–1124	CAN-107	KOR-84	UK-75	JAP-52	HOL-46	GER-45	FRA-39	SWIS-37	AUS-27
Am J Sports Med	USA-819	KOR-117	JAP-84	GER-82	UK-49	AUS-40	ITA-25	CAN-34	SWE-32	SWIS-31
BJJ	UK-545	USA-115	KOR-76	JAP-75	HOL-50	CAN-46	AUS-43	GER-41	CHINA-35	SWIS-31
Arthroscopy	USA-513	KOR-105	JAP-63	GER-55	CHINA-40	CAN-34	ITA-27	UK-22	FRA-18	SPAIN-18
KSSTA	USA-242	GER-195	KOR-157	ITA-149	JAP-144	UK-85	HOL-76	TURK-70	SWE-64	CHINA-62
J Orthopaedic Research	USA-535	JAP-107	GER-96	CAN-69	CHINA-67	UK-48	TAIW-45	AUS-37	KOR-31	HOL-31
Acta Orthopaedica	SWE-125	DEN-76	NOR-69	HOL-59	FIN-40	GER-34	UK-34	USA-21	JAP-17	AUS-13
CORR	USA-1155	CAN-110	KOR-98	JAP-71	UK-60	SWIS-60	GER-59	FRA-49	ITA-45	HOL-33
J Arthroplasty	USA-934	JAP-136	CAN-124	UK-117	KOR-114	AUS-72	CHINA-64	GER-37	SPAIN-29	HOL-26
Spine Journal	USA-491	KOR-78	CHINA-62	JAP-56	CAN-48	HOL-29	UK-24	SWIS-23	INDIA-21	ITA-21
Spine	USA-1168	JAP-307	CHINA-255	CAN-166	KOR-163	UK-73	GER-65	AUS-59	HOL-57	TAIW-49
J Shoulder Elbow Surg	USA-659	JAP-79	UK-72	KOR-65	CAN-60	SWIS-49	FRA-42	GER-36	ITA-35	BELG-34
Injury	UK-215	USA-126	GER-114	ITA-89	CHINA-78	HOL-57	GREEC-48	SPAIN-37	SWIS-34	AUS-34
International Orthopaedics	GER-232	CHINA-198	UK-101	USA-97	FRA-97	JAP-81	ITA-76	A-76	CRO-54	SWIS-49
European Spine Journal	CHINA-251	JAP-182	GER-161	USA-150	ITA-133	UK-124	FRA-104	KOR-90	SWIS-84	HOL-81

Excluded journals: Osteoarthritis Cartilage ( No. 3 - IF: 4.165); Journal of Physiotherapy (No.4 - IF: 3.708); Journal of Orthopaedic Sports Physiotherapy (No. 8 - IF: 3.011); Gait Posture (No. 12 - IF: 2.752); Journal of the American Academy of Orthopaedic Surgeons (No. 14 - IF: 2.527); Physical Therapy (No 15 - IF: 2.526); Clinical Journal Sports Medicine (No 19 - IF: 2.268). USA: United States; UK: United Kingdom; SWE: Sweden; GER: Germany; Can: Canada; Kor: Korea; JAP: Japan.

Table 3.

Highest number of publications and impact points for each country

Rank	Country	Publications	Rank	Country	Impact points
1	United States	8149	1	United States	24744
2	United Kingdom	1644	2	United Kingdom	4776
3	Japan	1467	3	Japan	4053
4	South Korea	1354	4	South Korea	3765
5	Germany	1272	5	Germany	3491
6	China	1222	6	China	3034
7	Canada	930	7	Canada	2774
8	Italy	737	8	Holland	2155
9	Holland	663	9	Italy	1982
10	France	548	10	Switzerland	1507
11	Switzerland	527	11	Australia	1412
12	Australia	485	12	France	1382
13	Sweden	403	13	Sweden	1187
14	Spain	311	14	Spain	833
15	Austria	295	15	Austria	801
16	Taiwan	264	16	Norway	755
17	Denmark	254	17	Taiwan	729
18	India	246	18	Denmark	710
19	Norway	240	19	India	646
20	Turkey	235	20	Turkey	630
21	Belgium	219	21	Belgium	614
22	Greece	182	22	Greece	508
23	Finland	167	23	Brazil	408
24	Brazil	147	24	Finland	402
25	Hong Kong	130	25	Hong Kong	371
26	Israel	119	26	Israel	315
27	Ireland	98	27	Singapore	295
28	Singapore	84	28	Ireland	262
29	New Zealand	78	29	New Zealand	227
30	Croatia	74	30	Iran	174
31	Egypt	68	31	Egypt	168
32	Iran	65	32	Croatia	159
33	Poland	61	33	Poland	141
34	Thailand	52	34	Thailand	128
35	Czech Republic	39		Slovenia	128
36	Slovenia	32	35	Czech Republic	84
37	Hungary	29	36	Hungary	71
38	Portugal	25	37	Portugal	71
39	Chile	24	38	Chile	66
40	Malaysia	23	39	Malaysia	63
41	South Africa	21	40	South Africa	59
42	Argentina	20	41	Argentina	55
43	Serbia	19	42	Serbia	43
44	Luxemburg	14	43	Luxemburg	43
45	Saudi Arabia	12	44	Saudi Arabia	29
46	Mexico	10	45	Mexico	26
47	Lebanon	9	46	Lebanon	23
	Lithuania	9		Lithuania	23
	Russia	9	47	Russia	21
48	Estonia	7	48	Estonia	17
48	Nigeria	7	49	Nigeria	15
49	Pakistan	6	50	Romania	13
	Romania	6		Philippines	13
50	Columbia	5	51	Pakistan	12
	Kuwait	5	52	Columbia	11
	Philippines	5		Tunisia	11
	Tunisia	5	53	Kuwait	9
51	Bulgaria	3	54	Iceland	7
	Iceland	3	55	Bulgaria	6
	Iraq	3		Iraq	6
52	Malawi	2	56	Malawi	5
	Morocco	2		Nepal	5
	Nepal	2		Uganda	5
53	Ethiopia	1	57	Morocco	4
	Sudan	1	58	Ethiopia	3
	Uganda	1		Sudan	3

However, when adjusted for population size (PSPP), Switzerland was the leading country with one publication per 15300 people, followed by Norway with one publication per 21100, and Denmark with one publication per 22300. Switzerland was also the leader in the category of impact factor (PSPI), accumulating one impact factor point per 5400 people, followed by Norway with one impact factor point per 6700, and Holland with one impact factor point per 7800 (Table 4).

Table 4.

Number of publications (PSPP) and impact (PSPI) normalized for population size (publication/impact point per in thousand populations)

Rank	Country	PSPP	Rank	Country	PSIP
1	Switzerland	15.3	1	Switzerland	5.4
2	Norway	21.1	2	Norway	6.7
3	Denmark	22.3	3	Holland	7.8
4	Sweden	24.1	4	Denmark	7.9
5	Holland	25.4	5	Sweden	8.2
6	Austria	28.7	6	Austria	10.6
7	Finland	32.3	7	Canada	12.1
8	Canada	35.9	8	Luxemburg	12.6
9	Luxemburg	38.9	9	United States	12.9
10	South Korea	38.9	10	United Kingdom	13.4
11	United Kingdom	38.9	11	Finland	13.4
12	United States	39.3	12	South Korea	13.6
13	Australia	44.3	13	Australia	15.2
14	Belgium	51.1	14	Belgium	18.2
15	Hong Kong	55.3	15	Singapore	18.3
16	New Zealand	57.3	16	Hong Kong	19.4
17	Croatia	57.8	17	New Zealand	19.7
18	Greece	60.4	18	Greece	21.6
19	Germany	63.1	19	Germany	23
20	Singapore	64.3	20	Slovenia	24
21	Slovenia	64.3	21	Ireland	24.3
22	Ireland	65.1	22	Israel	25.6
23	Israel	67.7	23	Croatia	27
24	Italy	82.4	24	Italy	30.7
25	Japan	86.8	25	Japan	31.4
26	Taiwan	88.4	26	Taiwan	32
27	Iceland	107.7	27	Iceland	46.1
28	France	121.5	28	France	48.1
29	Spain	151.9	29	Spain	56.7
30	Estonia	185.7	30	Estonia	76.5
31	Czech Republic	269.2	31	Turkey	121.7
32	Turkey	326.2	32	Czech Republic	125
33	Lithuania	333.3	33	Lithuania	130.4
34	Hungary	341.4	34	Hungary	139.4
35	Serbia	379.5	35	Portugal	147.3
36	Portugal	418.4	36	Serbia	167.7
37	Lebanon	551.8	37	Lebanon	215.9
38	Poland	631.6	38	Chile	247.6
39	Kuwait	673.8	39	Poland	272
40	Chile	680.8	40	Kuwait	374.3
41	China	1110.5	41	Iran	443.5
42	Egypt	1176.5	42	China	447.3
43	Iran	1187.3	43	Malaysia	471.7
44	Thailand	1283.1	44	Egypt	476.2
45	Malaysia	1292.1	45	Brazil	491.2
46	Brazil	1363.2	46	Thailand	521.2
47	Argentina	2072.5	47	Argentina	753.6
48	Tunisia	2178.0	48	South Africa	915.2
49	Saudi Arabia	2402.5	49	Tunisia	990
50	Bulgaria	2421.7	50	Saudi Arabia	12108.3
51	South Africa	2571.4	51	Bulgaria	15353.8
52	Romania	3326.7	52	Romania	15353.9
53	India	5089.4	53	India	19380.8
54	Malawi	8180.0	54	Malawi	32720
55	Ethiopia	9410.0	55	Columbia	43649
56	Columbia	9602.8	56	Mexico	45536.5
57	Iraq	11140	57	Nepal	55600
58	Mexico	11839.5	58	Iraq	55700
59	Nepal	13900	59	Russia	68333.3
60	Russia	15944.4	60	Uganda	75160.0
61	Morocco	16505	61	Philippines	75684.6
62	Philippines	19678	62	Morocco	82525.0
63	Nigeria	24800	63	Nigeria	115733.3
64	Pakistan	32695.7	64	Sudan	126533.3
65	Uganda	37580	65	Pakistan	163478.3
66	Sudan	37976	66	Ethiopia	313666.7

The number of publications, when normalized with respect to economic activity (GDPP), was highest for Croatia, with one publication per $772000, followed by Korea with $1042000, and Greece with $1294000. For impact factor (GDPI) Croatia was again the leader, and produced one impact factor point per $359000, followed by South Korea with $375000, and Holland with $408000 (Table 5). When adjusting for both GDP and population simultaneously (GDPCP) China was the leader, producing one publication per $6200, followed by India with $6400, and the USA with $6700. The United States was the leader in the impact factor category (GDPCI), producing one impact factor point per $2200, followed by India with $2400 and China $2500 (Table 6). However, these results need to be interpreted carefully, and it is probable that the extremely large population size of both China and India resulted in data distortion.

Table 5.

Number of publications (GDPP) and impact points (GDPI) related to GDP ( in thousand dollars)

Rank	Country	GDPP	Rank	Country	GDPI
1	Croatia	772	1	Croatia	359
2	South Korea	1042	2	South Korea	375
3	Greece	1294	3	Holland	408
4	Holland	1326	4	Greece	464
5	Switzerland	1330	5	Switzerland	465
6	Denmark	1348	6	Sweden	481
7	Sweden	1417	7	Denmark	482
8	Slovenia	1417	8	Slovenia	576
9	Austria	1547	9	Austria	579
10	Finland	1630	10	United Kingdom	626
11	United Kingdom	1818	11	Canada	644
12	Taiwan	1852	12	Norway	662
13	Canada	1920	13	Taiwan	671
14	Norway	2083	14	Finland	677
15	Malawi	2129	15	United States	704
16	United States	2138	16	Hong Kong	784
17	Hong Kong	2237	17	New Zealand	829
18	Serbia	2309	18	Malawi	852
19	New Zealand	2412	19	Belgium	866
20	Belgium	2427	20	Israel	970
21	Ireland	2559	21	Ireland	975
22	Israel	2569	22	Serbia	1020
23	Italy	2905	23	Australia	1032
24	Australia	3003	24	Singapore	1044
25	Germany	3041	25	Italy	1080
26	Japan	3137	26	Germany	1108
27	Turkey	3398	27	Japan	1135
28	Singapore	3665	28	Turkey	1267
29	Estonia	3784	29	Luxemburg	1509
30	Egypt	4213	30	Estonia	1558
31	Spain	4442	31	Spain	1658
32	Hungary	4471	32	Egypt	1706
33	Luxemburg	4634	33	Hungary	1949
34	Lebanon	5081	34	Lebanon	1988
35	France	5163	35	France	2047
36	Czech Republic	5263	36	Lithuania	2102
37	Lithuania	5372	37	Iceland	2434
38	Iceland	5679	38	Czech Republic	2444
39	Iran	6543		Iran	2444
40	Thailand	7785	39	Thailand	3163
41	India	8327	40	India	3171
42	China	8474	41	Portugal	3241
43	Poland	8933	42	China	3413
44	Portugal	9204	43	Poland	3865
45	Tunisia	9722	44	Chile	3910
46	Nepal	9884	45	Nepal	3954
47	Chile	10752	46	Tunisia	4419
48	Malaysia	14700	47	Malaysia	5367
49	Brazil	15960	48	Uganda	5400
50	South Africa	16671	49	Brazil	5750
51	Bulgaria	18906	50	South Africa	5934
52	Argentina	26833	51	Bulgaria	9452
53	Uganda	26998	52	Argentina	9757
54	Kuwait	32722	53	Romania	15311
55	Romania	33174	54	Kuwait	18179
56	Pakistan	40605	55	Ethiopia	18540
57	Morocco	55004	56	Pakistan	20303
58	Ethiopia	55621	57	Philippines	21906
59	Philippines	56955	58	Sudan	24734
60	Saudi Arabia	62187	59	Saudi Arabia	25733
61	Sudan	74202	60	Morocco	27502
62	Iraq	74503	61	Columbia	34340
63	Columbia	75448	62	Iraq	37251
64	Nigeria	81215	63	Nigeria	37901
65	Mexico	129469	64	Mexico	49796
66	Russia	206733	65	Russia	88600

Table 6.

Number of publications (GDPCP) and impact points (GDPCI) related to GDP per capita (in thousand dollars)

Rank	Country	GDPP	Rank	Country	GDPI
1	China	6.2	1	United States	2.2
2	India	6.4	2	India	2.4
3	United States	6.7	3	China	2.5
4	South Korea	20.7	4	South Korea	7.4
5	Japan	24.7	5	Japan	8.9
6	United Kingdom	28.2	6	United Kingdom	9.7
7	Germany	37.6	7	Germany	13.7
8	Turkey	44.7	8	Turkey	16.7
9	Egypt	47	9	Italy	17.6
10	Italy	47.4	10	Canada	18.1
11	Canada	54	11	Egypt	19
12	Brazil	77.4	12	Holland	24.2
13	France	78	13	Brazil	27.9
14	Holland	78.7	14	France	30.9
15	Iran	83.7	15	Iran	31.3
16	Spain	95.4	16	Spain	35.6
17	Thailand	114.9	17	Greece	42.3
18	Greece	118.1	18	Taiwan	43.8
19	Taiwan	120.8	19	Australia	43.9
20	Malawi	127.5	20	Thailand	46.7
21	Australia	127.7	21	Sweden	49.6
22	Sweden	146.2	22	Malawi	51
23	Switzerland	162.4	23	Switzerland	56.8
24	Austria	173.5	24	Austria	63.9
25	Croatia	182.1	25	Belgium	77.1
26	Belgium	216.2	26	Croatia	84.7
27	Pakistan	219.5	27	Denmark	85.5
28	Poland	235.1	28	Poland	101.7
29	Denmark	239	29	Hong Kong	108.3
30	Finland	298.3	30	Pakistan	109.7
31	South Africa	308.7	31	South Africa	109.9
32	Hong Kong	309	32	Israel	118.1
33	Israel	312.7	33	Finland	123.9
34	Serbia	323.8	34	Norway	128.9
35	Nepal	351	35	Nepal	140.4
36	Norway	405.4	36	Uganda
37	Nigeria	457.6	37	Serbia	143.1
38	Hungary	483.7	38	New Zealand	166.9
39	New Zealand	485.8	39	Malaysia	179.5
40	Malaysia	491.6	40	Singapore	190.8
41	Czech Republic	500.8	41	Ethiopia	191.3
42	Ireland	554.8	42	Hungary	197.6
43	Ethiopia	574	43	Ireland	207.5
44	Philippines	574.4	44	Nigeria	213.5
45	Chile	605.3	45	Chile	220.1
46	Argentina	625.4	46	Philippines	220.9
47	Singapore	670	47	Argentina	227.4
48	Uganda	715	48	Czech Republic	232.5
49	Slovenia	750	49	Slovenia	279.1
50	Tunisia	884.2	50	Portugal	311.7
51	Portugal	885.3	51	Sudan	371.7
52	Mexico	1032.6	52	Mexico	397.1
53	Sudan	1115	53	Tunisia	401.9
54	Lebanon	1117.6	54	Lebanon	437.3
55	Russia	1415.1	55	Russia	606.5
56	Columbia	1580.8	56	Lithuania	717.8
57	Morocco	1595	57	Columbia	718.5
58	Romania	1666.2	58	Romania	769
59	Lithuania	1834.1	59	Morocco	797.5
60	Saudi Arabia	2013.4	60	Saudi Arabia	833.1
61	Iraq	2140	61	Iraq	1070
62	Bulgaria	2617	62	Estonia	1186
63	Estonia	2880.3	63	Bulgaria	1308.5
64	Luxemburg	8333.1	64	Luxemburg	2713.3
65	Kuwait	8718.8	65	Kuwait	4843.8
66	Iceland	17334.5	66	Iceland	7429.1

When ranked with respect to GDPCP Poland was the median country, publishing 61 articles, and served as the median academic output benchmark. The results showed that 28 countries were able to achieve this academic output (Table 7). As an example, for the United States to achieve this benchmark a minimum of 235 publications were required; however, a total of 8149 publications were recorded, which was 3,468% greater than the requisite number. For Norway, to achieve this benchmark a minimum of 414 publications were required, but only 240 publications were recorded; this was only 58% of the number of publications necessary to have achieved an academic output equivalent to the median activity (Table 7).

Table 7.

Number of publications required to equivalent with the median (Poland n = 61) using the benchmark measure

Rank	Country	Published publica- tions 2010-2014	Papers to be published	% of published papers
1	China	1222	32	3783
2	India	246	7	3656
3	United States	8149	235	3505
4	South Korea	1354	119	1137
5	Japan	1467	235	952
6	United Kingdom	1644	197	833
7	Germany	1272	203	625
8	Turkey	235	45	525
9	Egypt	68	14	499
10	Italy	737	148	496
11	Canada	930	214	435
12	Brazil	147	48	303
13	France	548	182	301
14	Holland	663	222	298
15	Iran	65	23	280
16	Spain	311	126	246
17	Thailand	52	25	204
18	Greece	182	91	198
19	Taiwan	264	136	194
20	Malawi	2	1	184
21	Australia	485	263	183
22	Sweden	403	251	160
23	Switzerland	527	364	145
24	Austria	295	218	135
25	Croatia	74	57	129
26	Belgium	219	201	109
27	Pakistan	6	6	100
28	Poland	61	61	100
29	Denmark	254	258	98
30	Finland	167	212	79
31	South Africa	21	28	76
32	Hong Kong	130	171	76
33	Israel	119	158	75
34	Serbia	19	26	72
35	Nepal	2	3	67
36	Norway	240	414	58
37	Nigeria	7	14	50
38	Hungary	29	60	49
39	New Zealand	78	161	48
40	Malaysia	23	48	47
41	Czech Republic	39	83	47
42	Ireland	98	231	42
43	Ethiopia	1	2	50
44	Philippines	5	12	41
45	Chile	24	62	39
46	Argentina	20	53	38
47	Singapore	84	239	35
48	Uganda	1	3	33
49	Slovenia	32	102	31
50	Tunisia	5	19	26
51	Portugal	25	94	26
52	Mexico	10	44	23
53	Sudan	1	5	20
54	Lebanon	9	43	21
55	Russia	9	54	17
56	Columbia	5	34	15
57	Morocco	2	14	15
58	Romania	6	42	14
59	Lithuania	9	70	13
60	Saudi Arabia	12	103	12
61	Iraq	3	27	11
62	Bulgaria	3	33	9
63	Estonia	7	86	8.1
64	Luxemburg	14	496	2.8
65	Kuwait	5	185	2.7
66	Iceland	3	221	1.4

The United States was the leader when ranked with respect to GDPCP, publishing 8,149 articles, and served as the leading academic output nation. Using the NC_top to calculate the required number of publications to be equivalent with the global research leader (United States), only two other countries, China and India, were considered equivalent or superior (Table 8). For example, for Korea 4174 publications would have been needed to have an academic output equivalent to that of the United States, but only 1354 articles (32%) were published. Again, these results need to be interpreted carefully, and it is highly probable that the large population size of both China and India resulted in data distorsion.

Table 8.

Number of publications required to equivalent with the leader (United States) the benchmark measure

Rank	Country	Published publica-tions 2010-2014	Papers to be published	% of published papers
1	China	1222	1132	108
2	India	246	236	104
3	United States	8149	8149	100
4	South Korea	1354	4174	32
5	Japan	1467	5402	27
6	United Kingdom	1644	6915	24
7	Germany	1272	7138	18
8	Turkey	235	1569	15
9	Egypt	68	477	14
10	Italy	737	5210	14
11	Canada	930	7498	12
12	Brazil	147	1699	8.6
13	France	548	6378	8.6
14	Holland	663	7787	8.5
15	Iran	65	812	8
16	Spain	311	4429	7
17	Thailand	52	892	5.8
18	Greece	182	3208	5.6
19	Taiwan	264	892	5.5
20	Malawi	2	38	5.2
21	Australia	485	9243	5.1
22	Sweden	403	8797	4.6
23	Switzerland	527	12775	4.1
24	Austria	295	7640	3.9
25	Croatia	74	2011	3.7
26	Belgium	219	7068	3.1
27	Pakistan	6	197	3
28	Poland	61	2141	2.8
29	Denmark	254	9091	2.7
30	Finland	167	7436	2.2
31	South Africa	21	968	2.1
	Hong Kong	130	5995	2.1
	Israel	119	5553	2.1
	Serbia	19	918	2.1
32	Nepal	2	105	1.9
33	Norway	240	14523	1.6
34	Nigeria	7	487	1.5
35	Hungary	29	2094	1.4
	New Zealand	78	5656	1.4
	Malaysia	23	1688	1.4
36	Czech Republic	39	2915	1.3
	Ireland	98	8115	1.2
	Ethiopia	1	86	1.2
	Philippines	5	429	1.2
37	Chile	24	2168	1.1
	Argentina	20	1867	1.1
38	Singapore	84	8401	1
39	Uganda	1	107	0.94
40	Slovenia	32	3582	0.89
41	Tunisia	5	660	0.76
42	Portugal	25	3303	0.75
43	Mexico	10	1541	0.65
44	Sudan	1	166	0.6
	Lebanon	9	1502	0.6
45	Russia	9	1901	0.47
46	Columbia	5	1180	0.42
47	Morocco	2	476	0.42
48	Romania	6	1492	0.4
49	Lithuania	9	2464	0.36
50	Saudi Arabia	12	3606	0.33
51	Iraq	3	958	0.31
52	Bulgaria	3	1172	0.26
53	Estonia	7	3009	0.23
54	Luxemburg	14	17412	0.08
	Kuwait	5	6507	0.08
55	Iceland	3	7762	0.04

DISCUSSION

These results demonstrate that the United States collectively published more articles and accumulated the highest number of impact factor points during the study period from 2010 through 2014, and confirms its overwhelming dominance of publications in the fifteen highest ranked journals in the field of orthopaedics. However, after adjusting for population size, Switzerland was the most academically productive nation. Similarly, after adjusting the number of publications with respect to GDP, Croatia was the most productive, and “cost effective” country.

Over the last 30 years, English has become the international language of medical science[13]. Of the current top 50 highest impact journals in orthopaedics, 45 are based in English speaking countries; all 50 of these journals publish their manuscripts in English only[9]. The majority of those countries where English is the primary language also enjoy a high standard of living, and would appear to have advantages in terms of research funding and academic opportunity. Although this suggests an inherent bias towards authors from those countries where English is the principal language, over this 5-year period articles were published by a total of 66 different countries; in many of those countries English is not the main language. Strategies were employed here to attempt to eliminate or minimize any of these potential socio-economic advantages, and therefore obtain a better measure of the relative academic activity and orthopedic research output from various nations around the world. This study has revealed superior academic activity outcomes has been achieved by several of these countries, when adjusted for population size and GDP.

Both GDP and GDP per capita are indicators of economic strength, representing the value of all goods and services produced over a specified time period[7]. The cost of producing a research paper per GDP/capita is theoretically a better indicator of a country’s research productivity, one that takes into consideration some of the socio-economic conditions that might favor more populous or prosperous nations. After adjusting for GPD per capita both India and China were the leading countries, but due to their inordinately large population size the calculated figures are most likely biased. After eliminating these two countries, the United States, South Korea, Japan, Germany, and the United Kingdom ranked among the top five countries with the highest number of both publications and impact factor points. One possible explanation could be that the research output of these countries is directly related to economic vitality, although none of these five leading countries had the highest GDP per capita. For example, the United States, ranked 8^th, Germany 15^th, the United Kingdom 17^th, Japan 23^rd and South Korea 27^th. Earlier research by Meo et al[7] and Halpenny et al[8] also failed to demonstrate a correlation between GDP per capita, total number of publications, and h-index in different science fields and social science disciplines. However, they were able to confirm a strong and positive correlation between the number of publications and the percentage of GDP spent on research.

This study introduced a new metric to bibliographic analysis, normalizing the collective publications and impact factor points of individual nations to that of the output of the median nation, after first correcting for both population size and economic activity. Although this measure has not been validated yet and may lack the robustness of standard citation and content analysis, it is nevertheless similar to other accepted bibliometric measures. In our opinion it facilitates a better comparison between countries, by defining the number of publications that would be necessary for a particular country to produce to have an output equivalent to that of the median nation.

After normalizing research output, 28 countries exceeded this benchmark, whereas 38 were below the level of the median nation. These findings unequivocally demonstrated the dominance of the United States compared to all other countries. To have an output equivalent to the median nation, Poland, it was necessary for the United States to publish 235 articles: However, they collectively published 8149 and were the global leader by an overwhelming margin. China and India were ranked even higher by this metric, but this might demonstrate an inherent limitation of this methodology related to population size. Those countries with a very low GDP per capita, a large population size, and a relatively large number of publications will most likely result in a ceiling effect, and normalizing research output to that of the median nation would thus be unreliable. Therefore, further research is required to better define the extent of this problem and to validate this approach.

Research output is an important determinant of economic growth, and an increase in service delivery, education, and innovation is often an indicator of a society’s shift from a producing economy to a knowledge-based economy[14]. In fact, publications of scientific literature can indicate a nation’s growth and progress in science and technology[5]. Moir et al[15] observed a 21% increase in orthopaedic publications from 1980 to 1994 in six selected journals. More recently, Bosker and Verheyen[4] also reported an increased number of orthopaedic publications in the 15 major clinical orthopaedic journals from 2000-2004, with a total of 13311 articles. The present bibliometric analysis counted over 23000 articles, representing a 73% increase over a 10 years interval. Several authors have previously performed subspecialty analyses[1,16]. Luo et al[1] showed that high income countries published 90% of all articles in foot and ankle research, with the United States publishing the highest number; however, Switzerland took the lead when it was normalized to population size and GDP. Liang et al[16] reported that the United States published the largest number of publications in the subspecialty of arthroscopy, but when adjusted for population size Switzerland was again the country with the highest number of publications. Similar findings were reflected in our results, although in their study Korea ranked first when academic output was adjusted for GDP.

Bibliometric analysis has also been performed by other disciplines. In emergency medicine, the United States was the most productive country followed by the United Kingdom and Australia. When normalized to population size, Australia had the highest number of articles per million persons, but Germany had the highest mean impact factor and citations[17,18]. In the specialty of critical care medicine, the United States has published the most articles, followed by the United Kingdom, Germany, France, and Australia. The United States also had the highest number of randomized controlled trial publications, the highest total impact factor points, and the highest total citations[17,18]. Halpenny et al[8] performed a bibliographic analysis in radiology. In their study, the United States published 42% of the 10,925 papers, followed by Germany and Japan. When corrected for GDP, Switzerland (0.925), Austria (0.694), and Belgium (0.648) produced the most publications per billion of GDP. Robert et al[19] evaluated the pain medicine literature over a period of 30 years and reported that the United States, the United Kingdom and Germany were the highest ranking countries. The pattern of publication rates are comparable to orthopaedics and these findings can possibly be generalized to other disciplines of medicine.

This study has recognized limitations. While the total number of articles and cumulative impact factor points was determined for each nation, the value of individual articles was not assessed; it is possible that there was a significant discrepancy in the manuscript quality between countries, potentially introducing selection bias. Even the selection of impact factor as an outcome measure to evaluate publication quality has been criticized, as it is determined by technicalities that are not related to the scientific value of the research studies themselves[20,21]. Citation analysis was also not performed, and it is acknowledged that the number of citations are a proxy measure of influence reflecting the recognition and quality of the published research by its peers[22]. However, using the impact factor reflects citation counts indirectly, as article citation rates ultimately determine the journal’s impact factor[20]. Nevertheless, overcitation, biased citing, audience size, biased data, and ignorance of the literature are additional common criticisms of bibliometric studies[23]. Another potential limitation of this method is that the research output of the median nation was based on data collected over a specific five-year period from the fifteen currently highest ranked orthopaedic journals. These results will almost certainly change if more journals are included, or the time interval is either extended or shortened.

In conclusion, the results of this study demonstrate that five countries were responsible for 60% of the research output in orthopaedic surgery over a 5-year period, when restricted to the 15 highest ranked journals specific to the field. Only 28 of 66 countries were able to achieve a publication rate equivalent to that of the median nation, after first correcting for GDP per capita. The United States was unequivocally the global leader when judged by this measure, and exceeded the median production by more than 34 times. Although China and India ranked the highest after correcting for both GDP and population size, this probably reflects the inordinately large populations of both countries. The United States, United Kingdom, South Korea, Japan, and Germany placed in the top five countries with respect to both publication totals and cumulative impact factor points.

COMMENTS

Background

Bibliographic analysis of academic output has been performed for many indications and can be an indicator for academic excellence. However most studies have focussed on the total number of publications without accounting for gross domestic product or economic discrepancies between countries. The primary aim of this study was therefore to investigate the number of publications and total impact factor from each country, and to then relate these variables to population size, gross domestic product (GDP), and GDP per capita. Secondly they determined the minimum number of publications required to be comparable to the country producing the median number of publications, when normalized for GDP per capita. The final aim was to establish the number of publications that would be required from each country to be equivalent to the country having the highest research output, when normalized for GDP per capita.

Research frontiers

Over the last 30 years English has become the international language of medical science. In Orthopedics 45 of the 50 highest impact orthopaedic journals are based in English countries. Based on these facts the majority of publications in these journals should come from primary English speaking countries.

Innovations and breakthroughs

Based on the total number of publications and impact points the United States was the undebated leader for both the total number of publications and impact points. However when adjusting for publication size and GDP per capita, it was Switzerland respectively Croatia which were the most productive nations. When using a newly introduced benchmark to adjust for both population size and GDP, 28 countries exceeded and 38 nations were below the median nation.

Applications

This review suggests that the total number of publications and impact points are not representative of true research output and other factors should be included into bibliometric analysis.

Terminology

Bibliometric analysis is based on quantitative variables such as number of publications, impact points and citation rates. Analysis can be performed at the macro-level comparing countries performances, at the middle level analyzing Universities or other institutional output or at the microlevel investigating research output of departments or individuals.

Peer-review

The authors present a very interesting paper on the worldwide orthopaedic research activity. They relate the scientific production with the GDP, and per capita GDP. This sort of information, although known for general science, was unknown in the orthopaedic field. The relevance of this paper is not only related to science but also to politics.