Are UK and Ireland trauma and orthopaedic surgeons maintaining their research output?

Abstract

Introduction

Healthcare faces growing challenges. With reports of diminishing research output from the UK and Ireland in the leading surgical journals, this study aimed to ascertain whether this trend had been echoed in the trauma and orthopaedic literature.

Materials and methods

Citable research output from the 10 globally leading trauma and orthopaedic journals was analysed from five individual years, over a 20-year period, to ascertain trends in absolute output, geographical mix, and level of evidence.

Results

The overall number of published articles fell by 14.5%. North America saw the greatest decline (–8.0%), followed by Japan (–5.6%) and Europe (–3.3%). The UK and Ireland (+2.9%) and the rest of the world (+13.9%) saw rising output. A decline in lower (levels IV and V) and a rise in higher (levels I, II and III) quality evidence was observed. The UK and Ireland had a greater proportion of higher-quality studies than North America and Japan, but lower than Europe and the rest of the world. The impact factor of the leading journal rose from 4.47 to 7.01.

Discussion

The research landscape has evolved, with leading journals placing greater emphasis on higher-quality evidence. UK and Irish research output remains stable, contributing 14% of the most highly cited publications in 2018, and challenging North America’s dominance with a greater proportion of level I and II evidence in the leading journals.

Conclusion

With the growing emergence of Europe and the rest of the world, UK and Irish authors must build upon the region’s output despite political challenges such as Brexit. Increasing international collaboration will continue to play an important role.

Introduction

Healthcare services face increasing demand coupled with restricted resources. Expanding and ageing populations, increasing numbers with chronic long-term conditions, rising resource costs, and insufficient staffing all contribute to growing waiting lists and rising pressures upon staff.^1,2 Consequently, healthcare professionals have to increasingly balance service provision over academia, training and continuing professional development. Clinicians, particularly trainees, are required to publish research to progress in their development and maintain an evidence-based outlook to their clinical practice. With a critical need to maintain our research output to drive patient care, it is unclear whether these growing challenges have resulted in a change in the quality and quantity of research output that directly impacts patient care. Musculoskeletal disorders contribute to approximately 9% of the UK’s disability burden yet receive only 3% of the research spend.³ Despite this, trauma and orthopaedics in the UK have amassed a growing portfolio of high-profile randomised controlled trials (RCT) in recent years. With the need to continue to do more with less, we sought to evaluate the evolving geographical trends in research output within our specialty.

Materials and methods

Aims

The primary aim was to determine the overall change in UK trauma and orthopaedic research output over a 20-year period, in the context of global trends. The secondary aim was to determine any change in research mix (ie the quality of evidence).

Source data

The impact factor (IF) is a journal-level metric that represents the average citation count of the articles published in a journal. The IF of journals is published annually in the Clarivate Analytics (formerly Thomson Reuters) Journal Citation Reports (JCR), sourcing data from the Web of Science. Despite its limitations, the IF remains an established metric with which to gauge a journal’s relative influence.

To facilitate consideration of longer-term journal popularity, the five-year journal IF was used to determine the top 10 global trauma and orthopaedic journals over a 20-year period from 1998 to 2018, with the JCR analysed in five-year periods. The five-year journal IF is the average number of times articles published in the past five years from a given journal have been cited in the chosen JCR year. For example, in the 2018 JCR, the five-year window includes 2014, 2015, 2016, 2017 and 2018. It is calculated by dividing the number of citations in the JCR year by the total number of articles published in the five preceding years.

The 10 leading trauma and orthopaedic journals (ie those with the highest five-year IF for each of 1998, 2003, 2008, 2013 and 2018) were then selected (Table 1). Where the five-year journal IF was not available (1998 and 2003 publication years), the IF of the single chosen JCR year was used to rank the journals.

Table 1. Journal ranking by Impact factor for 1998, 2003, 2008, 2013 and 2018.

Year	Rank	Journal title	Impact factor		Region of origin
			5-year	1-year
1998	1	Journal of Bone and Joint Surgery – American volume	N/A	2.073	North America
	2	Osteoarthritis and Cartilage	N/A	2.014	UK and Ireland
	3	Journal of Orthopaedic Research	N/A	1.808	North America
	4	Spine	N/A	1.754	North America
	5	Journal of Bone and Joint Surgery – British volume	N/A	1.501	UK and Ireland
	6	Connective Tissue Research	N/A	1.418	UK and Ireland
	7	Spinal Cord	N/A	1.192	UK and Ireland
	8	Physical Therapy	N/A	1.192	North America
	9	Orthopedic Clinics of North America	N/A	1.145	North America
	10	Clinical Orthopaedics and Related Research	N/A	1.072	North America
2003	1	Osteoarthritis and Cartilage	N/A	2.964	UK and Ireland
	2	Spine	N/A	2.676	North America
	3	Journal of Orthopaedic Research	N/A	2.167	North America
	4	Journal of Bone and Joint Surgery – American volume	N/A	1.921	North America
	5	Clinical Journal of Sport Medicine	N/A	1.784	North America
	6	Arthroscopy – The Journal of Arthroscopic and Related Surgery	N/A	1.616	North America
	7	Physical Therapy	N/A	1.582	North America
	8	Gait & Posture	N/A	1.585	UK and Ireland
	9	European Spine Journal	N/A	1.527	Europe
	10	Journal of Bone and Joint Surgery – British volume	N/A	1.503	UK and Ireland
2008	1	Osteoarthritis and Cartilage	4.473	4.082	UK and Ireland
	2	American Journal of Sports Medicine	3.952	3.646	North America
	3	Journal of Bone and Joint Surgery – American volume	6.279	3.313	North America
	4	Journal of Orthopaedic Research	3.574	2.963	North America
	5	Spine	3.526	2.793	North America
	6	Gait & Posture	3.415	2.743	UK and Ireland
	7	Arthroscopy – The Journal of Arthroscopic and Related Surgery	2.732	2.503	North America
	8	European Spine Journal	2.514	2.396	Europe
	9	Journal of the American Academy of Orthopaedic Surgeons	N/A	2.250	North America
	10	Journal of Bone and Joint Surgery – British volume	2.837	2.196	UK and Ireland
2013	1	American Journal of Sports Medicine	5.009	4.699	North America
	2	Osteoarthritis and Cartilage	4.606	4.663	UK and Ireland
	3	Journal of Bone and Joint Surgery – American volume	4.384	4.309	North America
	4	Physical Therapy	3.896	3.245	North America
	5	Arthroscopy – The Journal of Arthroscopic and Related Surgery	3.552	3.191	North America
	6	Journal of Orthopaedic Research	3.257	2.972	North America
	7	Journal of Physiotherapy	2.972	2.894	Rest of the world
	8	Clinical Orthopaedics and Related Research	3.189	2.882	North America
	9	Knee Surgery, Sports Traumatology, Arthroscopy	2.862	2.837	North America
	10	Bone & Joint Journal	3.342	2.801	UK and Ireland
2018	1	American Journal of Sports Medicine	7.006	6.093	North America
	2	Journal of Physiotherapy	6.38	5.551	Rest of the world
	3	Osteoarthritis and Cartilage	5.701	4.879	UK and Ireland
	4	Journal of Bone and Joint Surgery – American volume	6.101	4.716	North America
	5	Arthroscopy – The Journal of Arthroscopic and Related Surgery	4.875	4.433	North America
	6	Bone & Joint Journal	4.056	4.301	UK and Ireland
	7	Clinical Orthopaedics and Related Research	4.478	4.154	North America
	8	European Cells & Materials	4.416	3.682	Europe
	9	Bone & Joint Research	3.35	3.652	UK and Ireland
	10	Journal of Arthroplasty	3.587	3.524	North America

Outcome measures

Citable research output was analysed from each journal. Abstracts were screened to determine the absolute output of papers from each of the 10 journals, the geographical mix and the article type.

The geographical source of a paper was categorised into one of five geographical regions based upon the country of origin of the first author: USA/Canada (North America); UK/Ireland; Europe; Japan; and the rest of the world. The articles were stratified according to the Joanna Briggs Institute levels of evidence into five main categories, ranging from level I to level V (Table 2).

Table 2. Classification of levels of evidence.

Level I	Level II	Level III	Level IV	Level V
L1 systematic review	L2 systematic review	L3 systematic review	Observational study	Case reporta
L1 meta-analysis	L2 meta-analysis	L3 meta-analysis	Comparative study
L1 (double-blinded) RCT	L2 cohort study	L3 RCT	L4 systematic review
	L2 RCT	L3 case–control study
		L3 comparative case series
		Pragmatic clinical trial
		Prospective controlled study
		Retrospective cohort study

^a Case reports included small case series (ie fewer than 10 cases)

Special commentaries, expert opinions, letters, (non-systematic) reviews, and other non-citable articles were excluded. Changes to journal names during the period of the study were accounted for; for example, in 2012 the Journal of Bone and Joint Surgery (British volume) rebranded itself as the Bone and Joint Journal.

Statistical analysis

Chi-square analyses were performed on the dataset. A p-value of 0.05 or lower was considered significant. Article types were categorised into levels I/II and levels III/IV/V for further statistical analyses. Odds ratios comparing geographical regions and their publication of articles of different types were determined. Statistical analysis was performed with SPSS version 23.0.

Results

Absolute output

A total of 6,406 published articles were included from the five years under consideration. The total number of published articles in the 10 leading global orthopaedic journals fell by 116 (14.5%) over the 20-year period from 802 in 1998 to 686 in 2018. However, this was not a linear decrease (Fig 1). 2018 saw the fewest articles published worldwide in comparison with the four other years under investigation, with publications from all five regions decreasing substantially compared with 2013. The numbers of publications across the 20-year period were not equally distributed (χ²(4) = 1635.10, p < 0.05), with most publications seen in 2003 (1,701) and 2013 (2,357), and the fewest in 2018 (686).

Figure 1. Trend in total publication output between 1998 and 2018 (NA, North America; UKI, UK and Ireland; EUR, Europe; JAP, Japan; ROW, rest of the world).

Geographical mix

North America, Europe and the rest of the world (predominantly Eastern Asia) had the highest overall publication output over the five study years. The number of publications across the five geographical regions were not equally distributed, (χ²(4) = 2776.60; p < 0.05).

Over the study period, North America had the greatest reduction in overall output (–8.0%), followed by Japan (–5.6%) and Europe (–3.3%). In contrast, the greatest rise in output was seen in the rest of the world (+13.9%), followed by the UK and Ireland (+2.9%; Fig 2; Table 3). While output from individual countries was not analysed in detail, for Europe the highest national output was observed from the Netherlands, Germany, Sweden and Norway. There was also an increase in research output from many Eastern and Central European countries, particularly Poland, Lithuania, Slovenia and Croatia. For the rest of the world, China, South Korea and Australia were consistently high-output countries, although a large increase in output was observed over the study period from others including Turkey, Brazil, India and Iran.

Figure 2. Overall change in output between 1998 and 2018 (NA, North America; UKI, UK and Ireland; EUR, Europe; JAP, Japan; ROW, rest of the world).

Table 3. Citable output by region.

Region	Articles
	1998		2003		2008		2013		2018
	(n)	(%)	(n)	(%)	(n)	(%)	(n)	(%)	(n)	(%)
North America	404	50.4	781	45.9	328	38.1	971	41.2	291	42.4
UK and Ireland	64	8.0	160	9.4	75	8.7	172	7.3	75	10.9
Europe	174	21.7	369	21.7	229	26.6	608	25.8	126	18.4
Japan	68	8.5	169	9.9	65	7.6	113	4.8	20	2.9
Rest of the world	92	11.5	222	13.1	163	19.0	493	20.9	174	25.4
Total	802		1701		860		2357		686

Note: Percentages are of the annual global total

Despite the UK and Ireland average output being maintained, there were fluctuations between the years studied. In 1998, the UK and Ireland contributed 64 articles (8.0% of global total), compared with 75 articles (10.9% of global total) in 2018. However, there was a notable variation in each of the five study years, with a peak output of 172 articles in 2013 (Table 3) and the highest global percentage contribution in 2018 (Fig 3).

Figure 3. UK contribution to global research output.

Article type

In 1998, level IV evidence was the most commonly published article type globally. This remained a consistent finding across the 20-year study period. However, the mix of articles published over the years did change (Fig 4). The most significant change observed was a decline in lower-quality evidence, with level IV output declining by 24.6% (60.6% in 1998 compared with 36.0% in 2018) and level V declining by 21.8% (22.8% in 1998 compared with 1.0% in 2018). Conversely, there was notable growth in higher-quality evidence. For example, the output of level I evidence increased by 14.6% (1.6% in 1998 compared with 16.2% in 2018). These changes in the types of articles published across the 20-year period were statistically significant, (χ²(16) = 1249.24, p < 0.05).

Figure 4. Change in article type by level of evidence over time.

After grouping articles into ‘higher’ (levels I and II) and ‘lower’ quality evidence (levels III, IV and V) and adjusting for absolute output, comparison of the UK and Ireland against the other regions demonstrated a greater proportion of higher-quality output compared with North America and Japan, and a lower proportion than Europe and the rest of the world (Table 4).

Table 4. Odds ratios comparing the UK and Ireland with other regions based on the proportion of higher- compared with lower-quality evidence.

Comparison region	Higher-/lower-quality articles (n)	Odds ratio	95% confidence interval		p-value
			Lower	Higher
UK and Ireland	73/473
Compared with:
Europe	265/1241	0.723	0.546	0.956	0.022
North America	249/2526	1.566	1.185	2.069	0.003
Japan	25/410	2.531	1.577	4.062	< 0.001
Rest of the world	215/929	0.667	0.500	0.889	0.003

Discussion

Between 1998 and 2018 there was an overall decrease in the total number of annually published articles in the leading 10 global orthopaedic journals. However, this was not a linear trend. Despite maintaining their status as the leading producers of trauma and orthopaedic publications, North American authors had the greatest fall in overall contribution by 8%, whereas the rest of the world increased their output by 13.9% over the 20-year study period, with a particular increase in output from Australia, Brazil, China and India. While European output demonstrated a 3.3% reduction, despite notable fluctuations over the 20-year period, the UK and Ireland countered this trend with an increase of 2.9%.

There was a trend towards higher-quality evidence, with a shift towards more level I publications in 2018, as compared with a greater proportion of level IV and V publications in 1998. However, despite a decline, level IV evidence (primarily observational and comparative studies) remained the most common article type. The five-year IF of the number-one ranking journal rose from 4.47 in 2008 to 7.01 in 2018.

The authors are not aware of a previously published scientometric study that has analysed the temporal changes in UK and Irish trauma and orthopaedic research output. However, other studies have observed variable trends in surgical and anaesthetic research output from the UK and Ireland, with the majority reporting decreases.^4–6 Several publications have analysed the trauma and orthopaedics research output of China in comparison to the leaders in this research field (the United States (US), the UK, Germany and Japan). They too observed the US to have maintained its contribution as the leading trauma and orthopaedics research producer, consistently ranking highest in terms of journal IF, average citations and percentage of articles in the top 10 journals. However, US dominance is decreasing, with Germany, South Korea and China submitting higher numbers of higher-quality publications.^7–10

This study does have several limitations. As a snapshot study, we considered a limited number of journals across selected years – not the entirety of the trauma and orthopaedic evidence base. Analysis of further journals and years, alongside consideration of other relevant factors (geopolitical, financial) might have permitted more detailed analysis.

As the first author’s address was used to determine the geographical origins of a study, we were unable to account for the growing trend towards international research collaborations; 21% of the overall global publications were internationally co-authored in 2018, compared with 11% in 1998.¹¹ This was most pronounced for the UK and Ireland, where 63% of publications in 2018 were the product of international collaboration compared with 26% in 1998. This disproportionately impacted certain regions; in 2018, international collaboration accounted for 63%, 60%, 55% and 54% of UK and Irish, French, Italian and German output, respectively. In comparison, only 30%, 42%, 28% and 32% of research from Japan, North America, China and South Korea, respectively, was a product of international collaboration.¹²

We evaluated the quantity of research published across different regions. However, quantity does not necessarily correlate with either quality or ‘popularity’ (ie citations) of a paper. For example, while the UK and Ireland contributed 7% of the total global publications in 2018, their output amounted to 14% of the most highly cited publications (those in the top-cited 1% of all publications globally).¹¹ Despite the obvious regional size differences, it was reassuring to see the UK and Ireland challenge North America’s dominance through a higher odds ratio of higher-quality output (ie a better focus on quality over quantity). However, with Brexit taking effect, the UK and Ireland must not ignore the growing dominance of Europe and the rest of the world, which continue to publish increasingly high-quality research in the leading journals.

The journal IF is not a universally accepted metric; critics regard it as oversimplified and imperfect. While it is helpful in producing a potential hierarchy of medical journals, it does not necessarily correlate with the quality of published research. Journals choose articles for publication that have a greater likelihood of being highly cited, which itself has a positive effect on their impact factor.¹³ As journals with higher IF are more likely to receive ‘popular’ papers, their brand allows their IF to rise further, thereby allowing the ‘rich to get richer’. This is compounded by the fact that a high IF does not necessarily infer that all articles within a journal are well-cited, as a few very highly cited papers can substantially elevate a journal’s IF.

This study emerged from concern over diminishing research output due to the growing challenges faced by UK and Ireland trauma and orthopaedics professionals. Nevertheless, the findings – an increase in UK and Irish output, with a trend towards higher-quality evidence – are highly encouraging and demonstrate the UK and Ireland’s resilience in maintaining their standing on the global academic stage. Coupled with the rising output from Asia, particularly China and South Korea, this has weakened North America’s dominance. The suggestion that more studies are being published ‘for the sake of publishing’ has been countered by the continuing trend towards more published level I evidence. The growing profile of cross-regional trainee research collaboratives, such as the British Orthopaedic Network Environment mirrors the rise in international collaboration in research and will help to maintain this trend, alongside driving down bias, minimising repetition, facilitating greater patient numbers, and reducing time to publication.^14,15

Conclusion

With high-quality research crucial in driving evidence-based improvements to patient care, our findings are promising for UK and Irish trauma and orthopaedics, a specialty with a growing repertoire of practice changing surgical RCTs. With the growing emergence of Europe and the rest of the world, UK and Irish authors must build upon the region’s output. Increasing international collaboration will continue to play an important role despite political challenges such as Brexit, to break down barriers and build a culture of robust, high-quality research.