Abstract
WHAT IS ALREADY KNOWN ABOUT THIS SUBJECT?
The USA, UK and Germany have a strong position in performance of drug and nondrug randomized controlled trials.
Europe's position in the quantitative and qualitative performance in drug randomized controlled trials in particular, and factors that drive the quantitative and qualitative performance of drug randomized controlled trials in Europe, are unknown.
WHAT THIS STUDY ADDS
Europe's position in the quantitative and qualitative performance of randomized controlled drug trials lags behind USA.
Factors are identified that are associated with the difference in publication output between countries.
The number of headquarters of pharmaceutical companies in a country, the research expenditures by pharmaceutical companies, as well as health-related R&D expenditures of a country appear to contribute to a relatively high scientific performance in randomized controlled drug trials.
AIMS
Performance of randomized controlled drug trials (drugRCTs) adds to the scientific output, scientific knowledge, scientific training and up-to-date status of healthcare and may drive economy. The purpose of this study was to benchmark Europe's position on drugRCTs relative to the rest of the world, and to identify factors that may drive this performance.
METHODS
The number of scientific publications on drugRCTs, indexed in PubMed and Thomson Scientific/Web of Science database over the period 1995–2004, was used as a proxy measure for the quantitative drugRCT output. The international citation impact of these publications was used as a proxy measure for the qualitative drugRCT output.
RESULTS
Country's origin of 103 211 publications was determined. After adjustment for population size, the number of drugRCT publications from Europe, USA and Australia/Japan was 102, 124 and 44 publications per million inhabitants, respectively. The proportional increase in publication output from 1995 until 2004 was lower in Europe compared with the USA and Australia/Japan (29.1, 40.1 and 63.4%, respectively). The number of citations per publication was 4.9 in Europe, 7.0 in the USA and 3.4 in Australia/Japan. Within Europe, the UK, Germany and Italy produced most publications. Country-specific factors associated with publication output in Europe were the number of pharmaceutical companies with headquarters in a country (R2 = 0.71, P < 0.001), national R&D expenditures by pharmaceutical companies (R2 = 0.63, P < 0.001) and health-related R&D expenditures by national governments (R2 = 0.22, P = 0.052).
CONCLUSIONS
When adjusted for population size, quantitative and qualitative performance of drugRCTs in Europe lags behind the USA but is ahead of Australia/Japan. Several factors appear to explain the differences, among which are the number of headquarters of pharmaceutical companies in a country, the research expenditures by pharmaceutical companies, as well as health-related R&D expenditures of a country. To enhance and strengthen Europe's position, researchers may strengthen their collaborations with local pharmaceutical companies, and national governments could increase their budgets for medical research funding.
Keywords: bibliometric analysis, randomized controlled trial
Introduction
Randomized controlled drug trials (drugRCTs) are of major importance for evidence-based treatment. Increasingly, the ‘evidence’ obtained in RCTs is used by clinicians and policy makers when setting guidelines and policies for patient care. Moreover, performance of drugRCTs adds to the scientific output, scientific knowledge, scientific training and up-to-date healthcare, and may drive the economy of an individual country or continent. To guarantee sufficient quantity and quality of drugRCTs for the future, and consequently to create a competitive knowledge-based economy in Europe, we need to establish which factors drive the number of (high-level) studies.
To date, there is no comprehensive database on all ongoing or finished drugRCTs worldwide. One of the closest benchmarks of clinical trial performance of a country or region is the number of scientific publications resulting from clinical drug trials. A recent publication has shown that the USA, UK and Germany have a strong position in performing drug and nondrug RCTs or controlled clinical trials, as derived from the number of scientific publications registered in the Cochrane database [1]. However, systematic data on the position of various countries in conducting drugRCTs in particular are lacking.
Given the aforementioned, we performed a bibliometric study to benchmark Europe's position in conducting drugRCTs and to identify factors driving drugRCT publication output. To evaluate Europe's position in conducting drugRCTs we compared Europe with other continents (USA and Australia/Japan). To analyse factors that determine output we performed a continental comparison and a country comparison within Europe.
Methods
Data sources
We searched the PubMed database to identify all research-based publications on drugRCTs in the period 1995–2004. We searched for the medical subject heading terms ‘Clinical Trial’ and (‘Pharmaceutical Preparation’, ‘drug’ or ‘drugs’). Selected publication document types included research publications, research notes, letters, and reviews. After completing the query of the PubMed database, the gathered publications were matched with Thomson Scientific/Web of Science-indexed publications (according to the name of first author, volume of journal, first page and year of publication) to obtain the full set of information on the country of origin of all authors. A publication was attributed to a country if that country was included in an affiliate address of one of the authors. Hence, a publication was assigned to all countries listed in the author's address information. We examined data for the USA, the 25 European member states that joined the European Union (EU) in 2004, plus Norway and Switzerland and Australia/Japan.
Bibliometric indicators
Quantitative and qualitative indicators were used to describe international drugRCT output. The number of drugRCT publications was used as a proxy measure for the quantitative drugRCT output. Citation rate, calculated as the ratio of the total number of citations that each country received over the total number of publications of that country, was used as a measure of international scientific impact and as a proxy measure for the scientific relevance and quality of drugRCT output.
Determining factors for publication output
We extracted average population size of each country in the period 1995–2004 from the Organisation of Economic Co-operation and Development (OECD) database and Eurostat database [2, 3]. The OECD database was also used to extract the average number of graduate students who participated in tertiary education programmes in life sciences.
Prespecified structural factors that were expected to be associated with publication output were established to examine how Europe can improve or sustain its position in conducting drugRCTs. National health-related R&D expenditures by governments were used as a proxy measure for the country's investment in clinical research. Average national health-related R&D expenditures of each country in 2004 were extracted from the OECD database [2]. These expenditures were expressed as percentages of gross domestic product (GDP). To investigate the influence of pharmaceutical companies, we obtained the number of headquarters of pharmaceutical countries allocated to a country from IMS Healthcare (Norwalk, CT, USA) [4, 5]. R&D expenditures by pharmaceutical companies in 2004 were obtained from the European Federation of Pharmaceutical Industry Associations [6].
Statistical analyses
Linear regression analysis was used to determine a trend over time in publication output within Europe, the USA and Australia/Japan and to determine differences between regions in trend in publication output over time. For this purpose the change in publication output in each region was calculated with 1995 as reference year. We performed linear regression analyses between the above-mentioned factors and the number of publications of drugRCTs. Data were analysed with SPSS version 14.0 software (SPSS Inc., Chicago, IL, USA).
Results
Database search
The PubMed query resulted in a total of 171 844 publications on drugRCTs in the period 1995–2004. After matching these publications with the Thomson Scientific/Web of Science database, we were able to determine the country of origin of 103 211 (60.1%) publications. The total number of publications increased from 8684 in 1995 to 12 218 in 2004.
Worldwide regional comparison
Europe appeared to produce the largest number of publications over the period 1995–2004, followed by the USA and Australia/Japan (Figure 1). After adjustment for the average population size in the period 1995–2004, the number of drugRCT publications in Europe was lower compared with that of the USA (102 vs. 124 publications per million inhabitants), but higher compared with that of Australia/Japan (102 vs. 45 publications per million inhabitants) (Figure 1). Trend analysis revealed an increase in publication output in each region (Figure 1; P for trend <0.01). Publication output in Europe showed an increase over the period 1995–2004 of 1210 publications, equivalent to 29.1%. The increase in publication output in the USA and Australia/Japan was 1198 publications (40.1%) and 313 (63.4%) publications, respectively (P < 0.01 for linear trends in publication output between Europe, the USA and Australia/Japan).
Figure 1.
Worldwide ‘quantitative’ publication output. Left panel, Total ‘quantitative’ publication output and ‘quantitative’ publication output adjusted for the average population size in Europe, USA and Australia/Japan over the period 1995–2004. Right panel, Trend in publication output in Europe, USA and Australia/Japan
Citation rate was determined to compare the international scientific impact and quality of drugRCT output. Citation rate was lower in Europe and Australia/Japan compared with the USA (Figure 2).
Figure 2.
Citation rate of drugRCT publications in Europe, USA and Australia/Japan
Several factors were analysed that may determine worldwide drugRCT activity. National health-related R&D expenditures as percentage of GDP was five times lower in Europe (0.052%) compared with the USA (0.256%), but slightly higher compared with Australia/Japan (0.043%). Furthermore, the number of pharmaceutical companies belonging to the top 100 and R&D expenditures by pharmaceutical companies were lower in Europe (36 companies, €21 106 million R&D expenditures) compared with the USA (39 companies, €23 758 million R&D expenditures), but higher compared with Australia/Japan (21 companies, €7016 million R&D expenditures).
European country comparison
When European countries were analysed separately, a wide variation in publication output was observed, with the UK, Germany and Italy producing the largest number of publications (Figure 3). After adjustment for average population size, Denmark, Finland and Sweden were the countries having produced the largest number of publications (Figure 3). Essentially similar country rankings were obtained if total publication output was adjusted for number of graduate students in life sciences. European countries that showed a marked absolute and percentage increase in publication output were Germany (347, 62.0%), Italy (198, 34.1%), Spain (155, 101.3%) and Greece (106, 207%) (Figure 3). The increase in publication output in other European countries was <50 publications or 15%. The citation impact of a country's publication output did not differ much between European countries (Figure 4).
Figure 3.
‘Quantitative’ publication output in Europe. Left panel, Total ‘quantitative’ publication output and ‘quantitative’ publication output adjusted for the average population size over the period 1995–2004 of top 15 European countries. Right panel, Trend in publication output in European countries
Figure 4.
Citation rate of drugRCT publications in 15 European countries with the largest publication output
To examine how Europe can improve its position in performing drugRCTs we tried to establish which factors were associated with a high publication output and appeared to drive the conduct of drugRCTs in Europe. Regression analyses, as shown in Figure 5, revealed that the publication output of a country was associated with the number of pharmaceutical companies with headquarters in a country (R2 = 0.71, P < 0.001; each additional company in a country resulted in ±900 additional publications), R&D expenditures by pharmaceutical companies (R2 = 0.63, P < 0.001; each million increment resulted in 1.4 additional publications) and modestly associated with national health-related R&D expenditures (R2 = 0.22, P = 0.052; each 0.01% increment resulted in ±50.000 additional publications).
Figure 5.
Factors associated with ‘quantitative’ publication output in European countries. (a) Association between publication output and number of pharmaceutical companies with headquarters in a country. (b) Association between publication output and research and development expenditures by pharmaceutical companies. (c) Association between publication output and health-related research and development expenditures. AT, Austria; BE, Belgium; CH, Switzerland; CZ, Czech Republic; DE, Germany; DK, Denmark; ES, Spain; FI, Finland; FR, France; GR, Greece; HU, Hungary; IR, Ireland; IT, Italy; NL, the Netherlands; NO; Norway; PT, Portugal; SE, Sweden; SI, Slovenia; SK, Slovakia; UK, United Kingdom. Health-related R&D expenditures data were not available for: Cyprus, Estonia, Hungary, Luxembourg, Lithuania, Latvia, Malta and Poland. Data on R&D expenditures by pharmaceutical companies were not available for Czech Republic, Cyprus, Estonia, Hungary, Luxembourg, Lithuania, Latvia, Malta, Poland and Slovakia
Discussion
This study was conducted to benchmark Europe's position in the world in performing drugRCTs and to identify factors that may drive this position. We showed that after adjustment for population size, quantitative and qualitative publication output in Europe was lower than that of the USA, but higher compared with Australia/Japan. Importantly, the proportional increase in publication output in Europe was lower compared with both the USA and Australia/Japan. Publication output in Europe was associated with the number of pharmaceutical companies with headquarters in a country and R&D expenditures by pharmaceutical companies, and modestly associated with health-related R&D expenditures of a country.
The association between publication output and these factors could be a consequence of the increased influence of pharmaceutical companies on clinical research over the past decade [7]. Recent bibliometric research of Europe's top 10 largest pharmaceutical companies has revealed a distinctive ‘home advantage’, where these companies tend to prefer local research partners [8]. Hence, the larger publication output after adjustment for population size in the USA may be partially attributed to the larger number of local pharmaceutical companies and higher levels of R&D investment by those companies. These observations highlight the importance of European researchers building and strengthening partnerships with local pharmaceutical companies in order to attract pharmaceutical research investment and retain their role in conducting drugRCTs.
Health-related R&D expenditures by national governments in Europe fall behind the USA, which may explain why the USA produces more publications per million inhabitants and its research attracts more citations in the international scientific literature. The low health-related R&D expenditures in Europe has recently led medical research councils to plea for an increase in public research funding so that it reaches 0.25% of GDP [9]. By drawing attention to the relatively low levels of research funding in Europe, medical research councils are hoping that national governments focus their resources on conducting high-quality (bio)medical and clinical research so that Europe's position in the performance of biomedical research and drugRCTs may improve. Our data, showing a borderline significant association between the number of drugRCT publications and health-related R&D expenditures, indeed suggest that increasing the level of public research funding might result in an increased number of drugRCTs.
Another factor that affects initiation of drugRCTs is regulation of drugRCTs. Recent research has revealed that the time taken to obtain regulatory approval to start a drugRCT was markedly shorter in the USS compared with Europe [10]. USA-based research may thus have benefited from the rapid trial application review process that facilitates the initiation of drugRCTs. A similar effect of trial application review process on publication practices can be observed within Europe. For example, European countries with high rankings in publication output after adjustment for population size, such as Sweden and the Netherlands, have established (already at that time) central ethics committee setting policies for local ethics committees and reviewing multicentre clinical trial applications [11, 12]. This has likely to have resulted in an environment in which the trial application procedure is rapid and consistent, ultimately leading to more drugRCTs conducted in these countries.
Few studies have assessed the relative country's contribution to clinical or biomedical research. Rahman et al. showed a decline in the USA's contribution to clinical research [13]. However, the authors evaluated only the number of publications in seven selected clinical journals and neither corrected for population size in each country, nor determined factors influencing publication output. Another bibliometric analysis has recently shown that the research productivity of the 25 countries to join the EU was 66% compared with the US, after adjustment for population size [6]. These data indicate that research productivity in Europe in general lags behind the USA.
We were able to attribute the affiliation address of 60.1% of the PubMed selected publications. This seemingly low percentage can be explained by the fact that the PubMed database covers a wider range of research literature than Thomson/Scientific Web of Science database with its focus on international peer-reviewed scientific journals. However, Thomson/Scientific Web of Science database is one of very few international multidisciplinary databases that contain the affiliate addresses of all authors as well as the full list of references (‘citations’) to the relevant information sources and research literature. The advantage of combining both databases is that PubMed allows us to search with Medical Subject Heading terminology, providing a consistent way to retrieve scientific publications that may use different terminology for the same concept, whereas the Thomson/Scientific Web of Science database can be used to retrieve the country's origin of all authors.
This study suffers from a few limitations, and some issues need to be addressed when interpreting our findings. A limitation is that it is influenced by publication bias, i.e by the fact that some drugRCTs are conducted but never published. Various reports have shown that about 30–60% of all clinical trials are never published [14, 15]. Such a significant loss of relevant information sources may have influenced our results. Second, no quantitative data were available to examine other factors potentially associated with publication output, such as differences in academic promotion policies, number of researchers in medical science in each country and the amount of research funding devoted to drugRCTs by private organizations and foundations. Strengths of this study include the use of a publication-based database. The advantage of using a publication-based database is that more weight is given to large-scale drugRCTs with major clinical and scientific impact since such trials are more likely to result in multiple publications. Hence, by calculating the number of publications a weight factor is included for the size and impact of each trial. However, it could be possible that we took multiple publications of the same drugRCT into account. Another advantage of the use of publication-based databases is that they allow for a longer observation period of drugRCT activity, since study based registers have become available only since 2000. Finally, factors that were associated with the worldwide publication output also appear to drive publication output within Europe. We believe that this consistency strengthens our observations.
In conclusion, quantitative and qualitative performance of drugRCTs in Europe lags behind the USA but is ahead of Australia/Japan. To enhance and strengthen Europe's position, researchers may strengthen their collaborations with local pharmaceutical companies, and national governments could increase their budgets for medical research funding.
Competing interests
None to declare.
Acknowledgments
This study received an unrestrictive grant from Nefarma. The funding body had no role in data collection, data analysis, data interpretation or writing of the study. The authors thank Nefarma for their unrestricted grant, and Bert van der Wurff (CWTS, Leiden University) for valuable assistance in collection and processing of the research publication data.
REFERENCES
- 1.Gluud C, Nikolova D. Likely country of origin in publications on randomised controlled trials and controlled clinical trials during the last 60 years. Trials. 2007;8:7. doi: 10.1186/1745-6215-8-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Organization for Economic co-operation and Development. OECD. [last accessed 4 April 2008];Stat. Available at http://www.oecdorg/topicstatsportal.
- 3.European Commission Eurostat. European Commission Eurostat data. [last accessed 13 April 2008];2008 Available at http://epp.eurostat.ec.europa.eu.
- 4.European Federation of Pharmaceutical Industries and Associations. The Pharmaceutical Industry in Figures 2006. [last accessed 4 April 2008]; Available at http://www.efpia.org.
- 5.IMS Health. Norwalk, CT: IMS Health; 2007. IMS World Health Review 2007. [Google Scholar]
- 6.Soteriades ES, Falagas ME. Comparison of amount of biomedical research originating from the European Union and the United States. BMJ. 2005;331:192–4. doi: 10.1136/bmj.331.7510.192. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Patsopoulos NA, Ioannidis JP, Analatos AA. Origin and funding of the most frequently cited papers in medicine: database analysis. BMJ. 2006;332:1061–4. doi: 10.1136/bmj.38768.420139.80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Tijssen RJW. Internationalisation of pharmaceutical R&D: how globalised are Europe's largest multinational companies. Technol Anal Strat Manage. 2008. forthcoming.
- 9.Watson R. Europe's research councils call for spending to be doubled to 0.25% of GDP. BMJ. 2007;335:1232. doi: 10.1136/bmj.39423.700926.DB. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Lambers Heerspink HJ, Dobre D, Hillege HL, Grobbee DE, De Zeeuw D. Does the European Clinical Trials Directive really improve clinical trial approval time? Br J Clin Pharmacol. 2008;66:546–50. doi: 10.1111/j.1365-2125.2008.03246.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Reghar E. Ethical diversity and regulatory harmonisation: an empirical exploration of research ethics committees following the Directive of Good Clinical Practice. [last accessed 28 March 2008]; Available at http://www.sussexacuk/sociology/documents/edreghar.pdf.
- 12.Centrale Commissie Mensgebonden Onderzoek. Centrale commissie mensgebonden onderzoek jaarverslag 2001. [last accessed: 30 March 2008];[Central Committee on Research Investigations in Human Subjects] Available at http://www.ccmo_online.nl/hipe/uploads/downloads_catc/ccmo-jaarverslag-2001.pdf.
- 13.Rahman M, Fukui T. A decline in the U.S. share of research articles. N Engl J Med. 2002;347:1211–2. doi: 10.1056/NEJM200210103471523. [DOI] [PubMed] [Google Scholar]
- 14.Bardy AH. Report bias in drug research. Therapie. 1996;51:382–3. [PubMed] [Google Scholar]
- 15.Decullier E, Lheritier V, Chapuis F. Fate of biomedical research protocols and publication bias in France: retrospective cohort study. BMJ. 2005;331:19. doi: 10.1136/bmj.38488.385995.8F. [DOI] [PMC free article] [PubMed] [Google Scholar]