Publication hyper-inflation in the field of intensive care

Dear Editor,

It has been about 30 years since the introduction of evidence-based medicine (EBM) [1]. Now, physicians carry out their daily clinical practice based on the newest and best available evidence. However, over the past two decades, the number of published evidence has exploded. In 2010, Bastian et al. reported that 75 randomized controlled trials (RCTs) and 11 systematic reviews (SRs) were published per day [2]. Moreover, the recent coronavirus disease 2019 (COVID-19) pandemic induced a further massive increase in the number of publications [3]. As a result, this might lead to the so called “publication hyper-inflation”. Yet, it is still unclear how many publications intensivists are exposed to.

We therefore conducted a meta-epidemiological study to examine the number of publications in the field of intensive care. We systematically searched Pubmed for RCTs and SRs published between 1990 and 2021. To search potentially relevant studies for intensivists, we used three Medical Subject Headings (MeSH) terms: “Critical Care”, “Intensive Care Units”, “Critical Care Nursing”. To retrieve RCTs and SRs, we used one of the search strategies as the study design filter which previous study used [2] (see supplementary materials for details). We then combined MeSH terms and the study design filter to examine the number of RCTs and SRs per year. We used Mann–Kendall test for trend analysis and binomial test for calculating the 95% confidence interval (CI) of the SRs to RCTs ratio. We performed two types of sensitivity analyses. First, we used Cochrane Central Register of Trials (CENTRAL) instead of Pubmed to retrieve the number of RCTs similar to the previous study [2]. Second, we excluded COVID-19-related studies to consider the influence of COVID-19-periods. We defined COVID-19-related studies based on MeSH terms (Supplement for details).

The results showed the number of RCTs per year was 66 in 1990, reached 331 in 2013, and then passed to 309 (25.8 /month) in 2021. The number of SRs, which was 2 in 1990, almost equated the number of RCTs in 2016 at 327 and reached 519 (43.3/month) in 2021 (both p for trend < 0.001) (Fig. 1, supplementary Table S1). Therefore, the ratio SRs to RCTs reached 1.68 (95%CI, 1.46 to 1.94) in 2021 (supplementary Fig. S1, Table S2). The results of sensitivity analyses also showed the explosion of publications (supplementary Figs. S2-S3, Table S1, Table S3).

Fig.1.

Annual trend in the number of published SRs and RCTs between 1990 and 2021. The number of RCTs significantly increased from 66 to 309 (25.8/month) over the periods (p for trend < 0.001). The number of SRs also significantly increased from 2 in 1990 to 519 (43.3/month) in 2021 (p for trend < 0.001)

The results of this study confirmed that there is an explosion in the number of publications in the field of intensive care. Ideally, intensivists need to read every article possible and catch up with newest knowledge. However, our study showed that the current number of publications far exceeds the number that can be read. The increase in the number of SRs is particularly noteworthy, with the number of SRs published relative to RCTs being greater than in any other disease area reported in previous studies [4]. Increased number of SRs may have a negative impact rather than contributing to the progress of EBM [5]. In the era of “publication hyper-inflation”, we caution that intensivists should examine the quality of papers and carefully select the ones that should be read more than ever before.

Supplementary Information

Below is the link to the electronic supplementary material.

Data availability

All data relevant to the study are uploaded as supplementary information.

Conflicts of interest

The authors declare that they have no conflict of interest.