Finding the ultimate truth: What role remains for systematic reviews and meta-analysis?

Systematic reviews and meta-analyses (SRMAs) are considered the most reliable form of evidence as these integrate the results of multiple well-designed original studies and are currently positioned at the top of the evidence-based medicine pyramid. However, this ideal has been dominated by the realities of mass production. The last three decades have witnessed a greater than 25-fold increase in the production of SRMAs, with their growth rate significantly outpacing the expansion of original biomedical research literature.¹ It is also shocking that, in a few specialties, the publication of SRMAs exceeds that of other original research. With roughly 80 SRMAs appearing daily,² this overwhelming volume reflects a fundamental change in academic incentives rather than a rise in research quality.

The exponential growth of SRMAs has led to an overflow of redundant, low-quality content. To date, over a quarter of a million SRMAs have been published, but only a few are novel in their respective topics. Despite well-defined methodological guidelines, a substantial proportion of published SRMAs have fundamental flaws. The problems are illimitable; most SRMAs results are either partial or inconclusive. Often, SRMAs are synthesized from primary studies already burdened by a high risk of bias. Combining data using sophisticated statistical methods does not overcome the inherent heterogeneity across studies conducted at different times, in various clinical and experimental settings, and across different populations. The process of data pooling automatically obscures many critical details about subjects, study protocols, and unaccounted confounders etc. It typically happens when SRMAs are based on non-randomized studies. No amount of statistical adjustment can rectify biased primary data. Evaluation suggests that only 3% SRMAs are methodologically sound and beneficial clinically; the rest are decent but not functional (17%), redundant and unnecessary (27%), misleading (13%), unpublishable (20%), or fatally flawed beyond repair (20%).¹

Why such an overwhelming volume of unreliable synthesis? The drivers are purely systemic. The pressure to publish, particularly for an early-career researcher, makes it easier, faster, and more feasible to conduct SRMAs than to conduct original research. Additionally, SRMAs are often treated as original research, and further, the entire technical process is simplified by user-friendly, sophisticated software. This culture promotes a high-volume, low-rigor approach, risking the integrity of scientific research, encouraging quantity over quality.

The number of SRMAs on a given topic is so much that there is an increasing trend to do systematic review of systematic reviews or umbrella reviews. The umbrella reviews also inherit and compound the limitations of their constituent SRMAs. A protocol registration platform like PROSPERO and various guidelines enhance the transparency and credibility of SRMAs, but these do not, in themselves, ensure quality. These procedural steps are not a solution to fundamental flaws. The hype surrounding SRMAs is so intense that even Government funding agencies are inviting proposals from researchers to do such reviews. Some researchers and libraries have made their careers and existence out of teaching and conducting SRMAs.

The ultimate responsibility of publishing SRMAs lies with the journal editor. The ethical and editorial accountability are critical for the reliability and integrity of the SRMA process. Isn't it unethical to freely analyse and repackage data from others' work without their consideration? The journals should adopt a policy requiring disclosure of all permissions for included studies in the review. More crucially, the editors must resist the temptation to publish SRMAs as citation-bait, ensuring that publication decisions are based on scientific merit rather than metric gains for the journal. The primary responsibility of a journal editor is to safeguard scientific integrity by rejecting reviews that ask the wrong research questions, compile inappropriate studies, or are authored by so-called researchers lacking genuine expertise in the field including lived-experience of the research topic.

The results of various SRMAs ultimately serve as the basis for formulating clinical guidelines and protocols for the management of a particular disease. The management guidelines based on such published literature can be disastrous and misguide the clinician in decision-making. For example, the current guidelines for the management of oral cancer recommend neck dissection in all patients with N₀ neck because of a 20% prevalence of occult metastasis resulting in overtreatment for 80% patients. Similarly, the British guidelines for alloplastic temporomandibular joint replacement recommend alloplastic temporomandibular joint replacement in cases of temporomandibular joint ankylosis. There are many more examples of such untrustworthy evidence-based guidelines. Such examples underscore that the ultimate danger lies not in the SRMAs method itself, but in its misuse by asking wrong questions or incorrect interpretation of data.

Therefore, the fundamental question is not whether SRMAs have a role, but what that role should be. The SRMAs are not an alternative to robust primary evidence. The uninterrupted production of SRMAs from weak primary data does not strengthen knowledge; it creates a convincing, citation-generating mill that risks distorting the evidence base itself. The right path to find the ‘ultimate truth’ requires a paradigm shift. The scientific community must place greater emphasis on doing high-quality original research. Without robust data from primary studies, SRMAs cannot fulfil their objective of providing genuinely reliable evidence to guide healthcare. Until then, we must recognize that many systematic reviews serve the metrics-based academia rather than actionable evidence-based medicine.