Reply to the Comments on “Meta-analysis Framework as a Form of Original Research”

Dear Editor,

We thank the commentators for their careful reading and comments on our published viewpoint, ¹ in which we discuss the similarities and differences between systematic reviews with meta-analysis (SRMA) and original (primary) research. We also thank them for reiterating the importance of SRMA as a “separate, high-value form” of research in medicine. Below, we offer our response to their comments.

The main thrust of their argument appears to be the difference in output of SRMAs versus original articles: While SRMAs generate new knowledge by summarizing evidence, identifying gaps, and providing high-level conclusions, they do not generate new empirical observations. We respectfully differ. As we already mentioned in our article, the result of an SRMA is typically a pooled estimate with a 95% confidence interval. This is obtained by statistically aggregating the results of individual studies included in the SRMA. ² The results thus obtained are new, novel observations not recorded in prior literature. Hence, an SRMA creates new inferential data or results from primary observations that can inform practice, research, and policy; it is not merely a synthesis of data, as the commentators suggest. This is the key innovation and value of an SRMA.

We also continue to assert that SRMAs are empirical; that is, based on observations rather than on opinion. Observations in empirical research can be obtained from inanimate, plant, animal, or human units, as well as from individual units (e.g., patients) or groups of units (e.g., hospitals, states, or countries). In SRMAs, studies are the units of observation from which data are obtained. ³ Arguing that only studies should be disallowed as units of observation is specious and prejudiced. Likewise, arguing that SRMAs do not generate new data but use data that have already been collected is also unwarranted; retrospective cohort and case-control studies are considered empirical even though they are based on data that have already been collected. Published studies are as valid a source of data as are insurance or healthcare databases. Mere availability of data in the public domain does not disqualify it for study.

The commentators also observe that original research results in the generation of novel primary data. So, should replicatory studies be disallowed from classification as original research because there is no longer novelty associated with the studies? Surely not. Whatever answer is given to support replicatory studies as original research will also support SRMAs as original research.

As a final note, we draw the readers’ attention to the apparent contradiction of separating SRMAs from original research for journal listings, research awards, and academic appointments, when the National Medical Commission has given similar status to both for academic promotions. ⁴ We reiterate that SRMAs are not mindless, mechanical exercises. They are probably best conceptualized as specialized research designs under the general heading of original research. For an overview of the limitations and unique sources of bias in an SRMA, we refer readers to our published article.1

For an obvious example of the public health importance of an SRMA, readers need look no further than the highly cited SRMA by Lau and colleagues, ⁵ published in 1992 in the New England Journal of Medicine; many lives could have been saved if this study had been conducted two decades earlier.