Publications on AYUSH systems are growing; however, the methodological quality of conduct as well as reporting of this research remains a major concern. There is a need to critically analyze the conduct and reporting of research in this field and assess its subsequent value for learning and implementation. The approach of research synthesis is one of the tools to improve research quality, minimize bias and avoid research waste.
Meta-research majorly involves conducting research synthesis of available evidence through systematic review and meta-analysis. Importantly, research synthesis also involves appraisal of methodological quality of research process and reporting and its impact on strength of evidence. It is a relatively evolving interdisciplinary research strategy that covers many aspects of research process along with its implementation. Professor John Ioannidis from Meta-Research Innovation Center at Stanford, discusses interdisciplinary nature and knowledge domains of meta-research. Meta-research includes investigating method-related aspects of studies (including quality, study design, bias), reporting standards (e.g. registration, transparency, conflicts of interests), reproducibility, evaluation (e.g. cost effectiveness analyses) and incentives (value of research). It covers diverse disciplines ranging from history, epistemology, ethics and philosophy of science to newer advances like data science, scientometrics, policy and economics [1]. The meta-research initiatives are positively impacting quality of health research. Many initiatives like pre-registration of studies (e.g. clinical trial registries), development of reporting standards (EQUATOR Network), and methods for systematic reviews for weighing evidence are the notable outcomes of the meta-research.
1. Research synthesis and AYUSH knowledgebase
Meta-research encompasses theoretical and experimental perspectives of the study. Meta-research based systematic reviews should not be a mere compilation of studies but are expected to provide research synthesis with the intent to aid informed decision making. A good review should present state of the quality of current available evidence and its strength, discuss its use in decision making, and provide newer insights. There are many types of reviews based on the objectives and methods adopted [2]. Rigorous meta-analysis and systematic reviews often offer translational recommendation for direct implementation in healthcare. Mixed methods reviews combine outcomes from qualitative and quantitative research. Umbrella reviews compile evidence from various published reviews. Even the traditional narrative reviews can provide enriched knowledge of the subject and help frame and fine-tune research questions.
Indeed, meta-analysis and systematic reviews include appraisal of research literature to answer predefined research questions. It assesses data quality to investigate risks of bias and pools data of relevant studies if the outcomes and methods are comparable. In the evidence hierarchy, systematic reviews and meta-analyses are expected to provide the most robust evidence of interventions. Well done systematic reviews also highlight the lacunae and learning from the available studies. Nonetheless, the findings of these reviews obviously depend on many factors such as studies included, defined outcomes to assess effects, methods of analysis and interpretation, and possibilities of bias in research appraisal. Hence, to bolster the quality of systematic reviews; the review protocols are peer reviewed and published on the platforms like Cochrane library and Prospero before initiating the reviews.
In the context of AYUSH systems, systematic reviews may include relevant inputs from a vast treasure of AYUSH knowledgebase along with its epistemological roots and philosophical foundations. Such reviews can provide a great learning for teachers, researchers and practicing clinicians. They can be used as a teaching material and can be discussed in journal clubs. It is highly pertinent to be able to read systematic reviews and hence this should also be taught to teachers, researchers and clinicians. We need systematic reviews on critical appraisal of published AYUSH research as well.
2. Improving AYUSH research
The J-AIM receives many manuscripts, some of them claimed as ‘systematic reviews’. The most common rejection comments on such manuscripts are unclear objectives, insufficient compilations, sub-optimal methods (e.g. methods of systematic review used for narrative review), insufficient literature search, lack of transparency and risk bias in analysis and interpretation. Such manuscripts indicate the dire need for training on conducting and reporting of meta-research based systematic reviews.
Meta-research in AYUSH can play a vital role to reduce research waste by clearly outlining the current body of evidence and thereafter informing future study designs. The meta-research studies in AYUSH sector need to address designing clear and specific research questions, unbiased research designs and methods that include AYUSH principles of diagnosis and therapeutics (such as whole system yet personalized interventions), use of AYUSH relevant and specific outcomes, data analysis and interpretation also with AYUSH principles and publication standards. Hence, the teachers, researchers and clinicians of AYUSH systems should receive formal training in methods of meta-analysis. Many online resources could be helpful for such training. For example, Cochrane Handbook describes systematic methods and guidance on developing various types of reviews. Many MOOCs (Massive Online Open Courses like edX and Coursera) are providing online courses for writing systematic reviews.
Indeed, there are many challenges for meta-research on AYUSH concepts and interventions. The methods of meta-research need to be adapted to suit the nature of AYUSH systems. For example, the interventions may not be general but specific to disease subsets (Rogavastha), the interventions may be complex and customized to patient needs, there may be change in dose, duration, and combinations of treatment components. The research question itself can focus on the management approach rather than a specific formulation building a platform for evidence-based AYUSH practice. The training for AYUSH researchers should be also designed considering these challenges and unique needs of AYUSH healthcare system.
Meta-research offers an approach for systematic appraisal of studies. However, there is no substitute for core values of research – quality, ethics and transparency. Meta-research is expected to create awareness and vigilance to curb sloppy science but at the same time the mass production of poor quality systematic reviews has itself become a new challenge [3]. Hence, AYUSH healthcare researchers need to receive robust formal training of evidence-based medicine tools that are in the context of AYUSH healthcare. Also, the publication of peer reviewed study protocols of AYUSH healthcare studies can improve the credibility and reproducibility of these and future studies.
Acknowledgements
The author acknowledges the inputs and suggestions of Dr Rahul Mhaskar, Dr Preeti Chavan-Gautam and Dr Kishor Patwardhan.
Footnotes
Peer review under responsibility of Transdisciplinary University, Bangalore.
References
- 1.Ioannidis J.P., Fanelli D., Dunne D.D., Goodman S.N. Meta-research: evaluation and improvement of research methods and practices. PLoS Biol. 2015;13(10) doi: 10.1371/journal.pbio.1002264. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Grant M.J., Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009;26(2):91–108. doi: 10.1111/j.1471-1842.2009.00848.x. [DOI] [PubMed] [Google Scholar]
- 3.Ioannidis J.P. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q. 2016;94(3):485–514. doi: 10.1111/1468-0009.12210. [DOI] [PMC free article] [PubMed] [Google Scholar]