How can meta-research be used to evaluate and improve the quality of research in the field of traditional, complementary, and integrative medicine?

Abstract

The field of traditional, complementary, and integrative medicine (TCIM) has garnered increasing attention due to its holistic approach to health and well-being. While the quantity of published research about TCIM has increased exponentially, critics have argued that the field faces challenges related to methodological rigour, reproducibility, and overall quality. This article proposes meta-research as one approach to evaluating and improving the quality of TCIM research. Meta-research, also known as research about research, can be defined as “the study of research itself: its methods, reporting, reproducibility, evaluation, and incentives”. By systematically evaluating methodological rigour, identifying biases, and promoting transparency, meta-research can enhance the reliability and credibility of TCIM research. Specific topics of interest that are discussed in this article include the following: 1) study design and research methodology, 2) reporting of research, 3) research ethics, integrity, and misconduct, 4) replicability and reproducibility, 5) peer review and journal editorial practices, 6) research funding: grants and awards, and 7) hiring, promotion, and tenure. For each topic, we provide case examples to illustrate meta-research applications in TCIM. We argue that meta-research initiatives can contribute to maintaining public trust, safeguarding research integrity, and advancing evidence based TCIM practice, while challenges include navigating methodological complexities, biases, and disparities in funding and academic recognition. Future directions involve tailored research methodologies, interdisciplinary collaboration, policy implications, and capacity building in meta-research.

1. Introduction

The field of traditional, complementary, and integrative medicine (TCIM) has garnered increasing scholarly and public attention in recent decades due to its holistic approach to health and well-being,1, 2, 3 encompassing a diverse array of therapeutic modalities that emerged from different geographical regions, histories, and epistemologies.⁴ The definition of “traditional medicine” by the World Health Organization encompasses “the entirety of knowledge, expertise, and customs rooted in the theories, beliefs, and experiences of various cultures, whether explainable or not, utilized in the preservation of health and the management, prevention, diagnosis, or treatment of physical and mental ailments”.⁵ The US National Center for Complementary and Integrative Health (NCCIH) distinguishes “alternative” health approaches as those outside mainstream medicine, “complementary” health approaches as those used alongside conventional medicine, and “integrative health” as the coordinated merging of complementary approaches with conventional medicine.⁶ The global acceptance of the integrative approach to medicine is on the rise, as both patients and practitioners acknowledge its capacity to complement and enrich conventional treatments by addressing the entirety of a person's well-being.¹ Throughout this article, we will refer to this group of therapies as TCIM.

Despite TCIM popularity among patients, and an exponential growth in the number of research articles published on the topic of TCIM in recent years,⁷ TCIM researchers are confronted by considerable challenges related to methodological rigour, reproducibility, and overall research quality.8, 9, 10, 11, 12 Addressing these challenges is crucial for establishing safe and effective TCIMs as a credible and evidence-based approach to healthcare. This necessitates the adoption of rigorous research methodologies, transparency in reporting, and collaboration among stakeholders to ensure that the research that informs TCIM practice meets robust standards.^11,13 By addressing these challenges effectively, we believe that the credibility and impact of TCIM research can be enhanced, fostering trust among patients, practitioners, and the broader healthcare community. Ultimately, this can contribute to the integration of TCIM into conventional healthcare, providing patients with access to evidence-based treatment options that prioritize holistic well-being. In this article, we propose that one solution to accomplishing this is through the conduct of meta-research.

2. Definition of meta-research

Meta-research, also known as research about research, can be defined as “the study of research itself: its methods, reporting, reproducibility, evaluation, and incentives”.¹⁴ It is sometimes used interchangeably in the literature with the term “metascience”,¹⁵ however, as the field of TCIM extends beyond laboratory and clinical science (e.g., sociology), for the purpose of this article, we will henceforth refer to this term as “meta-research”. Meta-research can be categorized into five primary areas, as follows: methods, reporting, reproducibility, evaluation, and incentives, which correspond to the processes of conducting, communicating, verifying, assessing, and incentivizing research, respectively.¹⁶ Meta-research collectively facilitates a comprehensive examination of research methodologies, biases, and gaps in the existing literature surrounding research quality.^14,16 While meta-research interfaces with established disciplines such as statistics, data science, and medicine, the overarching goal within this field is to enhance the reliability, credibility, and transparency of research practices.^14,16

2.1. Significance of meta-research in traditional, complementary, and integrative medicine

The proficiency, expertise, and aptitude of researchers represent a significant impediment. Factors contributing to diminished competency encompass inadequate training, limited experience, or deficient research literacy.^10,17,18 Additionally, bias poses a barrier, fostering adverse perceptions toward TCIM research vis-à-vis research in other (biomedicine) medicine fields.¹⁰ This bias often stems from the presumption that TCIM lacks evidential support, necessitating concerted efforts to rectify.^19,20 Other instances of bias include, but are not restricted to, the biomedical community's disregard for TCIM and its research, as well as the inadequate cooperation between TCIM researchers and those in other (biomedicine) medical fields.¹⁰ Although we do not argue that the overall quality of TCIM research is necessarily poor, these challenges sometimes pose great hurdles in reliably evaluating the efficacy, safety, and broader implications of TCIM modalities.

In response to these challenges, we argue that meta-research can serve as a key tool that can be used to address these complexities through thorough and meticulous examination of the research methodologies that have been used to study TCIM to date. By systematically assessing the methodological strengths and weaknesses inherent in studies, meta-research serves to study, promote, and defend high quality research.^16,21,22 Furthermore, meta-research can play a crucial role in identifying sources of bias and variability within TCIM studies,¹⁰ thereby shedding light on the factors influencing research outcomes.^23,24 Downstream, this synthesized evidence not only contributes to advancing knowledge but also informs clinical decision-making and policy formulation in the TCIM domain.¹² Thus, we argue that by conducting TCIM-specific meta-research, we can ultimately both evaluate and further improve the quality of research in this field.

2.2. Purpose and scope of the article

The purpose of this article is to explore the role of meta-research in evaluating and improving the quality of research in the field of TCIM. Specifically, this article aims to provide an in-depth overview of the application of meta-research in TCIM, encompassing a non-exhaustive list of seven major topics of interest, as depicted in Fig. 1: 1) study design and research methodology, 2) reporting of research, 3) research ethics, integrity, and misconduct, 4) replicability and reproducibility, 5) peer review and journal editorial practices, 6) research funding: grants and awards, and 7) hiring, promotion, and tenure of researchers. Table 1 summarizes all of the details about these seven key topics. By providing an overview and case example pertinent to each of these meta-research topics, we aim to shed light on how to achieve the following specific to TCIM research: identify methodological strengths and weaknesses; identify sources of bias and variability; and synthesize evidence to inform clinical practice and policy decisions in TCIM. Following this, we provide a number of benefits as well as challenges associated with conducting work at the intersection of meta-research and TCIM. Finally, we discuss future directions that can be pursued to advance knowledge, address gaps, and overcome challenges.

Fig. 1.

Key topics of meta-research applicable to traditional, complementary, and integrative medicine.

Table 1.

Seven key topics of meta-research applicable to traditional, complementary, and integrative medicine.

Meta-Research Topic	Key Summary Points
Study Design and Research Methodology	•Study design and research methodology are crucial for ensuring the quality and reliability of research findings, particularly in TCIM. •Meta-research in this area evaluates the methodological rigour, identifies biases, and proposes strategies to enhance research quality, aiming to improve the generalizability of findings and advance evidence-based practice. •Important considerations include the appropriateness of study designs, robustness of outcome measures, integration of diverse methodologies, and addressing ethical considerations such as informed consent and conflict of interest disclosures.
Reporting of Research	•Reporting guidelines are essential for ensuring transparency and integrity in TCIM research, facilitating accurate and complete reporting, which aids in replication, critical appraisal, and clinical decision-making. •Meta-research evaluates adherence to these reporting standards, identifying biases and questionable practices, and aims to enhance the credibility and reproducibility of TCIM research. •Frameworks such as the CITE Framework provide guidance for integrating traditional knowledge with evidence-based approaches, emphasizing rigorous evaluation and comprehensive reporting to bridge traditional knowledge with contemporary health practices.
Research Ethics, Integrity, and Misconduct	•The study of research ethics, integrity, and misconduct are vital in TCIM research, with ethics encompassing norms of conduct and integrity involving the practice of these norms, while misconduct includes fabrication, falsification, and plagiarism. •Meta-research in this area focuses on evaluating and improving processes such as informed consent, data management, and conflict of interest disclosures, which are critical for maintaining research credibility. •Addressing ethical concerns and promoting responsible conduct through meta-research helps maintain public trust, safeguard research participants, and uphold the integrity of TCIM research.
Replicability and Reproducibility	•Replicability and reproducibility are crucial for ensuring the reliability and validity of research findings, with both terms relating to the consistency of study results when replicated or reproduced by other researchers. •Meta-research in this area evaluates how well study findings can be replicated or reproduced, focusing on transparency of methods, availability of raw data, and result consistency across studies. •Key factors influencing replicability and reproducibility in TCIM research include methodological variations, publication biases, and selective reporting of outcomes.
Peer Review and Journal Editorial Practices	•Peer review and journal editorial practices are critical for maintaining the quality and integrity of research publications, with meta-research evaluating the effectiveness, reliability, and transparency of these processes. •Key considerations include the transparency of peer review procedures, reviewer expertise and diversity, consistency in editorial decision-making, and the identification of biases, conflicts of interest, and inefficiencies. •Promoting transparency and accountability in peer review and editorial practices helps prevent misconduct, ensures the robustness of TCIM research publications, and fosters trust in research dissemination.
Research Funding: Grants and Awards	•Research funding is essential for advancing TCIM research, with meta-research evaluating the distribution, impact, and outcomes of funding, and identifying biases and disparities in allocation. •Key considerations include the transparency and fairness of funding processes, alignment of funding priorities with research needs, and the influence of funding sources on research outcomes. •Meta-research helps optimize resource allocation, foster collaboration, and advance evidence-based practice in TCIM by assessing the effectiveness and impact of research funding.
Hiring, Promotion, and Tenure of Researchers	•Hiring, promotion, and tenure processes are crucial for academic advancement, with TCIM researchers facing unique challenges in this domain. •Meta-research evaluates the criteria, practices, and outcomes of these processes, focusing on transparency, fairness, recognition of research contributions, and alignment with evolving research priorities. •By identifying biases, disparities, and barriers, meta-research aims to optimize career advancement pathways, foster inclusivity, and enhance the quality and impact of TCIM research.

3. Key topics of meta-research applicable to traditional, complementary, and integrative medicine

In this section, we provide an overview of key topics of meta-research, within which projects can be initiated to evaluate and improve the quality of research in the field of TCIM. We have also included a relevant case example specific to TCIM following the discussion of each topic.

3.1. Study design and research methodology

Study design and research methodology play a fundamental role in shaping the quality and reliability of research findings^25,26 across all fields, including TCIM. Study design can be defined as “a framework, or the set of methods and procedures used to collect and analyze data on variables specified in a particular research problem”.²⁷ Meta-research in this area focuses on evaluating the methodological rigour of studies²⁸, identifying potential biases,^29,30 and proposing strategies to enhance research quality.^31,32 By scrutinizing the design and methodology of studies, the aim would be to improve the generalizability of findings, ultimately advancing evidence-based practice.^14,27 Key considerations include the appropriateness of study designs for investigating TCIM interventions, the robustness of outcome measures, and the integration of diverse methodologies to capture the complexity of TCIM approaches. Moreover, meta-research sheds light on ethical considerations, such as informed consent procedures and conflict of interest disclosures, ensuring transparency and integrity in TCIM research.

Case Example: In examining the effectiveness of TCIMs, meta-research has revealed methodological challenges as indicated by a couple recent studies. One study conducted by Zhang et al.³³ examined the use of pattern differentiation in WHO-registered traditional Chinese medicine (TCM) trials, revealing inadequate reporting and application of pattern differentiation. Among 376 trials including pattern differentiation, only 43.6 % reported pattern differentiation in outcomes, and 7.2 % presented diagnostic criteria for the pattern studied. These findings underscore the importance of meta-research in evaluating methodological rigour and identifying areas for improvement in TCIM research, particularly in the design and analysis of acupuncture and TCM trials, which can inform future research practices in this field. Another study conducted by Liu et al.³⁴ highlighted deficiencies in statistical methods employed in acupuncture randomized controlled trials (RCTs) published between 2010 and 2019. Among the 262 RCTs analyzed, only 50.4 % clearly predefined the primary outcome, while 27.5 % specified the use of intention to treat or modified intention to treat population for primary analysis. Moreover, in trials reporting missing participant data, 70.7 % used suboptimal methods for dealing with missing participant data, and only 6.6 % conducted sensitivity analysis. Additionally, only 13.0 % of trials with repeated measures design utilized advanced statistical models for handling repeated-measure data in the primary analysis. This indicates a substantial gap between recommended statistical practices and their implementation in acupuncture RCTs.

3.2. Reporting of research

Reporting of research is a critical aspect of ensuring transparency and integrity in TCIM research and can be aided using reporting guidelines. A reporting guideline has been defined as “a checklist, flow diagram, or explicit text to guide authors in reporting a specific type of research, developed using explicit methodology”.³⁵ Reporting guidelines exist for a variety of study types, with extensions developed for specific interventions. Meta-research in this area focuses on evaluating the completeness and accuracy of research reports, as well as the adherence to reporting standards and guidelines.^36,37 Effective reporting facilitates the replication of studies, enables critical appraisal of research findings, and informs clinical decision-making and policy development.³⁶ Specific to the field of TCIM, the implementation of traditional knowledge within research and practice requires rigorous evaluation and reporting standards. Frameworks such as the Contemporary Implementation of Traditional Knowledge and Evidence in Health (CITE) Framework³⁸ provide guidance on selecting, evaluating, and applying traditional knowledge in health contexts, aiming to bridge traditional knowledge with evidence-based approaches. In a systematic review, Foley et al.,³⁹ shed light on the criteria used for the selection, evaluation, and application of traditional knowledge in contemporary health practice, highlighting the importance of robust methodologies and comprehensive reporting in integrating traditional knowledge into evidence-based healthcare practices. Meta-research endeavours to identify biases and questionable practices in reporting TCIM research, such as selective outcome reporting.⁴⁰ By promoting comprehensive and transparent reporting practices, meta-research contributes to enhancing the credibility and reproducibility of TCIM research, ultimately advancing evidence-based practice in the field.

Case Example: The quality of herbal supplement RCTs have been found to vary following assessments of their efficacy in the management of various diseases/conditions.⁴¹ Additionally, it has been documented in the literature that researchers who conducted systematic reviews and meta-analyses of TCIM RCTs encounter challenges related to inconsistent and incomplete reporting of study methods and results.^42,43 Moreover, more recent research, exemplified by a systematic review evaluating the adherence of RCTs investigating herbal interventions for common dermatoses to the CONsolidated Standards Of Reporting Trials (CONSORT) extension criteria on reporting herbal interventions (hCONSORT) criteria, reveals persisting challenges in meeting reporting standards for herbal interventions in dermatology, underscoring the ongoing need for improved adherence to reporting guidelines in TCIM research.⁴⁴ To combat this, meta-research projects could both involve evaluating adherence to established reporting standards (such as the CONSORT statement⁴⁵ and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement,⁴⁶ in addition to relevant reporting guideline extensions specific to TCIM³⁷), but also exploring how best to encourage TCIM researchers to adhere to these reporting standards in cases where work has already identified that adherence is poor (e.g., RCTs of herbal interventions⁴⁴).

3.3. Research ethics, integrity, and misconduct

Research ethics and misconduct are critical considerations in TCIM research, ensuring the integrity and credibility of research findings. Research ethics “encompasses norms of conduct and concomitant discourse pertaining to the delineation of good or acceptable from bad or unacceptable behavior in research”,⁴⁷ research integrity is the practice of these norms,⁴⁸ while [scientific] misconduct can be defined as “fabrication, falsification, or plagiarism in proposing, performing, or reviewing research, or in reporting research results”.⁴⁹ Examples of meta-research in this area include evaluating or improving the following processes: informed consent procedures,⁵⁰ data management,⁵¹ and the disclosure of conflicts of interest.^52,53 Additionally, addressing falsification of data is crucial in maintaining the credibility of TCIM research, as it undermines the validity of study findings and erodes public trust in the field. By addressing ethical concerns and promoting responsible conduct, meta-research contributes to maintaining public trust, safeguarding research participants, and upholding the integrity of TCIM research.

Case Example: Investigations were carried out regarding the work of a group of authors who had published articles in scholarly journals on the topic of nutrition, including dietary and herbal supplements, following irregularities in study data discovered in an article published in 2017; it was later found that the principal investigator of the group had allegedly published just under 150 RCTs within a 4-year period.⁵⁴ To date, these investigations by other researchers have extended to the discovery of suspicious findings in 172 publications authored by this group. Besides the fact that the sheer volume of research output from this group is deemed astonishing and implausible, evidence was found to suggest that the random allocation of participants could not have resulted in the reported treatment groups, raising questions about the reliability of the study designs. The distribution of participant withdrawals from these RCTs also appears implausible, casting doubt on the accuracy of reported outcomes. Moreover, inconsistencies in the reporting of participant population sizes are prevalent, further undermining the credibility of the findings. Most concerning is the identification of unethical conduct in the execution of RCTs, raising serious ethical and moral implications. Lastly, frequent discrepancies between trial registration documents and published journal articles, concerning study conduct, location, participant demographics, and other critical aspects, further erode confidence in the integrity of the research conducted by this group.^55,56 These revelations underscore the importance of upholding research ethics and integrity to ensure the credibility and trustworthiness of research findings, highlighting the role of meta-research in identifying and addressing such misconduct.

3.4. Replicability and reproducibility

Replicability and reproducibility are fundamental aspects of ensuring the reliability and validity of research findings in research.^57,58 While the distinctions between replicability and reproducibility are contested,⁵⁹ both of these terms refer to the extent to which study findings are consistent with findings by other researchers using data that is either the same or newly collected. In general, however, meta-research in this area focuses on evaluating the extent to which study findings can be replicated or reproduced across different settings or by independent researchers. Key considerations include the transparency of study methods, availability of raw data, and consistency of results across studies. Specific to this topic, meta-research work can identify factors that may influence replicability and reproducibility in TCIM research, such as methodological variations, publication biases, and selective reporting of outcomes.

Case Example: Incomplete reporting in scientific publications on herbal drugs is a prevalent issue, with many authors failing to accurately document chemical, biological, and pharmacological aspects clearly or completely, including the composition and stability of herbal extract preparations.^60,61 Complete chemical and pharmacological characterizations of bioactive metabolites are necessary for developing modern drugs from herbal sources, necessitating the establishment, assessment, and standardization of research methods. A study examining the reproducibility of herbal drug research practices offers insight into the challenges of ensuring accurate reporting and reliable results in this field. Süntar⁶² established a systematic workflow for studying herbal medicines, emphasizing the importance of accurate plant identification, chemical composition determination, and validation of pharmacological methods. By following a step-by-step approach outlined in the study, this aims to enhance the reproducibility of herbal drug research and improve the reliability of reported bioactivity and pharmacological effects. Süntar's work underscores the importance of adhering to rigorous research practices in herbal drug studies to ensure transparency, integrity, and replicability of findings, thereby advancing evidence-based practice in herbal medicine.

3.5. Peer review and journal editorial practices

Peer review and journal editorial practices are essential components of maintaining the quality and integrity of research publications.63, 64, 65 Meta-research in this area focuses on evaluating the effectiveness and reliability of peer review processes,^66,67 as well as editorial practices⁶⁸ employed by TCIM journals. Key considerations include the transparency of peer review procedures, the expertise and diversity of reviewers, and the consistency of editorial decision-making.^69,70,66 Meta-research specific to this area involves the identification of potential biases, conflicts of interest, and inefficiencies in peer review and editorial practices, aiming to enhance the robustness and credibility of TCIM research publications. Moreover, concerns about misconduct in peer review or editorial practices, which can lead to the retraction of published articles, underscore the importance of promoting transparency and accountability in the peer review and editorial process. By promoting transparency, and accountability in the peer review and editorial process, meta-research contributes to upholding the standards of research rigour and fostering trust in TCIM research dissemination.

Case Example: A notable case exemplifying the role of peer review and journal editorial practices in TCIM research emerges from a qualitative interview study exploring challenges faced by TCIM researchers as they attempted to publish their research.⁷¹ The study's findings underscore the perceived biases encountered by participating TCIM researchers during peer review and the consequent influence on their research processes. Participants argued that biased reviewers and editors often hindered the publication of TCIM studies, while TCIM researchers resorted to strict adherence to established research methodologies to navigate these obstacles. Despite efforts to adhere to rigorous standards, TCIM researchers argued that within conventional biomedical paradigms they encountered difficulties in demonstrating that they conducted credible research.⁷¹ This case example highlights the need for meta-research that evaluates and addresses biases in peer review and editorial practices, ensuring the fair evaluation and dissemination of TCIM research. By scrutinizing the peer review and editorial practices of journals, this can identify areas for improvement and lead to the proposal of strategies to enhance transparency, fairness, and quality assurance in the publication process.

3.6. Research funding: grants and awards

Research funding, including grants and awards, plays a crucial role in supporting and advancing research endeavours in research.^72,73 Meta-research in this area focuses on evaluating the distribution, impact, and outcomes of research funding in TCIM, as well as exploring potential biases and disparities in funding allocation.^74,75 Key considerations include the transparency and equitableness of funding processes, the alignment of funding priorities with research needs, and the influence of funding sources on research outcomes.^76,77 By assessing the effectiveness and impact of research funding, meta-research in this area contributes to optimizing resource allocation, fostering collaboration, and advancing evidence-based practice in TCIM.

Case Example: An example illustrating the influence of research funding in TCIM can be observed in a hypothetical study examining the quantity and nature of research grants that have been awarded to study a commonly used TCIM therapy for a given disease/condition. This meta-research could involve assessing the distribution and impact of research funding on this TCIM therapy's research outcomes, including publication productivity, citation impact, and clinical translation.⁷⁸ By examining funding patterns and disparities, opportunities to address gaps in research funding can be identified, increased investment in underrepresented areas can be advocated for, and interdisciplinary collaboration can be fostered.^11,79 For example, such a study may reveal a lack of funding for clinical trials investigating the efficacy and safety of this TCIM therapy for a given disease/condition of interest, highlighting the need for targeted funding initiatives in this area. In general, meta-research in this area underscores the importance of transparent and fair research funding practices in driving innovation and addressing priority health needs in TCIM.

3.7. Hiring, promotion, and tenure

Hiring, promotion, and tenure processes are pivotal components of academic advancement and career development in research^80,81; it is worth noting that TCIM researchers face unique, field-specific challenges, as outlined in the case example below. More generally, however, meta-research in this domain focuses on evaluating the criteria, practices, and outcomes associated with hiring, promotion, and tenure decisions in academia.⁸² Key considerations include the transparency and fairness of evaluation criteria, the recognition of research contributions, and the alignment of institutional practices with evolving research priorities.^82,83 Meta-research endeavours to identify potential biases, disparities, and barriers in hiring, promotion, and tenure processes.⁸⁴ By assessing the effectiveness and impact of academic evaluation practices, meta-research contributes to optimizing career advancement pathways, fostering inclusivity, and enhancing the quality and impact of TCIM research.

Case Example: Researchers who study TCIM, may face unique field-specific challenges in the academic setting, relating to reconciling the epistemological differences between TCIM modalities and mainstream scientific frameworks.⁸⁵ Even before becoming a TCIM researcher, one hurdle to overcome includes gaining the necessary and appropriate training in this field.¹⁷ Unique to this field is the fact that university-based TCIM researchers come from two backgrounds: 1) individuals not trained as TCIM practitioners and 2) individuals trained as TCIM practitioners. TCIM researchers in the first category can be categorized as those who have never been a TCIM practitioner, but earned a graduate or terminal degree (e.g., such as a Master's or a Doctor of Philosophy (PhD) in a health field or an Doctor of Medicine (MD), a Doctor of Dental Surgery (DDS), their equivalents, or another conventional practitioner degree) and who hold an active interest in the topic of TCIM. Such individuals are proportionately few (when compared to all graduate or terminal degree-holding researchers in health fields), given that TCIM is not a commonly taught topic at the university, and an interest in TCIM may have instead been gained through experience with personal illness or interactions with patients who inquired about TCIMs. The second category of TCIM researchers, current and former TCIM practitioners, have also typically earned a postgraduate degree at a university, because completing a postgraduate degree with a research component is the typical route for the formal development and recognition of research skills. This is also thought to be the course of study that best prepares students for careers in research.⁸⁶ Although a doctoral program, such as a PhD, is the most prestigious research higher degree, admission to a PhD program requires research experience and training. Specific to countries such as the United Kingdom, Australia, and Canada, it is known that private colleges dominate TCIM practitioner training (e.g., naturopathy, herbal medicine, and homeopathy), and these colleges may be unable (or unwilling) to overcome the hurdles required to offer bachelor's degree and postgraduate degree programs.^17,85 In Australia, the government disallowed the teaching of less than Bachelor's degree programs for a range of TCIM professions in 2014 and this has seen a marked reduction in institutions offering TCIM practitioner training overall.⁸⁷ Even if TCIM practitioners are able to overcome these educational hurdles,⁷⁵ once they gain a researcher position at the university, they may encounter pressure to develop evidence bases for TCIM practices that align with evidence-based medicine, despite the inherent challenges in fitting TCIM into traditional biomedical models.⁸⁵ Moreover, skepticism from within academia, as well as media campaigns disparaging TCIM, undermines the legitimacy of TCIM research and the professional identity of TCIM researchers, regardless of whether they are/were (former) TCIM practitioners or not, adding to the complexities of their roles.^85,88 TCIM researchers who are/were (former) TCIM practitioners, like biomedical practitioners, are additionally required to navigate dual identities as both academics and practitioners while addressing the demands for evidence and recognition within universities.⁸⁸ By examining hiring, promotion, and tenure practices, meta-research can serve to identify potential biases and barriers faced by TCIM researchers, such as limited access to institutional resources, disparities in research funding opportunities, and a general lack of a clear career path for those in the academic field of TCIM.

4. Benefits of meta-research in the context of traditional, complementary, and integrative medicine

It is our view that meta-research offers a promising avenue for advancing the field of TCIM by leveraging established methodologies from other medical disciplines. While meta-research within the field of TCIM remains relatively nascent, drawing upon the wealth of knowledge accumulated in other medical fields can yield key benefits. Meta-research can serve as a critical tool for evaluating methodological rigour, identifying potential biases, and proposing strategies to enhance research quality, thereby improving the evidence base for TCIM. Meta-research initiatives contribute to maintaining public trust and safeguarding research participants by addressing ethical concerns and promoting responsible conduct in research endeavors. Meta-research endeavours to enhance the replicability and reproducibility of research findings, thereby fostering confidence in the reliability of TCIM research evidence. By evaluating peer review and journal editorial practices, meta-research can enhance the transparency, accountability, and fairness of research dissemination processes. Additionally, meta-research in the domain of research funding plays a crucial role in optimizing resource allocation, fostering collaboration, and advancing evidence-based practice in TCIM. Finally, by examining hiring, promotion, and tenure practices, meta-research can identify potential biases and barriers faced by TCIM researchers, ultimately contributing to the optimization of career advancement pathways and the enhancement of the quality and impact of TCIM research. Thus, while the field of TCIM may currently lack extensive meta-research efforts, leveraging the benefits derived from meta-research in other medical fields holds significant promise for advancing research practices and outcomes in TCIM.

5. Challenges and barriers to meta-research in the context of traditional, complementary, and integrative medicine

We acknowledge that conducting meta-research in the field of TCIM will not be without challenges, reflective of broader research hurdles that are known to exist. These challenges include the access to the necessary data and methodological complexities arising from the diverse array of therapeutic modalities encompassed within TCIM, which necessitate tailored meta-research methodologies to account for the unique characteristics of each modality. Additionally, biases and skepticism towards TCIM within the biomedicine community pose obstacles, leading to perceptions of TCIM as lacking evidential support and hindering cooperation between TCIM researchers and those in conventional medical fields, that will not be entirely remedied by meta-research alone. Furthermore, navigating the dual identities of many TCIM researchers as both academics and practitioners, particularly for those with backgrounds in TCIM practice, adds layers of complexity to hiring, promotion, and tenure processes within academia. Limited access to institutional resources and disparities in research funding opportunities further exacerbate challenges in conducting meta-research in TCIM itself, which can continue to hinder efforts to optimize resource allocation and foster collaboration. Overall, while the potential benefits of meta-research in TCIM are evident, we still believe that we must work towards addressing these challenges and barriers, as this is imperative to advancing the quality and credibility of research in this field.

6. Future directions

At the intersection of meta-research and TCIM, several future directions can be pursued to advance knowledge, address gaps, and overcome some of the aforementioned challenges and barriers.

Future efforts should focus on developing tailored meta-research methodologies that accommodate the unique characteristics and complexities of TCIM modalities. This includes exploring innovative approaches to assessing methodological rigour, addressing biases, and enhancing transparency in TCIM research practices. This may be achieved by developing and implementing standardized reporting guidelines for TCIM-relevant and TCIM-specific research which can improve the quality and consistency of reporting of research. This can include the implementation of currently existing reporting guidelines, in addition to the development of additional TCIM-specific extensions of existing reporting guidelines.

Fostering interdisciplinary collaboration between TCIM researchers, conventional medical researchers, and experts in meta-research methodologies can facilitate the exchange of knowledge and best practices, contributing to the advancement of evidence-based practice in TCIM. Furthermore, initiatives aimed at promoting research literacy and training in TCIM-specific research methods among researchers and practitioners can enhance capacity building within the field. Furthermore, building capacity in meta-research itself within the TCIM community through education and training initiatives is essential. Providing resources, workshops, and courses on meta-research methods can empower TCIM researchers to conduct research that evaluates and improves the research in their own field.

Investigating the policy and practice implications of meta-research findings in TCIM is essential for translating evidence into action. This can lead to a stronger case for increased funding and resources dedicated to TCIM research, which is essential to overcoming existing barriers and disparities in research funding allocation. Finally, ongoing evaluation and refinement of hiring, promotion, and tenure practices within academia to better recognize and support TCIM researchers, particularly those with diverse backgrounds and dual identities as academics and practitioners, are critical for advancing TCIM research.

7. Conclusion

This article has provided insight into how meta-research can be used to evaluate and improve the quality of research in the field of TCIM. By systematically evaluating methodological rigour, identifying biases, and promoting transparency, and optimizing resource allocation, meta-research can play a pivotal role in improving the evidence base and advancing practice within TCIM.

CRediT authorship contribution statement

Jeremy Y. Ng: Conceptualization, Writing – original draft, Writing – review & editing. Myeong Soo Lee: Writing – review & editing. Jian-ping Liu: Writing – review & editing. Amie Steel: Writing – review & editing. L. Susan Wieland: Writing – review & editing. Claudia M. Witt: Writing – review & editing. David Moher: Writing – review & editing. Holger Cramer: Writing – review & editing.

Declaration of Competing Interest

JYN, MSL, JPL, AS, LSW, and HC are part of the editorial board of this journal but this education article was externally peer reviewed and their role for the journal had no bearing on the editorial decision. The authors declare that they have no other competing interests.