Navigating uncertainty in evidence-informed decision-making for traditional, complementary and integrative medicine: An updated critical interpretive review

Abstract

Background

Evidence-informed decision-making (EIDM) presents unique challenges for traditional, complementary and integrative medicine (TCIM). To aid decision-making, a critical interpretive review was updated to examine how internationally and nationally endorsed consensus statements on guideline development and policy briefs, along with TCIM-specific recommendations may address these challenges.

Methods

The review drew on critical interpretive synthesis and rapid review methods. PubMed, CINAHL and AIMED were searched on 19 January 2024. Single reviewers conducted the screening and extracted descriptive data. Analysis began deductively from the original review themes and developed iteratively.

Results

Included were 167 publications. Over the past decade, evidence-to-decision (EtD) frameworks have considered a broader range of modifying factors (e.g., importance of the problem, stakeholder preferences, feasibility, costs, health equity, human rights, social impacts) that may warrant making a stronger recommendation despite lower certainty evidence, and vice versa. Additional strategies proposed by TCIM guideline developers included systematically incorporating multiple evidence sources and developing TCIM extensions for reporting guidelines. Four interrelated themes affirmed the importance of 1) transparent and inclusive decision-making, 2) ensuring the scope and framing of the problem are sensitive to TCIM contexts, 3) using diverse types of evidence in EtD frameworks and when there is evidence uncertainty, and 4) centering equity, including epistemic equity, throughout the EIDM process.

Conclusion

Advancing EIDM methodologies specifically for TCIM guideline development and policy can help decision-makers navigate evidence uncertainty to support the appropriate integration of TCIM into global health systems.

Protocol registration

International Platform of Registered Systematic Review and Meta-analysis Protocols: INPLASY202510066.

1. Introduction

Traditional, complementary and integrative medicine (TCIM) continues to be widely used across the globe.^1,2 The Word Health Organization recognizes the role of TCIM in achieving the United Nations’ sustainable development goals and universal health coverage, which ideally should be informed by evidence.³ Despite an exponential growth in TCIM research, clinical practice guidelines (CPGs) and health policies often overlook TCIM.4, 5, 6 This is partly due to evidence uncertainty. Notwithstanding, biomedical interventions are often recommended despite very low-certainty evidence.⁷

In response to this observation, Hunter et al.⁶ reviewed internationally endorsed consensus statements for guideline development and policy briefs to determine how they could further support TCIM evidence informed decision-making (EIDM), particularly when evidence is insufficient or uncertain. Key findings highlighted the importance of framing the question or problem to ensure it adequately considered TCIM contexts and reasons for use; not using an evidence hierarchy as a direct constrainer on the strength of recommendations; applying endorsed strategies for making recommendations despite insufficient evidence; including evidence about modifying factors (e.g., stakeholder values, demand, costs, equity and feasibility) that may warrant making a stronger recommendation despite uncertainties about clinical benefits and harms; and applying structured consensus processes to enhance transparency and reduce bias in the decision making process ⁶.

In the decade following, substantial methodological progress has been made to support EIDM. The ‘Grading Recommendations Assessment Development Evaluation’ (GRADE) working group and World Health Organization (WHO), along with other bodies, continue to update and expand their guidance. Concurrently, TCIM guideline developers are responding to the challenges of applying EDIM guidance designed predominantly for biomedical contexts.^4,5,8 This includes acupuncture⁹ and traditional Chinese medicine¹⁰ extensions to the ‘Reporting Items for practice Guidelines in HealThcare’ (RIGHT),¹¹ along with modifications and alternatives to improve the applicability of EIDM guidance to TCIM contexts.12, 13, 14

Given these methodological advances, it seemed timely to update the previous review by Hunter et al.⁶ The aim of this critical interpretive review was to examine how internationally endorsed consensus statements for guideline development and policy briefs, along with TCIM-specific recommendations, can support EIDM for TCIM guidelines and policy, particularly when the evidence of clinical benefits or harms is insufficient, uncertain, or absent.

2. Methods

This review drew on the principles of critical interpretive synthesis (CIS)¹⁵ to explore existing guidance and recommendations and critically reflect upon their underlying assumptions and values. The CIS approach supports a more nuanced analysis attuned to the complexity and cultural diversity of TCIM, in which the methods for knowledge generation may fall outside conventional evidentiary norms. For pragmatic reasons, rapid review (RR) methods were applied as reported below. A protocol was registered on 18 January 2025 (ID: INPLASY202510066).¹⁶ Reporting follows relevant PRISMA 2020 items¹⁷ and recommendations for reporting CIS reviews.¹⁸

2.1. Criteria for inclusion and exclusion

Eligible publications were formal consensus statements on the EIDM process in any healthcare setting that were endorsed by governmental or international bodies and published in peer-reviewed journals or grey literature. TCIM-specific formal or informal statements and recommendations published in peer-reviewed journals were also included. Examples of eligible EIDM guidance include recommendations for clinical practice guideline development, evidence to decision (EtD) frameworks or evidence-informed policy briefs.

2.2. Literature searches

RR methods were applied to constrain the search. This included not conducting a second-stage of screening as is typical for CIS,¹⁸ only searching three databases (PubMed, CINAHL and AIMED) from inception to 19 January 2024; structuring database search terms to optimise specificity; and only searching websites of previously included consensus statements for updates and new consensus statements (Supplementary File 1). Searches were augmented through citation and project tracking, and publications known to the reviewers. There were no limits on publication dates or languages.

2.3. Data selection

As per RR methods, a single reviewer (JH) screened the database search results and websites. A second reviewer (ML) screened the excluded publications and reincluded any they considered relevant.

2.4. Data extraction

After calibration, data were extracted from included publications by single reviewers (JH, ML) using a customized data extraction template and reviewed by another reviewer (JH, ML, DC, LF, JL). Any disagreements were resolved through consensus.

The data extracted included publication date, region and a summary of the content and focus, along with information relevant to framing the questions, the role of non-RCT evidence and evidence hierarchies, how certainty/confidence in the evidence was determined and rated, strategies for dealing with insufficient evidence, EtD frameworks and processes, approaches to qualifying and rating recommendations, equity considerations (including epistemic equity) and consensus processes.

2.6. Data analysis

The analytical approach was similar to the original review⁶ and is detailed in the protocol.¹⁶ Briefly, thematic analysis began deductively using the themes from the original review, which were further interrogated and developed iteratively. Following basic data extraction, publications were purposively selected for in-depth analysis based on methodological rigor, relevance and novelty. In keeping with the critical interpretive approach, analysis was “dynamic, recursive and reflexive”.¹⁵ Strategies were compared for similarities (reciprocal translational analysis) and contradictions (refutational synthesis), with lines-of-argument (synthesizing arguments) used to construct overarching themes. Drawing on our experience with TCIM research, synthesis and EIDM, we developed a list of contextual factors to further sensitize and ground the analysis (Table 1). These considerations evolved in parallel with data extraction and supported ongoing critical and reflexive reflection on how the complexities of different TCIM systems, contexts and worldviews shape EIDM.

Table 1.

Contextual factors relevant to TCIM evidence-informed decision-making.

1.	TCIM healthcare cultures and communities value a broad range of non-biomedical evidence sources such as traditional knowledge (TK) – be it codified or non-codified – expert opinion of esteemed practitioners, healers and community leaders, and non-biomedical research methods (e.g., Indigenous research methods, classical Ayurvedic logic, Homeopathic proving). These sources of evidence are not typically considered or valued in the EIDM process, reflecting a broader issue of epistemic equity.
2.	TCIM systems often use different terminologies and clinical assessment methods for classifying, diagnosing, managing and assessing the effectiveness of interventions, which can be challenging to integrate into biomedically focused CPGs.
3.	Outcomes valued by TCIM patients, practitioners and communities may not align with standard clinical or policy questions and will, therefore, not be considered during the EIDM process. Examples include restoring internal homeostasis and enhancing salutogenesis in the absence of disease, and broader concerns such as equitable access to TCIM, equitable opportunities for TCIM practitioners, implications for intellectual property rights, sustainability of wild-sourced ingredients and other factors impacting community, environmental and planetary health.
4.	Evaluating complex, multicomponent interventions is methodologically challenging and/or resource intensive, especially when there are long-term effects, the therapy is individualized or difficult to standardize, or a single intervention has multi-target effects on multiple diseases or conditions.
5.	It is unrealistic to anticipate that every TCIM intervention already in use, must or will be evaluated by multiple, gold-standard interventional trials and synthesis methods. Reasons include resource limitations, especially in low- and middle-income countries where they are widely practiced, the longstanding prioritization of biomedical research and development, and ethical and practical constraints (e.g., asking communities to withhold TCIM treatment or use a placebo when an intervention is widely accepted, culturally embedded and safely used).
6.	Broad questions such as “is TCIM effective?” are important yet may be less relevant to TCIM practitioners and in countries where the therapy is already accepted and funded. Instead, more refined questions about the clinical application of TCIM, comparative effectiveness of different TCIM options, and its safe and effective integration with biomedicine need to be answered.

TCIM: Traditional, Complementary and Integrative Medicine.

3. Results

3.1. Search results & characteristics of included publications

A total of 167 publications were included (Fig. 1). There were 15 internationally or nationally endorsed consensus statements (Table 2),^8,19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35 representing 132 of the included publications (see Supplementary File 1). Nine statements were new additions to the list,^{8,22,23,25,26,28,29,31,33,34} including one specific to TCIM (Korean Medicine).⁸ The remaining 35 publications,^4,5,9,11, 12, 13, 14^,36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63 including 14 published in Chinese,^5,51, 52, 53, 54, 55, 56, 57, 58^,60,62 provided informal TCIM guidance and recommendations. Further details of the search results, including publications included and excluded at full-text screening, and the characteristics of the included publications, are reported in Supplementary File 1.

Fig. 1.

PRISMA flow diagram of literature search.

Table 2.

Consensus statements on guideline development and policy.

Title, Region	Focus: Description
GRADE Handbook19
International	Clinical, Public Health, Health Systems: Grading of Recommendations, Assessment, Development and Evaluation (GRADE) provides detailed guidance on rating certainty in the evidence for quantitative studies (e.g., randomised controlled trials (RCTs), non-randomized studies of interventions (NRSIs), meta-analyses, network meta-analyses, economic evaluations), Confidence in the Evidence from Reviews of Qualitative research (GRADE-CERQual)66 and going from evidence to recommendations. Associated resources include the GRADEpro Guideline Development Tool (GRADEpro GDT) for producing guidelines, including evidence profiles, Cochrane summary of findings tables and GRADE Evidence to Decision (GRADE EtD) framework, and interactive Summary of Findings (iSoQ) tool for CERQual. GRADE is endorsed by WHO, Cochrane Collaboration and many countries across the globe.
WHO Handbook for Guideline Development20
International	Clinical, Public Health, Health Systems: Guidance on all stages of guideline development including planning, formulating questions, selecting outcomes, finding, synthesising and assessing evidence, developing recommendations, and providing rapid advice guidelines for a public health emergency. Associated resources include Qualitative Evidence Synthesis.21 Guidance supersedes the 2003 guidelines. Refers to GRADE.
WHO-INTEGRATE Evidence-to-Decision Framework22
International	Public health, Health Systems, Health Policy: Standalone EtD framework to be used alongside other WHO guidance on decision-making a guideline development. Aligns with WHO norms and values, adopts a whole-of-society perspective, takes an interdisciplinary and intersectoral view, is designed for decision-making about complex interventions within complex health systems and when human rights, ethical or equity issues are important, or non-heath sectors are involved or affected.
Evidence, Policy, Impact: WHO Guide for Evidence-Informed Decision-Making23
International	Clinical, Public Health, Health Systems, Health Policy: Guide and repository of vetted methods (e.g., SUPPORT Tool, SURE Guides, GRADE, WHO Handbook, Evidence Informed Policy Network (EVIPNet) Europe EBP manual) and tools (https://evidence-impact.org/) for evidence-informed decision-making in the clinical, public health and health systems policy and practice. Presented in the context of the 'Evidence Ecosystem for Impact' framework.
SUPPORT Tool24
International	Health Policy: Guidance and tools for evidence-informed policymaking including strategies for supporting organizations, priority setting, clarifying problems, framing options, planning implementation, finding and assessing evidence, engaging stakeholders and going from evidence to decision.
SURE Guides for Preparing and Using Evidence-Based Policy Briefs25
International	Health Policy: Supporting the Use of Research Evidence (SURE) Guides are designed for preparing and using an evidence brief for policy with a focus on African contexts. Includes guidance on prioritizing topics, clarifying the problem, literature searching, selecting policy options, implementation barriers, ongoing monitoring and evaluation, deliberative process, and stakeholder engagement. Refers to the SUPPORT tool and GRADE.
Evidence-Informed Policy Network (EVIPNet) Europe Evidence Brief for Policy (EBP) manual26
Europe	Health Policy: Guide for preparing an EBP with a focus on European contexts. Includes guidance on EBP preparation, priority-setting and problem definition, evidence synthesis, formulating EBP, and post EBP uptake and evaluation. Refers to SURE Guides, GRADE, GRADE CERQual.
National Health & Medical Research Council (NHMRC) Guidelines for Guidelines Handbook27
Australia	Clinical, Public Health, Health Systems: Guidance for all stages of guideline development including planning, developing, reviewing, implementing, and updating. Guidance supersedes the 2009 handbook, closely follows the GRADE approach and terminology, and is continually updated.
Canadian Task Force on Preventive Health Care (CTFPHC): Methods Manual28,29
Canada	Clinical, Health Services, Public Health: Describes the processes and methods used in the development, publication, dissemination and evaluation of CTFPHC guidelines to support primary care practitioners in their delivery of preventive health care for Canadians, including preparation, selecting, appraising and synthesising evidence, and drafting recommendations. Refers to GRADE methods.
AWMF Guidance Manual and Rules for Guideline Development30
Germany	Clinical: The Association of the Scientific Medical Societies in Germany (AWMF; Arbeitsgemeinschaft der Wissenschaftlichen Medizinischen Fachgesellschaften) provides guidance on all stages of guideline development including: planning and organisation, SRs, selecting evidence, forming recommendations, structured consensus development, dissemination, implementation, and evaluation. Refers to GRADE and Cochrane methods.
Manual on Development & Implementation of Evidence-Based Clinical Practice Guidelines31
Malaysia	Clinical, Health Services, Public Health: Describes the processes and methods used in the development, publication, dissemination, monitoring and evaluation of CPGs developed by the Malaysian Health Technology Assessment Section, including: preparation, selecting / appraising / synthesising evidence, and forming recommendations. Refers to GRADE and USPSTF methods.
SIGN 50 A Guideline Developer’s Handbook32
Scottland, UK	Clinical, Health Services, Public Health: Scottish Intercollegiate Guidelines Network (SIGN) provides guidance on all stages of guideline development including guideline development group, selecting topics, conducting SRs, assessing quality of evidence, making recommendations, consultation and peer review, presentation and publication, updating guidelines, implementation, and involving patients and their representatives. Since 2009 adopted the GRADE approach.
Manual for Clinical Practice Guideline Development33
Philippines	Clinical, Health Services, Public Health: Describes the processes and methods used in the development, dissemination and evaluation of CPGs in the Philippines, including: preparation, selecting / appraising / synthesising / grading evidence, forming recommendations, and consensus development. Refers to GRADE methods.
Manual for Developing Evidence-Based Clinical Practice Guidelines of Korean Medicine8
Republic of Korea	(TCIM) Korean Medicine - Clinical: Guidance from the National Institute for Korean Medicine Development (NIKOM) on all stages of CPG development from establishing the CPG development committee, to choosing topics and diseases, deciding on the scope, planning processes, developing clinical questions, finding, evaluating, and synthesising the evidence, determining the level of evidence certainty and strength of the recommendation, consensus decision-making, external expert review, final approval, updating CPGs and strategies for disseminations. Guidance focuses on the use of Korean Medicine with or without biomedical therapies, for managing biomedically defined diseases and conditions. GRADE is used for rating evidence certainty, with one additional category when the evidence is Classical Text-Based. Categories for strength of recommendations draws on USPSTF and SIGN.
Saudi Arabian Handbook for Healthcare Guideline Development34
Saudi Arabia	Clinical, Health Services, Public Health: Describes the framework used to inform the development, publication, dissemination, implementation and evaluation of CPGs in the Kingdom of Saudi Arabia, including planning and organisation, selecting/grading evidence, forming recommendations, and consensus development. Refers to GRADE methods.
U.S. Preventative Services Task Force (USPSTF) Procedure Manual35
United States of America	Clinical, Public Health, Health Systems: Describes the methods and processes used by USPSTF for selecting topics, searching, reviewing and appraising evidence, engaging public input, and making a recommendation. Methods for systematic evidence reviews are according to the Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Centers (EPCs).

AWMF: Arbeitsgemeinschaft der Wissenschaftlichen Medizinischen Fachgesellschaften (Association of the Scientific Medical Societies in Germany) CPGs: Clinical Practice Guidelines; CTFPHC: Canadian Task Force on Preventive Health Care; EBP: Europe Evidence Brief for Policy; EtD: Evidence to Decision; EVIPNet: Evidence Informed Policy Network; GRADE: Grading of Recommendations, Assessment, Development and Evaluation; GRADE-CERQual: Confidence in the Evidence from Reviews of Qualitative research; GRADEpro GDT: GRADEpro Guideline Development Tool; NHMRC: National Health & Medical Research Council; NIKOM: National Institute for Korean Medicine Development; SIGN: Scottish Intercollegiate Guidelines Network; SURE Guides: Supporting the Use of Research Evidence Guides; TCIM: traditional, complementary and integrative medicine; U.S.: United States of America; USPSTF: U.S. Preventative Services Task Force; WHO: World Health Organization.

3.2. Thematic results

Building on the five themes identified in the original review, the updated thematic analysis revealed both continuity and advancements in EIDM and strategies for managing evidence uncertainty. Analysis reorganized and expanded upon the original themes. These are presented across four interrelated themes that affirm the importance of 1) transparent and inclusive decision-making, 2) ensuring the scope and framing of the problem are sensitive to TCIM contexts, 3) using diverse types of evidence in EtD frameworks and when there is evidence uncertainty and 4) centering equity, including epistemic equity, throughout the EIDM process.

Across all themes, there was broad alignment between internationally and nationally endorsed consensus statements, as well as informal TCIM-specific guidance and recommendations. Additionally, TCIM guideline developers often drew upon, extended and/or modified established biomedical frameworks for EIDM, to accommodate TCIM contexts and manage evidence uncertainty.

3.2.1. Facilitating transparent and inclusive decision-making

Transparent and inclusive decision-making processes are critical for credible and contextually relevant guidelines and policies. Decisions about committee composition and consensus methods shape whose knowledge counts, how evidence is interpreted, and how recommendations are ultimately formulated. In the context of TCIM, these processes help ensure that diverse epistemologies and stakeholders’ perspectives are fairly represented.

The consensus statements all detailed processes for establishing a steering group, decision-making committee, guideline development panel, and/or policy briefing team, which should comprise of a diverse range of stakeholders. To enhance credibility and methodological rigor, technical and methodological experts, healthcare professionals, and patient representatives were commonly listed. In the context of TCIM, the primary focus was ensuring there were members with relevant clinical and methodological expertise in TCIM. Some TCIM guideline developers recommended appointing two chairpersons - one with biomedical and the other with TCIM qualifications.¹⁴

Statements often recommended establishing a priori decision-making processes across the whole EIDM process.^20,23 However, there was little guidance on specific processes. This may in part reflect an assumption that by applying a structured framework for appraising evidence certainty and making recommendations such as GRADE, will naturally lead to consensus.³³ However, even GRADE notes that subjectivity is unavoidable.¹⁹ For reasons that are unclear WHO²⁰ and NICE²⁹ provided less explicit guidance than earlier versions.⁶

As noted in the previous review, the Association of the Scientific Medical Societies in Germany (AWMF) provided early detailed guidance on structured consensus processes.^6,30 Formal methods include nominal group processes, Delphi surveys, and structured consensus conferences. Notably, Delphi surveys were frequently recommended by TCIM guideline developers.^{13,40,46,48,49} Voting was another approach used to aid decision-making, especially when there were disagreements on important decisions.^14,29,34,35

TCIM guideline developers further emphasized the limitations of brainstorming by guideline panels when framing CPG research questions and recommended more rigorous methods such as scoping reviews and Delphi surveys to identify and prioritize research questions.^38,48,49 Criteria for reporting expert consensus used in TCIM CPGs were also proposed.⁵³

3.2.2. Framing the problem and expanding the scope for TCIM contexts

How a problem is framed fundamentally shapes the scope of a guideline or policy brief, including the questions asked, the interventions and solutions considered, the types of admissible evidence, and stakeholders’ perceptions of importance and relevance.^23,25 For TCIM guideline developers, framing the problem often involved expanding the scope to include traditional knowledge (TK) and practices, and ensuring problem statements and CPG reporting aligns with TCIM contexts.

The rationale for including TK was twofold. Firstly, TK derived from ancient literature and/or passed down orally through the generations are a cornerstone of many TCIM health care practices.^{40,46,49,59,61,63} Notably, the value of these evidence sources for guideline development were far more than the supportive role of tacit (or colloquial) knowledge referred to in WHO guidance.²³ Secondly, TCIM guideline developers anticipated gaps in empirical evidence, especially for clinically important questions about TCIM diagnosis and the comparative effectiveness of different TCIM interventions and their components, dose, timing, intensity, duration etc..^{8,40,46,49,53,59,61,63}

Ensuring the questions are important to patients and other relevant stakeholders was widely emphasized. Standard approaches to question formulation, such as the application of the PICO (Population, Intervention, Comparison, Outcome) framework, were commonly recommended ^8,13,19,20,22, 23, 24, 25, 26, 27^,29, 30, 31, 32, 33, 34, 35^,59 Adaptations of these frameworks for TCIM included describing populations using TCIM diagnoses,^{9,10,13,14,36,50} specifying interventions according to TCIM rationale, clinical indications and timing of use,^{9,14,36,40,59} evaluating TCIM-defined clinical outcomes,⁵⁹ and aligning TCIM terminology with WHO ICD-11 whenever possible.⁵⁹

Included publications also suggested the framing of questions should aim to delineate the individual and combined strengths of TCIM and biomedicine to identify potential benefits (or harms) unique to their integration.^14,40 Table 3 builds on the previous review⁶ and the earlier “Guidelines for WHO guidelines,⁶⁴ to outline other instances when TCIM interventions should be considered.

Table 3.

Instances when guidelines and policy briefs should consider TCIM.

1.	Empirical evidence is available about benefits and/or harms.
2.	Commonly used or recommended interventions.
3.	Strong preference or perceived importance amongst patients or practitioners.
4.	Potential to reduce a high burden of disease.
5.	Potential to simultaneously address multiple conditions or symptom clusters, or reduce polypharmacy.
6.	Potential to improve outcomes important to patients, including self-efficacy, well-being and optimizing health.
7.	Potential to restore homeostasis and build salutogenesis prior to disease development.
8.	Potential for beneficial or unwanted interactions when integrated with biomedical interventions.
9.	Safety concerns (e.g., drug interactions, toxicity, quality control).
10.	Offers an alternative to interventions already widely used or recommended that are ineffective, very costly, or have low patient acceptability.
11.	Potential for substantial economic, social or environmental benefits.
12.	Already available or feasible to implement in the health care setting.
13.	Is one of only a few options that are available, affordable, or potentially effective.

TCIM: traditional, complementary and integrative medicine.

For more complex questions, consensus statements recommended logic models such as an ‘analytic framework for assembling indirect evidence,³⁵ or a ‘problem tree’ to clarify and frame the problem.²⁶ WHO,^20,23 SUPPORT,²⁴ SURE,²⁵ and EVIPNet²⁶ statements highlighted the importance of considering modifying factors and contextual information from the outset (e.g., prognosis, costs, values, preferences, equity, local contexts, and stakeholders’ perceptions). Despite the relevance to TCIM, which is typically characterized by high intervention complexity, these internationally endorsed strategies were yet to be recommended by the TCIM guideline developers included in this review.

3.2.3. The role of diverse types of evidence

3.2.3.1. Incorporating broader evidence for modifying factors

There has been greater recognition over the past decade that evidence about health benefits and harms is only one, albeit an important factor, when moving from evidence to decisions.^19,20,22 EtD frameworks consider a wide range of modifying factors that may warrant making stronger or weaker recommendations despite uncertainty about benefits or harms, and vice versa. Except for the USPSTF, that does not usually consider modifying factors,³⁵ consensus statements generally aligned with, or adopted the approaches used in the GRADE¹⁹ or WHO^20,22 EtD frameworks summarized in Table 4.

Table 4.

Examples of Evidence to Decision (EtD) Frameworks.

Benefits & Harms	Modifying factors*
GRADE Handbook19
•Evidence certainty (effect estimate) •Balance net benefits / harms (desirable / undesirable effects)	•The problem (importance, burden disease, baseline risk) •Values/preferences (outcome importance) •Resource implications (costs, cost-effectiveness) •Equity •Acceptability •Feasibility
WHO Handbook for Guideline Development20
•Quality/certainty in the evidence of effect •Balance of benefits and harms	•Priority of the problem •Values and preferences about the health outcome •Resource implications •Equity and human rights •Acceptability •Feasibility
WHO-INTEGRATE22
•A meta-criterion ‘quality of the evidence’ applies to each of the criteria
•Balance of health benefits and harms	•Human rights and sociocultural acceptability •Health equity, equality and non-discrimination •Societal implications •Financial and economic considerations •Feasibility and health system considerations

EtD: Evidence to Decision; GRADE: Grading of Recommendations, Assessment, Development and Evaluation; WHO: World Health Organization.

^⁎Modifying factors are those that may warrant making a weak recommendation despite high certainty evidence about the balance between health benefits and harms, and vice versa.

Although the USPSTF does not explicitly consider modifying factors,³⁵ when a recommendation is required despite insufficient evidence, decisions may be structured around four domains: burden of suffering, potential harm, costs, and current practice.⁶⁵ This approach was adapted for TCIM to also include potential harms of doing nothing, opportunity costs from using a less effective intervention, feasibility, and implications of introducing or ceasing an intervention.³⁸

TCIM guideline developers also noted that even when clinical evidence is limited, there may be robust evidence about modifying factors (e.g., quantitative and/or qualitative studies on TCIM use, treatment preferences, reasons for use, and important outcomes),³⁸ hence, the importance of systematically identifying this evidence to inform decisions. Indeed, while systematic reviews (SRs) remained the gold standard for evidence synthesis,^{22,23,25,26,66} this was not limited to SRs of interventions. For example, GRADE-CERQual⁶⁶ assesses SRs of qualitative research (e.g., reporting stakeholders’ preferences and values), and the ‘WHO Guide for EIDM’²³ outlined a range of primary and secondary study designs for answering different research questions.

3.2.3.2. Reframing evidence hierarchies through a whole-of-evidence approach

Although greater attention has been given to evidence sources beyond randomized controlled trials (RCTs), across the board, RCTs remained the preferred study design for assessing the health effects of interventions.^{8,9,11,13,14,19,20,}22, 23, 24, 25, 26, 27^,29, 30, 31, 32, 33, 34, 35, 36, 37, 38^,40,46, 47, 48, 49, 50^,59,61,63 However, non-RCT evidence (e.g., interrupted time series analyses, controlled before-after studies, cohort studies, case-control studies, qualitative research, real-world data, modelling studies, preclinical research) may be required when interventional trials or randomization are unethical or unacceptable, or to answer questions about rare adverse events, toxicity, drug-interactions, long-term outcomes, generalizability to other populations, diagnosis and screening. Due to the typical narrow inclusion criteria of RCTs, non-randomised studies of interventions (NRSIs) may also be required to evaluate preventive interventions, especially for under-represented subgroups and when there are equity considerations.³⁵

The GRADE approach was widely endorsed for rating evidence certainty.^{11,14,19,20,23,}25, 26, 27^,29,31, 32, 33, 34, 35^{,37,40,46,50,63} However, some limitations were noted, including GRADE’s inability to discriminate evidence derived from study designs when everything is rated ‘very low certainty evidence’; here it was argued by some TCIM guideline developers that evidence from RCTs should still be considered more trustworthy than other study designs.^37,47

Rather than bundling together all uncertain evidence about the benefits and harms of an intervention, and then relying on modifying factors to inform recommendations, TCIM guideline developers proposed modified GRADE approaches and alternative strategies for including multisource evidence.^{8,13,37,40,46,47,49,61} They emphasized that the classic evidence pyramid is just the ‘tip of the evidence iceberg’ (Fig. 2).¹⁴ For instance, whilst the ‘Hierarchies of Evidence Applied to Lifestyle Medicine’ preferred RCTs, it also considers the quality and quantity (plurality) of other evidence sources, including preclinical studies, RCTs and NRSIs assessing interim/surrogate outcomes, observational studies supporting generalizability, and observational studies about long-term effects.³⁷

Fig. 2.

Whole of evidence approach: the classical evidence pyramid is just the tip of the evidence iceberg.

Numerous strategies were also recommended for rigorously incorporating TK. These included triangulating evidence from literature reviews, text-mining ancient literature, analyzing clinical records (real-world evidence), and eliciting expert consensus via interviews and Delphi surveys ^{13,40,46,49,61,63}.

3.2.3.3. Other strategies for dealing with evidence uncertainty

GRADE, WHO and USPSTF explored various strategies for dealing with evidence uncertainty as “decision makers do not have the luxury of waiting for certain evidence”.⁶⁵

An often-overlooked strategy was the GRADE discordant recommendations (i.e., paradigmatic situations for making strong recommendations despite low quality evidence) ¹⁹. Other examples of its application are given in the WHO Handbook⁶⁷ and the previous review.⁶

The USPSTF developed the ‘analytic framework’ to assemble and link indirect evidence along the causal pathway.³⁵ Other approaches presented in the “GRADE notes for scenarios of ‘no’ evidence”, included surveying clinical experts to collate data from their unpublished observations and case series,⁶⁸ and using disease registry data.⁶⁹

3.2.4. Centering equity across the decision-making process

Equity was consistently emphasized across the decision-making process. In theory, equity could be an outcome in-and-of-itself, however, statements generally focused on assessing inequalities in health outcomes and access, and underrepresented populations in research that limit the generalizability of the available evidence.^19,22,23 As highlighted in Table 4, the GRADE EtD framework only referred to ‘equity’,¹⁹ while the WHO Handbook also considered human rights,²⁰ which was further elaborated and expanded upon in WHO-INTEGRATE to support broader health and policy decisions.²²

Although not explicitly discussed in EtD frameworks, nor by TCIM guideline developers, many of the TCIM-specific recommendations addressed another equity issue – epistemic equity – that involves the fair recognition and inclusion of TK as a legitimate evidence source. Strategies aimed at promoting epistemic equity included ensuring the guideline committee chair has relevant TCIM experience, using systematic methods to incorporate and appraise TK, and challenging the dominance of biomedical evidence hierarchies.

For example, guideline developers of Traditional Chinese Medicine (TCM) guidelines for common pediatric diseases, systematically incorporated both ancient texts and modern TCM expert opinions alongside clinical trial data. In some cases, due to the central importance of TK, these sources could be rated higher than the lowest level of ‘expert opinion’ in standard evidence hierarchies.⁴⁹ Other guideline developers created and applied a “Stratified Evidence Scoring Method of Acupuncture Clinical Practice Guidelines”^61,62 that weighted the quality of evidence from ancient literature, medical experience of modern famous experts, as well as modern empirical evidence based on a combination of the GRADE evidence certainty assessment and a STRICTA reporting score (i.e., Revised STandards for Reporting Interventions in Clinical Trials of Acupuncture⁷⁰ CONSORT extension).

4. Discussion

4.1. Summary of the findings

This updated review highlights methodological advancements in EIDM and guideline development, its relevance to TCIM, and strategies for dealing with uncertain or insufficient evidence. The findings affirm the importance of transparent and inclusive decision-making, ensuring the scope and framing of the problem are sensitive to TCIM contexts, applying a whole of evidence approach, and centering equity, including epistemic equity, throughout the process. The past decade has also provided more guidance for structured, transparent EIDM methods (e.g. EtD frameworks, incorporating evidence for modifying factors, consensus methods). In addition, TCIM guideline developers have proposed strategies for systematically incorporating multiple evidence sources to inform clinical decisions, including TK that challenge evidence norms. Importantly, these developments were not separate from biomedical EIDM frameworks and instead built upon and adapted them to systematically incorporate traditional knowledge, multisource evidence, and other context-specific considerations.

4.2. Gaps and implications for research

Despite these advancements notable gaps were identified. Although there is an increasing emphasis on health equity, the explicit recognition of epistemic equity in EtD frameworks remains overlooked. This has implications not only for TCIM and Indigenous communities who have different ways of knowing, being and doing, but also nursing and other allied health professions who are disadvantaged by persisting medical dominance.⁷¹ Addressing institutional epistemic injustice within EDIM could lead to more inclusive and effective healthcare policies and interdisciplinary healthcare that benefits communities, patients and practitioners alike.

TCIM guideline developers and RIGHT extensions for TCIM^9,36 supported epistemic equity by offering strategies for including TK (e.g., theory, diagnosis, indications and outcomes). Formal consensus methods involving TCIM experts and literature syntheses of ancient texts were also proposed. Missing however, were strategies for incorporating uncodified TK, protecting intellectual property, and considering community and environmental benefits and harms.

Also not well explored was the suitability of existing EtD frameworks for making recommendations about TCIM interventions that are multi-component and/or individualized, and approaches for assimilating the net benefits and harms of TCIM interventions that have multi-target effects. This is in contrast to existing CPGs and policy questions that typically focus on assessing the use of single interventions for single conditions.³⁸ Further research is indicated to explore the application of logic models (e.g., analytic framework³⁵ and ‘problem tree’²⁶) and identify other possible approaches.

The need for TCIM-specific recommendations for guideline development was affirmed through the RIGHT extensions for acupuncture ⁹ and traditional Chinese medicine (TCM) ³⁶, CPG reporting criteria for TCM expert consensus ⁵³ and tailored criteria for evaluating the quality, interpretation and applicability of TCM CPGs.⁵ Coupled with work already underway by TCIM guideline developers, the findings suggest that TCIM extensions to existing GRADE and WHO consensus statements on guideline development and policy briefs might be helpful.

4.3. Limitations of the review

Limitations of this review reflect the flexibility and subjectivity imbedded in CIS methods that are often criticized.¹⁸ To enhance transparency and systematicity we published a review protocol,¹⁶ declared our perspectives and potential biases (Table 1), and used standard SR methods when applicable. Other limitations include the use of RR methods, including a single-stage sampling process, and no quality appraisal. The latter was partially offset by restricting the inclusion criteria to only international or nationally endorsed consensus statements and TCIM recommendations published in peer-reviewed journals.

Whilst the review did not aim to be exhaustive nor develop a theoretical framework, it is likely that other relevant TCIM strategies and recommendations were missed, especially gray literature and those published in languages other than English. Given the richness of the data already identified, a more comprehensive review of the TCIM literature is warranted. Notwithstanding, the findings form a solid basis for improving the applicability of current consensus statements for TCIM contexts.

4.5. Conclusion

Many TCIM interventions lack high-certainty empirical evidence about their health benefits and harms, yet decisions must still be made. The judicious use of EtD frameworks, alongside other strategies that systematically incorporate multiple evidence sources, offer a pathway for navigating TCIM evidence uncertainties. Ensuring these methodologies respect and preserve the integrity of TCIM knowledge systems is crucial to advancing epistemic equity and preventing the marginalization of valuable insights from these rich knowledge traditions. Advancing EIDM methodologies specifically for TCIM guideline development and policy will not only enhance the credibility, uptake and impact of TCIM recommendations, but also ensure they are inclusive and culturally relevant, and support the appropriate integration of TCIM into global health systems.

Funding

None.

Ethics statement

Not applicable.

Data availability

The data that support the findings of this study are presented in the supplementary files. Additional data may be made available upon request.

CRediT authorship contribution statement

Jennifer Hunter: Conceptualization, Methodology, Data curation, Validation, Formal analysis, Writing – original draft, Writing – review & editing, Visualization. Matthew Leach: Conceptualization, Data curation, Validation, Formal analysis, Writing – original draft, Writing – review & editing. Dennis Chang: Validation, Writing – review & editing. Myeong Soo Lee: Validation, Writing – review & editing. Yutong Fei: Writing – review & editing. Jianping Liu: Validation, Writing – review & editing.

Declaration of competing interest

JH, DC, and JL are editorial board members and MSL is an editor of this journal, but these roles had no influence on the editorial process or decision. The authors declare that they have no other conflicts of interest.

Appendix. Supplementary materials

Supplement 1. Literature Search Strategy

Supplement 2. Publications Included at Full Text Screening Stage

Supplement 3. Publications Excluded at Full Text Screening Stage

Supplement 4. Characteristics of included publications