Development and Preliminary Validation of the Chinese Medicine Interventional Clinical Trials Research Question Importance Tool (CMICT‐RQIT)

ABSTRACT

Objective

Evidence‐based medicine emphasizes clinical research driven by important questions, yet Chinese medicine lacks practical quantitative tools to identify such questions. We developed the Chinese Medicine Interventional Clinical Trials Research Question Importance Tool (CMICT‐RQIT) to support topic selection and provide transparent criteria for proposal review, promoting high‐quality clinical research in Chinese medicine.

Methods

This study followed internationally accepted procedures of conceptualization and operationalization. Using a mixed‐methods approach—including a literature review, qualitative interviews, Delphi surveys, expert consensus meetings, and the analytic hierarchy process—we developed the CMICT‐RQIT. CMICT‐RQIT was prespecified and interpreted as a formative/composite multicriteria decision‐support tool.

Results

Following a standardized development process comprising three stages—framework construction, tool development, and tool evaluation—the CMICT‐RQIT V1.0 was established. It consists of four domains, 10 facets, and 24 items, each with corresponding composite weights, item explanations, scoring criteria, and an operations manual. Exploratory traditional psychometric analyses showed limited internal consistency and poor structural fit, consistent with the formative/composite nature of the tool and should not be interpreted as evidence that all items measure a single latent trait.

Conclusions

CMICT‐RQIT V1.0 can assist researchers in selecting research topics, with its greatest value lying in providing a concise summary of the importance of research question along with a comprehensive 24‐item checklist for clinical investigators. At the same time, it offers reviewers transparent criteria for assessing the importance of research questions, thereby promoting the objectivity and fairness of evaluations. CMICT‐RQIT V1.0 should be used primarily as a structured checklist and decision‐support index.

1. Introduction

Interventional clinical research holds an important position within the evidence system that supports clinical decision‐making [1]. The research questions it addresses directly influence the development of clinical practice guidelines and health policy decisions, playing a critical role in promoting the healthy and continuous advancement of the evidence system [2, 3]. Over the past decade, the number of registered and published clinical studies in Chinese medicine (CM) has escalated in the wake of strong national policy support. As of July 2023, more than 4500 CM clinical trials had been registered on the Chinese Clinical Trial Registry Platform. However, the large volume of studies has also revealed numerous issues—such as unclear research questions, inappropriate methodologies, nonstandardized implementation, and poor reporting quality—which have hindered the translation of research findings [4, 5, 6]. In particular, there remains a shortage of high‐quality research outputs with substantial international influence [7, 8, 9, 10, 11]. As noted by some researchers, as of 2024, only 30 randomized controlled trials (RCTs) related to CM had been published in four top authoritative journals—The New England Journal of Medicine (NEJM), The Lancet, The Journal of the American Medical Association (JAMA), and The British Medical Journal (BMJ)—and only seven of these studies were conducted in China [12]. This situation may be associated with factors such as suboptimal evidence quality, cultural differences, and the unique characteristics of the CM theoretical system, which collectively hinder international understanding and acceptance [9, 12]. More importantly, the research questions addressed in CM clinical studies generally lack sufficient importance, making it difficult for their findings to generate high impact S1.

Evidence‐based medicine has consistently advocated for conducting clinical research driven by important clinical questions [13]. The starting point of scientific research is the formulation of a research question of significant clinical relevance. When a research question lacks sufficient clinical importance or specificity, it often results in vague research objectives, weakening the value of the evidence and potentially wasting time and resources [9]. Although Chinese scholars have proposed strategies such as principles for topic selection, sources of research questions, and the population, intervention, comparison, and outcome (PICO) framework, a unified consensus has yet to be established [14, 15]. Since the late 1990s, researchers have gradually explored methods for establishing priority research areas and applied these approaches in health policy‐making, guideline development, and priority setting in CM clinical research [16, 17, 18]. Researchers have proposed steps for selecting priority areas in CM and have developed assessment tools for prioritizing clinical questions in guideline development [9, 19]. Other scholars have suggested optimizing international methodologies to better align with China's context [20]. However, constructing a scientifically sound and rational approach for setting priorities in clinical research still requires further exploration and practice. Existing evaluation tools for CM clinical research primarily focus on research design, implementation processes, or efficacy assessment, while often neglecting the critical step of starting from important clinical issues to formulate scientific research questions and define research objectives. Yet identifying important questions and applying standardized methodological approaches is the essential first step in conducting high‐level clinical research in CM [21]. Only through this process can high‐quality and translational research evidence be generated.

In May 2023, the Australia and New Zealand Musculoskeletal Clinical Trials Network developed an assessment tool for quantifying the importance of research questions [22]. We attempted to apply this tool to clinical studies of musculoskeletal diseases in CM, but found its applicability limited [23]. A possible reason is that the principles and methodology underlying the tool do not fully align with the characteristics of CM or integrative Chinese–Western medicine research. CM clinical research often involves syndrome differentiation, individualized interventions, integration of CM theory with biomedical concepts, and complex intervention strategies, which may not be adequately captured by existing biomedical‐oriented prioritization frameworks. Therefore, an assessment tool is needed to evaluate the importance of research questions in the CM field. Importantly, the Chinese Medicine Interventional Clinical Trials Research Question Importance Tool (CMICT‐RQIT) is a decision‐support tool, not a reflective psychometric scale, designed to help researchers and reviewers systematically assess all relevant aspects of research question importance, rather than relying solely on a unidimensional psychometric total score, because the importance of CM clinical research questions is determined by multiple heterogeneous and complementary considerations that are not expected to function as indicators of a single latent construct. Accordingly, we employed a mixed‐methods approach—drawing on conceptualization and operationalization frameworks—to develop the CMICT‐RQIT. This tool is designed to assist clinical researchers in making structured decisions about topic selection and to provide project reviewers with a practical and transparent basis for assessing research question importance. In doing so, it aims to guide CM clinical research toward addressing important clinical issues and enhancing the clinical value and translational potential of research outputs.

2. Methods

2.1. Study Design

Based on the complex and abstract concept of research question importance, we followed the processes of conceptualization and operationalization [24, 25]. Through mixed‐methods research (MMR)—including literature review, qualitative interviews, Delphi surveys, expert consensus meetings, and the analytic hierarchy process (AHP)—we developed the CMICT‐RQIT and conducted a preliminary evaluation using selected exploratory psychometric and reproducibility assessments grounded in classical test theory principles [26, 27]. CMICT‐RQIT is a decision‐support tool designed to guide researchers and reviewers in systematically assessing the importance of research questions, rather than functioning as a reflective psychometric scale that produces an inferential total score. Because the importance of CM clinical research questions is conceptualized as a multidimensional formative/composite construct rather than a single latent variable, the purpose of these analyses was exploratory and pragmatic, focusing primarily on scoring reproducibility, structural coherence, and practical usability during preliminary tool development. The detailed process is shown in Figure 1.

FIGURE 1.

Development flowchart. AHP, analytic hierarchy process; CMICT‐RQIT, Chinese Medicine Interventional Clinical Trials Research Question Importance Tool; ICC, intraclass correlation coefficient.

2.2. Construction of the Tool Framework

2.2.1. Literature Review

The literature review aimed to establish the framework for the tool. Considering representativeness, workload, and standardization and completeness of the report, we retrieved RCTs of CM from China National Knowledge Infrastructure (CNKI) and PubMed over 5 years (from January 1, 2019 to December 31, 2023) to represent the current status of interventional clinical research.

The inclusion criteria for the literature were: (1) Articles published in high‐impact journals. Chinese articles were required to be published in journals included in the 2023 Chinese Science Citation Database (CSCD) core collection, and English articles in journals ranked in Quartile 2 or above by the Chinese Academy of Sciences in 2023 with an impact factor ≥ 10; (2) The study design was an RCT; (3) The intervention was CM, for example, Chinese herbal medicine, acupuncture, tuina (massage), traditional exercises; (4) No restrictions were placed on study population, control interventions, or outcome measures. This sampling strategy was intentionally designed as a form of purposive sampling to identify information‐rich cases. High‐impact journals were used as a pragmatic proxy to capture well‐developed, explicitly justified, and methodologically transparent research questions, thereby facilitating the extraction of conceptually clear and diverse candidate items during the framework development stage. In research priority setting and structured decision tool development, purposive sampling targeting high‐quality and well‐reported publications is a recognized strategy to extract rich and theoretically grounded candidate items. Any of the following conditions will be excluded: Duplicate publications; secondary analyses of RCTs; abstracts or short reports; animal experiments; and articles for which the full text could not be obtained.

We first screened the articles by reading titles and abstracts, and then reviewed the full texts to further determine eligibility. The first author independently performed literature screening and data extraction, and a second team member conducted random checks. In case of disagreement, a third party was consulted to resolve the issue, thereby ensuring the reliability and accuracy of data extraction. We used a predesigned Excel spreadsheet to extract basic information from the included studies, such as title, publication date, journal source, authors, affiliations, regional distribution, and funding support. We used a descriptive method to analyze the characteristics of scientific questions in the included CM interventional clinical studies.

We extracted PICO information related to the research questions. The diseases investigated in the studies were classified according to the 11th Revision of the International Classification of Diseases (ICD‐11). Two researchers collected and carefully read the introduction/background sections of all included articles. Using grounded theory methodology, coding was conducted in NVivo 20.0 across three sequential stages: open coding, axial coding, and selective coding. First, during open coding, two researchers independently reviewed the introduction/background sections line by line and extracted concepts related to clinical need, evidence gaps, innovation, CM characteristics, feasibility, and anticipated impact. Second, during axial coding, semantically similar codes were compared, merged, and organized into preliminary categories and subcategories according to their conceptual relationships. Third, during selective coding, higher order categories were further integrated around the core concept of “importance of research questions in CM interventional clinical studies,” thereby generating preliminary candidate items, dimensions, and domains for the CMICT‐RQIT framework [28, 29]. Disagreements were resolved through discussion or consultation with a third researcher. These categories were not assumed to reflect manifestations of a single latent construct; rather, they were conceptualized as complementary dimensions jointly contributing to the overall assessment of research question importance. SPSS 18.0 software was used to perform descriptive statistics of the basic characteristics of the literature and PICO features, which were presented as frequencies and percentages.

2.2.2. Qualitative Interview With Expert Panel

To mitigate potential selection bias introduced by this strategy, the literature‐derived items were subsequently triangulated and expanded through qualitative expert interviews, ensuring that the framework was not solely driven by high‐impact publication characteristics. After reviewing the relevant literature and considering the research team's suggestions, we developed a preliminary interview outline. We then conducted a preinterview with two experts and used their feedback to revise the outline, ensuring its completeness and applicability. The interview invitation letter was subsequently finalized. Convenience sampling and purposive sampling were adopted to select interviewees. Factors such as age, gender, years of work experience, and interviewees’ professional titles were comprehensively considered to ensure the sample's representativeness and balance. Drawing on experience from similar qualitative studies, we estimated the expected sample size at 10–15 participants. The final number was determined when information saturation occurred—meaning recruitment ceased when no new information was emerging from additional interviews. During experts’ discussions, we focused the core content of the interviews on the following questions: “What do you think are the similarities and differences between Chinese and Western interventional clinical research?” and “When assessing the importance of research questions in CM interventional clinical studies, what do you usually consider?” Although specific questions in each interview could be adjusted as needed, the core content always remained unchanged.

2.2.3. Drafting the Tool Framework

After the interviews were completed, the coding procedures and NVivo 20.0 operations followed the same steps as those used in the literature research portion. Specifically, we used NVivo 20.0 to extract the frequencies of the coding terms. Building on grounded theory, we then employed content analysis, thematic analysis, and logical analysis to develop the tool framework of CMICT‐RQIT. The qualitative interview data were subsequently coded and integrated with findings from the literature analysis. Finally, we developed a framework (V0) for the CMICT‐RQIT, providing the theoretical foundation for the subsequent stages of tool development.

2.3. Tool Development

2.3.1. Conducting the Delphi Survey

The tool framework generated during the qualitative exploration stage was organized and converted into a Delphi survey questionnaire. Following the principles of expertise and representativeness, we planned for experts meeting the following criteria for inclusion: (1) Individuals engaged in long‐term clinical research or guideline development in either CM or integrative Chinese–Western medicine; (2) Holding a professional title of intermediate level or above; (3) Having a minimum of 5 years of work experience for those with a doctoral degree, ≥ 10 years for those with a master's degree, and ≥ 30 years for those with a bachelor's degree; (4) Willing to participate and having sufficient time and energy to complete the expert consultation questionnaire. These criteria were designed to ensure that the participating experts possess sufficient clinical research experience to have a profound understanding of the importance of the research questions. Using convenience and purposive sampling and group discussion, we prepared an initial list of 50–60 experts. Experts rated the importance and operability of each item on a five‐point Likert scale. We emphasized that the Delphi process was conducted independently of the research team's preferences to maintain objectivity, and items were further discussed during expert meetings.

2.3.2. Data Analysis

After collecting the questionnaire in the Delphi survey during the tool development stage, we conducted statistical analyses in five key areas: (1) experts’ characteristics were summarized using frequencies and percentages, covering age, gender, professional title, academic degree, specialty, years of experience, and workplace to assess disciplinary representativeness and authority; (2) expert enthusiasm (positivity coefficient) was evaluated through response and opinion submission rates, defined respectively as (number of returned questionnaires)/(number distributed) and (number of questionnaires with comments)/(number returned), to assess participation; (3) expert authority coefficient was calculated as the mean of experts’ familiarity with the content (Cs) and judgments (Ca), with Cr = (Ca + Cs)/2, and Cr > 0.70 indicating high authority; (4) degree of consensus (concentration) among experts was assessed using each item's mean importance score and proportion of ratings ≥ 4, where higher values indicated stronger agreement; (5) degree of coordination among experts relied on the coefficient of variation (CV) and Kendall's coefficient of concordance (Kendall's W), with CV ≤ 0.25 indicating consistency, and a Kendall's W with p < 0.05 indicating statistically significance. Item retention required satisfaction of both of the following: (1) mean importance score ≥ 4.0, proportion of ratings ≥ 4 ≥ 50%, and CV ≤ 0.25; (2) mean operability score ≥ 3.5, proportion of ratings ≥ 4 ≥ 50%, and CV ≤ 0.25. The tool versions achieved after the two rounds of Delphi consultation were named V0.1 and V0.2, respectively.

It should be noted that item retention decisions were not based solely on quantitative thresholds. In line with recommendations for complex construct development and formative/composite tools, theoretical relevance, domain representativeness, and preservation of CM‐specific characteristics were also considered during item selection. Items not meeting Delphi statistical criteria were provisionally flagged as candidate items for structured re‐evaluation during the subsequent expert consensus meeting, where final decisions were made through predefined consensus procedures and group voting.

2.3.3. Conducting a Consensus Meeting

We set expert inclusion criteria based on authority, representativeness, and familiarity: (1) participation in both previous Delphi surveys rounds; (2) associate senior professional title or higher; (3) master's degree or higher; (4) willingness and availability to attend. The meeting was in a hybrid online–offline format, and the main procedures included voting on items retained or excluded during the Delphi phase, proposing optimizations and revisions, and providing suggestions for subsequent evaluation.

The standards for reaching consensus were: (1) For items retained during the Delphi survey stage: if ≥ 70% of experts agreed, the item was included; if ≥ 70% disagreed, the item was excluded; items not meeting either threshold required further discussion followed by revoting. (2) For items excluded during the Delphi survey stage: if ≥ 70% of experts agreed with the exclusion, the item was removed; if ≥ 70% disagreed, the item was retained; all other cases similarly required repeated voting after discussion. We designated the version of the tool revised through the consensus meeting as V0.3.

2.3.4. Preparing the Test Version

Throughout the Delphi surveys and the expert consensus meeting, revisions were made to items at all levels, as well as to the scoring criteria, until the expert opinions converged. Ultimately, we finalized the tool's dimensions, domains, items, their explanations, and scoring criteria. Then, following the principles of authority, representativeness, and familiarity, experts weighted the consultation survey. The inclusion criteria for experts were: (1) having participated in the consensus meeting; (2) holding an associate senior professional title or above; (3) possessing a master's degree or higher; (4) being willing and sufficiently available to complete the questionnaire. A total of 17 experts who had participated in the previous consensus meeting participated in the weighting consultation. Based on the experts’ input, we first established a hierarchical structural model comprising the goal, criterion, and scheme layers in a top‐down manner. Then, using the Saaty 1–9 scale, we constructed judgment matrices for factors within the same level and performed pairwise comparisons. For example, when constructing the judgment matrix for the four domains A, B, C, and D, if A21 = 3, then 1/A21 = 1/3, and the remaining matrices follow the same logic.

The original consultation data collected from experts were entered into a predesigned Excel spreadsheet and verified by another member of the research team. Then we used R software to conduct an AHP analysis, assigning weights to the tool's dimensions, domains, and items in a stepwise manner. The composite weight of an item represents its contribution to the overall goal, calculated using the multiplicative method. The composite weight of each item equals: (domain weight × facet weight × item weight), representing the item's contribution to the goal layer. This process resulted in the weighted test version of the tool (V1.0).

2.4. Tool Evaluation

2.4.1. Testing Sample

The evaluation of the test version of the tool (V1.0) faced several challenges, including the complexity of evaluation targets, the lack of prospective application cases, and the absence of an external gold standard. To address these limitations, we turned to clinical trial study protocols, which outline a trial's plan and comprehensively reflect the background, objectives, design, and implementation. Since a study protocol precedes the publication of the final report, it serves as a feasible source of research questions for this evaluation. With this approach in mind, and considering both sample representativeness and sample size requirements, we retrieved RCTs on CM included in PubMed between January 1, 2019 and September 8, 2024.

The search terms and search strategy were consistent with those used in the literature review. Subsequently, we obtained the study protocols corresponding to the identified RCTs using three approaches: (1) Checking whether the RCT article directly cited its study protocol; if so, the protocol was obtained based on the citation. If not, the keywords related to the title combined with “protocol” were used to retrieve the study protocol published online; (2) Checking the Supporting Information of the RCT article; (3) Contacting the corresponding author or first author of the RCT article via email to request the study protocol. If none of these three methods yielded the protocol, the RCT was excluded. From the perspective of project reviewers, and based on the information reported in the study protocols, we used the test version of the tool (V1.0) to evaluate the importance of research questions. We conducted a descriptive analysis to depict the basic information on the research questions and the item scores.

2.4.2. Assessing Reliability and Validity

We applied classical test theory to evaluate the reliability, validity, discriminative capacity, and inter‐rater consistency of the tool using both pre‐ and postweighting evaluation data. Although CMICT‐RQIT was conceptualized as a formative/composite decision‐support tool rather than a reflective latent‐variable scale, selected psychometric analyses were conducted as exploratory assessments of scoring reproducibility, structural coherence, and practical usability during preliminary tool development. For exploratory external validity assessment, 20% of the study protocols were randomly selected for testing, stratified by high‐ and low‐impact groups. All other evaluation analyses used the full set of included study protocols. Test–retest reliability was assessed by having the same evaluator reassess the study protocols after a 1‐month interval. During the second‐round assessment, the evaluator was blinded to all previous scoring results, and the first‐round evaluation data were concealed until completion of all reassessments and statistical analyses. The analyses included:

(1) Descriptive Analysis: Frequencies and percentages were used to describe the protocol's basic characteristics and the scoring distribution of each item.

(2) Reliability: Internal consistency was assessed using Cronbach's α. Test–retest reliability was examined by having the same evaluator reassess the protocol 1 month later. Depending on data distribution normality, either Pearson's correlation coefficient (r) or Spearman's coefficient (r _s) was used. Values of r or r _s > 0.70 indicated good reliability.

(3) Validity: Content validity refers to the extent to which the items accurately reflect the intended objectives and purposes of the tool. It mainly evaluates the scientific rigor and comprehensibility of the tool's content, typically based on expert judgment. For an exploratory external validity assessment, the ANZMUSC‐RQIT served as a pragmatic reference tool rather than a definitive gold standard. Although originally developed for research prioritization in the musculoskeletal field, several of its evaluation dimensions—such as clinical relevance, evidence gap, innovation, feasibility, and potential impact—show partial conceptual overlap and general applicability to the assessment of clinical research questions. At the time of this study, no validated criterion tool specifically designed for evaluating the importance of CM interventional clinical research questions was available. Therefore, ANZMUSC‐RQIT was used as an external comparative framework for preliminary validation purposes only. Thus, we evaluated the randomly sampled study protocols using both the CMICT‐RQIT V1.0 and the ANZMUSC‐RQIT. Depending on data normality, we used either Pearson (r) or Spearman (r _s) correlation coefficients, with r or r _s > 0.70 to explore the degree of conceptual convergence between the two tools. As an exploratory structural assessment, we first conducted exploratory factor analysis (EFA) using principal component analysis (PCA) and varimax rotation for factor extraction. Then, we performed confirmatory factor analysis (CFA) and assessed model fit (CFI, TLI > 0.90; RMSEA, SRMR < 0.08, indicating good model fit). These analyses were conducted to explore potential structural coherence and evaluator scoring patterns rather than to confirm a strict reflective latent‐variable structure.

(4) Discriminative Capacity: Based on the reasonable hypothesis that “research questions of greater importance are more likely to be published in higher impact journals and thus generate greater real‐world influence,” data verified by two evaluators were used. Depending on data normality and homogeneity of variance, we employed either the independent‐samples t‐test or the Wilcoxon rank‐sum test to examine differences in total scores and dimension scores of the CMICT‐RQIT V 1.0 between the high‐ and low‐impact groups. We also used logistic regression to examine the relationship among total scores, dimension scores, and journal impact grouping. A two‐tailed p < 0.05 indicated statistical significance.

(5) Inter‐Rater Consistency: We used the intraclass correlation coefficient (ICC) to evaluate the consistency between two evaluators, with ICC > 0.70 indicating good agreement.

2.4.3. Preparing the Final Version

Based on the results and feedback from using the test version, we drafted a revised version. After a final round of discussion from the research team, the final version of CMICT‐RQIT (V 1.0) was established.

3. Results

3.1. Development of the Tool Framework

A total of 39,805 original articles were retrieved, and after screening based on the inclusion and exclusion criteria, 539 studies were included—488 in Chinese and 51 in English. Using grounded theory, we systematically summarized the original coding terms extracted from the introductions/backgrounds of the 539 studies. To clarify the process, we followed the transformation logic: “original text in the introduction/background → coding vocabulary → researcher notation → items.” These coding terms were further synthesized to produce 22 initial items for the preliminary framework. Next, the 22 items obtained through open coding were repeatedly examined, with a focus on evaluating the importance of research questions in CM interventional clinical research. Through continuous conceptual refinement, seven facets were derived: A‐1 Attention, A‐2 Disease (syndrome) burden, A‐3 Research gap, B‐2 Approach, B‐3 Technology/methods, C‐1 Advantage positioning, and C‐2 Research foundation. Further analysis and organization of these seven facets led to their aggregation into three domains: A Demand, B Innovation, and C Relevance.

We conducted one‐on‐one, semistructured interviews with 13 CM interventional research experts. Following grounded theory, we coded responses and summarized the experts’ insights and evaluation suggestions regarding importance of research question. Besides the items from the literature review, experts added 10 new items: B‐1.1 CM theory, B‐1.2 Modern medical theory, C‐3.1 Syndrome standardization, C‐3.2 Intervention measure standardization, D‐1.1 Medical decision‐making, D‐1.2 Patients, D‐1.3 Social resources, D‐1.4 Modernization of CM, D‐2.1 Medical products/technology, and D‐2.2 Academic results. Notably, several items—particularly those related to CM theoretical foundations, standardization, and broader societal value—were primarily contributed by expert interviews rather than high‐impact literature, indicating that the final framework was not exclusively shaped by the literature sampling strategy.

Integrating the 32 items derived from both the literature review and the qualitative interviews, and maintaining a consistent focus on evaluating the importance of research questions in CM interventional clinical research, these items were further synthesized into 11 facets: A‐1 Attention, A‐2 Disease (Syndrome) burden, A‐3 Research gap, B‐1 Theory, B‐2 Approach, B‐3 Technology/methods, C‐1 Advantage positioning, C‐2 Research basis, C‐3 Standardization degree, D‐1 Research value, and D‐2 Anticipated results. We then refined and aggregated these 11 facets into four domains: A Demand, B Innovation, C Relevance, and D Benefits, ultimately formed the CMICT‐RQIT framework (V0).

3.2. Delphi Survey

The two rounds of Delphi surveys yielded 50 and 44 valid questionnaires, respectively. The proportions of experts with authority coefficients (Cr) ≥ 0.7 were 98% and 100%, respectively. The Kendall's W values for importance and operability ratings in the second round were 0.22 and 0.30, respectively, showing improvement compared with the first round (0.16 and 0.18).

After the first round of the Delphi survey, 26 of the 32 items met the inclusion criteria, while six items—B‐3.1 New technology, B‐3.2 New method, C‐2.1 Theoretical basis of CM, C‐2.2 In vitro/animal experiments, D‐1.3 Social resources, and D‐1.4 Modernization of CM—were removed for not meeting the criteria. The item “Patients” was removed after discussion due to overlap with other items. One new item (D‐1.2 Research methodology guidance) was added, one domain and three facets were modified, and other items and their explanations were revised in accordance with expert suggestions. This resulted in CMICT‐RQIT V0.1, comprising four domains, 10 facets, and 25 items with accompanying explanations.

Although the items B‐1.1 CM theory, B‐1.2 Modern medical theory, and D‐1.2 Research methodology guidance did not meet the inclusion criteria after the second round of Delphi surveys, these items were not immediately discarded. Instead, based on expert comments emphasizing their potential theoretical importance, contribution to domain completeness, and relevance to CM‐specific research characteristics, these items were provisionally retained as candidate items for structured re‐evaluation during the subsequent expert consensus meeting, rather than being finalized by the research team alone. Based on expert recommendations, items A‐3.1 to A‐3.3 were merged into a single item (A‐3.1: Lack of Evidence). Additionally, one new item (C‐3.1 Disease diagnostic standardization) was added, four facets were modified, and slight revisions were made to item explanations along with supplemental scoring criteria.

3.3. Expert Consensus Meeting

A total of 17 experts from 10 different provinces in China participated in the consensus meeting. All decisions regarding item retention or exclusion, including those items that did not meet Delphi criteria, were finalized through predefined consensus rules and group voting, ensuring that no single investigator or research team preference determined the final structure. All 24 items retained during the Delphi phase reached consensus for inclusion. Five items excluded during the Delphi phase—New technology, New method, In vitro/animal experiments, Social resources, and Modernization of CM—were unanimously confirmed for exclusion in a single round of voting. The item “Theoretical basis of CM” reached exclusion consensus during the second round of voting. However, consensus could not be reached on the item “Patients,” even after discussion and three rounds of voting. Ultimately, based on the voting results (52.94% of participants supported exclusion) and expert opinions (e.g., the item overlapped with several other items), the research team decided to remove it.

Following these voting outcomes, experts participating in the consensus meeting provided several suggestions for modifications to the CMICT‐RQIT, leading to the development of CMICT‐RQIT V0.3 and its accompanying operation manual (Supporting Information).

3.4. Weighting by AHP Method

A total of 17 experts from 10 regions in China completed the AHP‐based weighting consultation survey. Based on analysis, the composite weights of the 24 items in the tool were determined, resulting in the test version with weighting (Table 1). The weighting structure showed that the Demand and Advantages domains received relatively greater priority within the overall framework. When using this version to evaluate the importance of research questions in CM interventional clinical studies, if item D‐1.1 “Health Policy Decision‐making,” received a score of 1, its weighted score was calculated as 1 × 0.083 = 0.083. Weighted scores for other items were calculated in the same way. The score for each domain was obtained by summing the weighted scores of all items under that domain. For example, the domain score for D Benefits is the sum of the weighted scores of items D‐1.1, D‐1.2, D‐2.1, and D‐2.2. The calculations for other domains follow the same logic.

TABLE 1.

CMICT‐RQIT V1.0.

Domain	Facet	Item (composite weight)	Item description	Scoring criteria
A Demand	A‐1 Stakeholders	A‐1.1 Patient‐doctor needs (0.112)	Based on the current situation, do the proposed interventions (plans) address the unmet needs between healthcare providers and patients?	1 Both doctor and patient have needs
				0.5 Either doctor or patient has needs
				0 Neither has needs
		A‐1.2 Public health decision‐maker needs (0.084)	Are there any national, provincial, or local government health policy documents, meeting minutes, health department planning documents, public health reports, or project application guidelines that address this research issue?	1 National level
				0.5 Other level
				0 None
	A‐2 Disease burden	A‐2.1 Epidemiological characteristics (0.060)	What are the epidemiological characteristics of the disease to be studied?	1 Severe
				0.5 Typical
				0 Mild
		A‐2.2 Financial burden (0.055)	What is the financial burden of the disease to be studied?	1 Severe
				0.5 Typical
				0 Mild
		A‐2.3 Patients’ physical and mental burden (0.060)	What physical and mental burden does the disease impose on patients?	1 Severe
				0.5 Typical
				0 Mild
	A‐3 Research gap	A‐3.1 Lack of evidence (0.118)	Systematically review domestic and international clinical practice guidelines, expert consensuses, systematic reviews, meta‐analyses, and original research related to the proposed research question. Is there a lack of strong evidence for the proposed research question?	1 Yes
				0 No
B Innovation	B‐1 Theoretical innovation	B‐1.1 Chinese medicine theory (0.048)	Does the research question have the potential to expand or establish new Chinese medicine theories? Does the research question have the potential to expand or establish new Chinese medicine theories?	1 Yes 0.5 Maybe
				0 No
				1 Yes 0.5 Maybe 0 No
		B‐1.2 Modern medical theory (0.037)	Does the research question have the potential to expand or establish new modern medical theories?	1 Yes
				0.5 Maybe
				0 No
	B‐2 Design innovation	B‐2.1 Targeting new indications and/or expanding treatment populations (0.011)	Compared to existing related clinical research, does the proposed intervention (plan) target new indications and/or expand the treatment population?	1 Yes
				0 No
		B‐2.2 New intervention measures (plans) (0.009)	Compared to existing related clinical research, is the proposed intervention (plan) a new approach for the disease under study?	1 Yes
				0 No
		B‐2.3 Clinically important outcomes (0.015)	Compared to existing related clinical research, does the proposed research question attempt to explore other important outcomes of the disease?	1 Yes
				0 No
C Advantages	C‐1 Advantage positioning	C‐1.1 Efficacy (0.047)	Compared to existing interventions (plans), does the proposed intervention (plan) have any advantage in terms of effectiveness?	1 Yes
				0 No
		C‐1.2 Safety (0.059)	Compared to existing interventions (plans), does the proposed intervention (plan) have any advantage in terms of safety?	1 Yes
				0 No
		C‐1.3 Accessibility (0.011)	Compared to existing interventions (plans), does the proposed intervention (plan) have any advantage in terms of ease of access for patients and/or ease of application for healthcare providers?	1 Both doctor and patient have advantages
				0.5 Either doctor or patient has an advantage
				0 Neither has an advantage
		C‐1.4 Affordability (0.011)	Compared to existing interventions (plans), does the proposed intervention (plan) have any advantage in terms of cost‐effectiveness?	1 Yes
				0 No
		C‐1.5 Patient preferences (0.014)	Compared to existing interventions (plans), does the proposed intervention (plan) have any advantage in terms of patient preferences and values?	1 Yes
				0 No
	C‐2 Research foundation	C‐2.1 Relevant clinical research foundation (0.048)	Is there any direct or indirect clinical research evidence supporting the formulation of the research question?	1 Direct evidence
				0.5 Indirect evidence
				0 None
	C‐3 Standardization degree	C‐3.1 Disease diagnostic criteria (0.013)	Does the disease to be studied have any internationally/nationally/industrially/organizationally recognized diagnostic standards?	1 International/national level
				0.5 Other levels
				1 Direct evidence
		C‐3.1a Syndrome diagnostic criteria (if applicable) (0.008)	Does the disease type to be studied have internationally/nationally/industrially/organizationally recognized diagnostic standards?	1 International/national level
				0.5 Other levels
				0 None
		C‐3.2 Intervention measure standardization (0.008)	What is the level of difficulty in standardizing the proposed intervention (plan)?	1 Easy
				0.5 Typical
				0 Difficult
D Benefits	D‐1 Research impact	D‐1.1 Medical decision‐making (0.083)	What is the likelihood that this research will supplement, update, or change existing healthcare decisions?	1 Very likely
				0.5 Somewhat likely
				0 Not very likely
		D‐1.2 Research methodology guidance (0.031)	Does this research have the potential to provide methodological guidance for future traditional Chinese medicine and integrative medicine clinical research?	1 Yes
				0.5 Maybe
				0 No
	D‐2 Anticipated results	D‐2.1 Medical products/technology (0.039)	How likely are the expected outcomes to have the potential to develop new medical products/technology?	1 Very likely
				0.5 Somewhat likely
				0 Not very likely
		D‐2.2 Academic results (0.017)	What is the likelihood that the expected results will produce high‐impact academic outcomes?	1 Very likely
				0.5 Somewhat likely
				0 Not very likely

3.5. Reliability and Validity of CMICT‐RQIT

According to the inclusion criteria, a total of 256 RCT study protocols (research questions) were included after screening. Among them, 201 (78.52%) were obtained from online publications, 49 (19.14%) from Supporting Information, and six (2.34%) through email contact with the authors. A total of 112 and 144 study protocols were published in journals ranked in Q1–Q2 and Q3–Q4, respectively.

The reliability assessment showed the overall Cronbach's α of CMICT‐RQIT V1.0 was 0.59 before weighting and 0.52 after weighting. The Cronbach's α of all four domains was low (≤ 0.5). These findings suggest limited internal consistency under the assumptions of traditional reflective psychometric models. The test–retest reliability for the total score and the A Demand domain was high (rs > 0.75), moderate for the C Advantages, and low for the B Innovation and D Benefits domains. The relatively lower stability observed in the B Innovation and D Benefits domains suggests that evaluators may have had greater variability in interpreting or applying these scoring criteria. Before and after weighting, the correlation coefficients (r) between the total scores of CMICT‐RQIT V1.0 and ANZMUSC‐RQIT were 0.37 and 0.38, respectively. The weighted data showed poor CFA model fit, whereas the unweighted data also demonstrated poor CFA fit indices. These results indicate that the underlying structure of CMICT‐RQIT may not conform to a conventional unidimensional reflective latent‐variable model and may instead reflect a multidimensional formative/composite framework.

Before and after weighting, statistically significant differences were observed between the high‐impact and low‐impact groups in all four domain scores and the total score (p < 0.01). Furthermore, logistic regression analysis showed that the four domain scores and the total score were positively associated with the impact level of the journals in which the study results were published (p < 0.01). Finally, regarding inter‐rater consistency, before and after weighting, consistency for both the total score and the A Demand domain was high (ICC > 0.70); it was moderate for the C Advantages, and low for the B Innovation and D Benefits domains.

Scope and Purpose of Tool Use: This tool is designed to assess the importance of scientific questions in clinical studies involving CM interventions, supporting researchers in making topic selection decisions. The primary significance of this tool is not to provide researchers with an overall score for their research question, but to help them thoroughly examine whether all items have been considered. It also provides project reviewers with a transparent evaluation tool, thereby improving the objectivity and fairness of the review process. Conducting research to address important scientific questions can enhance the value of studies and promote the high‐quality development of clinical research in CM. Intended users of the tool: This tool is suitable for researchers in CM interventional clinical studies, including clinicians, nurses, and other related practitioners. If the tool is used by project reviewers, researchers will need to provide the necessary relevant information in their project proposals.

C‐3.1a is a conditional item applicable only to studies involving CM syndrome differentiation. For studies not involving syndrome differentiation, this item is automatically assigned 1 point to ensure score comparability across research questions. Therefore, the total number of items in the CMICT‐RQIT V1.0 remains 24.

4. Discussion

The importance of a research question is the key starting point for determining whether a clinical study is worth initiating. In this study, we developed CMICT‐RQIT V1.0 through conceptualization, operationalization, literature review, qualitative interviews, two rounds of Delphi surveys, a structured expert consensus meeting, and AHP‐based weighting. The final tool consists of four domains, 10 facets, and 24 items with corresponding weights, scoring criteria, item explanations, and an operation manual. Unlike traditional reflective psychometric scales, CMICT‐RQIT is intended as a formative/composite multicriteria decision‐support tool. The importance of a CM clinical research question is determined by multiple complementary considerations, including stakeholder demand, disease burden, evidence gaps, innovation, comparative advantages, standardization, and anticipated benefits. Therefore, its primary value lies in structured judgment, item‐level checking, and support for reviewer deliberation, rather than producing a stand‐alone psychometric score.

The use of high‐impact journals during the literature review phase should be interpreted cautiously. This strategy was not intended to equate research question importance with publication impact, but rather to identify clearly articulated and well‐justified research questions for initial item generation. We acknowledge that this approach may underrepresent locally relevant, practice‐oriented, or emerging CM research topics. However, qualitative interviews and expert consultations supplemented CM‐specific theoretical, clinical, and societal perspectives that may not have been captured in the initial literature sample. Furthermore, journal impact was used only as a pragmatic external grouping variable during validation rather than as a gold standard for research question importance.

The retention of several items that did not fully meet Delphi statistical criteria may also raise concerns regarding subjectivity. However, these items were not retained solely by the research team; instead, they were provisionally carried forward based on expert qualitative comments and theoretical considerations and subsequently evaluated through predefined consensus procedures and structured group voting during the expert consensus meeting. For formative/composite tools, strict reliance on statistical thresholds may remove conceptually important but heterogeneous components. Therefore, item selection in this study integrated quantitative evidence with structured expert consensus to preserve clinically and theoretically meaningful dimensions.

The psychometric findings should also be interpreted within the conceptual framework of formative/composite measurement. Low Cronbach's α values and suboptimal CFA fit would generally indicate poor measurement performance in reflective latent‐variable scales, where items are expected to be highly correlated and represent a single underlying construct. In contrast, high interitem correlation is not theoretically required for CMICT‐RQIT because important CM clinical research questions may derive their significance through different pathways. For example, a research question may be highly important because it addresses a major evidence gap and disease burden despite limited theoretical innovation, whereas another may be important because of methodological or theoretical innovation despite modest immediate policy impact. Such multidimensional and heterogeneous characteristics are common in CM clinical research, which often integrates theoretical rationale, clinical applicability, feasibility, comparative advantages, and policy relevance.

Accordingly, traditional reflective‐scale indices such as Cronbach's α, EFA, and CFA have limited interpretability in this context. These analyses were conducted primarily as exploratory structural assessments and indicators of scoring reproducibility rather than as definitive tests of latent construct validity. Nevertheless, the relatively low test–retest and inter‐rater consistency observed in the Innovation and Benefits domains suggest practical challenges in scoring reproducibility. To address this issue, we provided the operation manual, strengthened evaluator training and calibration procedures, and incorporated clearer scoring guidance. Future studies should prospectively evaluate the tool across diverse CM research settings and further optimize evaluator guidance, scoring calibration, and external validation strategies.

The criterion‐related validity findings also require cautious interpretation. Because no universally accepted gold‐standard instrument exists for evaluating the importance of CM interventional clinical research questions, the ANZMUSC‐RQIT was used only as a pragmatic external comparator. Although some dimensions of ANZMUSC‐RQIT have cross‐disciplinary applicability, important differences remain between musculoskeletal research and CM clinical research in terms of theoretical foundations, intervention characteristics, and evaluation priorities. Therefore, the observed correlations should be considered preliminary and exploratory rather than confirmatory evidence of criterion validity. Future studies should establish external reference standards through structured consensus ratings from independent experts experienced in CM clinical research prioritization.

The weighting results (Demand > Advantages > Benefits > Innovation) reflect a “demand‐oriented, strengths‐focused, balanced‐benefits, and steady‐innovation” philosophy in CM clinical research. Demand was considered the most important dimension because clinically relevant research should address disease burden, patient needs, and evidence gaps identified through systematic reviews, guidelines, and stakeholder involvement. This emphasis aligns with the increasing recognition of patient‐centered and evidence‐informed research prioritization. Innovation highlights the integration of CM theory with modern biomedical concepts and encourages the development of new indications and optimized intervention strategies while maintaining clinically meaningful outcomes as the core objective. Within the Advantages dimension, safety received the highest weight, followed by efficacy and patient preferences, indicating that CM intervention evaluation should prioritize safety while increasingly incorporating patient values and participation throughout the research process. Standardization was also emphasized because it directly affects reproducibility, methodological rigor, and international recognition of CM research. The Benefits dimension reflects the broader impact of research on health policy, clinical practice, methodological development, and translational applications such as medical products and technologies.

Compared with existing research‐prioritization frameworks developed primarily for biomedical or musculoskeletal research, CMICT‐RQIT attempts to incorporate characteristics specific to CM clinical research, including the integration of CM theory and modern medicine, individualized interventions, syndrome differentiation, and the evaluation of complex interventions. Existing frameworks may insufficiently address these CM‐specific dimensions, particularly the balance between theoretical inheritance, clinical practicality, standardization, and translational value. Therefore, CMICT‐RQIT may provide a more contextually appropriate framework for prioritizing CM interventional clinical research questions.

This study has several limitations. First, the initial literature sample was restricted to CM RCTs published in selected high‐impact journals, which may have introduced spectrum bias and may not fully represent routine CM interventional research or locally influential research with strong CM characteristics. Second, the validation sample was limited to accessible protocols corresponding to PubMed‐indexed CM RCTs, and the low protocol retrieval rate may have introduced selection bias. Therefore, the results should not be directly extrapolated to all TCM interventional studies, particularly complex interventions, real‐world studies, and nonrandomized trials. Third, the journal impact grouping is an imperfect proxy for the importance of research questions and may introduce circularity in external hypothesis testing. Fourth, although recoded protocols were used during retesting, residual memory effects cannot be entirely excluded. Fifth, the ANZMUSC‐RQIT was only an external comparator rather than a gold standard, and the modest correlation should not be interpreted as indicating satisfactory criterion validity. Finally, because CMICT‐RQIT is formative/composite in nature, traditional reflective psychometric indices have limited interpretability as indicators of latent construct validity, although they may still provide preliminary information regarding scoring reproducibility and structural coherence during early‐stage tool development. Future studies should prospectively evaluate CMICT‐RQIT across diverse CM interventional study designs, compare tool scores with independent expert‐panel importance ratings, assess weighting sensitivity and item contribution, and further refine scoring guidance for domains with lower inter‐rater agreement. Alternative approaches may also be explored to better capture the complex characteristics of CM clinical research question importance.

CMICT‐RQIT V1.0 is intended as a structured checklist and decision‐support tool for both researchers and reviewers. Researchers may use the tool to assess the importance of proposed research questions systematically and provide evidence‐based justification during protocol development, while reviewers may use it to support transparent, objective, and consistent proposal evaluation. Overall, CMICT‐RQIT V1.0 provides an initial methodological framework for systematically assessing the importance of CM clinical research questions. Its formative/composite design accommodates the multidimensional and heterogeneous characteristics of CM clinical research and may contribute to improving the relevance, quality, and translational value of future CM clinical studies.

In conclusion, CMICT‐RQIT V1.0 is a formative/composite multicriteria decision‐support tool for evaluating the importance of research questions in CM interventional clinical studies. It includes four domains—Demand, Innovation, Advantages, and Benefits—and 24 items with composite weights. In the absence of a universally accepted gold standard, the current evaluation provides preliminary evidence regarding content coverage, expert consensus, total‐score reproducibility, external discriminative capacity, and inter‐rater agreement. However, low internal consistency and poor CFA fit indicate that CMICT‐RQIT should not be interpreted as a unidimensional reflective psychometric scale. For researchers, the tool's greatest value is to provide a structured checklist and transparent summary of research question importance. For reviewers, it offers a standardized decision‐support framework to improve transparency and fairness. Further prospective validation and calibration are needed before broad implementation across all CM interventional research settings.

Funding

This research was supported by the Science and Technology Co‐construction Project of the National Demonstration Zone for Comprehensive Reform of Traditional Chinese Medicine (GZY‐KJS‐GD‐2025‐007) and the State Key Laboratory of Traditional Chinese Medicine Syndrome (SKLKY2026C0007, QZ2025ZZ43).

Ethics Statement

This study has been reviewed and approved by the Ethics Committee at Guangdong Provincial Hospital of Chinese Medicine (AF/02‐07.0/12.0). All consulting experts agreed to publish the views expressed in the manuscript.

Conflicts of Interest

The authors declare no conflicts of interest.

Supporting information

Supporting Information: jebm70156‐sup‐0001‐SuppMat.docx