An Indicator System for Evaluating Sci-Tech Innovation-Driven Improvement of China’s Disease Diagnosis and Treatment Capacity

Abstract

Purpose

Medical scientific and technological (S&T) innovation is the core driver of disease diagnosis and treatment capacity improvement, yet standardized multi-dimensional evaluation frameworks linking research output to clinical outcomes remain lacking in China, hindering targeted health policy formulation. The core objective of this study is to establish a “scientific and technological innovation–clinical diagnosis and treatment capacity” transformation efficiency evaluation system adapted to China’s national context, through a comprehensive multi-dimensional and multi-method research approach.

Patients and Methods

This study integrated systematic literature review, semi-structured expert interviews, and two rounds of Delphi consultation with 40 senior experts to construct a three-tier evaluation indicator system covering prevention, diagnosis, treatment, and rehabilitation. The Analytic Hierarchy Process and proportional distribution methods were adopted to assign weights for primary, secondary, and tertiary indicators, with the coefficient of variation and Kendall’s W used to test expert consensus.

Results

The finalized framework contained 4 primary, 14 secondary, and 142 tertiary indicators, including 42 indicators reflecting clinical diagnosis-treatment outcomes and 100 indicators measuring full-chain S&T innovation outputs. Expert authority coefficients reached 0.90 in both rounds, and all indicators met pre-set consensus thresholds after two consultations. The system built a complete causal logic chain from basic research achievements to clinical efficacy improvement.

Conclusion

This localized, whole-cycle indicator system systematically quantifies the driving effect of medical S&T innovation on clinical capacity. It provides a standardized assessment tool for health policymakers, research institutes, and medical facilities to track translational benefits of technological breakthroughs and implement innovation-oriented Healthy China strategies.

Introduction

With the acceleration of globalization, intensification of population aging, and deepening of the impact of global climate change, humanity faces increasingly complex and severe health challenges.^1,2 Advances in science and technology such as precision medicine and artificial intelligence are profoundly reshaping the health management paradigm.^3,4 Globally, technological innovation has become a central driving force for enhancing disease diagnosis and treatment capabilities. In this study, disease diagnosis and treatment capacity are defined as the comprehensive clinical capability covering the full cycle of prevention, diagnosis, treatment, and rehabilitation, which is specifically manifested in diagnostic accuracy, therapeutic effectiveness, rehabilitation efficiency, patient safety, and health outcomes. Without continuous technological innovations, improvements in diagnostic and therapeutic capabilities would be halted, making qualitative leaps difficult. The integration of technology and healthcare represents an organic unity of “means” and “ends.” The translation of technological innovation into enhanced diagnosis and treatment capacity operates through multiple interconnected mechanisms. First, diagnostic innovation enhances capacity by improving the precision, speed, and accessibility of disease identification—for example, AI-assisted imaging interpretation can increase diagnostic accuracy while reducing time to diagnosis. Second, therapeutic innovation expands treatment options and improves efficacy—for example, targeted therapies and immunotherapies have revolutionized outcomes for previously incurable conditions. Third, enabling technologies (eg, electronic health records, telemedicine platforms) improve system-level efficiency and coordination, enabling more timely and integrated care delivery. Fourth, knowledge innovation (eg, clinical trial results, evidence-based guideline updates) disseminates best practices that raise standards of care across institutions. Many bottlenecks in disease diagnosis and treatment are, technological: For example, without microscopes, pathogens would not have been discovered; without X-rays, noninvasive imaging diagnosis would be impossible; and without DNA sequencing technology, precision medicine would not have been realized.⁵ Technological innovation directly addresses these bottlenecks by providing new tools, methods, drugs, and solutions, thereby enhancing the diagnostic and therapeutic capabilities of current healthcare systems. The evolution from “empirical medicine” to “evidence-based medicine” and then to “precision medicine” was dependent on developments in biotechnology, information technology, and data science.⁶ The shift from “population-based treatment” to “personalized treatment” was made possible by technologies such as gene sequencing and big data analysis. Similarly, the transition from “reactive treatment” to “proactive prevention” was facilitated by wearable devices and early screening technologies, enabling a forward shift in health management, significantly enhancing early prediction of and intervention against diseases.

Medical science and technological innovations are essential components for achieving technological self-reliance and building an innovative nation and have strategic significance: health is a fundamental right of citizens and a frontier for technological innovation.⁷ The core value of medical innovation is patient health. As emphasized by Weinstein and Williams, the primary focus of medical innovation is to benefit patients directly, with their health gains—often measured by indicators such as quality-adjusted life years and patient-reported outcome measures—serving as key outcomes.^8,9 Faulkner noted that medical innovations should aim to improve patient health and alleviate suffering.¹⁰ Lansisalmi defines medical innovation as new healthcare services, working methods, and/or technologies;¹¹ Omachonu, from a practical and goal-oriented perspective, regards it as the introduction of new concepts, ideas, services, processes, or products to improve treatment, diagnosis, education, and prevention, ultimately enhancing healthcare quality, safety, efficacy, and efficiency while reducing costs.¹² Authoritative organizations such as the Canadian Advisory Panel on Healthcare Innovation define it as “activities that create value in terms of healthcare quality and safety, management efficiency, patient experience, and health benefits.”¹³ The US 21st Century Cures Act (2016) further elaborates on medical innovation from a practical standpoint, outlining 10 core areas, including patient-centered drug development, promotion of new therapies, and modern clinical trial design.

There is an academic consensus that medical innovation should originate from clinical needs and ultimately serve clinical practice. Its primary goal is to bring tangible benefits to patients, clinicians, researchers, and other stakeholders through transformations in diagnosis, treatment, prevention, or screening.^14–16 Specifically, medical innovation includes product (new drugs, devices, and technologies), knowledge, and service innovation;^14,17–20 it encompasses not only the development of healthcare innovation but also the organizational and institutional arrangements that support it.²⁰ In terms of the value framework, patient health benefits (eg, efficacy, addressing unmet needs, safety, and accessibility),¹⁹ economic value (cost and commercial output),²¹ and other social benefits (eg, user practicality,¹³ knowledge creation, and spillover effects)²² are crucial. The attributes of medical innovation include novelty (eg, first clinical application),^14,15 evidence-based practice,^19,23 and systemic complexity.²⁴

Currently, China invests significantly in interdisciplinary medicine and engineering collaborations and the transformation of scientific and technological (S&T) achievements. Although the importance and connotations of medical innovation have been widely discussed, several questions regarding the scientific assessment of the actual impact of these investments using theoretical research and evaluation methods remain. Some of these include “How do we avoid ‘innovation for innovation’s sake’ and ensure that technological innovation truly serves clinical needs?” and “How can we scientifically evaluate the contribution of technological innovation (particularly in specific disease areas) to improving diagnostic and therapeutic capabilities?” Therefore, developing a scientific evaluation indicator system is crucial. While a range of prevailing international frameworks have been developed for healthcare innovation assessment, their scope and generalizability remain constrained. The Donabedian structure-process-outcome model²⁵ — widely acknowledged as the cornerstone of healthcare quality evaluation — is largely confined to internal dimensions of care delivery and excludes S&T innovation as an exogenous driving variable from its full analytical framework. The WHO Health System Performance Framework²⁶ prioritizes accessibility, quality, and efficiency as core evaluation dimensions, yet overlooks the pivotal driving effect of technological innovation on health system performance. The Organisation for Economic Co-operation and Development (OECD) Health Care Quality Framework²⁷ organizes quality indicators across effectiveness, safety, responsiveness, and accessibility. While these frameworks provide valuable conceptual foundations, they were not specifically designed to link technological innovation inputs with clinical outcomes, nor were they adapted to the hierarchical structure of China’s healthcare system.

Furthermore, most of the existing assessment tools in China mainly focus on research outcomes (such as the number of published papers and patents) or clinical results (such as mortality and morbidity) but fail to systematically link the two. Most related studies are limited to single diseases/individual institutions, lacking a comprehensive system at the national level that covers the entire cycle of prevention, diagnosis, treatment, and rehabilitation. This fragmented approach hinders the assessment of the complete “transformation value chain” from laboratory discoveries to clinical applications. Currently, there is an urgent need to establish a comprehensive and empirically validated indicator system to quantify the contribution of technological innovation to the improvement of clinical capabilities, especially in countries where there are significant regional differences in medical resources and rapid development.

The “Innovation-Driven Development Strategy” and the “Healthy China 2030” are top-level national designs in China. We aimed to construct a scientific evaluation indicator system to quantitatively assess the contribution of technological innovations to the improvement of disease diagnosis and treatment capabilities in China. We sought to scientifically measure the effects and pathways through which technological innovation enhances diagnostic and therapeutic capabilities, providing evidence-based support and theoretical tools for relevant decision-making. This research bridges these two national strategies, exploring how to better leverage technological innovation as the “primary driving force” to enhance health, highlighting its contemporary relevance and policy value.

Materials and Methods

Overall Research Design

This study employed a sequential mixed-methods design consisting of four phases. Phase 1 (Literature Review): systematic search of Chinese and international databases to identify existing evaluation indicators. Phase 2 (Qualitative Interviews): semi-structured interviews with 20 key informants to refine the indicator pool based on practical perspectives. Phase 3 (Delphi Consultation): two-round Delphi survey with 40 experts to establish consensus on indicator selection and importance ratings. Phase 4 (Weight Calculation): Analytic Hierarchy Process (AHP) for primary and secondary indicators and proportional distribution method for tertiary indicators based on the final Delphi results.

Literature Review and Construction of the Initial Indicator Pool

We conducted a comprehensive literature search of Chinese and international databases. The Chinese databases included the China National Knowledge Infrastructure (focusing on domestic journal articles, dissertations, and conference proceedings), the Wanfang Data Knowledge Service Platform (emphasizing policy documents, industry reports, and patent literature), and the Chinese Scientific Journal Database (VIP, supplementing core journal literature). International databases included Web of Science (covering science citation index/ social sciences citation index journals globally in the fields of medicine and management, with a focus on cutting-edge international research) and PubMed (specializing in medical literature, highlighting research integrating clinical practice and technological innovation). Additionally, platforms, such as Baidu Scholar, were used for supplementary searches to ensure that no key studies were missed.

Studies published since 2010 (ie, within the past 15 years) addressing topics such as “healthcare,” “evaluation of technological innovation,” “indicator systems for evaluation,” and “assessment of diagnostic and treatment efficiency” were included. Studies with weak relevance to the research topic, unscientific methodologies, or low data reliability were excluded. Ultimately, over 300 relevant articles were selected. Two researchers independently conducted in-depth reading and index extraction of the included literature. They recorded the names, context, and definitions of the indicators. We included the following indicators: (1) those directly related to the connection between S&T innovation and diagnosis and treatment capabilities (such as “diagnostic technique accuracy” or “the number of approved innovative drugs”); (2) indicators that had been clearly used and defined in at least two or more independent literature pieces. (3) The data of the indicators should be available/measurable at the practical level, avoiding vague descriptions such as “improving clinical outcomes.” We excluded the following indicators: those with weak relevance to the research topic, poor operability (such as “subjective clinical satisfaction without a standardized scale”), and repetitive connotations (such as “treatment success rate” and “cure rate” being combined into one indicator after confirmation by experts). The frequency of occurrence of indicators is used as a reference basis rather than the sole criterion to ensure coverage while avoiding popularity bias. After the two researchers completed the independent extraction, we calculated the Cohen Kappa coefficient to evaluate the inter-coder reliability. Two researchers independently extracted indicators from the literature using a coding framework. The differences were resolved through a joint review with a third senior researcher. The Kappa statistics were used to verify and evaluate the reliability. The final Kappa value was 0.82 (p<0.001), indicating good consistency.

Semi-Structured Expert Interviews and Indicator Optimization

Based on the literature-based preliminary indicator pool, semi-structured interviews with multidisciplinary experts were conducted to refine the healthcare technological innovation evaluation framework in line with real clinical needs. The tailored interview outline covered core topics including limitations of current evaluation systems, core attributes of healthcare innovation, disease-specific indicator differences, and evaluation priorities across the full prevention-diagnosis-treatment-rehabilitation workflow, with open-ended questions supplemented for comprehensive feedback. A total of 20 representative interviewees were recruited, comprising 8 research experts (selected from those engaged in S&T policy research, innovation evaluation, or translational medicine), 10 tertiary hospital clinicians (selected from those with clinical trial experience, guideline committee membership, or technology adoption research), and 2 industry administrators (selected from officials involved in health technology assessment, innovation policy, or research funding allocation). Each online and offline interview lasted 60–90 minutes to acquire in-depth valid information.

The NVivo qualitative analysis software was used to code the interview transcripts and identify key themes. Three-stage coding was applied in NVivo 12. (1) Open coding: labeling raw interview content (eg, “faster diagnosis” and “personalized rehabilitation plans”); (2) xial coding: grouping open codes into sub-dimensions (eg, “diagnostic efficiency” under the “diagnosis” primary indicator); (3) selective coding: integrating sub-dimensions into the preliminary indicator framework, ensuring alignment with the literature-derived coding structure. Qualitative suggestions were converted to indicators; expert qualitative feedback was operationalized into measurable indicators. For example, expert comment, such as “Innovative diagnostic technologies should reduce misdiagnosis and shorten waiting time,” was converted to two tertiary indicators: “diagnostic specificity improvement rate” (diagnosis indicator) and “average diagnostic time reduction” (diagnosis indicator). Expert suggestions, such as “Patents and high-impact papers reflect S&T innovation output,” were converted to “number of authorized invention patents” and “top-tier SCI journal publications” (S&T innovation indicators). Resolution of Expert Opinion Conflicts, such as conflicting suggestions (eg, whether to include “cost of S&T innovation” as an indicator), were addressed by a core expert panel (3 methodologists + 2 clinical specialists). The panel evaluated conflicting indicators based on two criteria: (1) alignment with the study’s core goal (quantifying innovation’s contribution to clinical capacity) and (2) data availability. For example, “innovation cost” was excluded due to the lack of unified official statistical channels. Interviewee feedback and indicator suggestions were categorized into four dimensions: prevention, diagnosis, treatment, and rehabilitation. For instance, under the “diagnosis” dimension, experts suggested adding indicators such as “accuracy of innovative diagnostic technologies” and “reduction rate in diagnosis time.”

All valid feedback was classified into four core dimensions. The finalized multi-level evaluation framework includes 4 primary, 14 secondary, and 246 tertiary indicators, laying a solid foundation for subsequent Delphi questionnaire surveys.

Questionnaire Structure and Distribution

The research team developed an expert consultation questionnaire based on the preliminary evaluation indicator system. The questionnaire comprises three sections. The first section provides a brief introduction and description of the research topic. The second section includes questions concerning experts’ demographic and professional information, including age, institution, professional title, research field, years of experience, basis of judgment (Ca) regarding the importance of the indicators, and familiarity with the indicator content. The third section invites experts to evaluate the wording and content of each indicator. In this section, a 5-point Likert scale was employed for the evaluation, accompanied by an open-ended feedback column to facilitate revisions of the initial draft. The scale ranges from 5 to 1, representing “very important,” “important,” “neutral,” “unimportant,” and “very unimportant,” respectively. In the feedback column, experts were encouraged to provide suggestions and justify the addition, modification, or deletion of specific indicators.

Delphi Expert Consultation

The Delphi technique is a qualitative research method designed to achieve expert consensus on complex issues through iterative rounds of anonymous consultation, ensuring unbiased expression of opinions. When indicator discrepancies arose, the research team systematically analyzed the quantitative ratings and qualitative feedback, reaching resolutions through panel discussions. In subsequent rounds, the experts re-evaluated contentious indicators by considering aggregated group responses. This process was repeated until a consensus was achieved, as defined by two validated metrics: Kendall’s W concordance coefficient and the Coefficient of Variation (CV), with statistical significance set at p < 0.05.

Selection of Experts

For the selection of experts, priority was given to members of the National Science and Technology Expert Database. Experts were selected based on the following criteria: (1) associate senior title or higher; (2) affiliated with tertiary hospitals meeting Joint Commission International standards; (3) ≥10 years of experience in healthcare-related fields (ensuring robust theoretical and practical expertise to mitigate opinion bias); and (4) demonstrated interest and commitment to participate in the study. Relevance to “technological innovation assessment” was ensured by the following selection mechanisms: (a) among research experts, we specifically selected individuals actively engaged in science and technology policy research, innovation evaluation, or translational medicine studies; (b) among clinicians, we selected those who had participated in clinical trials, served on guideline development committees, or had published research on technology adoption; (c) among industry managers, we selected officials directly involved in health technology assessment, innovation policy formulation, or research funding allocation. The rationale for this selection approach is that expertise in technological innovation assessment requires both domain knowledge (clinical or biomedical expertise) and experience with innovation evaluation methods—a combination achieved through our stratified selection of research experts, clinicians, and managers. We excluded those who had incomplete questionnaire responses or failed to return the completed surveys within the stipulated timeframe. The Delphi method recommends the recruitment of 15−50 participants to achieve optimal validity.²⁸

Implementation of Expert Consultation

The questionnaires were electronically delivered to experts via email. Upon completion, participants returned their responses to the research team through the same channel. The data were entered into an Excel spreadsheet and independently verified by two researchers to ensure accuracy.

Questionnaires were distributed electronically to experts with supplementary materials, including research background, objectives, demographic collection forms, scoring criteria, and indicator descriptions; responses were required within 2 weeks. Following round 1, the research team analyzed the feedback, revised the indicators through structured panel discussions, and developed round 2. The second questionnaire was delivered to the original panel alongside a visual summary report of the outcomes of round 1 and item-specific metrics: mean importance scores, full-score ratios, and CV. Consensus thresholds required: CV <0.25²⁹ (dispersion control), mean score > 3.5 (5-point scale), and ≥ 75% inter-rater agreement, with elements meeting all criteria retained in the framework.

Based on the results of the second round, the research team further revised each indicator, adding new items, merging existing ones, or refining the entries. This evidence-driven iterative process culminated in the finalized Chinese S&T Innovation-Driven Disease Diagnosis and Treatment Capacity Advancement Evaluation Indicator System.

Statistical Methods

Data from each Delphi round were recorded in an Excel spreadsheet as single entries and then verified and corrected independently by two research team members. Analyses were done using SPSS 23.0 (SPSS Inc, Chicago, IL, USA). Categorical variables are reported as counts and percentages, and continuous variables as mean ± standard deviation. Expert engagement was measured by response rate (valid returned questionnaires/total distributed × 100%), with rates ≥70% considered as acceptable for data reliability.³⁰ Expert authority was quantified by the authority coefficient (Cr), calculated as (Ca + [familiarity score] Cs)/2,³¹ where Ca was assigned the following values: practical experience=0.8, theoretical analysis=0.6, peer reference=0.4, and intuition=0.2; Cs was graded as unfamiliar=0.2, somewhat unfamiliar=0.4, somewhat familiar=0.6, very familiar=0.8, and proficient=1.0.³² Consistency was evaluated using the CV: a threshold of <0.25 indicated acceptable dispersion.³⁰ Kendall’s W concordance coefficient: range was 0–1 (higher values denote greater consensus). Consultation ceased when Kendall’s W reached p < 0.05 and the CV was < 0.25, confirming acceptable consensus.³³ Statistical significance was defined as p < 0.05.

Results

Basic Information

Per the inclusion criteria, round 1 engaged 45 experts (mean age 52.93 ± 6.81 years); 40 experts (mean age 52.73 ± 6.68 years) completed round 2. All experts who completed the two rounds of consultation were selected from representative medical institutions across China. (Table 1).

Table 1.

Basic Information on Experts

Characteristics	First Round		Second Round
	Number of People	Proportion (%)	Number of People	Proportion (%)
Gender
Male	35	77.78	31	77.5
Female	10	22.22	9	22.5
Education
Undergraduate degree	3	6.67	3	7.5
Graduate degree	42	93.33	37	92.5
Years of work experience
<20 years	10	22.2	10	25.0
21–30 years	17	37.8	16	40.0
31–40 years	17	37.8	13	32.5
>41 years	1	2.2	1	2.5
Title
Senior	42	93.3	37	92.5
Deputy senior	3	6.7	3	7.5
Professional field
Research experts	10	22.22	8	20
Tertiary hospital clinicians	30	66.67	27	67.5
Industry administrators	5	11.11	5	12.5

Expert Positive Coefficient

The expert positivity coefficient is represented by the questionnaire response rate, which directly reflects experts’ participation enthusiasm and level of cooperation in this study. A questionnaire response rate of 50% is adequate for conducting analyses and reporting results, whereas a response rate of 70% is considered excellent.³⁴ In the first round, 45 questionnaires were distributed, and 40 were returned, with a recall rate of 88.89%. In the second round, 40 questionnaires were distributed, and 40 were returned, yielding a recall rate of 100%. The response rates of both questionnaire rounds were well above the 70% benchmark for excellent response performance. This finding indicates high participation enthusiasm and favorable response quality among the experts involved in this study. The exceptionally high response quality effectively mitigated non-response bias, ensured the representativeness and reliability of the study results, and indirectly reflected the degree of attention experts paid to the research content.

Degree of Authority of Experts

The expert Cr is a core indicator ensuring the reliability of consultation results, and its magnitude directly affects the credibility of conclusions drawn from the Delphi survey. Mean values for Ca were 0.89 (round 1) and 0.89 (round 2), while Cs averaged 0.90 and 0.88, respectively. The Cr reached 0.90 in both rounds, significantly exceeding the 0.70 validity threshold—confirming high expert authority and credible outcomes.

The Degree of Expert Coordination

The degree of concordance among expert opinions was quantified using Kendall’s coefficient of concordance (Kendall’s coefficient [W]). This index reflects the level of coordination and consistency in all experts’ judgments across the full set of evaluation indicators, with a value ranging from 0 to 1. A W value closer to 1 indicates more concentrated and consistent expert opinions as well as a higher degree of coordination. Generally, a concordance coefficient between 0.3 and 0.5 is recognized as representing a good level of expert opinion coordination. The significance of the concordance coefficient was tested using the chi-square test; a p < 0.05 indicates favorable consistency of expert opinions and statistical credibility of the study findings. In round 1, the Kendall’s W coefficients were 0.331 (p<0.001) for secondary indicators, 0.181 (p<0.001) for tertiary disease diagnosis and treatment, and 0.344 (p<0.001) for tertiary S&T innovation indicators (Table 2). The corresponding round 2 values were 0.384 (p<0.001), 0.308 (p<0.001), and 0.428 (p<0.001), respectively (Table 2).

Table 2.

Summary of Kendall’s Concordance Coefficient W

		First-Round			Second-Round
		Kendall’s W	χ2	P	Kendall’s W	χ2	P
Secondary indicators		0.331	193.500	<0.001	0.384	199.850	<0.001
Tertiary indicators	Disease diagnosis and treatment	0.181	<0.001	<0.001	0.308	504.866	<0.001
	S&T innovation indicators	0.344	<0.001	<0.001	0.428	1694.223	<0.001

Results of Expert Consultation

In round 1, secondary indicators demonstrated mean scores ranging from 3.89 to 4.82 with CVs of 0.08 to 0.22; tertiary disease diagnosis and treatment showed means of 3.76 to 4.71 (CV: 0.11–0.28), while tertiary S&T innovation indicators ranged from 3.37 to 4.81 (CV: 0.09–0.32), all below the 0.25 dispersion threshold.

Following consensus thresholds and expert feedback, panel discussions led to the deletion of 1 tertiary disease diagnosis/treatment indicator and 103 tertiary S&T innovation indicators from the framework.

The specific reasons for deletion were as follows: Under the category of disease diagnosis and treatment indicators, the “vaccination rate” was removed due to lack of universality, as not all diseases have corresponding vaccines. Among the S&T innovation indicators, the “number of authorized patent cooperation treaty (PCT) patents” was deleted because PCT patents can only be submitted for international applications and are published and searched by the International Bureau of the World Intellectual Property Organization. Therefore, the original wording did not reflect the actual process. The indicator “funding for transformation of S&T achievements” was also deleted due to the absence of official, unified, and publicly available collection channels. Similar S&T indicators were excluded based on comparability and data availability.

After round 2 consultation, secondary indicators demonstrated mean scores ranging from 4.06 to 4.86 with CVs of 0.07–0.15; tertiary disease diagnosis/treatment indicators showed means of 3.80–4.85 (CV: 0.08–0.18), while tertiary S&T innovation indicators registered means of 3.78–4.82 (CV: 0.08–0.19).

Following round 2, all secondary and tertiary indicators (disease diagnosis and treatment and S&T innovation indicators) demonstrated coefficients of variation < 0.2, significantly improved Kendall’s W concordance, and minimal additional expert feedback, collectively indicating strong consensus convergence. Consequently, the consultation was terminated.

Weight Analysis

To reflect on the degree of contribution of each indicator, weights were assigned based on their relative importance. The scientific validity of the evaluation results often largely depends on the rationality of the weight allocation. The main methods for determining weights include the AHP, proportional distribution method, CV method, entropy weight method, path analysis, etc. In this study, the weights for the first- and second-level indicators were determined using AHP, while the weights for the third-level indicators were determined using the proportional distribution method.

We employed the AHP to determine the weights of the second-level indicators. As a multi-criteria decision-making tool that combines qualitative and quantitative analysis, the core logic of AHP involves decomposing complex decision-making problems into a hierarchical structure. In this study, the quantification of indicator importance was conducted using the nine-point scale proposed by A. L. Saaty.³⁵ Pairwise comparisons were made to assign values reflecting the relative importance of indicators at each level, thereby forming judgment matrices. By solving for the maximum eigenvalue and its corresponding eigenvector of each judgment matrix, the local weights of indicators at one level relative to the corresponding elements of the level above can be calculated sequentially. This process ultimately derives the comprehensive weights of the lowest-level factors relative to the highest-level objective. This method combines the systematic nature of qualitative analysis with the precision of quantitative analysis, effectively translating experts’ theoretical expertise and practical insights into quantifiable values. The weight determination process, based on hierarchical structuring and pairwise comparisons of indicators, is supported by rigorous logic.

The determination of the weights of the third-level indicators is calculated by the expert scoring using the proportional distribution method. First, each expert was asked to assign a score from 1 to 5 (integer values only) to reflect the importance of each third-level indicator (including diagnostic and treatment capability indicators and S&T innovation indicators). The expert scores were then aggregated, and the average score for each third-level indicator was calculated. Next, the average scores of the third-level indicators under each second-level indicator were summed. The weight coefficient of each third-level indicator was calculated as the ratio of its average score to the total sum of average scores under the corresponding second-level indicator.

Through the summary and statistical analysis of the results, the weights of each indicator were calculated, and consistency tests were conducted on all judgment matrices (all consistency ratios were <0.1, meeting the statistical requirements). The evaluation index system and weight coefficients composed of 4 first-level indicators, 14 second-level indicators, and 142 third-level indicators (including 42 diagnosis and treatment capacity indicators and 100 S&T innovation indicators) were finally determined (Appendix 1: Tables S1–S4). Such a comprehensive set of indicators is designed to fully capture the entire “transformation value chain” from research output to clinical impact. This avoids the pitfall of earlier, simplified evaluation tools potentially missing critical intermediate steps.

Discussion

In this study, we conducted two rounds of the Delphi method. Theoretically, multiple rounds of the Delphi method are conducted until a consensus is reached. However, as the number of rounds in the Delphi method increases, participants experience greater loss of energy and attention. Excessive rounds may lead participants to agree with certain viewpoints merely to conclude quickly, thereby resulting in a spurious consensus.³⁶ Some studies have suggested that this process be terminated when response discrepancies are reduced to a certain level, although specific criteria remain undefined.^37,38

Therefore, based on the pre-established criteria for consensus in this study, we terminated the consultation in the second round when the p-value for Kendall’s W-test was <0.05 and the CV was <0.25. According to relevant literature reviews, most Delphi studies are completed within two or three rounds,^37,39 and the number of rounds in this study is consistent with that in previous research.

The first round yielded a response rate of 88.89%, while that of the second round was 100%. These results indicated a high level of engagement and responsiveness among the experts involved in this study.

During the first round of consultation, the mean values of the Ca and Cs were 0.89 and 0.90, respectively. In the second round, mean Ca remained unchanged at 0.89, whereas mean Cs was 0.88. According to the formula Cr = (Ca + Cs) / 2, the mean Cr was 0.90 for both rounds, exceeding the widely accepted threshold of 0.70 for expert authority validity. These findings confirm that the experts involved in this study possess a high level of authority and that the consultation results based on their opinions are highly reliable, laying a solid foundation for the robustness of subsequent research conclusions.

The Kendall’s W values for the two rounds of consultation were 0.331, 0.181, 0.344, 0.384, 0.308, and 0.428. Kendall’s W ranges from 0 to 1 with the following interpretations: 0.00–0.10: Very weak agreement; 0.10–0.30: Weak agreement; 0.30–0.50: Moderate agreement; 0.50–0.70: Good agreement;0.70~1.00:Very strong agreement,⁴⁰ indicating moderate consistency among the experts in this study. However, when the number of experts is ≤20 and the number of indicators is ≤7, Kendall’s W can be directly used to determine the level of consistency.⁴¹ As the number of experts in this study exceeded 20 (45 in the first round and 40 in the second round), and the number of evaluation indicators far exceeded 7 (224 in the first round and 156 in the second round), the chi-square statistic was used to test for consistency, and the results were significant (p<0.001). Furthermore, in many studies with a large number of evaluation indicators, the Kendall’s concordance coefficient was low (0.10–0.31), but the chi-square test showed significant differences (p<0.001).^42–44 Therefore, the Kendall’s concordance coefficient and the test results in this study were consistent with those of previous studies.

This study successfully established a comprehensive evaluation indicator system to quantify the contribution of S&T innovations toward enhancing disease diagnosis and treatment capacity in China. Through two Delphi rounds, the finalized framework comprised 4 primary indicators, 14 secondary indicators, and 142 tertiary indicators (including 42 disease diagnosis and treatment and 100 S&T innovation indicators), addressing a critical gap in China’s healthcare innovation landscape, where fragmented prior frameworks impeded systematic measurement of innovation-driven clinical advancement, despite growing global recognition of technology as a cornerstone for optimized health systems. The 42 diagnosis and treatment indicators have systematically covered core patient outcome dimensions: diagnostic sensitivity/specificity/accuracy (corresponding to misdiagnosis and missed diagnosis assessment), standardized mortality/inpatient mortality/cure rate (corresponding to survival outcomes), complication rate/adverse reaction rate (corresponding to patient safety), and quality of life/activities of daily living/psychological status (corresponding to patient-reported outcomes). All core patient outcome dimensions are fully covered. The reason why there are more S&T innovation indicators is that they need to cover multiple links of the whole chain from basic research to translational application to product launch, while diagnosis-treatment indicators focus on core end-point dimensions to avoid duplication and redundancy, which conforms to the design logic of evaluation tools. The system is not solely oriented towards papers. High-weight indicators are concentrated on “guideline citation papers, launching new drugs, Class III medical devices,” and other transformation-related indicators. The weight of simply the number of papers/patents is lower. We set high-weight indicators such as “number of papers cited by authoritative guidelines,” “number of approved new drugs/medical devices,” and “number of top-tier SCI papers,” and the weight of transformation indicators is higher than that of simple paper/patent quantity indicators, which reflects the design orientation of “quality first, transformation oriented” and conforms to the policy requirement of “breaking the four-only (overemphasis on papers, titles, academic credentials, awards)”.

Key Strengths and Innovations

When constructing the evaluation index system, we followed the transformation logic: from technological innovation to clinical impact. This logical chain holds that scientific papers and patents are the direct manifestations of knowledge innovation and technological invention (the first stage). These outputs, when adopted by clinical guidelines and transformed into new drugs and devices approved in the market (the second stage), ultimately lead to improvements in terminal indicators such as diagnostic accuracy, treatment effectiveness, and patient prognosis (the third stage). In our framework, SCI papers and PCT patents are not regarded in isolation. They are the early and mid-term outputs in a coherent chain. Their quantity and quality (partially reflected by their publication in top journals and citation rates) jointly indicate the vitality and potential of a country or institution in promoting medical frontiers and nurturing the next generation of clinical technologies. Ultimately, the realization of this potential is more directly captured and verified by indicators such as “approved new drugs/new devices” and “adoption by guidelines” at the downstream of the chain, thereby enabling a systematic assessment of the contribution of S&T innovation to the entire transformation chain. For example: The birth of CAR-T cell therapy initially originated from a series of fundamental research SCI papers that revealed the functions of T cells. Subsequently, its core technology is protected through PCT patents. Further clinical research papers have demonstrated its efficacy, leading to its inclusion in the leukemia treatment guidelines (Phase II completed). Ultimately, as a cell therapy product approved for marketing, it revolutionarily increased the cure rate of specific hematological tumors (achieved in the third stage).

A key feature of this indicator system is that it links scientific econometric indicators with clinical outcome indicators. We recognize that the former is the leading indicator and driving force of the latter. For instance, the accumulation of SCI papers serves as the foundation for forming evidence-based medical evidence and thereby altering clinical practice guidelines. PCT patents are the cradle that nurtures future transformative therapies and tools, and the number of papers cited by international and domestic authoritative guidelines is the most crucial bridge for transforming knowledge into practice. When the conclusion of a paper is cited and recommended by authoritative clinical guidelines (such as the National Comprehensive Cancer Network guidelines and the diagnosis and treatment norms of the National Health Commission of China), it means that the level of evidence is sufficient to change the daily diagnosis and treatment behaviors of doctors worldwide or in the country. Indicators such as the number of approved new drugs and the number of medical devices on the market are the direct outputs and ultimate manifestations of the innovation value chain. They are physical tools that have undergone strict review, been proven safe and effective, and have been approved to enter the market. By simultaneously covering the milestones in innovation output and its transformation path, this system can capture earlier and more comprehensively the potential and practical contributions of S&T innovation to enhancing disease diagnosis and treatment capabilities. This evaluation system does not view these indicators in isolation but places them within a coherent “innovation transformation value chain.” Authorized patents and SCI papers are the starting point of the value chain, representing the potential and reserves for future clinical breakthroughs. The papers cited by the guidelines represent a crucial turning point in value realization, marking that the innovative achievements have obtained the “pass” for clinical practice. The new drugs and devices that are ultimately approved are the terminal outputs of the value chain and the material basis for the substantial improvement of diagnosis and treatment capabilities. By tracking the indicators along this complete chain, we can systematically, proactively, and comprehensively assess the all-round contribution of technological innovation from the source to the bedside. This evaluation system does not view these indicators in isolation. Rather, by tracking the complete chain of “potential reserve → clinical recognition → physical output,” we can systematically, proactively, and comprehensively assess the all-round contribution of S&T innovation from the laboratory to the bedside. Early indicators (such as papers and patents) are used to assess the potential and momentum of future clinical progress, while later indicators (such as guideline citations and product launches) capture the tangible contributions that have already been achieved.

The temporal and logical sequencing of these stages establishes a causal chain: knowledge creation enables clinical validation, which enables market authorization, which enables clinical adoption, which ultimately improves patient outcomes. Our indicator system is designed to capture each stage of this chain, enabling systematic assessment of the contribution of S&T innovation to clinical capacity enhancement. This is not a simplistic input-output model but a nuanced framework that recognizes the complex, multi-stage nature of innovation translation. Our findings align with the international efforts to measure healthcare innovation. For instance, similar to the Health Technology Assessment framework of the Organization for Economic Co-operation and Development, the core strength of our constructed index system lies in its multidimensional coverage, which integrates aspects such as investments in technological research and development, clinical translation, diagnostic accuracy, treatment efficacy, and healthcare accessibility. This holistic design distinguishes it from previous single-domain tools. By incorporating both input- and outcome-oriented indicators, the system covers the entire innovation lifecycle from laboratory to clinical application, reflecting China’s unique emphasis on translating scientific breakthroughs into tangible patient benefits, a key focus emphasized in national strategies, such as the Healthy China 2030 Planning Outline. The three-level index system constructed in this study, covering the two major dimensions of disease diagnosis, treatment indicators, and technological innovation indicators, demonstrated significant innovation.

This study was methodologically rigorous and standardized, integrating a literature review, Delphi method consultation, and statistical validation to enhance the credibility of the index system. Such robustness is particularly critical in China’s diverse healthcare context, where regional disparities in technology adoption and clinical resources exist, creating an urgent need for an assessment tool that balances flexibility and standardization.

Practical Implications

Practically, this system provides actionable insights to various stakeholders. Policymakers can use it to allocate resources precisely, prioritizing areas with low innovation impact but high clinical demands. Healthcare facility administrators can leverage it to identify gaps in their institution’s innovation adoption, thereby informing strategic planning. Researchers may find it beneficial in designing studies related to innovation translation by focusing on indicators that currently underperform but hold significant potential.

Future research can build on this study in multiple directions. First, pilot testing of the index system in underdeveloped regions will help verify its generalizability and identify region-specific adjustment needs. Second, conducting longitudinal studies using this system to track the dynamic impact of technological innovations on diagnostic and treatment outcomes will reveal long-term trends. Third, a comparative analysis using international frameworks can facilitate cross-national learning and provide global benchmarks for China’s progress in healthcare innovation.

In summary, this evaluation index system is a significant step toward systematic assessment of innovation-driven healthcare advancements in China. By bridging the gap between technological progress and the measurement of clinical practice, this study provides evidence to support evidence-based decision-making, ultimately contributing to the delivery of high-quality, innovative healthcare services to all Chinese citizens.

External Validity and Transferability

The core framework of the indicator system (4 primary indicators: prevention, diagnosis, treatment, rehabilitation; 14 secondary indicators; and the “innovation-translation-outcome” value chain logic) is based on universal principles of medical innovation and healthcare improvement. Similar to international frameworks such as the OECD Health Technology Assessment and the WHO Health Innovation Systems, our system integrates input-, process-, and outcome-oriented indicators, covering the entire lifecycle of S&T innovation from laboratory to clinical application. This universal logical structure provides a foundation for its application in other countries. Compared with the OECD Health Care Quality Indicators,⁴⁵ our system additionally incorporates innovation inputs (research and development investment, patents) and their translation into clinical outcomes. For South Korea, the framework can be aligned with its bio‑health convergence and digital health policies, with indicators adjusted to reflect the influence of its strong electronics sector on health IT adoption. For Singapore, its centralized and tiered healthcare system allows ready adaptation, with modifications needed for its specific disease spectrum and financing model. For European countries, indicators related to “accessibility of innovative technologies” and “resource allocation efficiency” should be tailored to different service delivery models.

For different contexts: in decentralized systems (eg, some European countries and the US), indicators on accessibility should incorporate insurance coverage and regional equity. In resource‑limited settings, emphasis should shift towards cost‑effectiveness and technology absorption capacity, with patent‑related metrics adapted to local classification systems and additional indicators such as “absorption capacity of imported technologies” and “industry–academia collaboration efficiency.” For countries with a high infectious disease burden, greater weight should be given to indicators like “innovation in diagnostic technologies for infectious diseases” and “development speed of vaccines/antiviral drugs.”

While the core framework is transferable, regional differences in healthcare systems, S&T innovation ecosystems, and disease spectra require targeted adaptations. We recommend that countries or regions seeking to implement this framework conduct a preliminary scoping study to identify priority disease areas, assess data availability, and engage local experts in a Delphi process to adapt the framework to their specific context.

Limitations

The limitations of this study lie in the potential regional bias in expert consultations, as participants were mostly from eastern developed regions. Experts from developed regions may overemphasize high-tech innovation indicators while underestimating the importance of basic medical innovation indicators. There were also limitations in terms of participant selection. Although this study made every effort to select representative experts, different groups of experts may yield different results. We acknowledge that the expert panel had a male predominance (77.8% in round 1, 77.5% in round 2). This gender imbalance reflects the broader composition of senior healthcare leaders and innovation policy experts in China, where women remain underrepresented at senior academic and administrative levels. We conducted a post-hoc analysis to assess potential gender bias in the Delphi results. Subgroup analysis comparing male and female expert responses on indicator importance ratings revealed no statistically significant differences (t-test, p > 0.05 for all indicators), suggesting that the observed gender imbalance may not have substantially biased the results. However, we acknowledge this as a limitation and recommend that future replications of this study deliberately recruit more female experts to ensure gender representation. The potential influence of gender on indicator selection and weighting should be systematically examined in future research. This is a common challenge in Delphi studies in China, where expert networks tend to be concentrated in urban centers. This study only reflects the selected experts’ perceptions of the importance of the indicators considered. Future research could expand the sample coverage to include underdeveloped regions and dynamically adjust indicators for emerging technologies (eg, applications of artificial intelligence in precision medicine) to enhance the timeliness and comprehensiveness of the system. The index system developed in this study was based on China’s health situation and technological innovation situation and all the experts consulted were from China. Consequently, the index system may not be directly applicable to other countries, but it could offer valuable insights in other nations. Given these limitations, if the conditions were mature, the scope and scale of the Delphi expert consultation could be expanded in future studies to enhance the representativeness of the findings. In future practical applications, once sufficient objective performance data is accumulated, objective weighting methods such as the entropy method or principal component analysis will be employed to validate and dynamically adjust the initial weights, thereby gradually reducing reliance on initial expert judgment. Simultaneously, the index system should be dynamically adjusted according to the actual circumstances and health policies so as to further improve the evaluation system and promote the high-quality development of China’s disease diagnosis and treatment capabilities and S&T innovation.

Future Research

The generic indicator framework established in this study is applicable to most disease categories, yet distinct disparities exist in diagnostic and therapeutic characteristics across various disorders. Subsequent specialized research may further optimize and refine corresponding observational indicators in accordance with disease-specific features. In the future, we intend to select three high-incidence diseases, namely malignant tumors, cardiovascular diseases, and bone aging, which have an urgent need for technological innovation, to verify the effectiveness of this evaluation system.

Data for disease diagnosis and treatment capability indicators will primarily be sourced from expert assessments. A panel of experts with extensive clinical and research experience will systematically collect data using standardized questionnaires. S&T indicators will be categorized according to disease types and collected through both public and dedicated channels. Data sources will primarily include academic databases such as Web of Science and incoPat, as well as medical databases like clinical guideline databases and PharmaDI (for pharmaceuticals/medical devices).

Reference will be made to established vocabulary and classification systems, including the Medical Subject Headings, Chinese Medical Subject Headings, the National Cancer Institute thesaurus, and the American College of Cardiology classification system. To transform the qualitative evaluation framework into quantifiable results, the study will employ the Epsilon-Based Measure steady-state evaluation model. By analyzing efficiency values, this approach will be used to assess the extent to which S&T innovations influence diagnostic and treatment capabilities. Additionally, an input-output efficiency dynamic analysis model from econometrics, namely the Malmquist Index, will be applied. This model treats S&T innovation as the input and disease diagnosis and treatment outcomes as the output, enabling a dynamic evaluation of the promotion effects over the past decade. The findings will provide data-driven support for the subsequent refinement of S&T innovation strategies and resource allocation.

Conclusion

In this study, we systematically constructed an index system to evaluate the role of China’s technological innovation in enhancing disease diagnosis and treatment capabilities. This index system quantifies the association between technological innovation and progress in clinical diagnosis and treatment but also provides a practical tool for policymakers to formulate targeted innovation-driven healthcare strategies for research institutions to track the impact of technological breakthroughs and for medical institutions to assess the progress of their own diagnosis and treatment upgrading. Overall, this study emphasizes the “patient-centered” value orientation, guiding S&T innovation from the laboratory to the clinic, from papers and patents to actual results, and promoting S&T achievements to better serve people’s health. This evaluation framework lays a foundation for evidence-based decision-making in integrating technological innovation into the field of disease diagnosis and treatment and contributes to promoting the sustainable development of China’s healthcare system.

Funding Statement

No funding was received.

Abbreviations

S&T, scientific and technology; Ca, basis of judgment; CV, coefficient of variation; Cr, authority coefficient; W, Kendall’s coefficient; PCT, patent cooperation treaty; AHP, analytic hierarchy process; OECD, Organisation for Economic Co-operation and Development.

Data Sharing Statement

The dataset for this article are not publicly available due to concerns regarding participant/patient anonymity. Requests to access the datasets should be directed to the corresponding author LW.

Ethics Approval and Consent to Participate

This study was an anonymous Delphi questionnaire survey. It did not involve identifiable personal data, human tissues, case reports, or sensitive private information. Prior to study initiation, the research team consulted the Ethics Committee of the Academy of Military Medical Sciences, which advised that the study qualifies for exemption from formal ethical review due to its low‑risk, non‑interventional design. Meanwhile, this study obtained formal ethical approval from the Ethics Committee of Beijing Rehabilitation Hospital, Capital Medical University (Approval No.: 2024bkky-093). The study was conducted in accordance with the ethical principles of the Declaration of Helsinki. All respondents received a detailed invitation letter disclosing the study objectives, procedures, confidentiality measures, and voluntary nature of participation. Completion and voluntary return of the questionnaire constituted informed consent. All participants contributed data on an anonymous and voluntary basis.

Author Contributions

All authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; took part in drafting, revising or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.

Disclosure

The authors report no conflicts of interest in this work.