Development and testing of a post competency scale for traditional Chinese medicine physicians undergoing standardized training

Abstract

Objective

To develop and test a post competency scale for traditional Chinese medicine (TCM) physicians undergoing standardized training to provide an applicable tool for scientific evaluation.

Methods

Based on literature analysis, behavioral event interviews, and expert consultations, measurement questions were formulated and the initial scale was designed. A questionnaire survey was conducted from July 2022 to May 2023 among TCM physicians undergoing standardized training in China. The rationality of the scale was confirmed through item purification, factor analysis, and tests of reliability and validity.

Results

The post competency scale consisted of three dimensions (TCM fundamentals and research abilities, TCM thinking and skill abilities, and personal traits and communication abilities) with 21 items. Exploratory factor analysis identified three common factors, accounting for a cumulative variance contribution rate of 62.165%. Confirmatory factor analysis demonstrated that the fit indices of the three-factor model fell within a relatively ideal level. The Cronbach's alpha coefficient of the scale was 0.885. Through convergent validity analysis, the standardized loading coefficients of the 21 items were >0.5, and the average extracted variance (AVE) of the three factors was also >0.5. Moreover, the square roots of the AVE values for each dimension exceeded the correlation coefficients between it and the other dimensions.

Conclusions

The findings suggest that the post competency scale of TCM physicians undergoing standardized training can provide a reliable scientific basis for training and assessment within China.

1. Introduction

Traditional Chinese medicine (TCM) standardized training is a transitional stage for TCM students, improving their clinical practice ability after completing institutional education [1] and playing an essential role in the cultivation and delivery of competent TCM physicians [2,3]. It has gradually become important for the implementation of the Healthy China 2023 initiative and improvement in the overall level of medical and health services [4,5].

The concept of “competence,” first proposed by Professor McClelland in 1973, refers to the deep-level characteristics that a person possesses, such as knowledge, skills, self-image, traits, and motivation [6]. This concept has been expanded and extended by later scholars. At present, the widely acknowledged definition of post competence pertains to the skills and attributes necessary for performing effectively and attaining exceptional outcomes within a particular role, organizational setting, and cultural milieu. The evaluation results of post competency can better reflect the potential and characteristics of TCM physicians undergoing standardized training [7,8]. However, most existing studies on post competency focus on other healthcare positions [[9], [10], [11]] and there is a lack of post competency measurement and evaluation tools for TCM physicians undergoing standardized training.

This study utilized literature analysis, behavioral event interviews, and expert consultations to develop measurement questions and create the initial scale. Subsequently, the item refinement, exploratory factor analysis, confirmatory factor analysis, as well as assessments of reliability and validity were conducted to provide a reference for the education, training, and assessment of TCM physicians in China undergoing standardized training.

2. Materials and methods

2.1. Literature review, behavior event interview, and expert consultation

We conducted a comprehensive search of the relevant literature on the development and validation of an evaluation index system for the post competency of TCM physicians in standardized training programs, both domestically and internationally, from 2002 to 2022. Relevant policies and documents issued by the National Health Commission were included to deepen our understanding of the research progress on standardized training for TCM physicians and post competency.

A stratified random sampling method was used to select 34 TCM physicians undergoing standardized training in tertiary grade A TCM hospitals in various provinces across the country as interviewees. The behavioral event interview method was used to explore key indicators of the post competency of TCM physicians undergoing standardized training programs through in-depth communication.

Two rounds of expert consultations were carried out, involving the participation of 20 TCM experts (twelve deputy chief physicians and eight chief physicians) who were familiar with TCM standardized training and questionnaire development. The selection of experts was based on the following criteria: (1) a master's degree or above; (2) 10 years or more in TCM clinical, teaching, or administrative work; (3) a senior title; and (4) participated voluntarily.

2.2. Scale and questionnaire design

2.2.1. Scale design

The scale design required multiple steps. (1) The research team compiled, summarized, and analyzed literature, initially forming three dimensions and 22 items. An individual score was allocated to each item, with a higher score indicating the item's greater significance in TCM standardized training. (2) The scale underwent an initial round of correction. Experts read and evaluated the items, eliminating language errors and ambiguities, and searched for any measurement items that could supplement relevant dimensions. The total number of measurement items increased to 24. (3) Finally, the scale underwent a secondary round of correction. Experts discussed the 24 measurement items, modifying and merging item entries based on the readability of relevant terms and accuracy of language expression. This resulted in 21 measurement items to be used within the initial TCM standardized training post competency scale (Table 1). The initial scale had ≥3 measurement items for each dimension, consistent with management research recommendations (at least three items per dimension in the scale), ensuring content validity.

Table 1.

Initial scale of post competency of physicians in TCM standardized training.

Dimensionality	Item	Measurement question
TCM fundamentals and research abilities	JK1	I have a basic understanding of TCM theory, such as “Fundamentals of Traditional Chinese Medicine,” “TCM Diagnosis,” “Prescription Studies,” “Pharmacology of Traditional Chinese Medicine,” “Internal Medicine of Traditional Chinese Medicine,” and so on.
	JK2	I am familiar with TCM thinking (holistic thinking, dialectical thinking, analogical thinking, reverse thinking, etc.), and can apply it in clinical work.
	JK3	I am familiar with classical Chinese and can read ancient texts and understand their meanings correctly.
	JK4	I can spend time reading TCM classical texts every week.
	JK5	I have a certain knowledge of epidemiology and medical statistics.
	JK6	I have a certain knowledge of foreign languages.
	JK7	I have critical thinking skills and am willing to question illogical facts or phenomena.
	JK8	I have a certain ability to perceive and insight into disease research.
	JK9	I can use opportunities to treat and save people in clinical work to explore the practical efficacy and treatment advantages of Chinese medicine, accumulate experience in TCM treatment, and discover the value of disease research.
TCM thinking and skill abilities	SJ1	I have a multi-perspective analysis of problem-solving ability and can analyze, and judge based on knowledge and experience.
	SJ2	I can master the TCM “pattern differentiation and treatment” thinking ability and diagnose and treat based on the patient's clinical manifestations and their own medical knowledge.
	SJ3	I can master TCM diagnostic techniques and characteristic diagnostic methods, including tongue diagnosis, pulse diagnosis, questioning, face diagnosis, voice diagnosis, smell diagnosis, and abdominal diagnosis, etc.
	SJ4	I can use TCM and Western medical knowledge and skills to conduct physical examinations and emergency treatments.
	SJ5	I can comprehensively and meticulously inquire about patients and formulate TCM treatment plans and adjust them.
	SJ6	I am familiar with the characteristics and usage taboos of different Chinese and Western medicines, and pay attention to gradual progression in medication.
	SJ7	I can write case reports in a timely, comprehensive, and meticulous manner to reflect the thinking process.
	SJ8	I can flexibly respond to common diseases and sudden medical situations.
Personal traits and communication abilities	TG1	I love and recognize the need for TCM to continue to learn and improve through further education, mentorship, and clinical practice, pursuing excellence.
	TG2	I can self-regulate emotions and stress and maintain confidence in my medical skills.
	TG3	I can communicate effectively and in a timely manner with colleagues or superiors regarding medical activities to avoid conflicts.
	TG4	I can convey the patient's condition to their family members in a timely and proactive manner, using accurate and easy-to-understand language to make the treatment plan clear.

TCM, traditional Chinese medicine.

2.2.2. Questionnaire design

The questionnaire consisted of three parts. (1) Design of questionnaire fill-in prompts. The purpose of the survey was clearly explained, participants were promised that their data would only be used for academic research, and basic requirements for complete responses to each question were described. (2) Basic information survey form. The form included questions regarding the participants’ sex, age, educational background, and relevant work experience. (3) Evaluation index system scale regarding post competency. The system scale contained 21 items and covered three dimensions of post competency. The questionnaire adopted the Likert 5-point scoring method [12]. Participants checked the corresponding area based on their feelings and degree of conformity described in the item statement. A score of “1” signified that it was very inconsistent with the item description and “5” indicated that it was very consistent with the item description.

2.3. Data collection

From July 2022 to May 2023, TCM physicians undergoing standardized training were selected as subjects for a paper-based questionnaire survey using a convenience sampling method. The questionnaires were distributed with the assistance of the academic affairs office of each hospital, and the research team provided instructions to ensure quality and completion. The head of the academic affairs office, after training, conducted the survey, explaining the purpose and providing instructions to the TCM physicians undergoing standardized training before the survey, ensuring their voluntary participation. The completed questionnaires were collected immediately. Any questions that arose during the survey process were promptly addressed by the research team to ensure the validity and accuracy of the survey. Inclusion criteria: (1) The standardized training hospital was a tertiary Class A TCM hospital; (2) The participants signed the informed consent form and willingly agreed to take part in the study. Exclusion criteria: (1) Workers who had not worked in their positions for more than 6 months; (2) Incomplete questionnaire respondents; (3) Repeated questionnaire respondents; (4) Respondents who obviously did not take the questionnaire seriously, such as those whose cumulative answer time was <2 min; (5) Respondents who had obvious logical errors in answering the questionnaire.

3. Data analysis

3.1. Initial scale item purification

SPSS 26.0 was used to calculate Cronbach's α coefficient and corrected item-total correlation (CITC) value of each item to determine the internal consistency of the scale and correlation of the total score. If the Cronbach's α coefficient of the item was ≥0.6 and the CITC value was >0.5, it was retained; otherwise, it was deleted.

3.2. Exploratory factor analysis (EFA)

From the sample data, 215 questionnaires were randomly selected and labeled as sample A for EFA. Before performing EFA, determining whether there was a correlation among the variables in the dimensions covered by the scale was necessary. Therefore, Kaiser–Meyer–Olkin (KMO) and Bartlett's sphericity tests were carried out to assess the suitability. The main component method was used to extract factors with the eigenvalues >1, and the scale dimensions were extracted based on the scree plot obtained. To achieve a factor structure with high theoretical significance and value, we used maximum variance rotation analysis and validated the rationality of the scale.

3.3. Confirmatory factor analysis (CFA)

The remaining 215 questionnaires in the sample data were labeled as sample B. A first-order CFA was performed using AMOS 21.0 to further examine the rationality of the dimensions of the scale and determine if there was a possibility of additional factor structures. Therefore, various first-order factor models with different factor structures were introduced for comparison. Based on previous research experience, if there was no obvious correlation between first-order factors during the validation process of scale dimensions, it indicates that there was no problem of multicollinearity or that the scale does not have a higher-level factor structure.

3.4. Validation of the reliability and validity

The Cronbach's α coefficient was used to determine the reliability of the scale. The validity test refers to the degree of closeness between the measurement results and intended target, including content and construct validity. Construct validity was evaluated in depth through two distinct aspects: convergent and discriminant validity.

4. Results

4.1. Basic information of the survey questionnaire

Out of 468 survey questionnaires collected, 430 were valid, resulting in an effective questionnaire recovery rate of 91.9%. The number of valid questionnaires met the basic requirement for empirical analysis (a ratio of 1:10 between the number of measurement items and the sample size). Among the 430 valid questionnaires, 204 were completed by men (47.4%), and 226 by women (52.6%); 265 aged ≤25 years (61.6%), 97 aged 26–30 years (22.6%), and 68 aged >30 years (15.8%); 293 held a bachelor's degree (68.1%), and 137 held a master's degree (31.9%); 194 had work experience (45.1%), and 236 had no work experience (54.9%).

4.2. Results of item purification

The CITC value for all items in the scale exceeded 0.5, and the Cronbach's α coefficient of the scale was within a reasonable range; therefore, all items in the scale were retained (Table 2).

Table 2.

Item purification of initial scale.

Dimensionality	Item	CITC	Cronbach's α coefficient (After removing this item)
TCM fundamentals and research abilities (0.911)	JK1	0.733	0.898
	JK2	0.686	0.901
	JK3	0.675	0.902
	JK4	0.724	0.898
	JK5	0.693	0.900
	JK6	0.698	0.900
	JK7	0.690	0.901
	JK8	0.654	0.903
	JK9	0.679	0.902
TCM thinking and skill abilities (0.900)	SJ1	0.657	0.890
	SJ2	0.682	0.887
	SJ3	0.689	0.887
	SJ4	0.696	0.886
	SJ5	0.664	0.889
	SJ6	0.706	0.885
	SJ7	0.672	0.888
	SJ8	0.717	0.884
Personal traits and communication abilities (0.832)	TG1	0.668	0.784
	TG2	0.654	0.790
	TG3	0.630	0.801
	TG4	0.690	0.774

CITC, corrected item-total correlation.

4.3. Results of EFA

Based on sample A, the KMO value was 0.913, and the χ² value of Bartlett's sphericity test was 2382.07 with 210 degrees of freedom and P < 0.001, suggesting that the presence of common factors among the items, the sample data was suitable for factor analysis. Three common factors with eigenvalues >1 were extracted through principal component analysis, with a cumulative variance contribution rate of 62.165% (Table 3). The scree plot in Fig. 1 drops sharply before component 4 and then becomes flat, indicating that extracting three common factors from the 21 items was appropriate and preserved the three dimensions of the model. The results of varimax rotation showed that the factor loadings of each item on its respective factor all met the criterion of >0.5; thus, all items of the scale were retained.

Table 3.

Principal component analysis.

Component	IE			ESSL			RSSL
	Total	VP (%)	CVP (%)	Total	VP (%)	CVP (%)	Total	VP (%)	CVP (%)
1	6.883	32.778	32.778	6.883	32.778	32.778	5.776	27.503	27.503
2	3.856	18.361	51.139	3.856	18.361	51.139	4.621	22.003	49.506
3	2.316	11.026	62.165	2.316	11.026	62.165	2.658	12.659	62.165
4	0.752	3.583	65.748	–	–	–	–	–	–
…	…	…	…	…	…	…	…	…	…
21	0.230	1.098	100	–	–	–	–	–	–

IE, initial eigenvalue; ESSL, extraction sum of squared loading; RSSL, rotation sum of squared loading; VP, variance percentage; CVP, cumulative variance percentage. “—” indicates no data available here; “ …” indicates that there is too much content, and only a part of the data is listed.

Fig. 1.

Scree plot of factor analysis.

4.4. Results of CFA

Based on sample B, the first-order factor models of all items in the scale were fit using the maximum likelihood method (Fig. 2A–C). The difference between the coefficient values of relevant indicators and standard thresholds was used to evaluate the superiority of the first-order factor models. As shown in Table 4, the first-order three-factor model demonstrated the best fit. Specifically, the absolute fit index χ²/df was 1.391 (acceptable value < 3) and the root mean square error of approximation was 0.043 (<0.05 indicates a “close fit”), indicating good fitting. Additionally, the goodness-of-fit index was 0.899 (a value > 0.7 suggests a reasonable fit), indicating that the model was acceptable. The parsimony goodness-of-fit index was 0.724 (acceptable value > 0.5), indicating that the model was relatively simple. The parameter evaluations of the three-factor model were shown in Fig. 2D.

Fig. 2.

Factor model. (A) Single-factor model. (B) Two-factor model. (C) Three-factor model. (D) The parameter evaluations of the three-factor model.

Table 4.

Fit index of first-order factor model.

Factor model	Factor combination	χ2	df	χ2/df	RMSEA	GFI	PGFI	NFI	TLI	CFI
Single-factor	JK + SJ + TG	1266.884	189	6.703	0.163	0.482	0.394	0.478	0.460	0.514
Two-factor	JK, SJ + TG	545.113	188	2.900	0.094	0.784	0.638	0.775	0.820	0.839
Three-factor	JK, SJ, TG	258.735	186	1.391	0.043	0.899	0.724	0.893	0.963	0.967

CFI, comparative fit index; GFI, goodness of fit index; NFI, normed fit index; PGFI, parsimonious goodness of fit index; RMSEA, root mean square error of approximation; TLI, Tucker–Lewis index; “+” indicates the combination of items corresponding to multiple factors.

4.5. Results of reliability and validity testing

4.5.1. Reliability testing

The Cronbach's α coefficient of the scale was 0.885. The Cronbach's α coefficients of the three dimensions (TCM fundamentals and research abilities, TCM thinking and skill abilities, and personal traits and communication abilities) were 0.911, 0.900, and 0.832, respectively (Table 5), suggesting that the scale had good reliability.

Table 5.

Reliability testing analysis.

Dimensionality	Number of entries	Score range	Cronbach's α coefficient	Split-half reliability coefficient
TCM fundamentals and research abilities	9	9–45	0.911	0.905
TCM thinking and skill abilities	8	8–40	0.900	0.900
Personal traits and communication abilities	4	4–20	0.832	0.818

4.5.2. Validity testing

The convergent validity test results showed that the standardized loading coefficients of all items were >0.5. The average extracted variance (AVE) of the three factors, namely, TCM foundation and research ability, TCM thinking and skill ability, and personal traits and communication ability, were also >0.5 (0.515, 0.566, and 0.558, respectively). The composite reliability (CR) of each factor was >0.7 (0.905, 0.912, and 0.834, respectively).

Regarding discriminant validity, the maximum correlation coefficient among the three dimensions of the scale was 0.360, which was far below the standard threshold of 0.85 (Table 6). Therefore, there were no high correlation coefficients among the dimensions of the scale, and there was no problem of multicollinearity within the scale. Additionally, the square root values of the AVE factors for the three dimensions were 0.718, 0.752, and 0.747, respectively, with each dimension's square root values of AVE exceeding the correlation coefficients between it and the other dimensions, indicating that there was both a certain degree of correlation and discrimination among the factors.

Table 6.

Validity testing analysis.

Dimensionality	TCM fundamentals and research abilities	TCM thinking and skill abilities	Personal traits and communication abilities
TCM fundamentals and research abilities	0.718a	–	–
TCM thinking and skill abilities	0.360b	0.752a	–
Personal traits and communication abilities	0.120b	0.260b	0.747a
AVE	0.515	0.566	0.558

Represents <0.05.

^aRepresents the arithmetic square root of AVE data, and other data represents the correlation coefficient between dimensions.

^bRepresents <0.01; － indicates that there is no need to repeat the correlation data.

5. Discussion

We developed a post competency scale for TCM physicians undergoing standardized training based on detailed literature review, and behavioral event interviews. Experts conducted two rounds of modification and refinement to enhance the pertinence of the measurement items, ensuring that the content and wording were optimized before the questionnaire was formed. To improve the accuracy and validity of the scale, the questionnaire was purified by ensuring that all items had a CITC value greater than 0.5. The factor analysis confirmed that the fitting indicators of the first-order three-factor model of the post competency scale for TCM standardized training were at a relatively ideal level. The correlation coefficients among the three dimensions (TCM fundamentals and research abilities, TCM thinking and skills, and personal traits and communication abilities) were 0.12, 0.26, and 0.36, respectively, all of which were <0.5, indicating that there was no problem of multicollinearity among the first-order factors of the structural model, and no higher-level factor structure, rendering higher-order confirmatory factor analysis redundant. Moreover, the factor loading, AVE, and CR indicators in this study all reached the standard threshold, and the square root values of AVE for each factor were greater than the correlation coefficients between factors, indicating that there were good convergent and discriminant validity between the corresponding dimensions. Based on the above analysis, it can be concluded that the TCM standardized training post competency scale constructed in this study has good reliability and validity and could be used for post competency measurement in subsequent influencing factor studies to improve the quality of TCM physician standardized training.

The requirements for the development of TCM physicians undergoing standardized training in the new era are met by this scale, and it serves as a guiding tool, to a certain extent, for the cultivation of the TCM physicians undergoing standardized training in key areas of TCM research. Firstly, the scale recognizes the importance of sustaining interest in specific fields and remaining unaffected by external influences. This ability ensures that the TCM physicians undergoing standardized training maintain a strong dedication to their chosen areas of expertise, allowing them to continually enhance their knowledge and skills. Secondly, the scale emphasizes the ability to trace back to the origins and extensively read classical TCM texts. This not only promotes a deep understanding of TCM theory but also encourages the integration of traditional knowledge into modern TCM practice. Furthermore, the scale also emphasizes that TCM physicians undergoing standardized training should continuously explore and accumulate TCM treatment experience through clinical cases, thereby forming a diverse and rich knowledge base.

There are several limitations to consider in this study. Firstly, the data collection for the developed TCM standardized training post competency scale relied on self-evaluation, which is a subjective method. In order to enhance the reliability and validity of measuring TCM standardized training competency, it would be beneficial to collect non-self-evaluation data through questionnaires that pair TCM standardized training assessments with TCM standardized training physicians. Additionally, incorporating questionnaires that involve TCM standardized training physicians and patients could further improve the accuracy of the assessment. Secondly, it is important to acknowledge that there may be sampling bias in the survey questionnaire process due to limitations in data availability during the development and collection of the post competency scale. In order to address this, future research should focus on strengthening the scientific rigor of regional sampling. This would involve ensuring that the sample is representative of the target population and reducing any potential biases that may arise during the sampling process. Furthermore, increasing the sample size would also contribute to enhancing the reliability and applicability of the post competency scale. By including a larger number of participants, the findings of the study would be more representative and generalizable to the broader population. This would provide a more comprehensive understanding of the TCM standardized training competency and increase the overall validity of the study. In the future, the scientific rigor of regional sampling should be strengthened, and the sample size should be increased to enhance the reliability and applicability of the post competency scale.

6. Conclusions

The findings suggest that the post competency scale of TCM physicians undergoing standardized training can provide a reliable scientific basis for training and assessment within China.

Ethics declarations

This study has been reviewed and approved by the Ethics Committee of the First Affiliated Hospital of Anhui University of Chinese Medicine (2022AH-51). Written informed consent has been obtained from all participants, who have agreed to the publication of any data included in the study.

Funding

The 2021 Traditional Chinese Medicine Talent Training Base Construction Project (zyyggwsfw20210045), the University Synergy Innovation Program of Anhui Province (GXXT-2020-025), the National Administration of Traditional Chinese Medicine: 2019 Project of building evidence-based practice capacity for TCM (2019XZZX-NB001), the Key Project of Natural Science Research in Anhui Universities (KJ2021A0547), and Anhui Province Western Learning Senior Talent Training Program (2019qgxxzggrcpxxm20220104).

Data availability statement

The data supporting the findings of this study can be obtained from the corresponding author upon a reasonable request.

CRediT authorship contribution statement

Ting Dong: Writing – review & editing, Writing – original draft, Supervision, Resources, Project administration, Methodology, Funding acquisition, Conceptualization. Chenling Zhao: Writing – original draft, Validation, Supervision, Software, Investigation, Formal analysis, Data curation. Mengting Wu: Writing – review & editing, Methodology, Investigation, Data curation. Guofang Yu: Writing – review & editing, Methodology, Investigation, Data curation. Danqing Liu: Writing – review & editing. Peng Huang: Visualization, Validation, Data curation. Wenming Yang: Methodology, Funding acquisition. Mingxiang Han: Funding acquisition.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.