Abstract
Objective
To develop a Critical Care Practical Competency Scale (CCPCS) for clinical nurses, and to assess its reliability and validity.
Methods
The scale was developed through a literature review, semi-structured interviews, and Delphi expert consultation. A convenience sampling method was employed to survey in-service clinical nurses from three tertiary Class A general hospitals in Anhui Province, followed by an item analysis and assessment of the scale’s reliability and validity.
Results
The CCPCS for nurses comprised 3 dimensions and 22 items. The Cronbach’s α coefficient was 0.97, the partial coefficient was 0.91, and the retest reliability coefficient was 0.85. The Scale-Level Content Validity Index/Average (S-CVI/AVE) was 0.98, Scale-Content Validity Index/Universal Agreement (S-CVI/UA) was 0.93, with the Item-Level Content Validity Index (I-CVI) ranging from 0.83 to 1.00. Exploratory factor analysis revealed three common factors, explaining a cumulative variance contribution rate of 73.21%.
Conclusion
The CCPCS for nurses exhibited robust reliability and validity, laying a foundational basis for training and assessing critical care nursing skills.
Supplementary Information
The online version contains supplementary material available at 10.1186/s12912-026-04339-y.
Keywords: Nurses, Critical care nursing, Clinical competence, Scale development, Validation study
Introduction
Given the escalating frequency of disasters and public health emergencies, the nursing team has assumed a critical role in emergency prevention and control [1–3]. Throughout the COVID-19 pandemic, a surge in critically ill patients necessitated additional intensive care beds nationwide [4, 5]. Despite the deployment of numerous caregivers to intensive care units (ICUs) lacking critical care training or experience, the number of experienced critical care nurses (CCNs) remains insufficient relative to the clinical complexity of their patients [6]. Nurses are confronting a diverse array of challenges stemming from both the disease and the pandemic [7]. In addition, the management of critically ill patients is evolving with the rising age of the population, increasing comorbidities, and greater complexity of care [8–10]. Consequently, there is an escalating demand for enhanced critical care capabilities among clinical nurses in both intensive care units (ICUs) and general wards [11].
Furthermore, a survey indicated a ratio of 3.6 critical care beds per 100,000 population across Asian countries and regions [12]. Given the shortfall of intensive care unit beds, prompt identification of critical illness by general ward nurses, coupled with immediate reporting to physicians for interventions, can enhance patient clinical outcomes and mitigate the need for ICU admissions among decompensated patients [13, 14]. Nurses in emergency and intensive care units possess robust skills in intensive care, whereas those in general wards exhibit strong specialization in specific diseases they encounter, with limited opportunities to manage patients facing acute and critical conditions. Their proficiency in intensive care tends to be less developed. Consequently, nursing managers must strengthen critical care training for all nurses, including those in general wards and ICUs, to enhance their critical care competencies. The traditional training approach, characterized by the one-way teaching model of “see one, do one, teach one”, excessively prioritizes practical skills over professional knowledge and theoretical training. This imbalance may result in a deficiency of essential professional judgment and problem-solving abilities in complex scenarios [12]. The latest systematic review highlights the urgent need to enhance the core competencies of nurses in intensive care to effectively address the care needs of critically ill patients in hospitals. Currently, the absence of internationally standardized educational regulations, coupled with the increasing demands for enhanced competencies, underscores the need to reinforce educational practices that foster the development of intensive care nursing capabilities [15]. Therefore, to align with the evolving needs of contemporary nursing practices, it is essential to develop a comprehensive training and assessment framework for nurses, focusing on their critical care competencies. Current research tools fail to evaluate the critical care competencies of nurses employed in the general ward and ICU [16–19]. Therefore, this study endeavors to develop a scale that measures clinical nurses’ practical competence in critical care, with the aim of offering a valuable reference for nursing managers in designing assessments for nurses’ critical care training.
Methods
Theoretical foundation
The competencies required in critical care nursing can be defined as an integrated combination of knowledge, skills, and attitudes [15]. Consequently, the development of this scale is grounded in the KPA theory. The Knowledge, Attitude, and Practice (KAP) theory, introduced by British scholar Costel in the 1960s, provides a conceptual framework. The KAP theory comprises three stages: acquiring knowledge, shaping attitudes, and developing practices. It is grounded in knowledge and influenced by individual attitudes, ultimately leading to changes in practice. The KAP theoretical model highlights the progressive and interdependent relationship among knowledge, attitudes, and practices, offering robust theoretical support and practical guidance for individual behavior change [20]. The KAP theory underscores that practical operational skills form the cornerstone of nursing practice. By means of simulation training and practical experience [5], nursing staff can enhance their operational proficiency, ensuring their capability to efficiently and accurately perform diverse nursing tasks in clinical settings. The critical care practice competency scale developed in this study is grounded in the KAP theory that clinical nurses must acquire essential knowledge of critical care, develop emergency awareness and attitudes, and ultimately facilitate changes in practice.
Design and procedures
This study employed a mixed-method sequential explanatory design. The Critical Care Practical Competency Scale (CCPCS) was systematically developed and validated through a three-stage process (Fig. 1): Phase 1, item development; Phase 2, scale development; and Phase 3, scale validation. During the item development stage, the initial item pool of the scale was constructed through literature analysis, semi-structured interviews, topical group discussions, and a Delphi survey. Following the assessment of content validity by experts, a pilot survey of the items was administered to a sample of nurses. During the scale development stage, a primary survey was conducted by administering the CCPCS scale to clinical nurses. Through the process of item analysis, high-quality items are carefully selected, and invalid items are systematically eliminated. Exploratory factor analysis (EFA) is employed to investigate the underlying structure of the scale, specifically aiming to identify the number of potential dimensions that underlie the items. During the scale validation stage, confirmatory factor analysis (CFA) is performed to assess the extent to which the dimensional model derived from exploratory factor analysis aligns with the actual data.
Fig. 1.
A schematic figure summarizing the three phases of CCPCS
Phase 1 item development
Establishing a research group
The research group comprised the chief nurse, deputy chief nurse, nursing department director, ICU head nurses, general ward head nurses, master tutors specializing in critical care, and graduate students in critical care. The primary roles of the group included defining research topics, developing indicators across all levels, engaging expert consultations, managing data, performing statistical analysis, and deliberating and synthesizing expert opinions.
Formation of the item pool
Initially, a comprehensive literature review was conducted to develop a scale for assesssing nurses’ critical care competency. Relevant literature was retrieved from PubMed, Web of Science, CNKI, and WANFANG databases as of October 30, 2022. The inclusion criteria for the literature required that articles be published in peer-reviewed journals, with a focus on intensive care capabilities, scale development, and related topics. Articles were excluded if they were conference abstracts, grey literature, or if the full text could not be obtained. The search terms used were (‘nurse’) AND (‘critical care competency’ OR ‘critical care capacity’ OR ‘intensive care ability’) AND (‘scale’ OR ‘questionnaire’ OR ‘questionnaire survey’ OR ‘scale development’). Equivalent Chinese terms were employed for searches in the CNKI and WANFANG databases. A total of 2441 articles were initially retrieved from the database. After the removal of 2340 duplicates and articles that did not meet the inclusion criteria, 101 articles were ultimately included in the final selection. The remaining articles primarily focused on content related to intensive care capabilities.
Semi-structured interview
Conducted semi-structured interviews with 30 registered nurses, ICU head nurses, and teaching staff from tertiary hospitals. The interview outline included the following: ① For nurses: Have you received training in critical care? What critical care knowledge and skills have you acquired? What additional critical care knowledge and skills do you believe are necessary to learn? ② To ICU head nurses and teaching staff: What knowledge and skills do you believe are essential for enhancing critical care capabilities?
Topic group discussion
Following a thorough literature review and semi-structured interviews, the research team deliberated on and identified key elements. Subsequently, they compiled a scale comprising 40 items focusing on three domains: capability for collaborative rescue, proficiency in operating instruments, and foundational critical care skills.
Delphi expert consultation
Between November and December 2022, ten critical care experts from Anhui, Jiangsu, and Hunan provinces were consulted. The inclusion criteria for experts are as follows: a senior professional title or above; at least 5 to 10 years of relevant experience in critical care; experts should specialize in critical care-related fields, including ICU clinical nursing, critical care education, critical care management, and critical care medical treatment. Ten expert panel comprised one specialist in critical care education, three head nurses from the ICU, three specialized ICU nurses, and three chief physicians from the ICU. An initial questionnaire survey was administered to the selected experts, who were asked to rate, suggest additions or deletions, and provide revision recommendations. The completed questionnaires were compiled into a revised version and returned to the experts for reevaluation. The expert consultation was concluded once all experts had provided no further revision recommendations. The initial questionnaire comprises three sections. Part 1 introduces the research background, purpose, and significance, and provides illustrative examples. Part 2 of the CCPCS inquiry questionnaire features sections for “Supplementary Opinions”, “Revision Opinions”, and “Importance Rating”. Experts assessed item significance using a Likert 5-point scale, ranging from 5 (very important) to 1 (completely unimportant). Part 3 covers expert demographics, criteria for judgment, and familiarity with the questionnaire. Criteria for retaining items: Items must meet the following conditions simultaneously: (1) an average importance score exceeding 3.5, (2) a coefficient of variation less than 0.25, and (3) a full score rate greater than 20%. These items will be selected based on expert opinions and group discussions, with potential modifications (additions or deletions) made as deemed necessary [21].
Content validity test
Six experts were invited to assess the content validity of the scale using the Delphi method. Experts rated the correlations between each item’s content and the measurement dimensions and concepts on a scale from 1 to 4 (from “not relevant” to “very relevant”), and provided modification comments on the items. The item-content validity index (I-CVI), Scale-Content Validity Index/Average (S-CVI/Ave), Scale-Content Validity Index/Universal Agreement (S-CVI/UA), and Fleiss’s Kappa were used to evaluate the degree of expert agreement. The criteria for item selection were I-CVI ≥ 0.78, S-CVI/Ave ≥ 0.90, and S-CVI/UA ≥ 0.80 [22, 23]. Fleiss’s Kappa values ≤ 0 as indicating no agreement and 0.01–0.20 as none to slight, 0.21–0.40 as fair, 0.41– 0.60 as moderate, 0.61–0.80 as substantial, and 0.81–1.00 as almost perfect agreement [24].
Pilot survey
A preliminary survey was conducted among 30 clinical nurses from a tertiary hospital in Anhui, China, to assess whether the scale included any content that was difficult to comprehend, repetitive, or unclear.
Phase 2 scale development
Participants
Between May and August 2023, a convenience sampling method was employed to survey clinical nurses currently working at three tertiary, Class A general hospitals in Anhui Province. The research team administered and collected questionnaires via Wenjuanxing. Inclusion criteria: (1) Registered nurses who had passed the National Nurses Qualification Examination and had at least one year of work experience. Exclusion criteria: (1) Non-clinical nurse, (2) Nursing interns.
Sample size calculation
Various guidelines exist in the literature for determining the appropriate sample size for factor analysis. One well-known guideline is the Rule of 300, recommending a minimum of 300 cases. Another perspective categorizes sample sizes as poor (100), fair (200), good (300), very good (500), and excellent (1,000 or more) depending on their appropriateness for factor analysis [25]. 820 nurses were recruited for this study. The final sample was randomly split into two equal parts using SPSS 18.0 software. 410 samples were allocated for exploratory factor analysis, while an additional 410 samples were designated for confirmatory factor analysis.
Item analysis
Three methods were employed to screen the items: (1) Correlation coefficient analysis retained items where the correlation coefficient (r) with the total score was ≥ 0.4. (2) Critical ratio analysis involved ranking scale total scores from high to low, focusing on the top and bottom 27%. Items meeting criteria—t values exceeding 3 with a significant intergroup difference (p < 0.05)—were retained [26, 27]. (3) Cronbach’s α coefficient analysis: Items that increased the Cronbach’s α coefficient when removed were considered for potential exclusion [28].
Exploratory factor analysis (EFA)
Kaiser-Meyer-Olkin (KMO) measurement and Bartlett’s test of sphericity assessed the appropriateness of employing principal component analysis (PCA) with varimax rotation in exploratory factor analysis (EFA). Varimax rotation is the predominant orthogonal technique known for reducing factor complexity while maximizing variance in factor loadings. The dataset was deemed suitable for PCA only if the Bartlett test of sphericity yielded significance (P < 0.05) and the KMO value exceeded 0.70 [29]. Criteria for retaining items included initial eigenvalues > 1.0, factor loadings > 0.4, communality > 0.5, cumulative variance > 60%, and cross-factor loadings differing by < 0.2 [22, 30–32].
Phase 3 scale validation
Confirmatory factor analysis (CFA)
The Analysis of Moment Structure (AMOS) was utilized to validate the hypothesized factor model in Confirmatory Factor Analysis (CFA). Additionally, both convergent and discriminant validity were examined to evaluate the construct validity across item measures [33].
The model fit was assessed using a battery of fit indices: the chi-square test, relative chi-square (CMIN/DF), root mean square error of approximation (RMSEA), comparative fit index (CFI), normed fit index (NFI), Tucker-Lewis index (TLI), and incremental fit index (IFI) [34]. Generally, a χ²/df ratio of less than 3 indicates excellent model fit, while ratios between 3 and 5 suggest acceptable fit. Values above 0.9, 0.8–0.89, and 0.7–0.79 for NFI, IFI, TLI, and CFI indicate “excellent,” “good,” and “acceptable” fit, respectively. An RMSEA below 0.08 also signifies an acceptable model fit [35–37].
Convergent and discriminant validity
The assessment of convergent validity was based on composite reliability (CR) and average variance extracted (AVE). Typically, CR > 0.7 and AVE > 0.5 indicate good convergent validity. Discriminant validity was evaluated by comparing the correlation coefficient between factors with the square root of AVE. A correlation coefficient lower than the square root of the corresponding AVE indicates acceptable discriminant validity for the scale.
Tests of reliability
Internal consistency reliability was assessed via Cronbach’s α coefficient. Items were split into odd and even orders, and split-half reliability was evaluated by correlating the divided items. After two weeks, the stability of the scale was assessed among 30 nurses, and a test-retest correlation analysis was conducted to ensure stability and consistency across the entire data collection period. Cronbach’s α coefficient and test-retest reliability exceeding 0.7 were considered desirable [38, 39].
Results
Item development
Delphi expert inquiry results
Ten critical care experts participated in two rounds of inquiry, achieving a 100% questionnaire response rate. The authority coefficient (Cr) of the experts was 0.94. The experts’ ages ranged from 38 to 55 years (mean ± SD: 46.10 ± 5.43), with an average of 25.80 ± 6.96 years of working experience. Among them, 3 held senior professional titles (30%), 7 held deputy senior professional titles (70%), and 7 had attained a graduate degree or higher (70%). Following two rounds of consultations, the Kendall harmony coefficient for the entire scale achieved 0.61, a statistically significant finding as determined by the Chi-square test (P < 0.01), suggesting a high level of consensus among experts regarding this scale. During the initial expert inquiry round, experts recommended the inclusion of four additional items: “Operation of ECMO and coordination of nursing care”, “Operation of the fiberoptic bronchoscope and dialysis machine with nursing coordination”, “B-ultrasound evaluation of gastric and bladder volumes, pulmonary diaphragmatic excursion, and diaphragm integrity”, and “Assessment and prevention of nutritional risks in critically ill patients”. Five experts concluded that the dimension of " critical care ability” overlapped with the central theme of the scale, as all items pertained to basic nursing care. They recommended revising the dimension to " foundational critical care skills”. Following the group discussion, it was agreed that the dimension overlapped with the name of the scale and could lead to potential confusion. Three experts identified the item “Correct identification of common arrhythmias in patients and correct interpretation of abnormal electrocardiograms” as overly complex, recommending its simplification to “Identification of life-threatening arrhythmias.” Following the group discussion, the experts’ recommendation was endorsed. Additionally, items with an average importance score below 3.5, a coefficient of variation exceeding 0.25, and a pass rate below 20% will be excluded. The first round of questionnaires will be analyzed and revised to create a new version, which will then be forwarded to experts for a second round of evaluation.
During the second round of expert inquiry, no objections were raised regarding the three dimensions. Items that were deemed cumbersome or inaccurately defined were revised, including changing “arteriovenous puncture technique” to “Centralized arteriovenous puncture catheterization technique”. The experts’ recommendations were accepted after group discussion. Following two rounds of counseling, the experts’ opinions converged, leading to the termination of the counseling process. Finally, a pilot version of the Critical Care Practical Competency Scale was developed, comprising 3 dimensions and 27 items.
Content validity
All items had an I-CVI ranging from 0.83 to 1.00, exceeding 0.78. S-CVI/Ave was 0.98, and the S-CVI/UA was 0.93, surpassing 0.9, demonstrating excellent content validity of the scale. The overall consistency among the six experts, evaluated using Fleiss’s Kappa, was 0.49, suggesting a moderate level of agreement.
Pilot survey
Based on the results of the pilot study, none of the 30 nurses raised objections regarding the content and expression of the scale, and no adjustments or additions were deemed necessary.
Scale development
Characteristics of the study participants
A total of 850 questionnaires were distributed in the survey, of which 820 were successfully collected, resulting in an effective recovery rate of 96.47%. A total of 30 questionnaires were manually excluded due to identical responses across all items, indicating a lack of attention during the completion of the questionnaire. Respondents had a mean age of 33.21 ± 1.78 years. The sample included 92 males (11.2%) and 728 females (88.8%). Educational backgrounds were represented by 25 college graduates (3.0%), 777 undergraduates (94.8%), and 17 master’s (2.1%). Among the participants, 117 (14.3%) were nurses, comprising 375 chief nurses (45.7%) and 50 deputy chief nurses (6.1%).
Items analysis
Item analysis revealed correlation coefficients ranging from 0.54 to 0.85 between each item and the total scale score, all exceeding 0.40, with no items meeting deletion criteria. Statistically significant differences were found among items in the high and low groups (p < 0.05). The t-values ranged from 18.12 to 41.63, all exceeding 3. No items were identified whose exclusion led to a significant increase in the Cronbach’s α coefficient. Consequently, all 27 items were retained.
Exploratory factor analysis
Results from the exploratory factor analysis (EFA) revealed a high Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy (0.96), along with a highly significant Bartlett’s Test of Sphericity (p < 0.01). The Bartlett test of sphericity yielded a statistically significant result (p < 0.05), affirming its appropriateness for factor analysis, given that KMO scores surpass 0.70 [29]. Principal component analysis with varimax rotation was conducted to determine the number of factors among the 27 items. Factor determination relied on eigenvalues, scree plots, and results from parallel analysis. In conjunction with the scree plot (Fig. 2), the findings suggested the presence of 3 to 5 factors. An optimal structure was observed when the analysis retained three factors. The cumulative variance explained was 73.21%. Five items (T12, T13, T3, T7, T8) exhibited loadings exceeding 0.4 on both factors, and their load discrepancies were less than 0.20, prompting their removal from the analysis. Although the item (T1) “Cardiopulmonary Resuscitation (CPR) Procedure” exhibits common loadings in Factor 1 (0.51) and Factor 3 (0.59), it was retained in Factor 3 due to its distinctive significance as a crucial skill for nursing staff. Returning to the core of the concept, “cardiopulmonary resuscitation (CPR)” is not an isolated individual skill. In clinical practice, it requires coordination within a team, with specific roles assigned to individuals (such as a designated person for chest compressions, another for providing ventilation, and a third for delivering equipment or calling for assistance). It inherently possesses a collaborative nature. Therefore, from a conceptual standpoint, CPR is categorized under the dimension of collaborative rescue.
Fig. 2.
Scree plot of the CCPCS
In alignment with theoretical underpinnings, the first factor was labeled foundational critical care skills (13 items), the second factor denoted proficiency in operating instruments (5 items), and the third factor characterized capability for collaborative rescue (4 items) (Table 1).
Table 1.
Exploratory factor analysis of the CCPCS
| Item | F1 | F2 | F3 | Communality |
|---|---|---|---|---|
| T23 | 0.84 | 0.84 | ||
| T25 | 0.83 | 0.79 | ||
| T19 | 0.83 | 0.79 | ||
| T21 | 0.82 | 0.81 | ||
| T22 | 0.81 | 0.76 | ||
| T26 | 0.79 | 0.76 | ||
| T20 | 0.78 | 0.77 | ||
| T18 | 0.77 | 0.71 | ||
| T27 | 0.76 | 0.75 | ||
| T9 | 0.66 | 0.66 | ||
| T24 | 0.64 | 0.63 | ||
| T16 | 0.59 | 0.61 | ||
| T15 | 0.55 | 0.54 | ||
| T11 | 0.86 | 0.75 | ||
| T14 | 0.82 | 0.72 | ||
| T10 | 0.80 | 0.74 | ||
| T4 | 0.62 | 0.66 | ||
| T17 | 0.61 | 0.63 | ||
| T2 | 0.64 | 0.79 | ||
| T1 | 0.51 | 0.59 | 0.61 | |
| T6 | 0.57 | 0.76 | ||
| T5 | 0.53 | 0.73 | ||
| Eigenvalue | 16.25 | 2.52 | 1.00 | |
| Variance(%) | 35.45 | 21.15 | 16.60 | |
| Cumulative Variance(%) | 35.45 | 56.60 | 73.21 |
Confirmatory factor analysis
Confirmatory factor analysis (CFA) was employed to evaluate the three-factor structure derived from exploratory factor analysis (EFA). The initial confirmatory factor analysis (CFA) model demonstrated χ²/df = 5.12, RMSEA = 0.09, NFI = 0.85, IFI = 0.87, TLI = 0.85, and CFI = 0.87, suggesting an acceptable model fit. After implementing several modifications, the results demonstrated an excellent model fit (χ²/df = 3.52, RMSEA = 0.07, NFI = 0.92, IFI = 0.94, TLI = 0.93, and CFI = 0.94). The standardized factor loading model resulting from confirmatory factor analysis is illustrated in Fig. 3. Each item exhibited a factor loading exceeding 0.40, with all items demonstrating statistical significance (P < 0.05), underscoring the robust structural validity of the CCPCS.
Fig. 3.
The structural equation modeling of the Critical Care Practical Competency Scale (CCPCS) for nurses (n = 410). Note: F1, Foundational critical care skills; F2, Proficiency in operating instruments; F3, Capability for collaborative rescue
Convergent and discriminant validity
The composite reliabilities (CR) for foundational skills in critical care, proficiency in operating instruments, and capability for collaborative rescue were 0.96, 0.84, and 0.85, respectively. Additionally, the average variance extracted (AVE) values of 0.65, 0.52, and 0.59 suggested that the model exhibited strong internal consistency and that the structural dimensions of the CCPCS demonstrated satisfactory convergent validity (see Table 2).
Table 2.
Convergent validity of the CCPCS
| Regression Weights Estimate |
Standard error | P-value | Factored load | AVE | CR | |||
|---|---|---|---|---|---|---|---|---|
| T21 | <--- | F1 | 1 | 0.87 | 0.65 | 0.96 | ||
| T19 | <--- | F1 | 0.93 | 0.04 | 22.83*** | 0.84 | ||
| T25 | <--- | F1 | 0.97 | 0.04 | 22.58*** | 0.84 | ||
| T23 | <--- | F1 | 0.96 | 0.03 | 25.74*** | 0.89 | ||
| T22 | <--- | F1 | 0.82 | 0.03 | 20.91*** | 0.80 | ||
| T26 | <--- | F1 | 1.01 | 0.04 | 23.10*** | 0.85 | ||
| T20 | <--- | F1 | 1.06 | 0.03 | 30.87*** | 0.89 | ||
| T18 | <--- | F1 | 0.79 | 0.04 | 18.99*** | 0.76 | ||
| T27 | <--- | F1 | 1.03 | 0.04 | 24.25*** | 0.87 | ||
| T9 | <--- | F1 | 0.83 | 0.05 | 17.79*** | 0.73 | ||
| T24 | <--- | F1 | 0.97 | 0.05 | 19.22*** | 0.76 | ||
| T16 | <--- | F1 | 0.79 | 0.05 | 16.91*** | 0.70 | ||
| T15 | <--- | F1 | 0.67 | 0.04 | 14.76*** | 0.64 | ||
| T11 | <--- | F2 | 1 | 0.63 | 0.52 | 0.84 | ||
| T14 | <--- | F2 | 1.31 | 0.08 | 15.77*** | 0.71 | ||
| T10 | <--- | F2 | 1.41 | 0.09 | 15.71*** | 0.68 | ||
| T17 | <--- | F2 | 1.71 | 0.14 | 12.41*** | 0.78 | ||
| T4 | <--- | F2 | 1.80 | 0.14 | 12.51*** | 0.79 | ||
| T2 | <--- | F3 | 1 | 0.79 | 0.59 | 0.85 | ||
| T1 | <--- | F3 | 0.47 | 0.04 | 11.01*** | 0.50 | ||
| T6 | <--- | F3 | 1.02 | 0.05 | 20.04*** | 0.88 | ||
| T5 | <--- | F3 | 1.01 | 0.05 | 19.27*** | 0.86 |
Although the correlation coefficient of 0.85 between foundational critical care skills and capability for collaborative rescue exceeds the square root of the average variance extracted (AVE) (0.77), the correlations for the other dimensions are all below the square root of their respective AVEs, indicating that the scale demonstrates reasonable discriminant validity (see Table 3).
Table 3.
Discriminant validity of the CCPCS
| F1 | F2 | F3 | |
|---|---|---|---|
| F1 | 0.77 | ||
| F2 | 0.39 | 0.72 | |
| F3 | 0.85 | 0.47 | 0.81 |
Note: The values in bold font were the square root of AVE
Reliability
The Cronbach’s α coefficient for the scale in this study was 0.97, with a split-half coefficient of 0.91. Thirty nurses were randomly selected for a retest after a two-week interval, yielding a retest reliability of 0.85.
Discussion
This scale is employed to assess the intensive care practice proficiency of nurses in both general wards and ICUs in China. A thorough literature review reveals the absence of comprehensive scales for evaluating the critical care capabilities of nurses, both domestically and globally. A Questionnaire to Assess Nursing Students’ Knowledge in Critical Care, the study enrolled 30 third-year nursing students from the University of Rome “La Sapienza” to validate an assessment instrument designed to evaluate the critical care knowledge of nursing students [17]. However, this instrument was designed to evaluate the effectiveness of various teaching methods, including gamification and role-playing, in comparison to traditional classroom-based teaching strategies. Consequently, it is not suitable for assessing the critical care competencies of clinical nurses currently practicing in the field. The First Aid Quality Assessment (FAQA) tool is centered on emergency interventions for injured patients, grounded in the ABC principle, and is assessed by the rescue personnel upon arrival at the scene. The primary assessment criteria include evaluation of airway management, control of external bleeding, recovery position, and prevention of hypothermia [16]. The FAQA tool is designed as a bystander emergency assessment instrument and is not suitable for evaluating the critical care competencies of ward nurses. The ICU Confusion Assessment Method (CAM-ICU) is a specialized tool developed to assist ICU nurses in the early identification of delirium [18].
A review concludes that there is a pressing need for reliable and effective tools to evaluate nurses’ critical care competencies. The development and ongoing research of effective, user-friendly assessment methods, including digital tools, are essential to address future demands and reduce disparities in global intensive care nursing education. The creation of universal and internationally applicable tools for describing and assessing educational competencies, with the flexibility to adapt to the evolving critical care environment, is essential [19]. The CCPCS employs a 5-point Likert scale, demonstrating high reliability and effectiveness. It serves as an effective tool for evaluating critical care competencies of nurses.
The CCPCS developed in this study exhibits robust scientific rigor. Following a comprehensive literature review and semi-structured interviews with teachers and head nurses from general wards and intensive care units, the research team deliberated on selecting characteristic items and establishing the item pool. Based on the outcomes of two rounds of expert consultation, it is evident that the experts displayed a high level of enthusiasm for participating in the project. The authority coefficient (Cr) of the experts in this study was 0.94, exceeding the threshold of 0.8 (35), signifying both their enthusiasm and credibility, thereby ensuring the reliability of the scale. The Kendall harmony coefficient among the experts was 0.61, with statistical significance (P < 0.01) after two tests, indicating minimal differences in expert opinions that generally converged.
The CCPCS demonstrates robust reliability and validity. Item analysis results indicated that all items correlated positively (> 0.4) with the total scale score. The differences between items in high and low groups were statistically significant (all P < 0.05). No item showed a significant increase in Cronbach’s α coefficient upon removal. Consequently, the items of the scale demonstrated considerable independence and representativeness, as well as a high degree of discriminative ability. The reliability analysis revealed a Cronbach’s α coefficient of 0.97 and a split-half coefficient of 0.91, both exceeding the threshold of 0.8, indicating excellent internal consistency of the scale. Additionally, the test-retest reliability coefficient was 0.85, demonstrating substantial stability over time.
The content validity results of this scale reveal that the I-CVI ranges from 0.83 to 1.00, exceeding the threshold of 0.78, signifying excellent content validity at the item level. The S-CVI/Ave was 0.98, and the S-CVI/UA was 0.93, surpassing 0.9, indicating good average content validity at the scale level. Exploratory factor analysis revealed 3 common factors and 22 items, with a cumulative variance contribution rate of 73.21%. All items demonstrated adequate factor loadings (> 0.50) and communalities (> 0.54), supporting the structural soundness of the scale. According to the item retention criteria, 5 items were deleted. Despite the factor loading difference of item (T1)”Cardiopulmonary resuscitation (CPR) procedure” between common factor 1 (0.51) and common factor 3 (0.59) being less than 0.2, given the critical importance of CPR as a skill that nurses must proficiently perform [40, 41], the research group decided to retain this item. It had been reassigned to common factor 3 due to the core concept of CPR. These three common factors align with the dimensions of the test scale constructed via the Delphi method, with minimal item reassignment, indicating strong construct validity of the scale.
During scale validation, the initial confirmatory factor analysis (CFA) model showed an acceptable model fit. Commonly, correlated residuals are introduced after a baseline model shows inadequate fit, and inspection of modification indices ensues to remedy the fit, necessitating the addition of covariance correlations based on modification indices [42]. According to the modification indices (MI), the initial model was modified in consecutive steps: e12 and e15, e11 and e17, e11 and e14, e10 and e16, e14 and e17, e15 and e20, e16 and e18, e21 and e22, e7 and e6, e7 and e5, e6 and e5, e2 and e1, respectively. The covariances introduced were all confined to the same dimension. Incorporating covariances within the same dimension is a widely used and theoretically conservative approach in modifying the CFA model. Items within the same dimension inherently share underlying latent factors, with residual covariances primarily arising from specific commonalities not fully captured by these factors. Thus, incorporating covariances within the same dimension enhances the model fit to a greater extent. Convergent validity analysis revealed that the composite reliabilities (CR) for foundational skills in critical care, proficiency in operating instruments, and capability for collaborative rescue were all exceeding 0.7. Similarly, the average variance extracted (AVE) values for these dimensions were all surpassing 0.5 [38, 43]. These results indicate strong convergence validity of the scale. In the discriminant validity results, F1(foundational critical care skills) exhibits poor discriminant validity. The square root of the Average Variance Extracted (AVE) was lower than its correlation with the capability for collaborative rescue dimension. This discrepancy may stem from nurses’ proficiency in rescue collaboration, which enhances their performance in basic critical care skills, thereby reducing the distinctiveness of CCPCS in the foundational skills dimension. Alternatively, the T1CPR technique may yield comparable scores in Dimension F1 (foundational critical care skills) and Dimension F3 (capability for collaborative rescue), with its distinguishing characteristics not being notably apparent. Future research should further investigate the underlying factors contributing to the overlap between these two dimensions and clarify the specific dimensions to which these factors pertain, thus enhancing the comprehensiveness of the tool. Overall, however, the three-factor structure of CCPCS was deemed acceptable.
The items developed for this CCPCS are purposefully crafted to evaluate and strengthen nurses’ competencies in critical care practice. It is well-established that nurses in general wards infrequently operate life-saving equipment, such as ventilators and defibrillators. However, through specialized critical care training, they are expected to attain proficiency in the operation of such equipment. All items represent essential content that must be mastered in critical care. They are comprehensive and effectively reflect nurses’ mastery of the critical knowledge and skills required in this field. By utilizing assessment tools, targeted training is implemented to address the specific knowledge gaps and skill deficiencies among nurses, aiming to improve their practical competencies in critical care nursing.
Limitations
Despite these strengths, this study is subject to several limitations. First, the nurse participants were solely recruited from three tertiary hospitals in Anhui, China, which may constrain the broader applicability of the findings. Second, the use of convenience sampling could limit the external validity of the study’s results. Future research should consider conducting a multicenter survey with a larger sample size to establish a national standard and validate the applicability of the CCPCS among clinical nurses.
Conclusions
The Critical Care Practical Competency Scale (CCPCS), developed in this study, comprises three dimensions and 22 items: foundational critical care skills (13 items), proficiency in operating medical instruments (5 items), and capability in collaborative rescue (4 items) (see supplementary material, Additional file 1). The reliability and validity testing results indicate that all dimensions and items meet established measurement standards, establishing the scale as a suitable reference for evaluating nurses’ critical care practical competency. Moreover, hospital nursing managers may implement targeted training programs and evaluations tailored to the specific areas of weakness identified in nurses, thereby optimizing the overall efficacy of training initiatives. Consequently, this scale provides a significant reference for the training and evaluation of nurses’ critical care competencies.
Supplementary Information
Below is the link to the electronic supplementary material.
Acknowledgements
The investigators express their gratitude to the committed participants and all research staff involved in this study.
Author contributions
Conceptualization: Miaomiao Yan, Xiubin Tao, and Dan Qin. Data curation: Miaomiao Yan, Pingping Jiang, Lijuan Zhang, and Zhongtao Zhou. Formal analysis: Miaomiao Yan, Zhongtao Zhou. Investigation: Miaomiao Yan, Lijuan Zhang, Xueli Bao, and Zhongtao Zhou. Methodology: Miaomiao Yan. Supervision: Xiubin Tao, Dan Qin. Writing – original draft: Miaomiao YanWriting – review & editing: Miaomiao Yan, Xiubin Tao, and Zhongtao Zhou.
Funding
This research was funded by the University-level Open Project of the Anhui Provincial Key Research Base of Humanities and Social Sciences (Project No. SJD202304); Management and Service Innovation Project (Project No. CX2024006); Three-new projects (Project No. Y25003).
Data availability
The datasets generated and/or analyzed in this study are not publicly accessible due to privacy concerns; however, they can be obtained from the corresponding authors upon reasonable request.
Declarations
Ethical approval
The study was conducted by the Declaration of Helsinki and approved by the Ethics Committee of Yijishan Hospital of Wannan Medical College. (No. 2023-153).
Informed consent
All participants provided informed consent and remained anonymous throughout the study.
Competing interests
The authors declare no competing interests.
Footnotes
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Jiang M, Sun M, Zhang X, Luan XR, Li RJ. Disaster nursing competency of intensive care nurses in Jinan, china: A multicenter Cross-Sectional study. J Nurs Res. 2022;30(3):e207. 10.1097/jnr.0000000000000492. [DOI] [PubMed] [Google Scholar]
- 2.Su Y, Wang L, Chen T, Liao L, Hu S, Yang Y. Development and validation of the nurse team resilience scale (NTRS) in the context of public health emergencies. BMC Nurs. 2023;22(1):489. 10.1186/s12912-023-01627-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.English M, Oliwa J, Khalid K, Onyango O, Willows TM, Mazhar R, Mkumbo E, Guinness L, Schell CO, Baker T, et al. Hospital care for critical illness in low-resource settings: lessons learned during the COVID-19 pandemic. BMJ Glob Health. 2023;8(11). 10.1136/bmjgh-2023-013407. [DOI] [PMC free article] [PubMed]
- 4.Grasselli G, Pesenti A, Cecconi M. Critical care utilization for the COVID-19 outbreak in Lombardy, italy: early experience and forecast during an emergency response. JAMA. 2020;323(16):1545–6. 10.1001/jama.2020.4031. [DOI] [PubMed] [Google Scholar]
- 5.Monesi A, Imbriaco G, Mazzoli CA, Giugni A, Ferrari P. In-Situ simulation for intensive care nurses during the COVID-19 pandemic in italy: advantages and challenges. Clin Simul Nurs. 2022;62:52–6. 10.1016/j.ecns.2021.10.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Bambi S, Iozzo P, Lucchini A. New issues in nursing management during the COVID-19 pandemic in Italy. Am J Crit Care. 2020;29(4):e92–3. 10.4037/ajcc2020937. [DOI] [PubMed] [Google Scholar]
- 7.Podgorica N, Zenzmaier C, Rungg C, Bertini B, Perkhofer S. Facing the unknown COVID-19 pandemic: A qualitative study with nurses and nursing assistants in three European countries. Front Public Health. 2022;10:985702. 10.3389/fpubh.2022.985702. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Woo BFY, Lee JXY, Tam WWS. The impact of the advanced practice nursing role on quality of care, clinical outcomes, patient satisfaction, and cost in the emergency and critical care settings: a systematic review. Hum Resour Health. 2017;15(1):63. 10.1186/s12960-017-0237-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Global burden and strength of evidence for. 88 risk factors in 204 countries and 811 subnational locations, 1990–2021: a systematic analysis for the global burden of disease study 2021. Lancet. 2024;403(10440):2162–203. 10.1016/s0140-6736(24)00933-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Jawad I, Rashan S, Sigera C, Salluh J, Dondorp AM, Haniffa R, Beane A. A scoping review of registry captured indicators for evaluating quality of critical care in ICU. J Intensive Care. 2021;9(1):48. 10.1186/s40560-021-00556-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Egerod I, Kaldan G, Nordentoft S, Larsen A, Herling SF, Thomsen T, Endacott R. Skills, competencies, and policies for advanced practice critical care nursing in europe: A scoping review. Nurse Educ Pract. 2021;54:103142. 10.1016/j.nepr.2021.103142. [DOI] [PubMed] [Google Scholar]
- 12.Phua J, Faruq MO, Kulkarni AP, Redjeki IS, Detleuxay K, Mendsaikhan N, Sann KK, Shrestha BR, Hashmi M, Palo JEM, et al. Critical care bed capacity in Asian countries and regions. Crit Care Med. 2020;48(5):654–62. 10.1097/ccm.0000000000004222. [DOI] [PubMed] [Google Scholar]
- 13.Connolly E, Kasomekera N, Sonenthal PD, Nyirenda M, Marsh RH, Wroe EB, Scott KW, Bukhman A, Minyaliwa T, Katete M, et al. Critical care capacity and care bundles on medical wards in malawi: a cross-sectional study. BMC Health Serv Res. 2023;23(1):1062. 10.1186/s12913-023-10014-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Schell CO, Khalid K, Wharton-Smith A, Oliwa J, Sawe HR, Roy N, Sanga A, Marshall JC, Rylance J, Hanson C, et al. Essential emergency and critical care: a consensus among global clinical experts. BMJ Global Health. 2021;6(9). 10.1136/bmjgh-2021-006585. [DOI] [PMC free article] [PubMed]
- 15.Kinnunen A, Hagman T, Paakkonen H, Saaranen T. Constructing critical care nursing expertise: an integrative literature review. Nurse Educ Today. 2025;149:106668. 10.1016/j.nedt.2025.106668. [DOI] [PubMed] [Google Scholar]
- 16.Idland S, Kramer-Johansen J, Bakke HK, Hjortdahl M. Assessing bystander first aid: development and validation of a first aid quality assessment (FAQA) tool. BMC Emerg Med. 2023;23(1):39. 10.1186/s12873-023-00811-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Colacci T, Bonavoglia F, Dibattista F, Gargaro V, Taborri S, La Torre G. Validation of a questionnaire to assess nursing students’ knowledge in critical care. Clin Ter. 2025;176(1):105–8. 10.7417/ct.2025.5172. [DOI] [PubMed] [Google Scholar]
- 18.Hebeshy MI, Gaballah S, Elsayed ER. Effect of educational training about CAM-ICU on nurses’ knowledge, confidence, and practice to detect delirium among critically ill patients in intensive care unit. BMC Nurs. 2025;24(1):673. 10.1186/s12912-025-03395-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Øvrebø LJ, Dyrstad DN, Hansen BS. Assessment methods and tools to evaluate postgraduate critical care nursing students’ competence in clinical placement. An integrative review. Nurse Educ Pract. 2022;58:103258. 10.1016/j.nepr.2021.103258. [DOI] [PubMed] [Google Scholar]
- 20.Kang K, Bagaoisan MAP. Research status of the Knowledge-Attitude-Practice theory model in gastric cancer prevention. Cureus. 2024;16(7):e64960. 10.7759/cureus.64960. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Miaomiao Y, Xiubin T, Juan B, Dan Q, Wei L, Gongxiang D. The application and effect of Delphi method in the evaluation system of the core competency for the fracture liaison service coordinator. J MuDanJiang Med Univ. 2021;42(01):119–22. 10.13799/j.cnki.mdjyxyxb.2021.01.029. [Google Scholar]
- 22.Jeon EJ, Sohng KY, Yeom HA. Development and validation of a self-care scale for older adults undergoing hip fracture surgery: the HFS-SC. BMC Nurs. 2022;21(1):197. 10.1186/s12912-022-00982-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Cai Y, Li Y, Zou J, Zhang J, Luo W, Zhang J, Qu C. Cross-cultural adaptation and reliability of the inventory of vicarious posttraumatic growth and research of its influencing factors: a cross-sectional study. BMC Nurs. 2024;23(1):763. 10.1186/s12912-024-02435-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb). 2012;22(3):276–82. 10.11613/BM.2012.031. [PMC free article] [PubMed] [Google Scholar]
- 25.Mohammadi MM, Nayeri ND, Varaei S, Rasti A. Design and validation of the presenteeism scale in nursing. BMC Nurs. 2023;22(1):290. 10.1186/s12912-023-01454-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Yang L, Zhou W, Gao Y, Wu T, Zhang H, Gan X. Development and validation of the missed intensive nursing care scale. BMC Nurs. 2024;23(1):165. 10.1186/s12912-024-01805-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Curtis AC, Keeler C. Measurement in nursing research. Am J Nurs. 2021;121(6):56–60. 10.1097/01.NAJ.0000753668.78872.0f. [DOI] [PubMed] [Google Scholar]
- 28.Li S, Gao Y, Lin Y, Wu W, Fang Q, Ni X, Zhou Y, Hong M, Zhang R, Lou Y. Development and preliminary testing of the cancer-related fatigue comprehensive assessment scale in cancer survivors. J Clin Nurs. 2023;32(7–8):1186–217. 10.1111/jocn.16272. [DOI] [PubMed] [Google Scholar]
- 29.Reay G, Rankin JA, Then KL, Fung T, Smith-MacDonald L. Emergency department triage decision-making by registered nurses: an instrument development study. J Adv Nurs. 2024;80(11):4725–35. 10.1111/jan.16252. [DOI] [PubMed] [Google Scholar]
- 30.Moon S, Chang SJ. Professional socialization of hospital nurses: A scale development and validation study. BMC Nurs. 2023;22(1):2. 10.1186/s12912-022-01169-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Gaskin CJ, Happell B. On exploratory factor analysis: a review of recent evidence, an assessment of current practice, and recommendations for future use. Int J Nurs Stud. 2014;51(3):511–21. 10.1016/j.ijnurstu.2013.10.005. [DOI] [PubMed] [Google Scholar]
- 32.Gázquez-López M, García-García I, González-García A, Martín-Salvador A, Pérez-Morente M, Martínez-García E, Álvarez-Serrano MA. Validation of the attitudes towards people living with HIV/AIDS scale in nursing students. BMC Nurs. 2023;22(1):245. 10.1186/s12912-023-01414-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Yang Z, Chen F, Lu Y, Zhang H. Psychometric evaluation of medication safety competence scale for clinical nurses. BMC Nurs. 2021;20(1):165. 10.1186/s12912-021-00679-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Luo L, Chen X, Xie H, Zhou J, Li L. Development and evaluation of a rehabilitation training compliance scale for patients with urinary incontinence. BMC Nurs. 2023;22(1):147. 10.1186/s12912-023-01326-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Jackson DL, Gillaspy JA, Purc-Stephenson R. Reporting practices in confirmatory factor analysis: an overview and some recommendations. Psychol Methods. 2009;14(1):6–23. 10.1037/a0014694. [DOI] [PubMed] [Google Scholar]
- 36.Atashzadeh-Shoorideh F, Parvizy S, Hosseini M, Raziani Y, Mohammadipour F. Developing and validating the nursing presence scale for hospitalized patients. BMC Nurs. 2022;21(1):138. 10.1186/s12912-022-00896-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Alharbi FS, Aljemaiah AI, Osman M. Validation of the factor structure and psychometric characteristics of the Arabic adaptation of the sense of coherence SOC-13 scale: a confirmatory factor analysis. BMC Psychol. 2022;10(1):115. 10.1186/s40359-022-00826-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Lu X, Wang L, Xu G, Teng H, Li J, Guo Y. Development and initial validation of the psychological capital scale for nurses in Chinese local context. BMC Nurs. 2023;22(1):28. 10.1186/s12912-022-01148-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Neuhaus C, Camathias C, Mumme M, Faude O. The German version of the KOOS-Child questionnaire (Knee injury and osteoarthritis outcome score for children) shows a good to excellent internal consistency and a high test-retest reliability in children with knee problems. Knee Surg Sports Traumatol Arthrosc. 2023;31(4):1354–60. 10.1007/s00167-022-07074-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Neumar RW, Shuster M, Callaway CW, Gent LM, Atkins DL, Bhanji F, Brooks SC, de Caen AR, Donnino MW, Ferrer JM, et al. Part 1: executive summary: 2015 American heart association guidelines update for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation. 2015;132(18 Suppl 2):S315–367. 10.1161/cir.0000000000000252. [DOI] [PubMed] [Google Scholar]
- 41.Alsabri MAH, Alqeeq BF, Elshanbary AA, Soliman Y, Zaazouee MS, Yu R. Knowledge and skill level among non-healthcare providers regarding cardiopulmonary resuscitation (CPR) training in the middle East (Arab countries): a systematic review and meta-analysis. BMC Public Health. 2024;24(1):2081. 10.1186/s12889-024-19575-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Finch WH, French BF, Hazelwood A. A comparison of confirmatory factor analysis and network models for measurement invariance assessment when indicator residuals are correlated. Appl Psychol Meas. 2023;47(2):106–22. 10.1177/01466216231151700. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Chen TT, Hsiao CC, Lu HJ, Chen YJ, Hung CC. The development and validation of a nurse preceptor core competency scale: A culturally compatible methodological study. Nurse Educ Today. 2025;150:106705. 10.1016/j.nedt.2025.106705. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The datasets generated and/or analyzed in this study are not publicly accessible due to privacy concerns; however, they can be obtained from the corresponding authors upon reasonable request.