Designing and psychometric evaluation of safe nursing care instrument in intensive care units

Abstract

Background

Providing safe care in a sensitive and high-risk unit such as the ICU is one of the most crucial tasks for nurses. One way to establish the criteria for safe care is by creating a instrument to assess it. Therefore, this study was conducted with the aim of designing and psychometrically evaluating an instrument for safe nursing care in the ICU.

Methods

The current study employed a sequential-exploratory mixed-method approach with two qualitative and quantitative phases. Based on the results of qualitative phase and the literature review, the primary instrument was designed. In the quantitative phase, the designed instrument underwent psychometric evaluation. Face, content and construct validity were assessed. Face validity was assessed by 20 nurses, and content validity was assessed by 26 experts. In the construct validity stage, the sample size for the exploratory factor analysis (EFA) included 300 nurses, and for the confirmatory factor analysis (CFA) included 200 nurses who work full-time in the ICUs of hospitals affiliated with Kermanshah University of Medical Sciences in western Iran. EFA sampling was conducted in three hospitals, encompassing six ICUs, while CFA sampling was carried out in two hospitals, covering four ICUs. Sampling was done using the convenience method. The reliability of the instrument was also assessed. Finally, the interpretability, feasibility, weighting, and scoring of the instrument were evaluated.

Results

The qualitative phase identified three themes, including professional behavior (with categories: Implementation of policies, organizing communication, professional ethics), holistic care (with categories: systematic care, comprehensive care of all systems), and safety-oriented organization (with categories: human resource management and safe environment). The primary instrument was designed with 107 items rated on a five-point Likert scale. In the quantitative phase, the psychometrics of the instrument were conducted. First, the face and content validity were assessed, and the average scale content validity index (S-CVI) was 0.94. Then, a preliminary test was conducted to assess the initial reliability (α = 0.92) and the correlation of each item with the total score. After completing these steps, the number of items in the instrument was reduced to 52. The results of the EFA explained 58% of the total variance, with 4 factors identified: professional behavior by following guidelines, comprehensive care, accurate documentation, and pressure ulcer care. At the CFA stage, the results of the calculation of indices and goodness of fit showed that the model had a good fit. The reliability of the relative stability by examining the intraclass correlation coefficient (ICC) for the whole instrument in 20 samples was 0.92 with a confidence interval of 0.97 − 0.81. To measure absolute stability and determine the responsiveness of the instrument, the standard error of measurement (SEM) was 4.39 and the minimum detectable change (MDC) was 12.13.

Conclusion

The instrument for safe nursing care in the ICU has favorable psychometric properties.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12912-024-02322-z.

Background

As recently defined by the World Health Organization (WHO) in 2021, patient safety is a framework of organized activities that create cultures, processes, behaviors, technologies, climates, and environments in healthcare organizations. This framework aims to continuously and sustainably identify risks, preventable harm, and reduce the likelihood of their occurrence [1]. According to this organization, unsafe care is a significant contributor to serious medical errors globally and ranks among the top ten causes of death worldwide [2]. According to available reports, 134 million serious incidents of unsafe care have been recorded in low- and middle-income countries. These incidents resulted in 2.6 million deaths per year [3]. There is evidence that patient safety is a global health issue that affects patients worldwide, including both developed and developing countries [4]. Researchers are evaluating interventions, designing health systems, and exploring creative methods to ensure patient safety. At a global level, the World Health Organization has implemented safety interventions in underdeveloped and developing countries [5].

In addition to the importance of safety in hospitals, this aspect becomes even more crucial in intensive care settings [6]. Intensive care units (ICUs) are specialized hospital units that care for critically ill patients. These patients require constant monitoring and specialized care due to their complex medical conditions [7]. The role of ICU nurses is crucial in ensuring patient safety and delivering quality care [8]. They are responsible for providing practical care, monitoring vital signs, administering medications, and assisting with medical procedures [9]. In addition to the acute condition of patients, nurses in these units are also required to deal with the emotional and psychological stress of caring for critically ill patients and their families [10]. However, providing safe care in these units can be challenging due to a number of factors [11]. One of the main challenges for nurses in these units is the requirement for high precision and the management of complex care needs [12]. The special conditions of patients hospitalized in ICUs, the equipment available in these units, and the care techniques used increase the risk of medical errors [13], but development and application of safe care criteria lead to increased patient survival, cost savings, and a reduction in preventable deaths [14]. One way to identify safe care criteria is by developing an instrument to review them. The utilization of patient safety review and assessment criteria not only enhances nurses’ and managers’ recognition and awareness of the current status of nurses’ safe care qualifications but also helps identify their skill and cognitive deficits and deficiencies [15].

Despite the availability of certain instruments for evaluating safe care in Iran [15] and other countries [16–19], these instruments are generic and lack specificity, failing to consider the variations among different care settings. Rashvand et al. (2016) conducted a study in Iran to develop and psychometrically evaluate a safety care instrument. The tool they developed consisted of 41 items with factors including assessment of nursing skills, assessment of psychological needs, assessment of physical needs, assessment of teamwork, and assessment of ethics [20]. Although this study is valuable, the instrument designed is a general tool to assess the safety of hospitalized patients, and it does not specifically address the care of patients in the ICU. Furthermore, since the tool’s design, there have been numerous changes in Iran’s health system structure and safe care standards. Instruments designed in other countries can be useful, but due to differences in the health system, facilities, and equipment, they may not always be compatible. For these reasons and due to the absence of a specific instrument thorough review of previous studies the researchers conducted this study with the aim of designing and psychometrically evaluating a safe nursing care instrument in the ICU.

Methods

Study design

The present study is a sequential-exploratory mixed-methods study conducted with the aim of designing and psychometrically evaluating a safe nursing care instrument in the ICU in Iran from October 2019 to October 2021. This study was conducted in two qualitative and quantitative phases. In the first phase, a qualitative approach was used to provide a comprehensive definition of the concept of safe nursing care in the ICU. In the next phase, a quantitative approach was used to design and psychometrically assess the safe nursing care instrument in these departments [21].

Qualitative phase and item generation

In the first phase of the study, conventional content analysis was utilized to gain a comprehensive understanding of the concept of safe nursing care and its dimensions. Through purposive sampling, in-depth and semi-structured interviews were conducted with 21 participants, including 7 nurses, 2 head nurses, 1 clinical supervisor, 1 nurse responsible for patient safety, 5 intensivists, 2 patients, 1 patient’s family member (patient’s son), 1 patient safety officer in the Ministry of Health, Treatment, and Medical Education, and 1 paramedic. The inclusion criteria for the healthcare personnel involved having a minimum of two years of professional experience in the ICU or in units associated with patient safety. The selection of the two-year threshold was based on the completion of the mandatory manpower plan course and the acquisition of sufficient experience and knowledge. Patients were included if they had a Glasgow Coma Score (GCS) of 15, demonstrated clear speech abilities, and received approval from the ICU intensivist to participate in the interview. Data collection persisted until data saturation was achieved, and no new codes emerged. A concluding interview was carried out to confirm data saturation.

Examples of interview questions for treatment staff include:

1. During each shift, what do you do for your patients?

2. How do you prioritize your nursing care?

3. What measures do you take to ensure patient safety?

4. What unsafe behaviors have you observed during your shift?

Qualitative data analysis was then carried out according to the steps suggested by Lindgeren et al. in 2020 [22]. First, decontextualization was carried out as follows. First, the text of the interviews was transcribed and read multiple times to grasp the main idea. The semantic units were then identified and coded. For recontextualization, the generated codes were compared and grouped into subcategories based on their similarities and differences. In the next step, the categories were extracted from the integration of subcategories, and finally, themes were extracted from the integration of categories. The first author conducted the data analysis, while the other authors reviewed and revised the codes, subcategories, and categories. A definition of the concept under study and its structures was then presented. The initial item pool was extracted by the first author and contained 130 items, 5 of which were extracted from other instruments. The item pool was refined several times by the research team, and a number of items were dropped. Finally, 107 items remained in the item pool.

At the end, the initial instrument was developed by establishing a 5-point Likert scale, with 1 = never, 2 = rarely, 3 = sometimes, 4 = often, 5 = always.

Quantitative phase and item reduction

In the quantitative phase of the study, the psychometric properties of the instrument were assessed according to the Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) criteria. Face validity, content validity, construct validity, and reliability were evaluated. The number of samples in each stage varied, and this is detailed separately in the respective sections.

Face validity

To assess face validity, two qualitative and quantitative methods were used. In this stage, the participants consisted of 20 nurses (10 nurses in qualitative and 10 nurses in quantitative method) from the ICU departments of hospitals affiliated with Kermanshah University of Medical Sciences in western Iran, who met the inclusion criteria. The inclusion criteria required participants to hold a bachelor’s degree in nursing and have less than two years of work experience. These criteria were chosen to ensure that nurses with minimal nursing education and work experience could comprehend the items. The samples were selected using the convenience sampling method.

For qualitative face validity ten nurses were asked to rate the items of the instrument in terms of difficulty, relevance, and ambiguity. For quantitative face validity, another 10 nurses were asked to rate the appropriateness of each item by responding to a Likert scale (completely appropriate = 5, somewhat appropriate = 4, moderately appropriate = 3, slightly appropriate = 2, not appropriate at all = 1). Quantitative face validity was assessed by calculating the items impact score (IIS). The formula (impact score = frequency (%) × suitability) was used to determine the impact score [23]. The acceptable impact score is greater than 1.5.

Content validity

To assess content validity, two qualitative and quantitative methods were used. At this stage, there were 26 expert participants (14 experts in qualitative and 12 experts in quantitative method), including head nurses, nurses in charge of the ICUs, intensivists, and faculty members with experience in working or teaching ICU courses at universities affiliated with Kermanshah University of Medical Sciences in western Iran, who met the inclusion criteria. The inclusion criteria required more than two years of work experience in the ICU department. Sampling was conducted using the convenience method.

Qualitative content validity was assessed with the participation of 14 experts. Adjustments were made to the item arrangement and recommendations were provided for deleting, modifying, and integrating several items. Quantitative investigation of content validity involved 12 experts. The content validity ratio (CVR) was calculated to assess the necessity of the item, while the content validity index (CVI), average content validity index (S-CVI), and adjusted kappa coefficient (K*) were used to evaluate the relevance of the items. The CVR of each item was calculated based on a 3-point Likert scale, which included not necessary = 1, useful but not necessary = 2, and necessary = 3. Considering the number of experts, the cut-off point of the minimum value of the acceptable index in the Lawshe table was 0.56, and items with CVR values less than 0.56 could be removed [24]. To determine the Content Validity Index (CVI) of each item, experts provided feedback on a four-point Likert scale as follows: not relevant = 1, somewhat relevant = 2, relevant but needs revision = 3, and completely relevant = 4. The minimum acceptable level of CVI was considered to be 0.80 according to Waltz and Basel [25]. The SCVI/average approach was used to calculate the SCVI. A value of 0.9 was established as an excellent criterion, while a value of 0.8 was designated as the minimum threshold for SCVI acceptance [23]. To determine the adjusted kappa coefficient, experts provided comments by selecting one of the options: relevant = 1 or not relevant = 0. An adjusted kappa statistic greater than 0.74 was considered excellent, between 0.60 and 0.74 good, and less than 0.60 poor [26]. In the quantitative content validity stage, decisions about the items were made based on the kappa statistic.

In both qualitative and quantitative content validity, experts provided feedback on the comprehensiveness of items using a five-point Likert scale as follows: Not comprehensive at all = 1, Not comprehensive = 2, Somewhat comprehensive = 3, Sufficiently comprehensive = 4, and Very comprehensive = 5.

Item analysis

This stage involves the initial evaluation of the instrument in the target community and is conducted before assessing construct validity. In the item analysis stage, each item was examined to check the initial reliability, the correlation between the items, and the correlation of each item with the total score. In this stage the samples included 50 nurses of the ICU departments of hospitals affiliated with Kermanshah University of Medical Sciences in western Iran and paper questionnaires were prepared and distributed to head nurses and charge nurses to complete the performance of the nurses under their supervision in ICUs. Sampling was conducted using the convenience method.

Construct validity

Samples

Samples in the construct validity stage consisted of nurses from ten ICU departments of five hospitals affiliated with Kermanshah University of Medical Sciences in western Iran. Exploratory factor analysis sampling (EFA) was conducted in three hospitals, encompassing six ICUs, while confirmatory factor analysis (CFA) sampling was carried out in two hospitals, covering four ICUs. The inclusion criteria were full-time employment in the ICU, not partial. Sampling was conducted using the convenience method.

The instrument developed in this study is designed to assess the performance of nurses in delivering safe care in ICUs. In most cases, the responsibility of evaluating nurses’ performance lies with head nurses and charge nurses. Therefore, the questionnaires are completed by head nurses and charge nurses about nurses under their supervision.

Measures

During the exploratory factor analysis stage, the online instrument was developed using Google Forms. Participants were then sent the URL link to access the scale through email or various social media platforms like WhatsApp or Telegram. Due to the non-completion of some samples and the prolonged sampling process at this stage, paper questionnaires were prepared to conduct confirmatory factor analysis.

At construct validity stage, the instrument used had two parts. The first part focused on the demographic characteristics of the participants, including age, gender, marital status, education, and work experience, particularly in the ICU. The second part also included 42 items related to assessment of nurses safe care in the ICU.

Construct validity was tested using exploratory and confirmatory factor analysis. The five-step guide by Williams et al. in 2010 was used to conduct the exploratory factor analysis (EFA). The sample size in factor analysis depends on the number of items in the designed instrument, and different sources consider at least 3–10 samples per item as acceptable [27]. The sample size in the EFA included 300 nurses of three hospitals, encompassing six ICUs. Initially, 300 online questionnaires were sent to head nurses and charge nurses. After one month and several follow-ups, 230 feedback responses were received, resulting in a response rate of 76.6%. To reach the desired sample size, an additional 70 online questionnaires were sent out, and the required sample was achieved. The questionnaires data were entered into SPSS software version 26.

The appropriateness of the sample was assessed using the Kaiser-Meyer-Olkin (KMO) sampling index, and KMO values above 0.9 were deemed excellent. The operability of the data was checked using Bartlett’s sphericity test. Experts have considered the significance of Bartlett’s sphericity test to indicate the factorability of the data [28]. The maximum likelihood method was also used to conduct factor analysis and extract factors [29]. For factor extraction, the Kaiser criterion, the scree plot, the percentage of variance of each factor, and the cumulative variance explained by all extracted factors were used. Factors with an eigenvalue greater than one, factors beyond the horizontal line indicated by the scree plot, and factors that account for at least 50% of the variance in the desired concept by the extracted factors were considered for evaluation [28]. The four-factor model with promax rotation also generated factors that offered the most accurate interpretation and aligned with the qualitative findings of the study. In addition, 58% of the variance in the desired concept was explained by the extracted factors. The minimum acceptable factor loading for the items was considered to be 0.3. Finally, the factors were named according to the common meaning of the items within them.

In the next step, the instrument was evaluated using CFA. The sample size in the CFA stage included 200 nurses of two hospitals, covering four ICUs. In this stage, 200 Paper questionnaires were distributed among head nurses and charge nurses. The aims of the study were explained, and written consent was obtained. After one week, 182 completed questionnaires were collected, resulting in a response rate of 91%. To reach the calculated sample size of 200, 18 additional questionnaires were completed. The questionnaires data was entered into SPSS software version 26 and LISREL version 8.8 software was used.

The most common indicators used to check model fit in CFA include Goodness of Fit Index (GFI), Adjusted Goodness of Fit Index (AGFI), Root Mean Square Error of Approximation (RMSEA), Comparative Fit Index (CFI), Incremental Fit Index (IFI), Non-Normed Fit Index (NNFI), Normed Fit Index (NFI), Relative Fit Index (RFI), and Parsimony Normed Fit Index (PNFI) were also calculated.

Reliability

The reliability of the instrument was assessed by examining internal consistency and stability (both relative and absolute). Internal consistency was evaluated by calculating the Cronbach’s alpha coefficient in a sample of 300 individuals (EFA samples). Cronbach’s alpha was computed for the entire instrument and individually for each factor. Although an alpha value of 0.7 is acceptable, some experts have recommended an alpha value of 0.8 and above [30]. In the present study, alpha values greater than 0.8 were considered.

Relative stability was assessed by calculating the intraclass correlation coefficient (ICC). The developed instrument was administered to 20 head nurses and charge nurses, and the same sample completed it again after two weeks for calculating the Intraclass Correlation Coefficient (ICC). The closer the ICC is to one, the greater the reliability [24]. In this study, the Intraclass Correlation Coefficient (ICC) was considered to be 0.8–0.9, indicating good reliability [30] .

Absolute stability was assessed by calculating the standard error of measurement (SEM) and the minimum detectable change (MDC); these two indicators also reflect the responsiveness of the instrument. The SEM was calculated using the formula SEM = SD × √(ICC-1), and the MDC was calculated using the formula MDC = SEM × z × √2 [31]. The preparation steps are shown in Fig. 1.

Fig. 1.

Steps in the development and psychometric evaluation of safe nursing care instrument in ICUs

Interpretability

To determine interpretability, the researchers used the following criteria: the percentage of unanswered items, the adequacy of the sample size, the distribution of total scores in the samples, and the identification of ceiling and floor effects [32]. The feasibility of the instrument was assessed by calculating two criteria: response time to the instrument and the percentage of forgotten items. To rank the instrument, the items were weighted, and the final instrument was scored.

Data analysis

Data analysis was conducted using SPSS version 26 software and LISREL version 8.8 software .

Ethical considerations

Written permission to conduct the research and approval from the ethics committee were obtained from Tehran Islamic Azad University of Medical Sciences with the code IR.IAU.TMU.REC.1399.481. The research objectives were explained to the participants, written informed consent was obtained from all participants and samples studied. Participants were informed of their voluntary participation in the research and their right to withdraw from the study. The research was conducted with respect for individuals, justice, and responsibility.

Results

Qualitative phase and item generation

The mean age of the participants was 37.47 years, and the mean experience of the healthcare professionals was 17.23 years. Half of the participants were female. The mean duration of the interviews was 36.42 min. The maximum interview duration was 80 min, and the minimum interview duration was 20 min.

After analyzing the data and reviewing the codes multiple times, the research team removed duplicates and merged similar cases. As a result, the number of codes extracted from the interviews decreased from 1997 to 1770, which were then categorized into 43 subcategories. In the next step, seven categories were merged, resulting in three themes derived from the merging of the categories. These themes include professional behavior (with categories: Implementation of policies, organizing communication, professional ethics), holistic care (with categories: systematic care, comprehensive care of all systems), and safety-oriented organization (with categories: human resource management and safe environment). In addition to utilizing the results of the qualitative section, patient safety instruments were also employed to compile the items. The initial pool of items was extracted by the first author, which included 130 items, of which 5 items were taken from other instruments related to patient safety. The pool of items was reviewed by the second author, and based on the comments provided, 17 items were removed due to similarities and inadequacy. Then the remaining items were reviewed by the third author, who suggested removing a number of other items. Finally, 107 items remained in the item pool, 102 items were taken from interviews and 5 items were taken from patient safety tools. Regarding the items taken from patient safety tools, 3 items were related to the subcategory of professional ethics and 2 items were related to the subcategory of organizing communication.

Quantitative phase and item reduction

Face validity

In the qualitative face validity stage, the items were assessed for ease of understanding and the potential for misunderstanding or ambiguity in the wording. Fortunately, the participants encountered no issues, and the items did not require any modifications. At the quantitative face validity stage, all items had an IIS greater than 1.5. Thus, all items were retained.

Content validity

During the qualitative content validity phase, the order of the items was adjusted based on the suggestions of the experts. At this stage, some items were modified, 20 items were merged and 2 items were eliminated due to overlap with other items and the number of items was reduced from 107 to 85. In the quantitative content validity phase, after calculating the CVR, 25 items were excluded. Also, in relation to the CVI values, since all items had a CVI greater than 0.8, no items were excluded and the number of items was reduced to 60. The average Scale Content Validity Index (S-CVI) was 0.94. Finally, an instrument with 60 items was obtained.

Item analysis

A preliminary test was conducted. At this stage, the instrument was administered to 50 head nurses and charge nurses in ICUs. The data analysis showed that the Cronbach’s alpha of the entire list was 0.92. Then, the correlation of the items with the entire instrument was checked using the correlation loop method, and the correlation between the items was examined using the correction factor method. At the end of this stage, 8 items were removed and a safe nursing care instrument with 52 items was prepared for construct validity.

Demographic characteristics of the participants

300 nurses, with a mean age of 31.35 ± 4.61 years and a mean work experience of 9.43 ± 5.59 years, participated in the exploratory factor analysis of this study. More of the sample had a bachelor’s degree (80.7%). In addition, the majority of them were female (73.3%) [Table 1].

Table 1.

Demographic characteristics of research samples in construct validity and exploratory factor analysis (N = 300)

Variable	Category	N (%)	Mean ± SD
Age (yrs)	-	-	31.35 ± 4.61
Experience (yrs)	-	-	9.43 ± 5.59
Experience in ICU (yrs)	-	-	6.40 ± 4.30
Marital status	Single	130(43.3)	-
	Married	161(53.7)	-
	Divorced	3(3)	-
Education level	Bachelor	242(80.7)	-
	Master or PhD	58(19.3)	-
Gender	Female	220(73.3)	-
	Male	80(26.6)	-
Type of shift	Fixed	50(16.7)	-
	Rotational	250(83.3)	-

In the confirmatory factor analysis 200 nurses participated with a mean age of 30.16 ± 5.89 years and a mean work experience of 6.33 ± 3.34 years. Like exploratory factor analysis more of the sample had a bachelor’s degree (84.8%) and the majority of them were female (69.7%) [Table 2].

Table 2.

Demographic characteristics of research samples in construct validity and confirmatory factor analysis (N = 200)

Variable	Category	N (%)	Mean ± SD
Age (yrs)	-	-	30.16 ± 5.89
Experience (yrs)	-	-	6.33 ± 3.34
Experience in ICU (yrs)	-	-	5.40 ± 2.30
Marital status	Single	68(34)	-
	Married	130(65)	-
	Divorced	2(1)	-
Education level	Bachelor	150(75)	-
	Master or PhD	50(25)	-
Gender	Female	135(67.5)	-
	Male	65(32.5)	-
Type of shift	Fixed	32(16)	-
	Rotational	168(84)	-

Construct validity

At the exploratory factor analysis stage, the KMO index was 0.970, and Bartlett’s sphericity test (BT) was statistically significant (p < 0.001). Four factors extracted from the factor analysis using the maximum likelihood method with Promax orthogonal rotation, with eigenvalues greater than 1, explaining 58% of the total variance. The first factor accounted for 50.43%, the second for 3.69%, the third for 2.48%, and the fourth for 1.73% of the total variance. This resulted in an instrument with 42 items and four factors [Table 3].

Table 3.

Results of exploratory factor analysis of safe nursing care instrument in ICU (N = 300)

Factors	Qn	Factor loading
Professional behavior by following the guidelines	17- The nurse uses Venous Thromboembolism Risk Assessment Scale to assess possibility of deep vein thrombosis.	0.836
	8- The nurse uses physical restraint alternately.	0.749
	7- The nurse physically restrains the patient according to the hospital’s standard instructions.	0.721
	6- The nurse uses the physical or chemical (pharmacological) restraint on the basis of the doctor’s order.	0.650
	3- The nurse is aware of the side effects of the medication.	0.636
	19- The nurse uses personal protective equipment such as gowns, masks, gloves, glasses and hats as required.	0.591
	22- To identify the patient, the nurse checks the wristband according to the hospital’s instructions.	0.588
	18- The nurse performs proper hand hygiene	0.577
	1- The nurse performs procedures such as sounding, gavage, suction, enema, dressing changes, etc. according to standard principles.	0.562
	37- The nurse checks the patient’s dietary pattern	0.532
	43- The nurse carries out the oral hygiene of the patient with a tooth brush, mouth wash, etc.	0.531
	2- The nurse administers the medication according to the 8 correct principles of medication administration.	0.528
	15- The nurse uses the fall risk assessment scale to assess the patient’s potential for falls.	0.511
	12- The nurse pays attention to peripheral and central venous catheter complications such as phlebitis and thromboembolism.	0.473
	33- The nurse pays attention to the pattern of sleep and rest of the patient.	0.443
	4- The nurse has the ability to perform medication calculations.	0.440
	5- The nurse administers the blood infusion according to the Haemovigilance chain.	0.416
Comprehensive care	26- In mechanically ventilated patients, the nurse is alert to the warning of increased airway pressure.	0.853
	30- The nurse checks the changes in the level of consciousness of the patient.	0.832
	28- The nurse responds to device alerts in a timely and accurate manner.	0.815
	34- The nurse checks the heart rate and rhythm of the patient.	0.789
	29- Nurse adjusts ventilator alarm settings to safe range	0.730
	25- The nurse adjusts the ventilator settings for mechanically ventilated patients based on the patient’s ventilatory status and the clinician’s instructions.	0.689
	27- For mechanically ventilated patients, the nurse ensures bilateral chest ventilation.	0.688
	23- The nurse is correctly managing the patient’s airway	0.596
	35- The nurse closely monitors the vital signs of the patient.	0.563
	16- Before removing the patient from the bed, the nurse pays attention to the vital signs of the patient.	0.554
	24- The nurse checks the patient’s breathing status (rhythm, number and blood oxygen saturation levels).	0.516
	31- The nurse recognizes symptoms of delirium in time.	0.505
	20- The nurse provides nursing care to the patients without interruption (handover of the patient to other colleagues when the patient is required to leave)	0.477
	13- The nurse pays attention to the patient’s pain symptoms.	0.466
	11- The nurse checks the function of connections, including airways (tracheal tube and tracheostomy), drains, catheters, ostomies and digestive tubes.	0.458
	14- The nurse assesses the effectiveness of pain control interventions.	0.449
	10- The nurse checks the effectiveness of the medications used in chemical restraint in terms of the patient’s clinical symptoms.	0.408
Accurate documentation	46- The nurse records documents correctly (initial assessment, preoperative care, handover, flow sheet and medication administration records).	0.823
	45- The nurse follows the instructions of the telephone orders.	0.753
	44- The nurse writes a nursing report on the basis of the ICU nursing report format.	0.705
	48- The nurse informs the doctor about the changing condition of the patient.	0.568
	47- The nurse explains to the patient or her companion about the complications of procedures that require informed consent.	0.405
Pressure Ulcer Care	42- The nurse takes care of the pressure injuries in a timely manner.	0.812
	41- The nurse prevents pressure injuries.	0.769
	40- The nurse uses the hospital’s standard scale to assess the patient’s risk of developing pressure ulcers.	0.461

Factors were named according to the common meaning of the items in each factor. For example, the first factor was named “Professional behavior by following guidelines “, the second “comprehensive care”, the third “accurate documentation” and the fourth “pressure ulcer care”. Based on the results of the EFA stage, a large number of items from the implementation of policies category led to the formation of a factor that the researchers named professional behavior by following the guidelines. The second factor was named comprehensive care. Despite the fact that a large number of the items in this factor were taken from the implementation of policies category and some from the holistic care theme, this name was chosen for this factor because of the breadth of the concept of comprehensive care. The third factor is accurate documentation. This factor has 5 items. 4 items are taken from the category of organizing communication, and the item that was related to providing information to the patient and the patient’s family before obtaining informed consent is related to the category of the implementation of policies. However, it can be said that the allocation of items to categories is based on the opinion of the researchers and is selective. Moreover, in the qualitative phase of this study, many items could be changed and moved to other categories. In relation to the fourth factor, Pressure Ulcer Care, the items in this factor relate to the prevention, care and treatment of pressure ulcers. These items are derived from the category of policy compliance and comprehensive care of systems, but the formation of a specific factor shows the importance of pressure ulcers in ICUs.

At the CFA stage, the results of the calculation of indices and goodness of fit indicated that the model had a good fit. The Chi-square index was 2499.13 with 813 degrees of freedom (P < 0.001). The indices used to assess the model fit and their corresponding values are listed in Table 4.

Table 4.

Results of model fit indices in confirmatory factor analysis (N = 200)

Fit Index	Acceptable range	Result
Minimum Discrepancy Function by Degrees of Freedom divided (CMIN/DF)	Good < 3 Acceptable < 5	3.25
Goodness of Fit Index (GFI)	0.9<	0.61
Adjusted Goodness of Fit Index (AGFI)	0.8<	0.57
Root Mean Square Error of Approximation (RMSEA)	Good < 0.08 Medium 0.08 to 0.1 Weak < 0.1	0.10
Comparative Fit Index (CFI)	0.9<	0.95
Incremental Fit Index (IFI)	0.9<	0.95
Non-Normed Fit Index (NNFI)	0.9<	0.95
Normed Fit Index (NFI)	0.9<	0.93
Relative Fit Index (RFI)	0.9<	0.93
Parsimony Normed Fit Index (PNFI)	0.5<	0.88

According to Jaccard and Wan (1996), if at least three indicators have acceptable values, we can conclude that the model fit is acceptable [33]. Based on this, and according to the values reported in Table 3, the minimum value of three indicators is appropriate, and therefore, the model fit is acceptable.

Reliability

Cronbach’s alpha coefficient was used to assess the internal consistency. The alpha value for the entire instrument was 0.97, for the first factor 0.94, for the second factor 0.94, for the third factor 0.89, and for the fourth factor 0.86, indicating a high level of internal consistency for both the instrument and all its factors. The ICC for the entire instrument was 0.92, with a confidence interval of 0.81–0.97. To measure the absolute stability and determine the responsiveness of the instrument, the SEM was 4.39 and the MDC was 12.13.

Interpretability

The ceiling and floor effect index was zero for the entire instrument and for factors 1, 2, and 4, and only 10% for factor 3. Examining the distribution of scores in various sample groups also demonstrated the instrument’s capability to detect differences between different groups in relation to the measured construct.

Feasibility

The average response time for the designed instrument was approximately 5.35 min in a sample of ten participants. The minimum response time was 4 min, and the maximum was 8.40 min. Considering the number of items in the list (42 items), the response time to the instrument appears reasonable. Moreover, since the questionnaire was distributed to the participants online via an electronic link, most of them managed to answer all the items. Only five questionnaires were answered, but they were considered biased because a large number of questions were left unanswered. These questionnaires were excluded from the study, and supervisors completed five new questionnaires.

Weighting

The ranking of the instrument was determined by assigning weights to the items. The results showed that the items with the highest weight were placed in the ‘comprehensive care’ dimension, indicating that this dimension was the most significant among the dimensions of the instrument in explaining the factors of safe nursing care.

Scoring

Finally, the instrument was scored using a 5-point Likert scale, which included the options of never, rarely, sometimes, often, and always. The scoring system ranged from 0 to 100, with the score for each option being never 1, rarely 2, sometimes 3, often 4, and always 5. With a total of 42 items, the total points achievable range from a minimum of 42 (if all items are answered as never) to a maximum of 210 (if all items are answered as always). A higher score on the scale indicated safer nursing care.

Discussion

Based on the results of the qualitative part of this study, the concept of safe nursing care in intensive care comprises three themes: professional behavior, holistic care, and safety-oriented organization. After completing the design and psychometric procedures, an instrument with 42 items and four factors was obtained. The factors extracted from the exploratory factor analysis using Promax rotation, explaining 58% of the total variance, were professional behavior by following guidelines, holistic care, accurate documentation, and pressure ulcer care. Internal consistency was assessed using Cronbach’s alpha coefficient, which was 0.97 for the entire instrument. The ICC for the entire instrument was 0.92, with a confidence interval of 0.81–0.97. In order to measure the absolute stability and determine the responsiveness of the instrument, the SEM was 4.39, and the MDC was 12.13. These values indicate the desirable psychometric properties of this instrument.

In this instrument, the first factor is referred to as professional behavior, which involves adhering to the guidelines. Based on the results of this study, this factor was identified as the most important aspect of safe nursing care in the ICU, consisting of 17 items. In the ICU, there are several procedures that must be performed correctly. Neglecting or making mistakes in performing these procedures seriously endangers the patient’s safety. Many patients admitted to these units are dependent on ventilators, and numerous procedures are performed on them. All these procedures are complex and require a high level of mental ability and skill on the part of the nurse. Based on the results of current research, safe procedures require adherence to the guidelines designed for them. However, many nurses rely on their personal skills and experience, which may compromise patient safety. The results of the research by Al-Omar et al. in Saudi Arabia in 2019 showed that nurses’ reliance on personal experience may compromise patient safety [34].

The results of Baccolini et al.‘s research in 2019 in Italy showed that the implementation of procedures to adhere to health standards in special care units was significantly influenced by following the established policies [35]. In this regard, Bignami et al. have argued, based on the results of their research in 2023 in Italy, that adhering to the policy is a secure approach to delivering clinical care, ensuring patient safety, and enhancing the quality of care in intensive care units [36]. Williams et al., in their study in the United States, concluded that adherence to guidelines can also result in a quicker diagnosis of sepsis [37]. However, it has been argued that adhering to guidelines may compromise the autonomy of the nurse, and that the nurse may not be able to effectively manage situations that were not anticipated in the guidelines [38]. For this reason, it seems that the nurse should be able to recognize and prepare for possible cases outside the procedure, in addition to following the guidelines set for the implementation of each procedure.

The second factor is comprehensive care, which also includes 17 items. The results of this study highlight the significance of advocating for safe nursing care in ICUs through a comprehensive care approach. These findings support previous research emphasizing the importance of systems thinking and safe care in enhancing patient safety and overall quality of care [39]. The results of this study emphasize that comprehensive care, which includes addressing the physical, emotional, social, and spiritual needs of patients, is valuable in ICUs. Implementing a comprehensive care model can lead to improved outcomes and patient satisfaction, as well as increased nurse satisfaction and reduced job burnout [40]. By focusing on the whole person, nurses can provide more personalized and effective care, ultimately contributing to a safer environment for patients and staff [41].

The third factor is accurate documentation, which consists of 5 items. The registration of accurate reports is essential for evaluating treatment and care measures. It is also the most effective document for judicial authorities, research affairs, supervisory affairs, and educating students. According to studies, out of every four cases of professional negligence, one case is related to incorrect registration of nursing reports and only 21.4% of nurses demonstrated proficiency in report writing [42]. A review study by McCarthy et al. found that the utilization of electronic reporting and documentation systems in acute care hospitals not only saves time but also reduces errors, as well as the risk of patient falls and infections [43].

The fourth factor is pressure ulcer care. Nurses are responsible for providing continuous and direct care in the prevention and management of pressure ulcers. To achieve quality and optimal care, they need to have evidence-based knowledge and practice. Risk assessment is recommended as the initial step in preventing pressure ulcers in nursing care. The reason for the poor quality of care regarding pressure ulcers is attributed to nurses’ lack of knowledge, absence of accountability, and inadequate monitoring of their performance. It is recommended that a systematic assessment be conducted at the patient’s bedside upon admission or whenever a change is feasible. The patient’s condition should be observed using a validated instrument [44]. In the current instrument, three items, which constitute the fourth factor of this instrument, pertain to assessment and nursing interventions regarding pressure ulcers.

In addition, the following table briefly compares patient safety instruments with the instrument designed and psychometrically tested in the present study based on the COSMIN criteria [Table 5].

Table 5.

Comparison of patient safety instruments with SNCI-ICU

Instrument	Scale development			Measurement properties
	General design requirements and development	Content validity	Comprehensibility	Internal consistency	stability	Measurement error	Criterion validity	Construct validity	Responsiveness	Cross-cultural validity
SNCI-ICU (Safe Nursing Care Instrument in theICU)	Interviews and literature review	✓	✓	✓	✓	✓	-	✓	✓	-
PS-ASK (patient safety attitudes, skills, knowledge ) Schnall et al. 2008 (USA) [19]	Using the “Reason” Model of human error	✓	-	✓	✓	-	-	✓	-	IN English version
NASUS (Nurses’ Attitudes and Skills around Updated Safety Concepts) Armstrong et al. 2017(USA) [17]	PS-ASK instrument And Health Professions Patient Safety Assessment Curriculum Survey (HPPSACS)	✓	-	✓	-	-	-	-	-	IN English version
NPPSS (Nursing Performance for Patient Safety Scale) Panthulawan et al. 2016 (Tailand) [18]	literature review	✓	✓	✓	-	-	-	✓	-	-
Safe Nursing Care Evaluation Questionnaire Rashvand et al. 2016 (Iran) [20]	Interviews and literature review	✓	-	✓	✓	-	-	✓	-	-
IHT Intrahospital Transport Lina Bergman et al. 2020 (Sweden) [16]	Systems Engineering Initiative for Patient Safety (SEIPS) and Observing the performance of nurses	✓	-	✓	-		-	✓	-	✓

In the table, the abbreviation SNCI-ICU (Safe Nursing Care Instrument in the ICU) refers to the instrument designed in the present study.

Comparing the process and methods of instruments

PS-ASK

It is a sequential exploratory combined study comprising two qualitative and quantitative phases. In the qualitative phase, researchers utilized studies on patient safety, Reason’s human error model, and the curriculum of the Columbia University School of Nursing in New York to extract items. However, the process of extracting items from the human error model, studies, and curriculum is not detailed, including the number of items extracted from each source and the methodology used. Additionally, no information is provided about the human error model.

NASUS

In this study, researchers extracted items and designed new tools utilizing PS ASK and HPPSACS instruments.

NPPSS

It is a sequential exploratory mixed-methods study comprising two phases - qualitative and quantitative. In the qualitative phase, the researchers solely relied on a literature review for data collection.

Safe nursing care evaluation questionnaire

This study is a sequential exploratory combined research comprising two phases - qualitative and quantitative. The qualitative phase involved interviews and literature review.

IHT

A sequential exploratory combined study with two qualitative and quantitative phases was utilized in the qualitative phase for literature review and observation of nurses’ performance.

In the design of the SNCI-ICU tool, a sequential exploratory combined method was carried out in two qualitative and quantitative stages. In addition to qualitative interviews with nurses, doctors, staff related to patient safety, patients, and patients’ family, the researcher also benefited from an extensive literature review. This review enriched the subject pool and covered all aspects of safe nursing care in the ICU department.

Comparison of Subscales of instruments

PS-ASK

It is a self-report instrument on knowledge, attitudes and skills and has 7 subscales and 26 items. Attitude subscales include error detection (4 items), time saving (2 items), and safety culture (3 items). Skill subscales include error analysis (6 items), threat to patient safety (3 items), and decision support technology (4 items). The knowledge subscales include 4 items. It seems that the number of 24 items is not sufficient to assess knowledge, skills and attitudes and that more items may be more useful. Despite the usefulness of self-report insruments for checking knowledge and attitudes, they do not seem to be as effective for checking skills.

NASUS

This tool is a self-report instrument and has two subscales and 24 items. The subscales include skill (7 items) and attitude (17 items).

NPPSS

It has 4 subscales and 64 items. The subscales include protection through communication (5 items), protection through risk management (11 items), protection through correct prescription of medications and solutions (4 items), and protection through the implementation of policies (4 items).

Safe nursing care evaluation questionnaire

This instrument consists of 5 subscales and 41 items. The subscales include assessment of nursing skills (16 items), assessment of psychological needs (8 items), assessment of the patient’s physical needs (8 items), assessment of nurses’ teamwork (5 items), and assessment of ethics (4 items).

IHT

This insrument comprises 5 subscales and 25 items. The subscales consist of organization (6 items), tools and equipment (5 items), transfer duties (4 items), environment (5 items), and teamwork (4 items). These subscales align with the dimensions outlined in the theoretical model of innovative engineering for patient safety, which the researchers utilized during the qualitative phase for concept definition and item generation.

In the present study, the SNCI-ICU comprises 42 items and 4 subscales. These subscales consist of professional performance following policies (17 items), comprehensive care (17 items), accurate documentation (5 items), and pressure ulcer care (3 items).

Comparison of psychometric properties based on COSMIN’s Criteria

PS-ASK

In this study, a Cronbach’s alpha value of 0.86 was reported in the internal consistency review, but no information was provided regarding the SEM and interpretability. Therefore, based on Cosmin’s criteria, the tool designed in the present study exhibits more comprehensive psychometric properties.

NASUS

In this study, the researchers only investigated content validity and internal consistency. Cronbach’s alpha values were reported as 0.67 for the attitude items, 0.71 for the skill items, and 0.73 for the entire instrument. The reliability of the attitude items is at an average level, indicating a need for item revision. Other measures of COSMIN, such as construct validity, criterion validity, stability and SEM have not been explored.

NPPSS

Content validity and construct validity have been assessed through EFA. The internal consistency review reported a Cronbach’s alpha value of 0.91. However, no information was provided regarding other criteria such as stability, SEM and responsiveness.

Safe nursing care evaluation questionnaire

In this study, content validity was assessed first, followed by structural validity using exploratory factor analysis. The internal consistency check reported a Cronbach’s alpha value of 0.97, and the tool stability check showed an ICC of 0.77.

IHT

In this study, content validity cross cultural validity and structural validity were assessed using exploratory factor analysis. Cronbach’s alpha ranged from 0.72 to 0.82 for each subscale, but no information was provided regarding measurement error and responsiveness.

In the present study, after establishing content validity, construct validity was assessed through EFA and CFA. Internal consistency was evaluated using Cronbach’s alpha, which was found to be 0.97. The relative stability was assessed by examining the ICC for the entire instrument and individually for each factor. The ICC for the entire instrument was 0.92, with a confidence interval of 0.81–0.97. To measure the absolute stability and determine the responsiveness of the instrument, the SEM and the MDC were calculated. They were found to be 4.39 and 12.13, respectively, for the entire instrument. All these features demonstrate that the instrument designed in the present study is not only specific for evaluating safe nursing care in the ICU but also possesses desirable psychometric properties.

Implications for practice

The use of a safe nursing care instrument is effective and practical not only for nursing assessors but also for nurses working in ICU departments. It can serve as a guide. In the field of education, utilizing the factors identified in this instrument, which are specific to ICU departments, can lead to more effective training for students and nurses. Additionally, it is beneficial in the field of management and can be used by nursing managers as a valid and reliable tool to assess safe nursing care in ICU departments. The evaluations generated can pinpoint areas where nurses may need improvement. Furthermore, researchers can use this instrument to assess safe nursing care in the ICU in descriptive studies. The results from the evaluations can provide fundamental data for decision-making by policymakers and safety planners.

Conclusion

The instrument for safe nursing care in the ICU has favorable psychometric properties based on the studies conducted using the Consensus based standards for the selection of health measurement instrument (COSMIN) criteria. The items in this instrument are derived from the results of a qualitative study and an extensive literature review. The outstanding feature of this instrument is its exclusivity to the ICU. On this basis, it can be said that this instrument is valid for evaluators and nurse managers to assess safe nursing care in the ICU.

Research limitations

In the present study, certain types of validity like convergent, divergent, comparison in known groups and experimental were not conducted, but they could be explored in future studies. Convergent validity was not achievable due to the absence of specific tools related to safe care in the ICU. However, this study has paved the way for other researchers to utilize this tool in their own studies.

Another limitation is the design of the instrument in the Persian language, which would necessitate translation and validation for cross-cultural applicability if it is intended for international use.

Due to the special conditions of the patients and the complexity of care measures, the interview process was interrupted several times during the patient interviews. The researcher’s experience of working in the ICU and familiarity with unsafe care in the ICU created a risk of bias in the researcher’s work. In the qualitative phase, to avoid bias during the interview, the researcher aimed to act as a listener and pose questions in accordance with the interview guide. Throughout this project, the researcher encountered numerous challenges. In the qualitative phase, many interviews were cancelled and rescheduled due to the concurrent interviews during the coronavirus epidemic. In the quantitative phase, it took 2 months to complete the questionnaires because they were submitted online to supervisors and nurses, and some questionnaires were not completed.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Abbreviations

WHO

World Health Organization

ICUs

Intensive Care Units

COSMIN

Consensus based standards for the selection of health measurement instrument

IIS

Item Impact Score

CVR

Content Validity Ratio

CVI

Content Validity Index

S

CVI)-Scale Content Validity Index

K*

Adjusted kappa coefficient

EFA

Exploratory Factor Analysis

KMO

Kaiser-Meyer-Olkin

BT

Bartlett’s sphericity Test

CFA

Confirmatory Factor Analysis

GFI

Goodness of Fit Index

AGFI

Adjusted Goodness of Fit Index

RMSEA

Root Mean Square Error of Approximation

CFI

Comparative Fit Index

IFI

Incremental Fit Index

NNFI

Non-Normed Fit Index

NFI

Normed Fit Index

RFI

Relative Fit Index

PNFI

Parsimony Normed Fit Index

ICC

Intraclass Correlation Coefficient

SEM

Standard Error of Measurement

MDC

Minimum Detectable Change

SNCI

ICU)-Safe Nursing Care Instrument in the ICU

PS

ASK)-Patient safety attitudes, skills, knowledge

NASUS

Nurses attitudes and skill around updated safety concepts

NPPSS

Nursing Performance for Patient Safety Scale

IHT

Intrahospital Transport

SEIPS

Systems Engineering Initiative for Patient Safety

HPPSACS

Health Professions Patient Safety Assessment Curriculum Survey

Author contributions

M.T, T.A and A.E contributed in study design. M.T contributed in data collection and wrote the manuscript. T.A, A.E and F.Z analyzed the data and revised the manuscript. All of the authors proved the final version of manuscript.

Funding

Not applicable.

Data availability

Due to university policies, the datasets generated and utilized for the present study are not publically accessible but are available from the corresponding author upon justifiable request.

Declarations

Ethics approval and consent to participate

This study was approved by the Ethics Committee of Tehran Islamic Azad University of Medical Sciences, under the code IR.IAU.TMU.REC.1399.481. The participants were informed about the possible duration of the interviews, their freedom and authority to stop the interview whenever they felt necessary, how to maintain the confidentiality of the information, and how the results of the study would be used. An informed consent form was also completed.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.