Evaluation and improvement of nursing undergraduates’ informatics competencies using a hybrid multi-criteria decision-making model

Abstract

Background

Nursing staff need to be constantly exposed to information systems at work and encounter patients who share medical data obtained from the internet; this was widely observed during the coronavirus disease 2019 (COVID-19) pandemic. Hence, nursing staff should have the necessary skills and education that can help them develop nursing students’ informatics competencies. However, research on assessing and improving nursing students’ informatics competencies remains scarce.

Objective

This study aimed to provide nursing educators with a refined evaluation model and targeted improvement strategies tailored to enhance undergraduate students’ informatics competencies.

Design

A cross-sectional study.

Methods

This study constructed a hybrid multiple-criteria decision-making model. The analytical hierarchical process was applied to obtain criteria weights; thereafter, the Visekriterijumska Optimizacija I Kompromisno Resenje with Aspiration-level (VIKOR-AS) method was used to assess undergraduate nursing students’ informatics competencies of in the case hospital.

Participants

Data were obtained from 22 clinically experienced nurses with experience in supervising undergraduate nursing students at a secondary public hospital in Zhejiang Province, China.

Results

According to the weighted results, “Skill (C₂)” is an important dimension with the highest weight ranking. The corresponding highest-ranking criteria for each dimension are “Knowing how to explain the information management strategies to ensure patient safety (C₁₂),” “Applying information technology tools to support patient safety management (wristband scanning to identify patients, patients’ electronic orders, etc.) (C₂₁),” and “Paying attention to the importance of information technology in clinical decision-making and preventing errors or facilitating patient care coordination (C₃₂).” In the case of the undergraduate nursing students’ performance assessment, Student E was the best overall performer from the perspective of overall utility value. The remaining students ranked as follows: Student C $\succ$ Student D $\succ$ Student F $\succ$ Student A $\succ$ Student B.

Conclusions

This study model remedies the shortcomings of previous studies on evaluating undergraduate students’ informatics competency dimensions, provides a reference for nursing colleges to develop nursing informatics-related curriculum content, and helps train nursing instructors to assess and train specific students. The results indicate that information skills are an important factor in the development of nursing students’ informatics competencies; hence, nursing educators should prioritize the development of nursing students’ informatics competencies, followed by information knowledge and attitudes.

Background

The coronavirus disease 2019 (COVID-19) pandemic triggered the development of internet-based hospitals that provide outpatient services through internet technology [1]. Internet hospitals played an important role in preventing and controlling the pandemic by providing essential medical support to the public [2]. These hospitals can provide a wider range of convenient outpatient services [3] by placing greater emphasis on personalized digital health and higher staff informatics competency requirements [4, 5]. With the development of information technology, the application of internet systems has been an integral part of the medical industry [6]. Nurses often encounter patients who provide medical data from the internet [7]. Therefore, the nursing staff should possess the necessary skills to obtain and process information. The degree of nursing informatics competence directly affects the quality and safety of nursing work [8]. Hence, nursing education is vital in developing nursing students’ information skills [9], and informatics competency is one of the core competencies undergraduate nursing students should possess [10]. Incorporating routine measurements of informatics competencies into the curriculum is critical, thereby ensuring that the graduating students are qualified in informatics [11]. However, research on the assessment of nursing students’ information competencies is scarce [12]. Therefore, this study focuses on the assessment of undergraduate nursing students’ informatics competencies.

Some of the previous studies have mainly used individual qualitative or quantitative methods to investigate the factors associated with informatics competencies. For example, Strudwick, Nagle [13] used the Delphi method to adapt and validate informatics competencies. Yen, Phillips [14] used the item reduction approach and psychometric analysis to assess nurses’ ability to lead nursing informatics. Penm, Ivey [15] used an email survey and follow-up interviews to investigate the perceptions of advanced pharmacy practice experience (APPE) instructors regarding students’ competencies in pharmacy informatics during their APPE rotations. Kaihlanen, Gluschkoff [16] conducted a cross-sectional study to investigate the nursing informatics competencies of Finnish registered nurses following a national education initiative. Additionally, some researchers have used a combination of qualitative and quantitative methods. Liu, Aungsuroch [10] developed a comprehensive indicator evaluation system for nursing students, which included informatics competencies. Patel, Branch [17] evaluated Woods Hole Medical Informatics and the extent to which the program’s goals were met using questionnaires, semi-structured telephone interviews, and participant observation methods. Dohan, Green [18] conducted an exploratory factor analysis to test competency models from a multilevel perspective by investigating the effect of informatics competency on dynamic competency.

However, few studies have specifically focused on the informatics competencies of undergraduate nursing students, and even fewer have used decision models to assess and improve informatics competencies. To fill this research gap, this study used a three-stage modeling approach to assess and improve undergraduate nursing students’ informatics competencies. First, informatics competency indicators from the existing literature were employed as the assessment system in this study. Second, the classical analytical hierarchical process (AHP) was used to obtain dimension and criteria weights from nursing experts with extensive clinical and didactic experience. Finally, the Visekriterijumska Optimizacija I Kompromisno Resenje with Aspiration-level (VIKOR-AS) method was used to synthesize the weights and performance of the indicators to obtain the overall utility, maximum regret value, and comprehensive indicator of undergraduate nursing students’ informatics competencies.

Methods

Design

The participants were 22 female clinical nurses with a bachelor’s degree employed at the case hospital. Among them, 82% of the respondents were over 30 years old, and 59% had more than 10 years of nursing experience. The backgrounds and characteristics of all the participants are presented in Table 1. In addition, all participants’ personal information was kept anonymous. They could withdraw from the study at any time, and there were no adverse consequences. Consent was obtained for this study, and their clinical experience was collected using a paper-based questionnaire. The expert questionnaire took approximately 30 min to complete. On this basis, this study further selected 6 experts from 22 experts through purpose sampling. Furthermore, another 6 nursing students were selected. The selection criterion is that these 6 nursing students have received guidance and training in nursing practice under these 6 clinical nurses. For this reason, in performance evaluation, according to the nursing practice process of 6 students, these clinical nurses evaluated them and put forward suggestions for improvement. Data were collected from July through September 2021 for the entire study.

Table 1.

Background and characteristics for 22 clinical nurses (n = 22)

Characteristics	Value (%)
Gender
Male	0 (0%)
Female	22 (100%)
Education
Bachelor	22 (100%)
Master or above	0 (0%)
Age
≤ 30	4 (18%)
> 30	18 (82%)
Professional title
Nurse Practitioner	5 (23%)
Head nurse	4 (18%)
Clinical Internship tutor	12 (55%)
Administrative staff	1 (4%)
Years of service
Less than 5 years	5 (23%)
5–10	4 (18%)
10 and above	13 (59%)

Ethical approval

All procedures were performed following the guidelines of the ethics committee of the institution and the principles of the Declaration of Helsinki. Data from all participants are anonymized. The Institutional Review Board (IRB) of Taizhou University, China (ID: TZXY2021007) approved the oral informed consent procedure for this study and the entire study.

Informatics evaluation model

In this study, the learning informatics model from the Nursing Student Quality and Safety Competency Assessment Framework developed by Liu, Aungsuroch [10] was selected as the evaluation framework. Regarding applicability, the framework is based on the Chinese nursing context and is a good fit for the Chinese undergraduate nursing students who are the subject of this study. The framework is based on the quality and safety education framework for nurses [19], which has good generalizability. In addition, it has good rationality by inviting the participation of 22 nursing education experts to develop assessment indicators and reach a consensus through semi-structured interviews and the e-Delphi method. The model had three dimensions and 15 criteria, as listed in Table 2.

Table 2.

Informatics evaluation model

Dimensions	Criteria
Knowledge (C1)	Knowing the availability of qualified information resources (C11)
	Knowing how to explain the information management strategies to ensure patient safety (C12)
	Knowing how to describe the basics of nursing information retrieval (C13)
	Mastering the hospital nursing information system-related knowledge (C14)
	Knowing how to help patients identify and obtain credible and high-quality health care information and resources (C15)
Skill (C2)	Applying information technology tools to support patient safety management (such as wristband scanning to identify patients, patients’ electronic orders, etc.) (C21)
	Applying information management tools to monitor the effectiveness of nursing care services (such as using the hospital’s information system to report pressure sores, falls, infusion reactions, and other nursing care deficiencies) (C22)
	Being able to use information technology to meet the needs of nursing work and use hospital information systems to handle patients’ treatment and nursing care needs adequately (C23)
	Being able to use the electronic patient record system to document patients’ care procedures (C24)
	Being able to search and retrieve the relevant information to make good and correct decision-making on relative complex nursing care problems (C25)
	Being able to provide written or other forms of knowledge information to patients and let patients understand the disease (e.g., using the WeChat platform to provide popular scientific knowledge) (C26)
Attitude (C3)	Recognizing the need for all medical and nursing staff to learn information technology throughout their lives (C31)
	Paying attention to the importance of information technology in clinical decision-making and preventing errors or coordination of patient care (C32)
	Focusing on protecting the confidentiality of electronic information records (C33)
	Paying attention to collect patient(s) information as much as possible to help doctors deal with problems (C34)

AHP method

Saaty developed a pairwise comparison-based weighting method known as the AHP method [20]. This method transforms a complex problem into a hierarchical indexing system. The relative weight value of each criterion was then obtained by pairwise comparison and by computing the eigenvalues and eigenvectors. This method helps better assess qualitative and quantitative criteria in the decision environment [21, 22]. Therefore, AHP is widely used in various decision-making situations, such as water resource management [23], doctor burnout [24], distance education evaluation [25], and evaluation of the quality of nursing teaching [26]. The calculation steps are as follows:

• Step 1: Construct the pairwise comparison matrix

Each expert assessed the relative importance of the criteria on a nine-point scale in a pairwise comparison matrix, as expressed in Eq. (1).

$$\mathit{E}={\left[\begin{array}{ccccc}{e}_{11}& \cdots & e_{1j}& \cdots & e_{1n}\\ ⋮& \ddots & ⋮& \ddots & ⋮\\ e_{i1}& \cdots & e_{\mathit{ij}}& \cdots & e_{\mathit{in}}\\ ⋮& \ddots & ⋮& \ddots & ⋮\\ e_{n1}& \cdots & e_{\mathit{nj}}& \cdots & e_{\mathit{nn}}\end{array}\right]}_{n\times n}=\left[\begin{array}{ccccc}1& \cdots & e_{1j}& \cdots & e_{1n}\\ ⋮& \ddots & ⋮& \ddots & ⋮\\ _{}^1\left./,_{e_{1j}}^{}\right)& \cdots & 1& \cdots & e_{\mathit{in}}\\ ⋮& \ddots & ⋮& \ddots & ⋮\\ _{}^1\left./,_{e_{1n}}^{}\right)& \cdots & _{}^1\left./,_{e_{\mathit{in}}}^{}\right)& \cdots & 1\end{array}\right]_{n\times n}^{}$$ 1

where $n$ is the number of criteria, and $e_{\mathit{ij}}=1/e_{\mathit{ji}}$ (positive reciprocal).

• Step 2: Compute the weight for each criterion

The weight of each criterion can be obtained by computing the maximum eigenvalue and the corresponding eigenvector for the pairwise comparison matrix, as expressed in Eq. (2).

$$W_i={\left(\prod _{j=1}^n{e}_{\mathit{ij}}\right)}^{\frac{1}{n}}\left./\phantom{{\left(\prod _{j=1}^n{e}_{\mathit{ij}}\right)}^{\frac{1}{n}}\sum _{i=1}^n{\left(\prod _{j=1}^n{e}_{\mathit{ij}}\right)}^{\frac{1}{n}}}\right)\sum _{i=1}^n{\left(\prod _{j=1}^n{e}_{\mathit{ij}}\right)}^{\frac{1}{n}}i,j=1,2,\dots ,n$$ 2

• Step 3: Compute the consistency of the pairwise comparison matrix

Based on the AHP method, it is assumed that decision-makers make rational judgments. However, in real society, their judgment is often irrational. Therefore, Eqs. (3) and (4) can be used to calculate the degree of inconsistency in the matrix, which is called the consistency index (CI) and consistency ratio (CR), respectively.

$$CI=\left({\lambda }_{max}-n\right)\left./\phantom{\left({\lambda }_{max}-n\right)\left(n-1\right)}\right)\left(n-1\right)$$ 3

$$CR=\mathit{CI}\left./\phantom{\mathit{CI}\mathit{RI}}\right)\mathit{RI}$$ 4

where ${\lambda }_{max}$ is the maximum eigenvalue of the matrix and the random consistency.

If CR is lower than 0.1, the judgments of the pairwise comparison matrix $\mathit{E}$ are acceptable. If CR is higher than 0.1, it means that they are highly close to randomness, and thus the matrix $\mathbf{E}$ is not credible.

VIKOR-AS method

The original VIKOR methodology was developed to evaluate the performance of multi-criteria decisions in complex systems [27]. The methodology integrates the criteria weights with the evaluation of alternatives to determine the performance and ranking of each alternative. Each alternative is defined according to the degree of the ideal solution for achieving the goal of compromise ordering. VIKOR has been widely used for many decision-making issues in various industries, such as sports tourism site selection [28], evaluation of integrated management systems [29], and self-driving vehicle risk measurement [30].

The original VIKOR method is based on $L_p-metric$ defining all the evaluation parameters. Assuming that the evaluation value of the alternative $A_i$ obtained under the evaluation criterion $c_j$ is $f_{\mathit{ij}}$, then $L_p-metric$ is as shown in Eq. (5).

$$\begin{array}{cc}{L}_p^u=\left\{\sum _{j=1}^n\left[w_j\left(\left|f_j^{\ast }-f_{\mathit{ij}}\right|\right)/\left(\left|f_j^{\ast }-f_j^{-}\right|\right)\right]_{}^u\right\}_{}^{_{}^1\left./\phantom{1u}\right)_u^{}},& 1\le s\le \infty \end{array}$$ 5

where $i=1,2,\dots ,m$; $j=1,2,\dots ,n$.

To overcome the drawback of setting the current best solution as the benchmark solution in the original VIKOR calculation, some studies based on the concept of aspiration level were added to the VIKOR calculation procedure, and the desired level and worst level were used as the evaluation benchmarks for the alternative solution, called VIKOR-AS [31–33]. This approach allows the gap between each alternative and the desired level to be known, and more effective management implications are discussed. The calculation steps are as follows:

• Step 1: Construct an initial decision matrix

Each expert obtains an evaluation score (ranging from very poor = 0 to very good = 10) for all alternatives based on the semantic variables and their corresponding evaluation scales. The arithmetic mean is used to aggregate the evaluations of all experts to obtain the initial evaluation decision matrix, which is expressed as follows:

$$\begin{array}{ccc}\mathit{F}={\left[f_{\mathit{ij}}\right]}_{m\times n},& i=1,2,\cdots ,m,& j\end{array}=1,2,\cdots ,n.$$ 6

• Step 2: Define the optimal and worst values

The conventional way to normalize VIKOR is to use the best performance value of the alternative solution as the denominator, as expressed in Eqs. (7) and (8).

$$\text{Optimal value:}f_i^{\ast }={\text{max}}_j\left\{f_{\mathit{ij}}\right\}$$ 7

$$\text{Worst value:}{f}_i^{-}={\text{min}}_j\left\{f_{\mathit{ij}}\right\}$$ 8

The desire-level concept is introduced in this step, and its VIKOR-AS regularization is expressed in Eqs. (9) and (10):

$$\text{Optimal value:}f_i^{\ast }=f^{\mathit{aspire}}=10$$ 9

$$\text{Worst value:}{f}_i^{-}=f^{\mathit{worst}}=0$$ 10

where $f^{\mathit{aspire}}=10$ (the highest level of the assessment scale) and $f^{\mathit{worst}}=0$ (the lowest level of the assessment scale).

• Step 3: Calculate sorted index $R_i$ 

The VIKOR-AS ranking is based on the group benefit $S_i$ and individual regret $Q_i$ to construct the ranking index $R_i$ (i.e., comprehensive indicator), where the weight $w_j$ is defined based on the AHP calculation. $\delta$ is the preference function, which is typically set to 0.5. When $R_i$ is small, the gap between the alternative and desired level is small. Conversely, when $R_i$ is large, the difference between the alternative and desired levels is larger. The relevant content is expressed in Eqs. (11)– (13).

$$L_i^{u=1}=S_i=\sum _{j=1}^n\left[w_j\left(\left|f_j^{\mathit{aspire}}-f_{\mathit{ij}}\right|\right)/\left(\left|f_j^{\mathit{aspire}}-f_j^{\mathit{worst}}\right|\right)\right]$$ 11

$$L_i^{u=\infty }=Q_i=\underset{j}{max}\left\{w_j\left(\left|f_j^{\mathit{aspire}}-f_{\mathit{ij}}\right|\right)/\left(\left|f_j^{\mathit{aspire}}-f_j^{\mathit{worst}}\right|\right)\right\}$$ 12

$$R_i=\delta (S_i-S_{}^{\ast })/(S^{-}-S_{}^{\ast })+(1-\delta )(Q_i-Q_{}^{\ast })/(Q^{-}-Q_{}^{\ast })$$ 13

where $S_{}^{\ast }=\underset{i}{\text{min}}\left\{S_i\right\}$, $S^{-}=\underset{i}{\text{max}}\left\{S_i\right\}$, $Q_{}^{\ast }=\underset{i}{\text{min}}\left\{Q_i\right\}$, $Q^{-}=\underset{i}{\text{max}}\left\{Q_i\right\}$.

VIKOR is a useful soft computing tool for multi-criteria decision analysis. In particular, when decision-makers are uncertain about expressing their preferences, a compromise solution can be used to obtain more scientific results. VIKOR offers the concept of maximum “group utility” and minimum “individual regret.”

Results

Results of weight by AHP method

The original comparison matrix of the informatics evaluation model for all clinical nursing lead specialists was used in Eqs. (1)– (4) and averaged to obtain a set of preference relationship weights (Table 3). Table 3 shows the average relative preference weights of the 22 clinical nursing lead specialists for the informatics evaluation model. The experience matrices of all experts meet the criterion of CR less than 0.1. First, from the dimensional point of view, the conformational surface with the highest local weight is “Skill (C₂)” (weights = 0.368) followed by “Knowledge (C₁)” (weights = 0.333) and “Attitude (C₃)” (weights = 0.298) respectively. Subsequently, the top three highest local weights in the “Knowledge (C₁)” dimension are “Knowing how to explain the information management strategies to ensure patient safety (C₁₂)” (Local weights = 0.321), “Mastering the hospital nursing information system-related knowledge (C₁₄)” (Local weights = 0.244) and “Knowing how to help patients identify and obtain credible and high-quality health care information and resources (C₁₅)” (Local weights = 0.158). The top three highest local weights in the “Skill (C₂)” dimension are “Applying information technology tools to support patient safety management (C₂₁)” (Local weights = 0.298), “Being able to use information technology to meet the needs of nursing work and use hospital information systems to handle patients’ treatment and nursing care needs adequately (C₂₃)” (Local weights = 0.176) and “Applying information management tools to monitor the effectiveness of nursing care services (C₂₂)”. Finally, the top three highest local weights in the “Attitude (C₃)” (Local weights = 0.17) dimension are “Paying attention to the importance of information technology in clinical decision-making and preventing errors or coordination of patient care (C₃₂)” (Local weights = 0.395), “Focusing on protecting the confidentiality of electronic information records (C₃₃)” (Local weights = 0.251) and “Recognizing the need for all medical and nursing staff to learn information technology throughout their lives (C₃₁)” (Local weights = 0.209).

Table 3.

Weight for dimensions and criteria

Dimensions	Local weight	Criteria	Local weight	Global weight	CR (min–max)
C1	0.333 (2)	C11	0.144 (4)	0.048 (12)	0.012–0.091
		C12	0.321 (1)	0.107 (3)
		C13	0.134 (5)	0.045 (13)
		C14	0.244 (2)	0.081 (4)
		C15	0.158 (3)	0.053 (10)
C2	0.368 (1)	C21	0.298 (1)	0.110 (2)	0.015–0.097
		C22	0.170 (3)	0.063 (7)
		C23	0.176 (2)	0.065 (6)
		C24	0.147 (4)	0.054 (9)
		C25	0.138 (5)	0.051 (11)
		C26	0.071 (6)	0.026 (15)
C3	0.298 (3)	C31	0.209 (3)	0.062 (8)	0–0.089
		C32	0.395 (1)	0.118 (1)
		C33	0.251 (2)	0.075 (5)
		C34	0.148 (4)	0.043 (14)

The value of () indicates the ranking

Results of performance evaluation by the VIKOR method

On this basis, this study further selected 6 experts from 22 experts through purpose sampling, and selected 6 students who had received guidance and training from these 6 experts in nursing practice. Table 4 shows the aspirated gaps for each undergraduate nursing student for each dimension and criteria. In the dimension, “Knowledge (C₁)” is the worst-performing dimension (i.e., the largest aspirated gap) for students A (aspiration gap = 0.439), B (aspiration gap = 0.457), E (aspiration gap = 0.312), and F (aspiration gap = 0.414). “Skill (C₂)” is student C (aspiration gap = 0.343) worst performing dimension. In the criteria, “Being able to search and retrieve the relevant information to make good and correct decision-making on relative complex nursing care problems (C₂₅)” is the worst-performing criteria for students A (aspiration gap = 0.533), B (aspiration gap = 0.533), C (aspiration gap = 0.417) and F (aspiration gap = 0.5). “Knowing the availability of qualified information resources (C₁₁)” is the worst-performing criteria for students D (aspiration gap = 0.45) and E (aspiration gap = 0.417). For overall evaluation, Student E is the best performer from the perspective of the comprehensive indicator. The remaining students are ranked as follows: Student C ($R_i$ = 0.339) $\succ$ Student D ($R_i$ = 0.363) $\succ$ Student F ($R_i$ = 0.407) $\succ$ Student A ($R_i$ = 0.425) $\succ$ Student B ($R_i$ = 0.435).

Table 4.

Performance and aspirated-gap of dimensions and criteria for six nursing students in the case hospital

Dimensions/Criteria	Local weight	A	B	C	D	E	F
C1	0.333	0.438	0.457	0.340	0.382	0.312	0.414
C11	0.144	0.450	0.483	0.383	0.450	0.417	0.400
C12	0.321	0.417	0.450	0.333	0.433	0.317	0.417
C13	0.134	0.433	0.467	0.367	0.350	0.317	0.400
C14	0.244	0.433	0.450	0.333	0.333	0.250	0.400
C15	0.158	0.483	0.450	0.300	0.317	0.300	0.450
C2	0.368	0.420	0.408	0.343	0.332	0.287	0.405
C21	0.298	0.300	0.283	0.283	0.217	0.150	0.300
C22	0.170	0.500	0.483	0.400	0.433	0.383	0.467
C23	0.176	0.433	0.433	0.300	0.317	0.283	0.417
C24	0.147	0.450	0.433	0.383	0.350	0.300	0.417
C25	0.138	0.533	0.533	0.417	0.433	0.400	0.500
C26	0.071	0.417	0.400	0.333	0.383	0.400	0.467
C3	0.298	0.372	0.368	0.319	0.320	0.294	0.382
C31	0.209	0.317	0.300	0.300	0.283	0.283	0.367
C32	0.391	0.383	0.383	0.317	0.333	0.317	0.367
C33	0.251	0.367	0.333	0.300	0.283	0.233	0.350
C34	0.148	0.433	0.483	0.383	0.400	0.350	0.500
Group utility $S_i$		0.412	0.413	0.335	0.345	0.297	0.401
Ranking		5	6	2	3	1	4
Maximum individual regret $Q_i$		0.438	0.457	0.343	0.382	0.312	0.414
Ranking		5	6	2	3	1	4
Comprehensive indicator $R_i$		0.425	0.435	0.339	0.363	0.305	0.407
Ranking		5	6	2	3	1	4

Discussion

Weight analysis

As shown in Table 3, “Skill (C₂)” (weight = 0.368) has the highest weight. Its weight is higher than “Knowledge (C₁)” and “Attitude (C₃).” In most cases, the internship of undergraduate nursing students is a transition process from the role of the student to that of a nurse; more emphasis is placed on practical application and acquiring basic skills [34]. Undergraduate nursing students with good information skills can adequately handle patient treatment and care needs using the hospital information systems. Good information skills can improve patient experience, reduce nursing errors and accidents, and improve patient safety [35]. Additionally, the application of information technology among nursing students will enable electronic access to patient cases, enabling them to search for and retrieve more comprehensive information, thereby providing a clearer understanding of relatively complex nursing issues. This helps in developing critical thinking and making more appropriate decisions [36]. Nurses with information skills can establish disease science platforms for patients and provide information on disease-related knowledge, such as scientific knowledge. Finally, several studies demonstrate that the use of big data and artificial intelligence will lead to more insightful and effective decision-making [37, 38]. Enhanced information skills will also benefit nursing students in making effective decisions [39]. Therefore, acquiring good information skills is crucial for developing information competency among undergraduate nursing students.

For the “Knowledge (C₁)” dimension, the important criteria are “Knowing how to explain the information management strategies to ensure patient safety (C₁₂)” (weight = 0.321) and “Mastering the hospital nursing information system-related knowledge (C₁₄)” (weight = 0.244). These two values were significantly higher than those of the other three criteria. Patient safety has always been a priority for nursing staff, and it is important to explain information management strategies to patients in an easy-to-understand manner to ensure their safety [40]. Sharing care information with patients can facilitate shared decision-making and help improve patient safety [41]. As a symbol of modern nursing, holistic nursing is a systematic project. For example, nursing records include the five steps of nursing assessment, diagnosis, planning, implementation, and evaluation, which contain very complicated and rich information, and traditional manual filling is time-consuming and laborious, which is contrary to the concept of holistic nursing. Mastering the hospital nursing information system can greatly improve the efficiency of nursing staff and allow nurses to go to the bedside to get close to patients, diagnose, and deal with existing and potential health problems [42]. Nursing educators and policymakers can develop standardized nursing informatics content and integrate informatics into nursing education using a sound framework to ensure that students are learning adequate informatics [43, 44].

For the “Skill (C₂)” dimension, the important criterion is “Applying information technology tools to support patient safety management (wristband scanning to identify patients, patients’ electronic orders, etc.) (C₂₁)” (weights = 0.298). In general, nursing students have not yet developed good safety care practices such as strict three- and seven-checking. The effective use of wristband scanning to identify patients and electronic patient orders, among others, will effectively ensure the safety of patient clinical outcomes [45, 46]. Universities can create interdisciplinary educational environments for engineering and healthcare professionals to develop nursing students’ applications and understanding of information systems [47].

For the “Attitude (C₃)” dimension, the most important criterion is “Paying attention to the importance of information technology in clinical decision-making and preventing errors or facilitating patient care coordination (C₃₂),” which is much higher than the other criteria. Nurse students without sufficient clinical nursing experience, incomplete knowledge, and unskilled skills are prone to errors, accidents, and tension in the nurse-patient relationship. If students pay attention to and are familiar with the error reminders in the information system, they can correct possible errors in a timely manner and maintain a harmonious nurse-patient relationship [48].

Performance analysis and management practice

As shown in Table 4, the best informatic performance of the nursing students in the case hospital was E (aspiration gap = 0.305), followed by C (aspiration gap = 0.339), D (aspiration gap = 0.363), F (aspiration gap = 0.407), A (aspiration gap = 0.425), and B (aspiration gap = 0.435). This study presents the following recommendations for informatics performance improvement from a specific guideline perspective using the bottom four ranked nursing students:

For student D, who ranked third, the gap in the main criteria was “Knowing the availability of qualified information resources (C₁₁)” (aspiration gap = 0.450). The rapid development of information technology, particularly the emergence of the big data era, and the infiltration of electronic information technology into clinical care requires the faculty to introduce nursing students to the available information resources promptly.

For student F, who ranked fourth, “Being able to search and retrieve the relevant information to make good and correct decision-making on relative complex nursing care problems (C₂₅)” (aspiration gap = 0.500) and “Paying attention to collecting patient information as much as possible to help doctors deal with problems (C₃₄)” (aspiration gap = 0.500) must be improved. Teachers must enable the student to develop a rigorous and conscientious work attitude and use information technology tools to make the collected information more comprehensive and objective.

For students A and B, who ranked fifth, the “Being able to search and retrieve the relevant information to make good and correct decision-making on relative complex nursing care problems (C₂₅)” (aspiration gap = 0.533) must be improved. Teachers should enable these nursing students to utilize information technology tools to make the information gathered more comprehensive and objective, as well as enhance decision-making skills in the face of complex nursing problems.

Based on the analysis of the above criteria, when establishing study groups among classmates, students E and C ranked first and second, respectively. They were the most informative and could team up with any of the other four students. However, the fourth-, fifth-, and last-, ranked students—F, A, and B—share similar problems (“Being able to search and retrieve the relevant information to make good and correct decision-making on relative complex nursing care problems (C₂₅)”) and are not suitable for teaming together.

Implications

This study provides a systematic scientific evaluation model for nursing student informatics. The model can assist nursing instructors and faculty in better assessing and recognizing nursing students’ informatics competence. It can also be used for the self-assessment of nursing students. The model proposed in this study can be used to identify informatics deficiencies among nursing students and make targeted improvements and enhancements, thereby improving the efficiency and effectiveness of informatics teaching.

Limitations

This study used the AHP and VIKOR-AS models to assess and improve the informatics competencies of undergraduate nursing students, bridging the gap between previous studies on quantitative analysis and targeted improvement of informatics competency assessment. This will help nursing educators efficiently analyze the informatics competencies of undergraduate nursing students and target specific criteria for developmental improvement. However, this study also has some limitations. First, the expert opinion in this study was derived from a single-hospital case, and the impact of regional differences should be considered in the replication process. Future studies that require random sampling of hospitals in a wider region would make these findings more reliable. Second, because few studies have quantitatively examined the weighting of the dimensions of informatics competency, it is difficult to make deep comparisons with previous studies during the discussion. More multi-criteria decision-making (MCDM) methods can be used to compare future studies. Additionally, with the emergence of big data and artificial intelligence, significant changes are expected in the medical and healthcare industries. This will require healthcare organizations and medical staff to be highly skilled in informatics to empower medical and healthcare with big data and artificial intelligence, and more relevant research is required.

Conclusions

The informatics competencies of undergraduate nursing students are important for providing quality, efficient, and safe nursing care to patients and are critical to their level of career development and career satisfaction. This study proposes a mixed MCDM model based on which undergraduate nursing students’ informatics competencies are assessed and suggestions for improvement are made. This study model remedies the shortcomings of previous studies on the evaluation of undergraduate students’ informatics competency dimensions, provides a reference for nursing colleges to develop nursing information-related curriculum content, and helps train nursing instructors to assess and train specific students. The results indicated that information skills are an important factor in the development of nursing students’ informatics competency, suggesting that nursing educators should place information skills at the top of the development of nursing students’ informatics competency, followed by information knowledge and attitudes.

Authors’ contributions

Chao Liu, Wei-Ling Hu and Youyou Hong conducted the study and drafted the manuscript. Chao Liu, Lili Feng, Yen-Ching Chuang and Bing-Long Wang revised the manuscript. Youyou Hong and Wei-Ling Hu participated in data collection. Yen-Ching Chuang and Chao Liu calculated the results of the AHP and VIKOR methods. Chao Liu and Lili Feng analyzed the results of this study and discussed its significance. Wei-Ling Hu, Yen-Ching Chuang, and Bing-Long Wang conceived the study and participated in its design and coordination. All authors read and approved the final manuscript.

Funding

No funding was accepted for this study.

Data availability

The original contributions presented in the study are included in the article materials, further inquiries can be directed to the corresponding authors.

Declarations

Ethics approval and consent to participate

All procedures were performed following the guidelines of the ethics committee of the institution and the principles of the Declaration of Helsinki. Data from all participants are anonymized. The Institutional Review Board (IRB) of Taizhou University, China (ID: TZXY2021007) approved the oral informed consent procedure for this study and the entire study.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.