Assessment of determinants predicting success on the Saudi Nursing Licensure Examination by employing artificial neural network

Abstract

BACKGROUND:

This study aims to use the artificial neural network as a novel approach to explore factors that determine and predict successful performance of nursing interns in Saudi Arabia on the Saudi Nursing Licensure Examination (SNLE).

MATERIALS AND METHODS:

The study employed a cross-sectional, analytic approach. A total of 62 nursing interns were recruited by convenience sampling from the University of Hail to participate. Data collection was conducted from September to December 2019. Descriptive statistics were used to describe the demographic characteristics of the nursing interns and their responses regarding examination determinants. Neural network analysis was used to identify factors that are highly predictive of the success of the nursing interns on the SNLE.

RESULTS:

Overall, the nursing interns were undecided (3.94 ± 0.14) about the influential factors determining their success. Their study hours (100%) and grade point average (GPA) (96.9%) were identified as strong determinants reflective of the tenacity and vigor of the nursing interns, based on the predictive power of the model. Meanwhile, age (45.7%), marital status (21.3%), gender (15.2%), and the type of academic program (5.9%) were considered the least important of the sociodemographic variables.

CONCLUSION:

Exam preparation activities such as preparation programs, review classes, and exam simulations must be promoted and enhanced to increase the passing tendencies of the nursing interns in the SNLE. The GPA and increased study hours make the most significant contributions to success on the SNLE as compared to other variables such as age, gender, marital status, and the academic program. This study serves as a springboard for nursing educators and administrators in laying tailored strategies to strengthen the nurse interns’ GPA and time management.

Introduction

Quality patient care emanates from a professional nurse who employs competent and safe nursing practices. To ensure competent and safe nursing practice, nurse regulatory boards establish requirements, rules, practice regulations, and laws.[1] A nurse's professional license is obtained after passing a nurse licensure examination, which bestows upon the nurse the right to practice the profession and guarantees entry-level aptitudes for safe nursing practice.[2] Many countries administer their own licensure exam for nurses. The National Council Licensure Examination-RN (NCLEX-RN) in the United States, the Canadian Practical Nurse Registration Exam (CPNRE) in Canada, and the Philippine Nursing Licensure Exam (PNLE) in the Philippines, are among the national exams certain countries require prior to nurse licensure. In contrast, other countries, including Australia, New Zealand, India, Ireland, and Jordan, do not require examinations for nurses.[1]

In Saudi Arabia, a graduate nurse obtains licensure by passing the Saudi Nursing Licensure Examination (SNLE). The SNLE is regulated by the Saudi Commission for Health Specialties, which is responsible for the assessment of licensure requirements, administration of the test, and updates of the exam's contents.[3] Recent statistics have shown a range of passing rates in nurse licensure examinations. Countries such as the US and Canada experienced a decline in the nursing licensure exam passing rates from 2017 to 2019.[4,5] Meanwhile, in the Philippines and Saudi Arabia, passing rates increased for the PNLE (41.9% in 2018, and 53.7% in 2019) and the SNLE (70% in 2018 and 83% in 2019).[6,7] Despite improvement in the passing rate, many still fail the nurse licensure exam, resulting in delayed entrance into the nursing workforce.[8]

Studies from other countries have sought to determine the predictors of a successful nurse licensure examination. For example, in the United States, Havrilla et al.[9] confirmed that grade point average (GPA) and taking part in comprehensive mock exams were predictors of success on the NCLEX-RN. In addition, nurse graduates with high scores on the Health Education Systems, Inc., exit exam who participated in stress management activities, prepared for the exam, and were proactive in test preparation activities were identified as being successful on their nurse licensure exams.[8,10] In the Philippines, the student–faculty ratio and other characteristics of the nursing school, such as location, size, type, and the year established, were linked with nurse licensure exam passing rates.[11] In other studies, terminal competency assessment, GPA,[12] intelligence quotient, college admission tests, and preboard examination showed significant associations with nurse licensure exam performance.[2,13,14] Likewise, GPA, school performance, time management, test-taking skills, and group studies were indicative of success on nurse licensure examinations.[15] Prelicensure exams, GPA, and test preparation strategies predicted higher success on the CPNRE.[16,17,18] While much has been written on the predictors of licensure examination success from countries with a strong foundation of nursing, no known similar studies have been conducted in Saudi Arabia. It is essential to look at the factors that influence the performance of nurse graduates on nurse licensure exams in Saudi Arabia. To the best of the researchers’ knowledge, this study is the first to be conducted in Saudi Arabia, in part because nursing is a young profession in the country.

The importance of this study is that it explores the determinants leading to passing the SNLE and may help academic institutions understand better how to prepare their graduates. In addition, it lays groundwork for future researchers aiming to enhance the passing rate on the SNLE through exam preparation programs, develop test-taking skills, and support Saudi nurse aspirants. Moreover, this study may be helpful for nurse educators and nursing program administrators to look into the readiness of nurse graduates in tackling the nurse licensure exam. Therefore, this study aims to explore the determinants or variables that predict the success of the nursing interns in passing the SNLE by employing Artificial Neural Network (ANN) analysis.

Materials and Methods

Design and Sample

This study employed a cross-sectional, analytic approach. A total of 62 nursing interns (75%) out of 83 from the University of Hail participated in the study through convenience sampling. These nurse interns were those who are about to take their SNLE and currently enrolled in the nursing intern program. Nursing interns who were not willing to participate were excluded from this study.

Instruments

The questionnaire was patterned on one used in the study of Nyangena et al.,[12] with liberal modifications. There are two parts in the questionnaire. The first gathers respondents’ demographic information–age, gender, marital status, academic program, GPA, and study hours–and the second covers the modified examination determinants consisting of 18 items divided into school factors (1–6), candidate factors (7–12), and examination factors (13–18). Items were answered on a 5-point Likert scale with corresponding verbal interpretations: (5) strongly agree; (4) agree; (3) undecided; (2) disagree; and (1) strongly disagree.

Five experts in the field were asked to validate the questionnaire. Two were associate professors in nursing, and three were researchers in the field of psychometrics working under the department of nursing. The five experts unanimously agreed that the questionnaire covers the concept that it seeks to measure. A Content Validity Ratio was computed at 0.79 which suggest that items are valid. Thereafter, the tool was subjected to the content validity index (CVI). It obtained an overall CVI score of 0.89 for relevance and 0.88 for clarity, affirming high content validity. Internal consistency was tested, and the Cronbach's alpha in this study was 0.887.

Data collection

Data were gathered over four months (September to December 2019) using the self-administered questionnaire. The questionnaire was directly distributed by the researchers during the breaktime of the nurse interns. With the consideration of the schedules and break time of the nurse interns, the questionnaires were distributed by schedules. The nurse interns were requested to drop the accomplished questionnaires in the box provided in the nursing station. All the questionnaire were retrieved with 100% return rate.

Ethical consideration

This research obtained ethical approval from the Institutional Review Board of the University of Hail (H-2016-098). Adequate information was provided prior to data collection, such as the aim of the study, willingness to participate, the time required for participation, and participants’ right to refuse or discontinue the study for any reason without consequences.

Data analysis

The Statistical Package for the Social Sciences, version 25 (IBM Software Group, Chicago, IL, USA) was used. Descriptive statistics such as frequency, percentage, mean and standard deviation were used to describe the demographic characteristics of the nursing interns and their responses on examination determinants. In predicting determinants to success to SNLE, the ANN was used. According to Chukwu and Nwachukwu, the ANN is commonly used to ascertain the linear relationship of data using a perceptron. It aims to elicit the connection pattern and network topology of data that mimic the action of a biological neuron.[19] Further, LeCun, Benigno, and Hinton suggest that ANN analysis employs solving that are nonlinear as well as complex, and that the analysis used simple mathematical processes such as addition and multiplication. The mathematical structure of ANN can predict the relationship among a set of input and output variables using an iterative learning process.[20]

Results

The majority of respondents were ages 21–30 (72.58%), female (59.68%), and unmarried (61.29%). Scholastic records showed that most were in the regular program (n = 41; 66.13%) and the rest were bridging students. Many reported studying more than 15 h a week (91.94%) and attained a total item score of 46–90 (96.77%) on the factors m [Table 1].

Table 1.

Nurse intern characteristics

Characteristics (n=62)	n (%)
Age
21-30	45 (72.58)
31-40	15 (24.19)
Over 40	2 (3.23)
Gender
Male	25 (40.32)
Female	37 (59.68)
Marital status
Unmarried	38 (61.29)
Married	24 (38.71)
Academic program
Regular	41 (66.13)
Bridging	21 (33.87)
GPA (mean±SD: 2.58±0.741)
Low (<2.000)	15 (24.19)
High (≥2.000)	47 (75.81)
Study hours (mean±SD: 17.06±4.420)
Low (<15.00)	5 (8.06)
High (≥15.00)	57 (91.94)
Total items score (mean±SD: 70.97±9.609)
Low (18-45)	2 (3.23)
High (46-90)	60 (96.77)

GPA=Grade point average, SD=Standard deviation

Table 2 presents the perceived examination determinants of the nurse interns. It shows that school factors has the highest mean which suggest to be the most influential to examination determinants. Most nursing interns agreed that school factors, such as the review program expert nursing faculty, and structure of questions in the review sessions had been helpful. However, they were undecided on whether factors such as personal time management balanced nutrition stress management activities and external peer review sessions were relevant contributing elements of their exam preparation, but they agreed that study time and self-study were useful strategies. Lastly, the respondents agreed that, in terms of exam factors, a concise question topics tackled while in the university and ample time allocation were critical to their success on the SNLE. Overall, the nursing interns were undecided about which examination factor was most influential to their success [Table 2].

Table 2.

Perceived examination determinants of the nurse interns

Item number	Items	Mean±SD	Interpretation
A	School factors	4.02±0.13	Agree
1	The review program is comprehensive and covered all subjects	4.02±0.90	Agree
2	The nursing faculties are specialists in their fields	4.23±0.82	Agree
3	The university encourages students to attend the review classes	3.89±0.98	Undecided
4	The review program enhances the test-taking skills of the students	4.06±0.67	Agree
5	The course program is patterned on the licensure examination framework	3.89±0.89	Undecided
6	The questions in the review class are structured on the licensure exam	4.03±0.85	Agree
B	Candidate factors	3.81±0.26	Undecided
7	I spent adequate time studying for the examination	4.10±0.86	Agree
8	I balanced my review time with my other tasks and social obligations (e.g., family, friends)	3.68±1.25	Undecided
9	I ate nutritious food and had a balanced diet while reviewing	3.73±1.19	Undecided
10	I engaged in stress management activities (e.g., exercise)	3.45±1.13	Undecided
11	I engaged in group review	3.79±0.85	Undecided
12	I did self-study modules in my free time	4.11±0.79	Agree
C	Exam factors	4.00±0.13	Agree
13	The examination questions were clear and easy to understand	4.19±0.83	Agree
14	The examination questions covered all topics in all nursing concepts	3.94±0.83	Undecided
15	The examination questions allowed me to think critically	3.94±0.79	Undecided
16	The examination questions were applicable to what I learned in the university	4.02±0.82	Agree
17	The time allocated for the entire examination was adequate	4.08±0.91	Agree
18	The examination questions focused on applied rather than theoretical knowledge	3.84±0.89	Undecided
	Overall	3.94±0.14	Undecided

SD=Standard deviation

The application of neural network analysis in predicting the important examination determinants showed that two variables were deemed important: GPA (96.9%) and study hours (100%). Other than study hours and GPA, the age variable (45.7%), which may be reflective of the tenacity and vigor of the nursing interns, was identified as the other determinant, based on the predictive power of the model, while marital status (21.3%), gender (15.2%) and academic program (5.9%) were recognized as the least significant of the sociodemographic variables [Table 3].

Table 3.

Independent variable importance to examination determinants score in a two-layer neural networks (perceptron/regression) model

Factors/covariates*	Importance	Normalized importance (%)
Age	0.160	45.7
Gender	0.053	15.2
Marital status	0.075	21.3
Academic program	0.021	5.9
GPA*	0.340	96.9*
Study hours*	0.351	100*

GPA=Grade point average . *Sig. at 90%

Figure 1 shows the two-layer neural network model of the association of examination determinants with demographic variables. The partitioning of input data included 61.3% on training (batch) optimized through scaled conjugate gradient, 29% testing (used to find errors and prevent overtraining), and 9.7% holdout (used to validate the model).[20] The input layer had 11 units excluding the bias unit. There was one hidden layer with two units and two units in the output layer. In the ANN model, the input layer receives information from the outside world, the hidden layer performs information processing, and the output layer produces the class label or predicts continuous values.[21] Prior to training, all covariates were normalized using the formula (x-min) ÷ (max-min). Adjusted normalized was set at 0.02 in the rescale method for scale dependents. Hyperbolic tangent was selected as the activation function for both input and output. The percentages of incorrect predictions obtained for multilayer perceptron training, testing, and holdout were 5.3%, 0%, and 0%, respectively. The low percentage of incorrect predictions or the cross entropy error is indicative of the power of the model to predict examination determinants.

Figure 1.

Two-layer neural network model diagram

Discussion

This study aimed to explore the determinants of SNLE success using the ANN as a novel approach in determining the important factors that may predict the success of nursing interns on the SNLE in Saudi Arabia. Study hours and GPA are determining factors that increase one's chances of passing the SNLE. That is, students with higher GPAs are more likely to pass the SNLE. Hinderer, DiBartolo, and Walsh agreed that GPA is an academic indicator that correlates with success on the NCLEX-RN,[22] and this was corroborated by Simon, McGinniss, and Krauss, who confirmed that students with higher GPAs have a greater likelihood of passing the licensure exam. The amount of effort exerted in studying for the licensure exam is reflected by the time spent.[23] Naturally, the more time invested in studying results in better chances of passing the exam. Other sociodemographic factors, such as age, gender, and marital status, were found to be nonpredictive of SNLE results. Several researchers have agreed with this finding,[24,25,26] but it contrasts with that of Kaddoura et al., who stated that bridging students and students with previous education at the college level have a higher probability of passing.[27] The present findings contribute firm bases for designing SNLE preparation programs to help graduating students and nurse interns. This will also help educators and policy makers develop strategies for the continual improvement of these variables toward the ultimate success of the students.

In terms of the perceived examination determinants, the nurse interns agreed that school and exam factors influenced passing the SNLE which is similar to the study in in Kenya by Nyangena et al.[12] Although the nurse interns’ responses were inconclusive about the “candidate factors,” possible reasons could be that they employed different strategies in dealing with their feelings or stress about an upcoming examination. Thus, it is important for nurse interns to deal practically with these issues as they affect their mental preparation. An earlier study suggested that the degree of expectation of the candidates puts pressure on them and can be channeled into physical symptoms.[12] This finding underscores the importance of eliminating such factors that negatively affect SNLE outcomes. School administrators and educators should be aware of such barriers and incorporate this knowledge into their strategies designed to help students succeed.

Preparation programs, review classes, and exam simulations were identified by the nurse graduates as particularly helpful. These have been recognized as increasing the likelihood of passing the nurse licensure exam.[28] According to Pike et al., students are now encouraged to incorporate the use of NCLEX-RN preparation resources to enhance their learning and increase their chances of passing.[17] Likewise, the mentorship of nurse educators in the familiarization of how the questions are structured and how to respond was seen as a significant factor in the success of nurse graduates.[29] This underscores that nursing schools need to conduct review classes and simulation exams prior to the SNLE to prepare the students by acquainting them with the examination process and to bolster their confidence prior to taking the examination.

The current study further showed that the respondents preferred self-study sessions rather than group learning as part of their preparation for the nurse licensure exam. This finding was supported by Lown and Hawkins, who found a relationship between failure on the nurse licensure exam and group learning.[30] This study contributes to the recognition that nurse interns need ample time and exposure to exam questions to have favorable outcomes from their exam preparation. This strategy allows nurse interns to become familiar with the structure and have full comprehension in mock examinations. It also helps prepare the nurse graduates to estimate the time required to finish the exam and have a better perspective from which to analyze exam questions.

Strength and limitations

The strength of this study lies on the novelty of the unit of analysis which deviates from the usual design and analysis. This gives a new and important perspective because in actuality, many of the relationships among inputs and outputs are nonlinear. The limitation of the study includes the setting and a low sample size. Future studies with larger samples that involve nursing graduates in different regions of Saudi Arabia would improve the generalizability of the data.

Conclusion

The GPA and more extensive study hours make significant contributions toward success on the SNLE. Variables such as age, gender, marital status, and academic program are not determinants of SNLE success. Exam activities such as preparation programs, review classes, and exam simulations must be promoted and enhanced to increase the passing tendencies of the nursing interns in the SNLE. This study serves as a springboard for nursing educators and administrators to tailor strategies to strengthen the nurse interns’ GPA and time management.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.