The effect of COVID-19 on nursing program passing scores

Abstract

The COVID-19 pandemic greatly challenged nursing education. Nursing programs had to continue providing quality nursing education in an altered environment where much of what was traditionally face-to-face learning was forced online and into simulation. The purpose of this quantitative comparative ex post facto designed study, using secondary data and guided by Knowles's adult learning theory, was to determine the effect of the COVID-19 pandemic on nursing students’ nursing program passing scores before and during the pandemic. The results of the independent t test were a statistically significant decrease (p < .001) in the nursing program passing scores for students during the COVID-19 pandemic compared to before the COVID-19 pandemic. The implications of this analysis provide information to nursing educators, and nursing programs can use this study to recognize there are students requiring extra support in the face of a challenge such as the COVID-19 pandemic.

The COVID-19 pandemic affected nursing students and nursing programs. The World Health Organization (WHO) characterized COVID-19 as a pandemic in March 2020 and colleges throughout the United States were suddenly forced to move classes and clinicals online (Tracy & McPherson, 2020). In nursing programs innovative practices were implemented to ensure that nursing student learning was not disrupted because of face-to-face teaching restrictions (Hill et al., 2020). Nursing students need to pass their nursing programs to be eligible for the National Council Licensure Examination for Registered Nurse (NCLEX-RN). For the nursing student a failure to pass their nursing program does not necessarily mean that the student will never be able to take the NCLEX-RN. This failure does delay the nursing education process as the student will need to remediate and then attempt again to pass their program.

The COVID-19 disruption led to nursing students experiencing unexpected changes in all aspects of their nursing education (Gaffney et al., 2021). Overall, what was face to face was moved online due to the effects of COVID-19 forcing quarantining, sheltering in place, and social distancing. Nursing students have varied abilities to cope with the challenges of the COVID-19 pandemic. Many students experienced increased anxiety due to the highly disruptive nature of the pandemic. The COVID-19 pandemic and its impacts caused stress and anxiety that challenged the process of nursing teaching and learning (Silva et al., 2021). The new situation also caused fear in students, including fear of illness and death and fear of ill family members and economic shortages (Silva et al., 2021). All of these contributing factors could cause a change in nursing students passing their nursing programs.

Theoretical Framework

The theoretical framework used is Knowles's (1980) theory of andragogy. This theory describes adult learners as self-directed and therefore expected to take responsibility for their own learning. Knowles's posited assumptions that adults need to know why they need to learn something, adults learn best when the topic is of immediate value, adults approach learning as problem solving, and adults learn experientially (Knowles's et al., 2005). Andragogy includes four principles of planning, experience, relevance, and content (Health Research Funding, 2020) and these are incorporated in prelicensure nursing education. Andragogy aligns with nursing education in the collaboration between adult learners and nursing educators, and in student-centered, problem based nursing education (Decelle, 2016). Knowles proposes steps of self-directed learning including diagnosing learning needs, formulating learning goals, implementing appropriate learning strategies, and evaluating learning outcomes (Bair, 2019). This mirrors the aspects of the nursing process of assessment, diagnosis, planning, implementation, and evaluation (ANA, 2020.) that is the foundation of nursing. This foundation generates the level of critical thinking necessary for nursing education to prepare safe and competent nurses ready to grow in their role as nurses.

Objectives

The purpose of this study was to determine the difference, if any, between the Associate of Science Nursing (ASN) students’ nursing program final passing scores before and during the COVID-19 pandemic. The results of this study may show the COVID-19 pandemic affected nursing education and its learning outcomes, and this affect may result in a decrease in nursing program passing scores. A decrease in nursing program passing scores would lead to a decrease in eligible candidates ready to become licensed and enter the nursing work force. This decrease in new graduate nurses would negatively affect the supply needed to fill the great demand for nurses. According to Spurlock (2020) there is a projected nursing shortage of 1.1 million registered nurses by 2022 in the United States. The nursing shortage has worsened since the COVID-19 pandemic. To combat the nursing shortage over half a million nursing students were needed to pass the NCLEX in 2020 and 2021 (Buerhaus et al., 2017). In 2020 and 2021 177,407 and 185,062 U.S. educated nurses entered the healthcare workforce by passing the NCLEX-RN (first time test takers; NCSBN, 2021, NCSBN, 2022). A decline in the total number of nurses ready to take the NCLEX-RN by failing their nursing programs worsens this problem facing the nursing profession.

Methods

Study Design

The research design was a quantitative comparative ex post facto analysis using secondary data (Trochim et al., 2016). Before the secondary data were acquired, IRB approval was granted from the university and the partner site. Secondary data was collected on nursing program passing scores of nursing students before and during the COVID-19 pandemic using purposive sampling. The comparative research allowed for the analysis of the nursing program passing scores prior to the pandemic and the comparison of these data to the nursing program passing scores during the pandemic (Trochim et al., 2016).

Setting and Participants

The source of data was an ASN program at a college in the southern region of the United States that graduates over 250 nursing students annually. The target population for this secondary data study was taken from the results of ASN program nursing students’ end of program passing scores. The purposive sampling strategy used takes the nursing program passing score data before and during the COVID-19 pandemic ensuring an adequate sample size. Inclusion criteria include data from nursing students that either passed/did not pass their nursing programs before or during the pandemic. The COVID-19 pandemic was defined as beginning in the semester including March 2020, or when the United States began to shelter in place due to COVID-19 (AJMC Staff, 2020). Prepandemic semesters were those prior to 2020. Fall 2018, spring 2019, and fall 2019 were included. During pandemic semesters included were fall 2020, spring 2021, and fall 2021. The semester including the COVID-19 pandemic shelter in place federal order (March 2020) was excluded as these students received their nursing education prepandemic and received their nursing program passing score during the pandemic.

Variables

The independent variable (IV, categorical) had two groups: before and during the COVID-19 pandemic (cohort year/term). The continuous dependent variable (DV) was the final program score (scale). For this study final nursing program grades were converted to final program scores using A = 4, B = 3, C = 2, D = 1, F = 0, W = 0, and WF = 0 and treated as a continuous variable. Ordinal level data were treated as an interval ratio level (see Robitzsch, 2020) because for items with 3–6 categories, using the linear factor model by treating variables as continuous is as defensible as treating them as ordinal (Robitzsch, 2020).

Data Sources

Data collection strategies included maintaining a unique student identifier, using deidentified data from a college in a southern state, and maintaining reliability. The data was deidentified to preserve privacy and ensure that no nursing student's identity were revealed. There were no missing data. The data are password protected and stored securely for 5 years before permanent deletion. The dataset included the student number, the cohort year, the term, first generation status, Comprehensive Predictor exam scores, and final program scores. Reliability, or the possibility to consistently reproduce results, was maintained as the final program score was measured consistently over time and without change. Ensuring reliability, or the consistent reflection of the measure being measured (Field, 2018) was achieved.

Study Size

A power analysis was conducted to determine sample size using a power level of 0.8 which indicates that if the study is conducted repeatedly it is likely to produce a statistically significant effect 80% of the time if a statistically significant effect exists (Field, 2016). The alpha level was set at 0.05. An alpha level of 0.05 indicates a 5% probability of a type I, or the incorrect rejection of the null hypothesis when it is actually true (Field, 2016). The effect size represents the strength of relationship between variables. The calculated power analysis for sample size using the two-tailed independent t test, using a 0.80 power, 0.5 effect size (medium), and 0.05 alpha yielded a sample of 128 (see Buchner et al., 2021).

Statistical Methods

SPSS Statistics version 28 (IBM) was used to analyze the data. An independent t test was conducted to test for any statistically significant difference by comparing the means of the dependent variable scores between the two independent and unrelated groups (Laerd Statistics, 2018). Prior to the analysis with the independent t test, the six assumptions of the independent t test were examined. The first assumption that the dependent variable was continuous was met. The second assumption that the independent variable consists of two independent categorical groups, prepandemic and during pandemic ASN students was met. The third assumption of the independent t test, independence of observations, was met as each ASN student was part of one group being studied and no ASN student was part of both groups.

The fourth assumption was to determine if there were any significant outliers for the study. The histogram revealed no significant outliers, and the fourth assumption was met.

The fifth assumption of the independent t test was that the dependent variable should be approximately normally distributed for each group of the independent variable. The Shapiro-Wilk test of normality statistic is .866, df 484, p < .001 prepandemic, and .886, df 623, p < .001 during the pandemic, and the Q-Q Plots show normality. Kurtosis is -.038 and if the kurtosis is close to zero then a normal distribution is assumed (SPC for Excel, 2022). Therefore, the fifth assumption was met. The sixth assumption of the independent t test homogeneity of variance was tested using Levene's test which indicated group variances can be treated as equal F = .198 and p = .656. Therefore, equal variances were assumed, and all six assumptions were met.

Results

Baseline characteristics of the final sample included n = 484 prepandemic and n = 623 during pandemic nursing program final scores from ASN students. Results of the nursing program final scores were examined to determine frequency distribution (see Fig. 1 ). The independent samples t test was conducted using IBM SPSS Version 28 and evaluated if there was any statistically significant difference between the mean nursing program final passing scores for the prepandemic and during pandemic groups. The results of the independent t test showed that there was a statistically significant difference between the prepandemic group (M = 2.58, SD = .903, n = 484) and the during pandemic group (M = 2.04, SD = 1.002, n = 623) t(1105) =-9.226, p < .001, 95% CI (-.651, -.423; see Table 1 ). Therefore, the null hypothesis was rejected. Effect size to determine the strength of the difference between the two groups was examined using Cohen's d and the effect size was large (Standardizer .960, Point Estimate -.559, 95% CI (-.680, -.438). This was expected with such large sample sizes.

Fig. 1.

Nursing Program final scores histogram, to be reproduced in color on the web (free of charge) and in black-and-white in print.

Table 1.

Independent Samples t Test.

		Levene's Test for Equality of Variances						t test for Equality of Means
		Significance						95% Confidence Interval of the Difference
		F	Sig.	t	Df	One-Sided p	Two-Sided p	Mean Difference	Std. Difference	Lower	Upper
Final Program Scores	Equal variance assumed	.198	.656	-9.226	1105	<.001	<.001	-.537	.058	-.651	-.423
	Equal variance unassumed			-9.347	1080.984	<.001	<.001	-.537	.057	-.649	-.424

Discussion

Key Results

The results from the study showed a decline in nursing student performance and outcomes during the COVID-19 pandemic. The mean scores decreased significantly during the COVID-19 pandemic. More nursing students did not achieve passing scores: 1 and 0 (D's, F's, W's, and WF's) during the COVID-19 pandemic compared to prepandemic (see Fig. 2 ). The passing scores of the students included fewer scores of 4 & 3 (A's and B's) and more scores of 2 (C's) during the COVID-19 pandemic compared to prepandemic (see Fig. 2).

Fig. 2.

Nursing program final scores, to be reproduced in color on the web (free of charge) and in black-and-white in print.

Limitations and Generalizability

The results of this study should be interpreted considering the following limitations. The external validity, or usefulness of this study for the broader group of prelicensure nursing education, is somewhat limited because the data is from one institution and from only an ASN group. Therefore, sampling bias is present and population validity is limited. Findings from this study cannot be generalized to students in a BSN program. Generalizing the findings of this study to additional localities, regions, and states could also be limited.

Interpretation

The results of this study align with (Knowles's Andragogy 1980) in that the nursing student is self-directed and learns experientially. The experiences available to nursing students during the pandemic changed compared to prepandemic. This change in learning experiences could explain the significant decline in during pandemic student outcomes. Of the four andragogical principles: planning, experience, relevance, and content (Health Research Funding, 2020) incorporated in prelicensure nursing education, planning, experience, and content could have been altered due to events surrounding the COVID-19 pandemic. While adult learners are self-directed and their learning is problem-focused, they need educator guidance, or a scaffolded approach (Dolan et al., 2021) to the planning and content delivered to support their learning experiences. The planning of nursing education was altered by shelter in place orders (AJMC Staff, 2020), social distancing, and clinical site availability during the pandemic (Dolan et al., 2021). As aspects of nursing education were forced online (Weston & Zauche, 2021) content could have changed and these andragogical changes could explain the differences in nursing student performance reflected in their program scores.

Implications

The findings of this study contributed original research into the challenges faced by the discipline of nursing education and by nursing students due to the COVID-19 pandemic. Identifying where COVID-19 affected the practice of nursing education and its outcomes has important implications in overcoming future challenges and pandemics. This study identified that nursing student final program scores decreased significantly during the pandemic. Data showed that nursing students struggled to achieve passing scores during the pandemic compared to before the pandemic, and a greater number of students did not pass in semesters during the pandemic. These differences in passing may have resulted in fewer students graduating and being NCLEX-RN ready. Those students not passing may be delayed entering the nursing workforce during COVID-19. The evidence this study provides in declining nursing student outcomes causes the need to evaluate what factors during the COVID-19 pandemic caused the difference. Positive social change implications derived from this research lie in identification of areas of improvement needed to mitigate these factors in challenging times of nursing education such as this pandemic. Nursing education can use what was learned during the pandemic to see weaknesses, develop action plans, and better support students in times of struggle. Greater support of nursing students that helps them to overcome challenging events may result in higher scores, more students passing their programs, and more new graduate nurses entering the workforce, thereby decreasing the nursing shortage.

Recommendations

Extending knowledge in the discipline of nursing education with regard to the COVID-19 pandemic continues and this study opens the door to further research to determine the causes of the difference in scores. Therefore, more research is needed to identify the factors that contributed to the decrease in nursing passing scores during the COVID-19 pandemic. Qualitative research is recommended for recognizing and assessing these factors from the nursing student and nursing education perspectives. Recommendations for further research include replicating this study in ASN and BSN programs to incorporate both aspects of prelicensure nursing education.

Conclusion

The COVID-19 pandemic had a negative impact on the passing scores of nursing students compared to prepandemic. The findings of this study can contribute to nursing education research by quantifying how the COVID-19 pandemic affected nursing students, nursing education, and its outcomes. This study may provide guidance for future studies to explore the effects of a pandemic on nursing students. The research findings help to show how the COVID-19 pandemic affected the passing scores of nursing students compared to prepandemic.