Deliberate practice and self-recorded demonstration of skill proficiency: One baccalaureate nursing school’s response to the COVID-19 pandemic

Abstract

Aim

The purpose of this study was to determine whether using self-video recording of the skill competency assessment would promote deliberate practice of nursing skills, clinical skill proficiency, limit virtual high stakes testing anxiety and facilitate progression of first semester baccalaureate nursing students during the global COVID-19 pandemic.

Design

A cross-sectional study was conducted during the Spring 2020 semester using self-reported student data from an end of course survey.

Methods

A 16 item instructor-designed survey of student’s perceptions of critical thinking, preparation, availability of materials, practice, video recording skill demonstrations, self-reported levels of anxiety related to the virtual testing environment and type of skill assessment exam was provided to all first semester baccalaureate nursing students enrolled in the basic nursing skills course.

Results

Approximately half of the cohort (N = 33) voluntarily responded to the instructor designed survey. The greatest relationship was observed between ‘adequate information for each version’ of the test and ‘adequate time to prepare’ (rho (32) = 0.729 p = 0.000). Although 54.6% (n = 18) of the respondents believed performing the demonstration in the home environment caused them to ‘think harder’ about the tasks, 78.8% (n = 26) reported feeling less anxiety than while performing previous demonstrations in the lab. A Wilcoxon test examined the results of the anxiety for demonstration on campus and anxiety for demonstration at home and a significant difference was found (p = 0.000, 95% CI) indicating anxiety levels were significantly less when demonstrating in the home environment. A moderate positive correlation was identified between opportunity to repeat with less stress at home (rho (32) = 0.61, p = 0.000), while moderate negative correlations were found between opportunity to repeat and anxiety levels related to recording (rho (32) = −0.60, p = 0.000), opportunity to repeat and anxiety related to demonstration at home (rho (32) = −0.53, p = 0.002) and concern about recording and opportunity to repeat (rho (32) = −0.49, p = 0.004).

Conclusions

Student success using remote assessment strategies during the Spring 2020 semester was similar to the success rate using traditional skill assessment methods in Fall 2019. Although the need for prompt feedback was identified as an area of improvement to promote deliberate practice, student video recording of skill proficiency was a viable solution for comprehensive remote assessment during the COVID-19 pandemic and campus closure. Although further study is recommended, findings have international implications for virtual teaching and learning in nursing education.

1. Introduction

The Spring 2020 semester presented challenges for nursing schools globally as the COVID-19 pandemic necessitated some campus closures, social distancing and a swift transition to alternative teaching and learning platforms. Although technology was available to provide theoretical instruction in the United States (Mariani, et al., 2020), Europe (Allande-Cussó, 2020; Hasalm, 2021) and other countries across the globe (Carolan et al., 2020), mechanisms to facilitate clinical skill practice and clinical practicum experiences during a global ‘lockdown’ were not widely known (Carolan et al., 2020; Hasalm, 2021). In the United Kingdom, the Nursing and Midwifery Counsel (2020) allowed nursing students the option of participating in paid clinical experiences through ‘Extended Clinical Placements’ which facilitated clinical hour attainment while contributing to the concerted national COVID-19 response. In the United States, each state determined acceptable substitutions for clinical requirements (National Council of State Boards of Nursing, 2020). Some states relaxed clinical hour requirements for upper division nursing courses while other states allowed on site and virtual simulation to replace all or part of the required clinical experience. Creating meaningful clinical and laboratory experiences proved to be more of a challenge during campus closures because clinical skills “are more difficult to replicate online” (Hasalm, 2021, p. 2). Student-created cell phone videos of skill demonstrations were successfully implemented in Korea and included students living in remote areas with limited internet access (Choi et al., 2015). To facilitate at home skill practice and demonstration of proficiency, one school of nursing in Southeast Texas used the sudden campus closure as an opportunity to explore the feasibility of using student created video recordings to validate the clinical skill performance of first semester nursing students.

2. Traditional clinical skill practice

Before COVID-19, students in the first semester clinical skill simulation lab course participated in a flipped classroom experience where lectures and skill demonstrations were viewed prior to lab. Students practiced clinical skills in the on-campus lab guided by faculty and during open labs staffed by peer mentors in the evening. During clinical skills lab classes, faculty were present to guide, instruct and demonstrate to improve and enhance student skill acquisition. Feedback from the instructor was an important part of the skills lab process and immediate feedback by faculty was highly valued by students (Choi et al., 2015, Furnes et al., 2018). Optional guided practice with peer mentors usually occurred in the evening during open lab hours.

3. Clinical Skill Performance Validation Before COVID-19

Prior to COVID-19, completion of the first semester skills course in the School of Nursing required satisfactory weekly performance on individual skills throughout the semester and a Comprehensive Skill Validation Exam (CSVE) at the end of the semester. Selected skills were validated by faculty during clinical lab time while other skill validations were recorded by peer mentors during open lab sessions and graded by faculty using course-developed rubrics. Students were required to successfully demonstrate skill proficiency each week to advance to the next skill.

Before COVID-19, retention of clinical skills in this nursing program was historically validated by a CSVE at the end of the semester. Open lab time was available for each student in addition to equipment for at home practice. Faculty traditionally discussed the four potential versions of the exam with students two weeks prior to administering it. Each exam version consisted of two simple and two complex skills. Simple skills include moving and positioning, range of motion, transfer and ambulation, hygiene and tube feeding while complex skills include sterile technique, urinary catheterization and wound care. On the exam day, students were randomly assigned to one of the four potential scenarios consisting of two simple and two complex skills. The CSVE was recorded for faculty review and to assess interrater agreement of this high-stakes exam. Students not achieving a passing score were required to attend mandatory remediation and allowed one additional opportunity to demonstrate skill proficiency on the CSVE.

Many students report this method of skill validation to be anxiety-producing and tend to associate errors with nervousness. In addition, many students report that the act of being recorded and viewed by faculty induces apprehension. In the end, if a student is not successful, the student is required to participate in additional practice and repeat the skill demonstration. Optimally, students must deliberately practice until proficiency is achieved, then participate in a second CSVE. The COVID-19 pandemic created an opportunity to explore alternative methods of skill validation in the literature to mitigate the impediments created by the closure of on campus facilities. Using the alternative method for the CSVE after the cohort had been exposed to the traditionally used on-campus demonstrations provided faculty the opportunity to compare the anxiety experienced by students using the two methods.

4. Literature review

Deliberate practice is a method used across the lifespan to promote the mastery of skills which are not inherent such as playing chess or learning a musical instrument (Ericsson et al., 1993). Immediate motivation to learn, preexisting knowledge and feedback are important characteristics which distinguish deliberate practice from simply gaining experience (Ericsson et al., 1993). Learning is improved when instructions are clear and followed up with feedback that is detailed enough to help the learner understand how to improve. Constraints such as limited resources, waning effort and conflicting motivations can hinder learning. Ericsson et al. (1993) tested the deliberate practice framework with two consecutive studies. In the first study interviews and practice diaries from professional violinists were analyzed: good violin students, best violin students and violin teachers. The authors believed these groups were a good fit for the theory because they were presumably able to self-critique during practice and not likely limited by resource constraints. As expected, the variable of accumulated hours of deliberate practice was the single best determinant of performance, with recent practice history also bearing importance. Sleep, naps and leisure time were less reliable variables in the determination of performance. The findings were replicated in a second study with amateur and expert pianists. Findings indicate expert pianists began playing at an early age and increased the amount of daily practice continuously, finally reaching a recent practice history of approximately 10 times that of the amateurs. These data support the importance of deliberate practice (Ericsson et al., 1993).

Deliberate practice has emerged as a concept in medical (Ericsson, 2015; Issenberg et al., 1999; Offiah et al., 2019) and nursing (Chee, 2014; Oermann et al., 2011; Wiggins et al. 2018) simulation-based education. Chee noted that Ericsson et al. (1993) focused on individual achievement, whereas Issenberg defined deliberate practice as “a stepwise process for teaching a cohort” (Issenberg, 2006, as cited in Chee, 2014, p. 249). Oermann et al. (2011) demonstrated that deliberate practice ensures skills retention over an extended time. In a study of 606 nursing students, deliberate practice significantly improved the quality of cardiopulmonary resuscitation skills for 12 months after the initial training compared with a control group Oermann et al. (2011). Offiah et al. (2019) studied 51 medical students throughout their final year of medical school and concluded that deliberate practice, under the supervision of qualified faculty, was an important component of skills mastery. Like Oermann et al. (2011) the authors emphasized the importance of continuous practice to maintain proficiency. In simulation-based education, skills are often practiced until each learner can perform them proficiently, but not necessarily until they have mastered it, before moving on to another skill (Chee, 2014).

5. Clinical skill acquisition and validation during COVID-19

During the Spring 2020 semester, the PI’s university was required to suddenly transition to virtual teaching and assessment strategies. Fortunately, throughout the semester, students were issued supplies and encouraged to practice at home (i.e. wound care kit, Foley catheterization kit), so this equipment was readily available to each student. The campus closure occurred after class instruction on the last new content (urinary catheterization) where all students received an indwelling catheter kit to practice setting up the sterile field and manipulating the equipment. To facilitate progression, course faculty needed a method for students to demonstrate skill performance. The School of Nursing surveyed students to assess computer and internet access. All students in the first semester cohort reported sufficient access; however, computers were available on loan from the School of Nursing if needed.

Students recorded their performance of catheterization of a water bottle in their home environment as a substitute for working with a manikin or task trainer in the Simulation Lab. This strategy allowed faculty to test the student video recording methodology prior to the CSVE. As the pandemic persisted, the faculty elected to conduct the CSVE in the same manner. Faculty conducted a synchronous web conference and demonstration using common supplies available to students in their homes in addition to their personal wound care and catheterization kits.

Despite the previous distribution of simulated sterile kits, the students did not have the moulage, manikin, or other medical equipment necessary to perform skills such as enteral feeding, catheterization, or wound care. To overcome this barrier (resource constraint) to demonstrating clinical skills at home, faculty offered synchronous instructional sessions via the online learning management system showing students how to substitute readily available objects for unavailable supplies and equipment. For example, a resealable food container replaced a denture cup and a water bottle simulated the meatus for urinary catheterization. Traditionally a classroom discussion occurred one to two weeks prior to the CSVE. The synchronous instructional sessions replaced the classroom discussion, ensuring that the same methodology was used during the COVID-19 closure semester.

Prior to grading the CSVE, interrater reliability of a previous CSVE was conducted which is consistent with recommendations from the National League for Nursing (Rizzolo et al., 2015). An intraclass correlation is the standard of practice for calculating interrater agreement in the School of Nursing and is calculated by the number of agreements/agreements plus the number of disagreements. Interrater agreement for the CSVE was 1.0 with five out of five faculty scoring the recorded skill performance ± 5 points from the mean. On the day of the CSVE, faculty emailed students the simulation scenario indicating the exam version and skills the student must demonstrate. Faculty also provided students with instructions on recording and uploading CSVE demonstrations to the learning management system by the end of the day.

Faculty anecdotally observed better performance on the Spring 2020 Comprehensive Skill Validation exam as compared with previous semesters despite the lack of lab practice time and student access to skills task trainers. Only one student out of 69 students (1.4%) was unsuccessful in the Spring of 2020 as compared with 7 students in the prior semester. Since the student recording process at home was new, faculty surveyed students to assess the student perspective and provide insight for the course evaluation.

6. Methods

After Institutional Review Board (IRB) approval was obtained, a descriptive and retrospective study was conducted to analyze survey data provided by first semester baccalaureate nursing students in the clinical skills course after the CSVE. Informed consent was not required by IRB because the survey data were anonymous and previously collected as part of the course evaluation. Also, a faculty-developed online survey was voluntarily completed by nearly half of the students in the cohort.

7. Instrument

The survey consisted of eight ordinal scale items [five items scored from 5 (strongly agree) to one (strongly disagree), two items reverse coded, one ordinal scale scored from 5 (extreme pressure) to 1 (totally relieved)], three items ranking anxiety from 0 (no anxiety) to 10 (highest anxiety), two categorical items and three interval items, for a total of 16 items on the instrument. The purpose of the survey was to understand student perspectives on the alternative format of the CSVE. Survey items are available in Table 1.

Table 1.

Comprehensive Skill Validation Exam Survey.

Survey Item
1.	When I learned that we would be recording our own final demonstration, I was concerned.
2.	The live discussion on Collaborate Ultra was helpful.
3.	The live discussion on Collaborate Ultra increased my anxiety.
4.	I had adequate information to prepare for each version of the comprehensive exam.
5.	I had adequate time to prepare for the Comprehensive Demonstration.
6.	How many minutes did it take you to gather the supplies needed for the Comprehensive Demonstration?
7.	How much anxiety do you usually have when performing a skill demonstration on campus? Rate your anxiety on a 0–10 scale.
8.	How much anxiety was caused by recording the demonstration? Rate your anxiety on a 0–10 scale.
9.	What was your level of anxiety in performing the demonstration at home? Rate your anxiety on a 0–10 scale.
10.	If you were able to pick up a packet with all the medical or specialty equipment needed for the demonstration, how would this affect your anxiety?
11.	It was less stressful to perform the skill demonstrations in my own environment with a friend, family member, stuffed animal (or other) than in the lab at school.
12.	Preparing to perform these skills at home actually caused me to think harder than when preparing to perform the skills in the lab at school.
13.	What or who did you use for your "patient" during the at home skill demonstration?
14.	How did you feel about having the opportunity to repeat the skills as many times as needed before submitting the video?
15.	On the day of the exam, how many times did you practice the demonstration before submitting it?
16.	What was your final score on the Comprehensive Demonstration?

8. Data analysis

Data were analyzed using the Statistical Package for Social Sciences (SPSS) version 24. Due to the nature of the data and levels of measurement, parametric and non-parametric tests were used. Descriptive statistics were examined for all items and Wilcoxon signed ranks tests were performed to compare the students’ reported anxiety levels during initial demonstrations in lab (prior to the university closure) with the CSVE recorded in the student’s own environment. Pearson’s correlation coefficient was used to assess the relationship between the number of times the skill was practiced prior to recording and the final demonstration score. Spearman’s rho was used to assess relationships between variables measured on an ordinal scale.

9. Results

Thirty-three of the 69 students enrolled in the course completed the survey. Responses to the faculty developed survey indicate 45.4% (n = 15) of the respondents were concerned about recording the final demonstrations. Most respondents (87.9%, n = 29) reported the faculty demonstration was helpful, while 9.1% (n = 3) reported it increased their anxiety. After the faculty demonstration, 78.7% (n = 26) of the respondents indicated adequate information was provided and 90.9% (n = 30) indicated having adequate time to prepare for the comprehensive demonstration. Although 54.6% (n = 18) of the respondents believed performing the demonstration in the home environment caused them to “think harder” about the tasks, 78.8% (n = 26) reported feeling less anxiety than while performing previous demonstrations in the lab.

Additional survey items revealed 81.8% (n = 27) of respondents felt somewhat or totally relieved to have the opportunity to repeat their performance, whereas 15.2% (n = 5) reported feeling some or extreme pressure to “get it right”. Although 66.7% (n = 22) of respondents indicate being provided with the missing supplies would decrease their anxiety, the remainder reveal it would not make a difference. To simulate skills, parents (n = 6, 18.2%), siblings (n = 7, 21.2%), significant others (n = 8, 24.2%) and friends (n = 6, 18.2%), served as simulated patients for the CSVE. Three percent (n = 1) of respondents used a pillow, 3% (n = 1) used a stuffed animal and 12% (n = 4) used other items. For more information on student perspectives on demonstration procedures ( Table 2).

Table 2.

Student perspectives on demonstration procedures.

Variable	N	Mean	Agree and Strongly Agree (%)
Concerned about recording	15	3.15	45%
Live discussion helped	29	4.45	88%
Live discussion increased anxiety	3	2.24	9%
Adequate info for each version	26	4.27	80%
Adequate time to prepare	30	4.45	92%
Less stress at home	26	4.27	79%
Home demonstration increased critical thinking	18	3.67	55%

Note: Items were scored on a Likert-type scale from 5 (strongly agree) to 1 (strongly disagree).

On average, respondents spent an average of 27.4 min (SD 25.7) gathering supplies and practiced the scenario an average of 4.6 times (SD 2.5) before submitting the final recording. The mean score on the CSVE for all respondents was 91.4% (SD 6.3), which was similar to the mean of 93% observed in the previous semester. The students rated anxiety during their initial demonstrations at school higher (M = 7.3, SD = 2.3) than that experienced during the comprehensive demonstration at home (M = 3.5, SD = 3.2), indicating respondents felt less stress at home. A Wilcoxon test examined the results of the anxiety for demonstration on campus and anxiety for demonstration at home and a significant difference was found (p = 0.000) indicating anxiety levels were significantly less when demonstrating in the home environment. For more information ( Table 3).

Table 3.

Anxiety Scores.

Variable	N	Mean
Anxiety for demonstration on campus	33	7.30
Anxiety about recording demonstration	33	4
Anxiety for demonstration at home	33	3.48

Note: items were scored on an 11-point Likert-type scale from 0 (no anxiety) to 10 (highest anxiety).

Spearman rho correlation coefficient was calculated for the relationship between student perspective variables measured on an ordinal scale. Moderate positive correlations were found between adequate time to prepare and live discussion helped (rho (32) = 0.40 p = 0.025), adequate info for each version and less stress at home (rho (32) = 0.45 p = 0.009), adequate time to prepare and less stress at home (rho (32) = 0.46, p = 0.007) and opportunity to repeat with less stress at home (rho (32) = 0.61, p = 0.000). The highest correlation coefficient was calculated for the relationship between adequate info for each version and adequate time to prepare (rho (32) = 0.73 p = 0.000). Moderate negative correlations were observed between opportunity to repeat and anxiety levels related to recording (rho (32) = −0.60, p = 0.000) and between opportunity to repeat and anxiety related to demonstration at home (rho (32) = −0.53, p = 0.002). In addition, a moderate negative correlation was calculated for concern about recording and opportunity to repeat (rho (32) = −0.49, p = 0.004). A Pearson’s correlation coefficient was calculated for the relationship between number of times practiced before recording and final demonstration score which revealed a very low and negative correlation (r (32) = −0.16, p = 0.373).

10. Discussion

Students enrolled in the first semester skills course live in urban, suburban and rural areas of Southeast Texas. Similar to Choi et al. (2015), it was feasible for all students in this study to record and upload their skill performance videos. Most students reported the synchronous web conference prior to the CSVE helped and did not increase anxiety. Respondents reported faculty provided adequate information about each potential exam version and that they had adequate time to prepare for the exam. Most respondents reported the home demonstration was less stressful and increased critical thinking to find items to simulate medical equipment. After participating in demonstrations in the Simulation Lab and at home, students reported the anxiety from demonstrations on campus was significantly greater than the anxiety with demonstrations at home. Analysis of anxiety levels may be skewed due to the abrupt onset of a global pandemic and the initiation of a new skill demonstration methodology; however, it is encouraging that students reported less stress in recording the demonstration at home.

Overall, students were relieved to have the opportunity to repeat the recording prior to the final submission; however, 15% reported some or extreme pressure to “get it right”. This finding reinforces previous recommendations (de Araujo Pastor Junior and de Melo Tavares, 2019) that students take a more active role in video facilitated learning and Choi et al.’s (2015) assertion that recording and reflecting can improve performance. The faculty were concerned that students feeling extreme pressure and lacking medical task trainers and medical equipment at home may be at a higher risk for failing the CSVE; however, only one student scored less than the 75% required to pass. This anxiety may have been mediated by the opportunity to practice the demonstration multiple times prior to submitting the final recording although increased practice before recording did not correlate with higher grades. This finding seems to conflict Ericsson et al.’s (1993) finding that increased practice leads to skill proficiency; however, the ‘practice’ addressed in this study was the number of times the skill was recorded prior to submission rather than overall practice time. The study may be strengthened by asking students the total amount of practice time prior to recording. A very small negative correlation between practice times and demonstration grades was observed which may indicate that students who were less prepared tended to have more attempts on the recording; however, the finding was not significant indicating the negative correlation may be due to chance.

Spearman rho correlation coefficient calculations indicate several moderate to strong relationships which are significant and not likely related to chance. The students reported having adequate time to prepare which was associated with the synchronous web conference discussion of expectations, feeling less stress performing the demonstration at home and adequate information given about each potential version of the exam. Overall, strategies used by faculty during the university closure were not significantly different from those used during a typical semester. Faculty used online technology to bridge the gap and create a sense of community with students at home. This validated Italla’s (2020) finding that schools of nursing which embraced technology were better able to implement online contingency education during the pandemic than those that did not embrace technology. Italla cautioned that while technology-based alternatives can supplement face to face learning experiences, the former may not completely replace the latter.

11. Limitations & recommendations

The study used a small convenience sample from one university in Southeast Texas which limits the generalizability of the findings. The faculty-developed survey instrument lacked established reliability which is also a limitation; however, a larger sample size would facilitate refinement of the instrument. Recommendations for future research include a larger sample size, multiple study sites, requiring a practice log to document student skill practice time, additional survey items regarding total practice time, using modular skill trainers for at home practice and replicating the study under traditional circumstances.

12. Conclusions

Along with its challenges, the COVID-19 pandemic created an opportunity to investigate innovative strategies for validating students’ skill performance under less-than-optimal conditions. Circumstances necessitated that the students develop a plan to demonstrate and record themselves at an improvised location off campus. Additionally, the students practiced integrating multiple skills into a patient care scenario without the benefit of immediate faculty feedback. The synchronous web conference about performing the CSVE at home helped students overcome resource constraints; however, students were required to rely on video review and self-reflection to improve performance. Although most students reported initial concern about demonstrating from home and self-recording, both may have been mediated by student control over the submission of the recording and opportunity for additional practice. If schools of nursing remain closed, or are closed for the entire semester, faculty need to provide a mechanism for specific and quality feedback on skill performance (Choi et al., 2015), require a log of practice time to document sufficient practice for proficiency (Ericsson et al., 1993) and consider modular skill trainers for at home practice. Student recordings of skill demonstrations at home may be a viable option for the future. Although the study findings warrant further investigation, the international nursing education community may identify future implications for the use of virtual competency assessment.

Role of Funding Source

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

CRediT authorship contribution statement

Dr. Troy J. Palmer, EdD, MSN, RN. Conceptualization, Methodology, Validation, Formal analysis, Investigation, Resources, Data curation, Writing - original draft, Writing - review & editing, Visualization, Project administration. LeAnn J. Chisholm, PhD, RN, CNE, CHSE. Conceptualization, Methodology, Validation, Formal analysis, Investigation, Resources, Data curation, Writing - original draft, Writing - review & editing, Visualization, Supervision. Carmen G. Rolf, MSN, RN. Validation, Investigation, Writing - original draft, Writing - review & editing. Christina Morris, MSN, RN, FNP-C. Methodology, Validation, Investigation, Writing - original draft, Writing - review & editing.