Student and instructor perspectives following a virtual objective structured clinical examination

Abstract

Background:

In March 2020, COVID-19 public health restrictions impeded in-person clinical assessment. In response, a dental hygiene program administered a virtual objective structured clinical exam (vOSCE) using Zoom to assess student competency in performing a health history. This study aimed to explore the vOSCE experience from both student and clinical instructor perspectives.

Methods:

This 2-part cross-sectional study gathered student and clinical instructor perceptions of the vOSCE. Forty-two students were invited to complete an online questionnaire. Basic descriptive statistics reporting percentages were tabulated. Twelve clinical instructors were invited to participate in focus groups, which were audiorecorded and transcribed verbatim. Qualitative data were analysed using inductive thematic analysis.

Results:

Questionnaires were received from 23 (55%) students. Students supported (91%) the vOSCE experience and believed it assessed their knowledge (87%), their ability to complete a health history (91%), and ability to communicate effectively (87%). Students reported high agreement (87%) with how the Zoom platform facilitated the examination. Some students (35%) felt the vOSCE was more stressful than an in-person OSCE. However, 43% indicated it wasn’t more stressful. Focus groups with clinical instructors revealed perspectives on using vOSCEs, which were captured under 4 themes: preparation, assessment suitability, authenticity, and future considerations.

Conclusion:

Based on student and instructor perspectives, vOSCEs could be a viable alternative to in-person OSCEs for health history evaluations. As technology applications continue to emerge for conducting virtual examinations, there may be increased use of and ease of use with a virtual platform to conduct other types of clinical evaluations.

PRACTICAL IMPLICATIONS OF THIS RESEARCH.

• Given the widespread use of technology in our changing world, educational institutions must determine how to make the most effective use of virtual evaluations.

• Conducting a vOSCE can be complex, requiring meticulous planning and preparation to avoid common pitfalls.

• Research is needed to ensure the scholarly development of novel ideas or educational processes that emerged during the COVID-19 pandemic.

INTRODUCTION

Objective structured clinical examinations (OSCEs) have been used for many years in health sciences educational programs.^{1, 2} Compared to more traditional types of examinations, an OSCE enables educators to identify students’ specific skills shortcomings and provide students with hands-on, interactive learning designed to reinforce and build upon basic concepts.³ Although the OSCE format may be time-consuming and resource-intensive, it provides invaluable feedback that can be utilized to inform curriculum modifications to improve student competency in essential skills and knowledge and produce competent entry-to-practice practitioners.⁴

Defined as a performance-based assessment in a simulated setting, where examinees are observed and assessed in a structured way,^{5, 6} the OSCE aims to evaluate professional skills, the application of clinical knowledge, problem solving ability, communication skills, and the ability of students to “think on their feet”.^{7 p135, 8–10} OSCEs employ standardized patients and scoring rubrics for consistency in the clinical examination setting and calibration of the examiners,⁸ and may involve either a series of short stations, each focused on a specific task, or a single station focused on a more complete or holistic assessment.⁶ An OSCE assesses not only the basic skills of students, but also their higher levels of cognition.¹⁰

When in-person learning at the University of Alberta, Canada, was suspended during winter term 2020 due to the COVID-19 pandemic, remote delivery of courses was required.¹¹ Conducting clinical evaluations became near impossible or very challenging. After much consideration of logistical factors and available resources, a decision was made to use the Zoom platform¹² to administer a virtual OSCE (vOSCE) in a dental hygiene clinical course. OSCEs often make use of trained actors hired to play a standardized patient in the specific scenarios.^{2, 13} However, due to public health restrictions during the first lockdown, the hiring process for actors capable of portraying standardized patients was temporarily put on hold. As a result, a decision was made to use clinical instructors from the dental hygiene program as the standardized patients for the vOSCE.

Process for the virtual objective structured clinical examination

The vOSCE is a timed, one-station evaluation activity during which students use a case vignette to complete a health history while conducting an interview with a standardized patient in a simulated clinical environment. The evaluation aims to assess students’ ability to identify significant health history findings and to demonstrate effective verbal and non-verbal communication skills when discussing the findings with a client as outlined in the Canadian Competencies for Baccalaureate Dental Hygiene Programs.¹⁴

For the vOSCE in 2020, clinical dental hygiene instructors served as either the standardized patient (SP) or the examiner. An orientation session was planned since a virtual clinical examination was novel to the instructors and the students. The course coordinator, administrative course support, and all 12 clinical instructors, irrespective of role in the vOSCE, met via Zoom¹² to discuss the evaluation rubric and the overall format for the examination. The course administrative support organized a practice session to allow instructors to experience their roles in the Zoom breakout room and calibrate the process for examination day. To calibrate the clinical instructors who were SPs, the mee,ting involved a discussion of the scenario script. Similarly, an orientation session was scheduled for the students to prepare them for the virtual clinical examination. The orientation included guidance on how to prepare for the examination, a description of the vOSCE process, and a practice client scenario. Lastly, 10 minutes prior to the student’s scheduled vOSCE, they received an email that contained the client’s health history. The client’s health history was a replica of an actual health history form and included medical, dental, and pharmacological information. In 2020, the simulated adult client responded affirmatively for having thyroid or parathyroid disease and having a blood sugar test but no diagnosis of diabetes, was taking 2 prescribed medications, reported bleeding gums, tissue soreness and swelling, and had not been to the dental hygienist for a few years but had seen a dentist about 2 years ago.

Forty-two (42) first-year dental hygiene students were scheduled into 6 breakout rooms, which were designed with a video filter to simulate a dental operatory. One SP and one examiner were present in each breakout room. An administrative support person admitted and moved each cohort of 6 students into the Zoom waiting and breakout rooms,¹² respectively, at 15-minute intervals. The examination proceeded with the student reviewing and assessing the health history with the client to identify and communicate significant findings. The examiner evaluated the student’s performance using a scoring rubric, monitored time, and provided a 2-minute warning when the student was approaching the maximum time allotted. The course coordinator also served as 1 of the 6 examiners. Once the examination was complete, students exited the breakout rooms and Zoom¹² and the next group of students were admitted. Of note was the importance of the administrative support to oversee the flow of the vOSCE and deal with any technical issues (e.g., webcam and microphone malfunctions, Zoom access¹²).

The overall aim of this study was to explore dental hygiene students’ and clinical instructors’ perceptions of and experiences with a vOSCE. The specific research questions guiding this study were as follows:

1. What were dental hygiene students’ experiences with the vOSCE?

2. What were dental hygiene clinical instructors’ experiences in the role as a standardized patient?

3. What were dental hygiene clinical instructors’ perceptions of and experiences with the vOSCE?

METHODS

This 2-part cross-sectional study was conducted with dental hygiene students and dental hygiene clinical instructors. The student cohort was in the first year of their Bachelor of Science (Dental Hygiene) program; all 42 students were invited to complete an anonymous online survey. All 12 dental hygiene clinical instructors were invited to participate in a focus group session. The University of Alberta’s Research Ethics Board (Pro00110642) approved the study.

Survey development, delivery, and data analysis

The student survey was modified from a questionnaire used by Furlong et al.¹⁵ who investigated nursing students’ experiences and perceptions after participating in OSCEs. A 12-item survey was created with 5-point Likert scale statements and open-ended questions. Statements were organized into 3 sections with 4 statements per section as follows: vOSCE content, utilization of Zoom¹² for the examination, and overall student experience in the vOSCE. Survey data were collected and managed using REDCap,^{16, 17} the university’s secure, online survey tool.

To maintain impartiality, a non-research team member with no involvement in the clinical course invited students to complete the survey, monitored survey completion, and cued reminders until the survey closed 8 weeks later. All students were invited to participate in the survey once they had completed the clinical course and received their final grades. Therefore, there was no perceived conflict between students and faculty members who were part of the research team.

The survey data were analysed using descriptive statistics. Ordinal variables were reported using frequency percentages and the mode for each item.

Focus group design and data analysis

Two focus groups were used to gain insight into the experience of the clinical instructors involved in the vOSCE. All 12 instructors were invited to participate. Instructors who consented to participate were asked to complete a priming activity before the focus groups to prompt their memory and reflections of their experience as there was a 3 to 4 month time lapse between the vOSCE and the focus group sessions.¹⁸ The priming activity contained a situational outline of the virtual examination (e.g., the conditions that prompted the transition to a virtual format, the content addressed, and the competencies assessed) and several reflective questions (e.g., how effective do you feel the virtual OSCE process was in assessing the student’s ability to complete an effective health history interview with a client? What worked well? What didn’t?).

Two 60-minute focus group sessions, one with 4 instructors and the other with 3 instructors, were conducted via Zoom,¹² audiorecorded using a secure handheld device, facilitated by an arms-length individual, and moderated by an administrative assistant who recorded field notes. The research team developed a semistructured interview guide that consisted of open-ended questions aligned with the 3 sections of the student survey.

The recorded transcripts were professionally processed by Transcript Heros,¹⁹ and verbatim transcript data were checked for accuracy, anonymised, and managed using NVivo 12 Pro²⁰. The data were analysed using inductive thematic analysis. Specifically, Braun and Clarke’s²¹ iterative 6-step process of thematic analysis was followed, which entailed becoming familiarized with the data, generating codes, searching, reviewing and defining themes, and producing the report. Although all team members thoroughly reviewed the transcripts to familiarize themselves with the data, qualitative data analysis was primarily completed by 2 research team members: the focus group facilitator (MH) and the clinical course coordinator where the vOSCE was used (SL).

MH and SL independently read the transcripts and developed open codes. Following open coding, MH and SL met and discussed their individual codes and, through consensus, further refined and agreed upon a coding scheme. These team members then independently applied this coding scheme to the transcript data and met again to discuss their interpretations and definitions of themes, subthemes, and associated aspects until unanimity was established. Thematic maps and tables were developed and shared with the research team. Subsequently, all 4 team members met to discuss the analysis and reach final agreement on the themes.

RESULTS

Twenty-three of 42 students (55%) in the dental hygiene student cohort completed the survey and 7 dental hygiene clinical instructors (6 SPs and 1 examiner) participated in the focus group sessions.

Survey findings

Frequency distribution of survey responses is shown in Table 1. The majority of students (87%) felt the vOSCE adequately assessed their knowledge and ability to complete a health history review. Over 90% agreed or strongly agreed that they were prepared to complete a health history prior to taking part in the vOSCE (Table 1). The majority of students indicated that the health history vOSCE was able to assess their abilities to identify significant health findings (91%) and communicate effectively (87%) with their client. When asked whether the online platform was conducive to completing a health history interview, 87% agreed or strongly agreed (Table 1). A majority of students felt that the overall transition within the vOSCE on the Zoom platform¹² was seamless (96%) and that they received adequate instructions (100%) for what to expect once they accessed the Zoom platform.¹²

Almost half of the students (43%) reported that moving the OSCE to the online platform did not affect their stress levels (Table 1). However, when asked what aspects of the vOSCE did affect their stress levels, potential internet connection instability and anxiety about receiving the OSCE preparatory documents were the 2 most frequently cited responses (Table 2). Overall, all students who completed the survey indicated that the Zoom platform¹² provided a fair way to complete this assessment (Table 1), with more than 90% indicating that the overall experience with the vOSCE was good or very good.

Focus group findings

Four common themes emerged from the focus groups and are described as preparation, assessment suitability, authenticity, and future considerations. A depiction of themes and subthemes is presented in Figure 1; representative comments from each theme are presented in Table 3.

Theme 1: Preparation

According to instructors, having the necessary support, information, training, and expertise was essential to feeling confident during the vOSCE process. They emphasized that the presence of strong administrative support facilitated vOSCE flow and built confidence that technological issues would not derail the experience. Instructors also highlighted that expertise specific to the role assumed (as SPs or examiners) was an important consideration. Instructors preferred the examiner role due to their experience and familiarity with the role but expressed that they received “good information [from the course coordinator]” to prepare as the SP. However, instructors also shared that a lack of SP experience prompted worries about providing a standardized performance.

Table 1.

Student responses about the experience of the vOSCE (n = 23)

Item	Strongly disagree (1)	Disagree (2)	Neutral (3)	Agree (4)	Strongly agree (5)	Mode
Content
I felt prepared to complete a medical history review/interview prior to taking part in the OSCE.	0.0	4.3%	4.3%	39.1%	52.2%	5
I felt that the OSCE assessed my knowledge with respect to the process of completing a medical history review with a client.	0.0	0.0	13.0%	43.5%	43.5%	4/5
I felt that the OSCE assessed my ability to identify any significant findings from the medical history.	0.0	0.0	8.7%	56.5%	34.8%	4
I felt that the OSCE assessed my ability to communicate effectively with the client.	0.0	0.0	13.0%	52.2%	34.8%	4
Online (Zoom) process
I received adequate direction on how to access the online OSCE.	0.0	0.0	0.0	22%	78%	5
I received adequate direction on what to expect with the OSCE process once I entered the online site (Zoom platform).	0.0	0.0	0.0	30.4%	69.6%	5
I thought the transitions between the waiting room and the breakout rooms were seamless.	0.0	0.0	4.3%	39.1%	56.5%	5
I thought the online platform was suitable for completing the evaluation for a medical history interview.	0.0	0.0	13.0%	56.5%	30.4%	4
Overall experience
I feel that moving this assessment to an online platform added to my stress level when completing the OSCE.a	4.3%	39.1%	21.7%	26.1%	8.7%	2
My experience in completing the online medical history interview was comparable to my experience when completing this interview face-to-face in my DH clinics in KEC.	0.0	30%	43%	22%	4%	3
I feel that using the Zoom platform to complete this OSCE provided a fair way to complete this assessment.	0.0	0.0	0.0	69.6%	30.4%	4
Student rating of overall vOSCE experience	Very poor (1)	Poor (2)	Neutral (3)	Good (4)	Very good (5)	Mode
Rank your overall experience with completing the online OSCE.	0.0	0.0	8.7%	47.8%	43.5%	4

^aRefer to Table 2 for more information on student stressors

Table 2.

Student responses regarding vOSCE stressors, by raw score of total responses (multiple responses allowed)

Item	Strongly disagree (1)
Internet connection instability	7
Familiarity with Zoom platform	0
Feeling “distanced” with the online interview (i.e., loss of personal connection)	5
Anxiety about receiving OSCE preparatory documents	6
Not applicable	6

Theme 2: Assessment suitability

Overall, instructors felt that the virtual platform was suitable for assessing the specified competencies but highlighted limitations in evaluating competencies related to students’ nonverbal communication, preparedness, and knowledge. Unanimously, instructors emphasized that nonverbal communication including eye contact, body language, and empathy was difficult to assess in the virtual environment. To mitigate this limitation, instructors recommended that, in future, students modify their camera angle so that more of their body would be visible. Additionally, instructors suggested that a different scenario could be developed requiring a greater display of emotion and vulnerability from the SP.

Some instructors felt that it was difficult to gauge the students’ preparedness and knowledge because they perceived greater “scripting’’ in the virtual environment, which they attributed to students’ ability to read notes from their screen. Others reported that the virtual platform did not affect their ability to assess this competency and opined that scripting was dependent on the student rather than the context. Instructors recommended withholding information “that would lead to scripting” to steer students away from “checklists’’ and towards “open-ended questions’’ to encourage a more natural conversation. Instructors also commented that the conversational nature of this vOSCE’s content (a health history) may be more suitable for the virtual platform compared to content that would require greater interaction through the use of additional props (e.g., learning aids for tobacco cessation counselling or oral health education).

Figure 1.

Instructor focus group qualitative data analysis

Table 3.

Instructor focus group representative comments

Theme 1: Preparation
Subthemes and aspects: Support Adequate information, training, and expertise: Technology Students Instructors	Participant 1: “You need the administrator to be excellent” Participant 4: “Technology wise…I felt nervous…but after doing it and then seeing how [the administrator] ran the rooms, I feel like it’s not a problem” Participant 6: “I found it easier being the evaluator than the standardized patient because you want to perform the same for each student, so you’re not giving any student advantage over the others” Participant 4: “I felt confident because [we had been given] some good information…I felt I really wanted to do well for the students and felt nervous that I wouldn’t be able to get them all of the information. But I feel like I did. I guess after you’ve done it, you feel good about it” Participant 3: “We were well guided by not just [the administrator] but were given the background of patients and all the information that we needed to act out” Participant 2: “In terms of preparing, it was more so just understanding the script so that you’re giving that correct information and trying to probe them [students] as best as you can without giving them the answers” Participant 5: “I was the standardized patient for this one and because I’ve been involved in a number of OSCEs in the past, working with a standard patient, they always had questions. So I think I was a bit more prepared to be a standardized patient because I knew what the standardized patient was asking in previous OSCEs and how to look at it”
Theme 2: Assessment suitability
Subthemes and aspects: Nonverbal communication Preparedness/knowledge Content	Participant 7: “Eye contact was difficult…I can look at the camera and watch what they’re doing, but I don’t think they knew where to look sometimes. So, I think that part is hard, for like the eye contact and connecting that way online. I still find – like even now I’m trying to look at people, and I find it hard” Participant 4: “Empathy is hard to assess in a virtual environment because leaning in, or something like that – the virtual environment doesn’t lend itself to doing that. The students were very professional and friendly and very open, but I think that sort of transition into empathy doesn’t come through necessarily, but it’s maybe not inspired by looking at a screen, the way it’s inspired looking face-to-face.” Participant 1: “I think the positioning of our cameras could be adjusted differently. So that you could see a bigger view of what the person’s doing with their hands.” Participant 4: “[The] standardized patient could be directed to give information that makes them more vulnerable…Maybe if one of the lines is, “I’m really afraid of the dentist, because of something that happened to me,” that could lead to a line of questioning by the student that could allow them to show some more empathy” Participant 2: “I think since it’s conducted online, on a platform, it’s really hard to gauge how prepared they are. Like even when I’m talking to you, I have a document that’s open. I feel like they were reading a lot from that document to really tick off certain things”
Theme 3: Authenticity
Subthemes and aspects: Environment Student–SP relationship	Participant 1: “Those little tweaks of changing your image, putting a different virtual image, I think are very cool…that’s a good idea.” Participant 4: “The energy, I guess of being there, having it be the in-person OSCE; waiting and seeing your peers; versus perhaps it’s a more calm energy when you’re at home and this is up to you and you’re going to do your thing [while they waited].” Participant 6: “Being removed and being in your own space [at home] can create a feeling of comfort and less anxiety…So, then you’re better paced, not as flustered.” Participant 5: “Would [they] show different emotions, different connection [with an actor]? Cause with the instructors, they maybe are a bit more relaxed and they might not be having that more nurturing kind of conversation potentially. So that might make a difference if it wasn’t the instructor [as SP].” Participant 3: “When we had hired SPs, it really produced an authenticity and a reality [for the students], like, “I don’t know this person”, and I’ve got to use my communication skills to sort of relax and get them, get them comfortable so that I can ask these, these even more tough questions and things [about their health history].”
Theme 4: Future considerations
Subthemes and aspects: Calibration Value: Flexibility Cost Role wisdom Tele-oral health and associated education	Participant 3: “The examiners took themselves off video, right? So, I think that was a good idea…Participant 4: Did everybody do that? Because we didn’t. Participant 6: I was going to say – we didn’t do that either” Participant 1: “The instructors were more expensive than [the actor SPs]. But I think their value might have been better. Participant 3: “I think there’s value in someone who does it and who has done it. Because they could maybe do a better job for the student, whether it’s an instructor who always is the SP, or someone we hire; that might be more effective than someone who’s doing it for the first time.” Participant 4: “I think [in-person] gives more value to the student, just because that’s how we practice dental hygiene. But I thought this OSCE and the other one we did virtually were wonderful, as far as the preparation and getting through those questions. I thought they were great. But yeah, I think in person there’s more value.” Participant 5: “A lot of doctors through COVID now are doing virtual visits. So it’s a skill that they have to learn, too. And I don’t think it was a bad thing to be able to do something like that.” Participant 3: “I don’t think we’ll ever probably return to [pre-COVID practices]. I think this worked too well in that respect. I think we’d all agree. We’d all question paying people to do what I think our instructors did a very, very good job at. Would I rather do it in person? Yeah. But the reality of our times, and finances, this virtual was a good thing.” Participant 4: “Maybe there is value in having one virtual, one in person, having both experiences for each class to be prepared for something like this having to happen.”

Theme 3: Authenticity

Instructors perceived differences in the realism and ambiance of the evaluation activity with respect to the examination environment (virtual versus in-person) and the student–SP relationship (instructor SPs versus hired actor SPs). They pointed out that the face-to-face environment was more realistic due to its capacity to authentically replicate a traditional clinical experience. However, instructors also felt that subtle “tweaks” could enhance the realism of the virtual environment, such as having students wear scrubs and changing their Zoom¹² background to a clinical image. Compared to in-person OSCEs, the ambiance of the vOSCE, according to the instructors, seemed more “at ease”, “calm”, and “better paced”. They suggested this relaxed atmosphere could be due to students being in their own space and away from the collective stress and “chatter” of pre-examination anticipation when surrounded by peers as they awaited their scheduled time slot. Instructors also inferred that students may have been more relaxed interacting with their instructor as the SP as there was an established relationship. However, they pointed out that this familiarity could affect the authenticity of the interaction since students may not have the same “cue” to build rapport and to “make the patient comfortable”. As a result, instructors conveyed that, in future, hiring an actor SP might be more realistic and appropriate.

Theme 4: Future considerations

Regarding the continued use of vOSCEs, instructors discussed calibration issues with examiners (e.g., having their camera on versus off) and instructor SPs (e.g., instructors potentially giving too much or too little information to students) during the vOSCE, as well as with previous actor SPs (e.g., not following their script). To address these calibration issues, instructors explored the value of having instructor versus actor SPs. Despite a greater associated cost, they surmised that instructor SPs may hold more experiential value than actor SPs due to the instructors’ expertise in the clinical content and competencies. However, they also recognized that there is value in hiring an experienced actor SP rather than a novice. As for the future of tele-oral health and associated education, some instructors felt that in-person OSCEs provide the most value to students. In contrast, others noted a growing emphasis on the need for remote health care skills post-pandemic. Instructors indicated that the virtual environment may remain commonplace in postsecondary education, especially considering the general ease of both technology use and the vOSCE process. Lastly, others felt that there may be value in implementing both virtual and in-person OSCEs.

DISCUSSION

At the onset of the pandemic in early 2020, it was critical to find alternative ways to conduct clinical evaluations. While most clinical evaluation components could not be completed virtually, one component was well suited to virtual evaluation: assessing a student taking a health history with a SP. This study gathered the perspectives and experiences of undergraduate dental hygiene students and dental hygiene clinical instructors after their participation in a virtual objective structured clinical examination (vOSCE).

Overall, students reported agreement with the vOSCE experience as a means of assessing their knowledge and ability to take a health history. Other studies that have examined student experiences in vOSCEs have similarly reported that students held positive views of the virtual examination^22–25 and felt they were able to fully demonstrate their knowledge in the online environment.^{24, 25} Although students in the current study had no prior exposure to any type of OSCE, the majority indicated they received adequate instructions on what to expect and reported an overall acceptable experience with the virtual examination. These results are consistent with other research on vOSCEs, which found that using strategies to familiarize students with vOSCE logistics (e.g., providing advance information on technical issues and time management and/or practice sessions) was important to support students’ preparation and to relieve their stress over the examination.^{13, 22, 23, 25}

As OSCE examinations often require students to synthesize and apply knowledge to problem-solve clinical scenarios,²² an encouraging finding was that the majority of students felt prepared to complete the vOSCE, which aligned with the instructors' comments that students held an appropriate level of “baseline knowledge” to complete the vOSCE. Similarly, in a study examining dental studen,t feedback following their experience in a vOSCE,²² students reported that the topics and scenarios covered in the examination aligned with their educational training and were clinically relevant. Moreover, a study by Majumder et al.²⁶ found the majority of medical students positively perceived the performance criteria of an in-person OSCE. However, one quarter of students were dissatisfied with the OSCE’s authenticity and context and expressed concerns about its utility as a practical experience. Considering these results in conjunction with the current study’s findings, it is crucial to align OSCE content, learning outcomes, and activities or tasks with students’ current theoretical and clinical competency and exposure to educational experiences. The literature highlights that clinical relevance is an important component of student engagement in learning activities²⁷ and that active engagement facilitates learning.²⁸ Thus, taking steps to emphasize the OSCE’s clinical relevance may support students’ examination preparation and overall OSCE experience in either environment.

Studies have reported that students often experience higher anxiety and stress in OSCE examinations compared to other types of examinations, such as written examinations.^{29, 30} Students in this study indicated that changing the pre-pandemic OSCE to a vOSCE did not affect their stress levels. However, when asked what aspects of the vOSCE did increase their stress levels, students reported possible internet instability as the highest stressor. Similarly, Hytönen et al.²² found that undergraduate dental students also had concerns about technical issues, such as connection problems in the online vOSCE environment, and reported that this contributed to their stress. Although Millennial and Generation Z students are very capable users of technology^{31, 32} and Zoom,¹² it is not surprising that students may experience stress during online evaluations due to the uncertainty that can arise when relying on technology.³³ To troubleshoot any technical issues during the vOSCE, an administrative assistant with a high level of technological expertise was always present during the sessions. Therefore, if anyone involved in the vOSCE experienced technical issues, the necessary support was available to resolve the problem. In parallel, other studies have highlighted the importance of administrative and/or instructional technology (IT) support to address technical issues and facilitate smooth vOSCE implementation.^{23, 24, 34–36}

When the dental hygiene program decided to conduct the OSCE virtually due to the COVID-19 pandemic, the need for detailed planning was recognized to ensure successful delivery of the examination. Subsequently, it was determined that dental hygiene clinical instructors would serve as the SPs instead of hiring paid actors to play those roles. Similar to Donn et al.¹³ the course coordinator chose to use instructors as SPs rather than actors to decrease the likelihood of disruptions to the vOSCE process, such as last-minute SP cancellations due to illness. For instance, it would be challenging to replace an actor SP at short notice due to the calibration, coordination, and technical training required for the event. However, if an instructor SP cancelled at the last minute due to illness, it would be less cumbersome and more efficient to recruit another instructor who is already familiar with the vOSCE content.

Clinical instructors in the role of either SP or examiner reported that thorough preparation was key to the success of the vOSCE. Similarly, other studies of examiner experiences in either virtual or in-person OSCEs have highlighted the importance of training, guidance, and support to ease anxieties and build confidence prior to the examination.²⁵

Clinical instructors reported the virtual platform was a suitable medium in which to complete the health history examination yet recognized the limitations of the online environment for assessing student preparedness and nonverbal communication competencies. The instructors in the group who had previously completed several in-person OSCEs prior to the vOSCE did not perceive any noticeable differences in student preparation and remarked that scripting is dependent on the individual student rather than the environment. Very little was found in the literature about whether the online context impacted students’ preparedness for vOSCEs compared to those conducted in person, which may be explained by the relatively novel use of virtual platforms to complete OSCEs. Prior studies on in-person OSCEs have reported that students engage in greater preparation for OSCEs compared to other forms of assessment.²⁹ However, reports also suggest that students strategically prepare for OSCEs by adopting a scripted or checklist approach, which makes it challenging for examiners to discern students’ true clinical competence in a dynamic environment.^{26, 37} To prevent scripting, instructors suggested OSCE scenarios should exclude information that would lead to checklists and scripting and include information that would be best identified through open-ended questions. Hopwood et al.³⁵ suggested that it may help to decrease scripting if students are provided with broader information or a resource list of what may be needed during the OSCE and not simply the exact information and resources needed for the exam.

The instructors unanimously reported that the online environment impeded the evaluation of nonverbal communication and suggested optimizing the camera field to enable body language interpretation and integrating specific OSCE scenario features (e.g., SP emotion) to encourage other communication aspects (e.g., empathy) to counterpoise this. Studies that collected instructor and other stakeholder feedback post-vOSCE found that respondents felt the virtual examination was well suited to assessing communication and a range of other skills that would traditionally be tested face-to-face (e.g., professionalism, written documentation, and practical skills).^{25, 35}

The instructors reported greater authenticity in the traditional, face-to-face OSCE environment compared to that of the virtual environment, which is a finding described in the literature.²⁵ To make the virtual environment more authentic, the instructors suggested students wear scrubs and change their Zoom backdrop to a clinical image. Similarly, Pante et al.³⁸ recommended neutral virtual backgrounds and appropriate clothing to professionalize the virtual environment. Hubbard³⁹ emphasized the importance of a neutral backdrop to ensure all students would appear equally professional and “level the online playing field”^p976 for students with inequitable access to more elaborate backdrops.

A recent study by Thampy et al.²⁵ investigating vOSCE stakeholder views and perspectives found that SPs perceived a calmer student demeanor during the virtual examination and that students reported high agreement with the comfort of taking the examination at home. Similarly, instructors in the current study reported that students seemed calmer in the virtual examination as compared to previous in-person OSCEs and attributed this difference to their being in their own space. However, it could be argued that these positive results were due to the extra preparation and thorough orientation associated with implementing a novel virtual examination. Therefore, further studies examining the relationship between the virtual environment, student comfort, and examination performance which take these variables into account will need to be undertaken.

There is little published data on perceptions of how the use of instructors as SPs contributes to students’ overall vOSCE experience and performance. In their pilot study of a vOSCE, Donn et al.¹³ found that staff were initially concerned that interacting with a known staff member as the SP would hamper students’ ability to relate appropriately and make the examination more dif,ficult. However, the authors noted the absence of such comments from staff and student in post-vOSCE feedback and further reported students felt at ease interacting with the staff SP. Consistent with these findings, instructors in the current study shared similar worries that the students’ existing rapport with an instructor might diminish the authenticity of the student–SP interaction but also highlighted that students might benefit from the comfort of seeing a familiar face during a novel and stressful experience. Although there is scant literature on the former, the latter aligns with findings in neuropsychopharmacology literature which indicate that socially familiar conspecifics serve as a safety signal and may play a role in reducing anxiety-like responses.^{40, 41}

Instructors in the current study commented that their vOSCE content expertise but lack of SP experience may have affected their ability to give a standardized performance as the patient. Traba et al.⁴² similarly drew attention to the lack of acting experience among medical residents utilized as SPs during a vOSCE but also reported the residents’ clinical expertise allowed for more robust feedback. Interestingly, one of the instructors in the current study reported feeling that their clinical expertise could enable enhanced feedback through the SP lens. The current study’s findings are also consistent with that of Adamo⁴³ who posited that actor SPs with experiential knowledge gained through numerous work events who are also trained to provide feedback may be tempted to prompt students and/or break character as the SP to assume the role of a “precepting teacher”^p265. Other challenges reported in the literature with using actors as SPs include training standards, tardiness, reliability, quality, and cost.⁴³

In reflecting on the future use of a vOSCE platform, instructors discussed considerations related to calibration, economics, and the emerging presence of technology. In terms of calibration, instructors emphasized the need to ensure that guidance is clearly detailed to indicate who should have their camera on. Because some examiners kept the camera on while others did not, the instructors wondered if this discrepancy had an impact on student performance. Furthermore, some instructor SPs provided more prompting than other SPs during the role-play scenario, also illustrating the need for better calibration in future OSCEs. Hopwood et al.³⁵ provided a comprehensive list of tips for delivering a vOSCE and recommended that the examiner remain off camera to lessen any interference in the examination process. The course coordinator for the dental hygiene program at the University of Alberta has now implemented this recommendation in the vOSCE.

In regard to economic considerations for future use of vOSCEs, instructors noted the cost to conduct the exams via Zoom was lower than for in-person exams since there were no physical room bookings or clinical chair utilization required. However, dental hygiene clinical instructors are more costly than paid actors who previously served in the SP role. Nevertheless, instructors pondered if there might be a cost trade-off in that the instructors may provide more value in the authentic experience that they bring to the setting given they are dental hygienists and experienced in the content area in contrast to hired actors. Pre-pandemic studies reported that the cost of SPs was often a limitation to more frequent use of in-person OSCEs and suggested that an alternative to reduce costs might be virtual OSCEs.^{43, 44}

Lastly, instructors discussed that using vOSCEs was important to a student’s overall skill development given emerging use of technology and telehealth possibilities in the future. Therefore, the vOSCE may foster a student’s ability to communicate effectively via telehealth platforms should they become increasingly used in oral health care settings. Similar projections have been reported in the medical and dental fields.^{35, 42, 45, 46} Overall, instructors believed it will be important to utilize both in-person and virtual OSCE sessions in a post-pandemic world.

A final suggestion for future consideration is how vOSCEs may be utilized for national clinic examinations. With the expanding discussion about a national performance-based examination by the Federation of Dental Hygiene Regulators of Canada, it may be imperative for implementation to consider how parts or all of the examination might be conducted virtually. A virtual platform may decrease some of the complexity associated with offering such an exam (e.g., cost) and reduce barriers for applicants in completing the exam (e.g., geographic location).

Limitations

The findings from this study need to be cautiously interpreted. First, the small sample size means that findings may not be indicative of experiences of vOSCEs everywhere. Second, the time gap between when the vOSCE was conducted and the data gathered from students and clinical instructors may have affected participant responses even when the instructors were given prompts to support recall of the vOSCE event prior to the focus groups. Third, the study would have benefitted from gathering the perspectives of students on the use of clinical instructors as SPs. Similar studies in the future should explore this aspect.

CONCLUSION

The results suggest that vOSCEs could be a viable alternative to in-person OSCEs for health history and other select clinical evaluations. Preparation for the virtual examination requires thorough planning and precise implementation. As technology applications continue to emerge for conducting virtual examinations, there may be increased use of and ease of use with a virtual platform to conduct aspects of clinical examinations.

CONFLICTS OF INTEREST

The authors declare no known conflicts of interest.