Structures, processes and outcomes of objective structured clinical examinations in dental education during the COVID‐19 pandemic: A scoping review

Abstract

Introduction

Objective structured clinical examinations (OSCEs) are an essential examination tool within undergraduate dental education. Fear of spread of the COVID‐19 virus led to dental institutions exploring alternative means of conducting OSCEs. The aim of this scoping review was to investigate what structures, processes and outcomes of dental OSCEs were reported during the COVID‐19 pandemic.

Materials and Methods

This scoping review was conducted and reported adhering to the Preferred Reporting Items for Systematic Reviews and Meta‐analyses extension for scoping review guidelines (PRISMA‐ScR). Published literature was identified through a systematic search of PubMed, Embase, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Education Resources Information Center (Eric), ProQuest and Google Scholar. Identified articles were independently reviewed by two authors (KS, AD), followed by synthesis in terms of the reported structures, processes and outcomes. Articles reporting cancellation or rescheduling were also included, extracting data on reasons and any suggestions/recommendations.

Results

The search yielded a total of 290 studies of which 239 sources were excluded after removal of duplicates, leaving 51 studies for title and abstract evaluation. Thirty‐four articles were excluded as they did not report on the topic of interest, leaving 17 for full‐text evaluation, of which nine were analysed according to the pre‐set themes. All dental OSCEs taking place (n = 6) were conducted online whilst the remaining (n = 3) were either cancelled or rescheduled. Data on structures reported specific online videoconferencing software used and provision of staff and student training. Processes on the execution of online OSCEs varied significantly from one study to the other, providing rich data on how dental institutions may carry out such assessments tailored to their need. Information regarding outcomes was sparse, as little attention was paid to the results of the students compared to pre‐pandemic, lacking investigation into reliability and validity of online dental OSCEs.

Conclusion

Dental OSCEs could be conducted online implementing well‐planned structures and processes; however, further evidence is needed to prove its reliability and validity based on outcomes. Dental institutions may need to consider alternative methods to assess practical competencies if online OSCEs are to take place.

1. INTRODUCTION

Assessment is an exercise designed to measure student's fulfilment of learning outcomes deemed necessary for the completion of their programme. ¹ In dental education, assessments evaluate abilities and competencies required for subsequent registration as a dental professional. ² There are various forms of assessments undertaken throughout the undergraduate dental curriculum, including the objective structured clinical examination (OSCE). OSCEs are universally utilised as an assessment method in dentistry for various reasons. They are considered gold standard as they are able to demonstrate the “shows how” level of clinical competence as described by Miller's Pyramid ³ and assess areas most critical to performance of healthcare professionals such as communication skills, history‐taking and physical examination. ⁴ They are also a fair method of assessment, ⁵ removing any patient–examiner interaction that may be experienced in other forms of assessment. ⁶

Over the years, the scope of OSCE examination has been broadened to satisfactorily complement cognitive knowledge and clinical testing, in order to cover areas such as the ability to handle unpredictable patient behaviour, obtain/interpret data, problem‐solve and educate patients. ⁷ , ⁸ , ⁹ , ¹⁰ , ¹¹ Most OSCEs use standardised patients (SP) for accomplishing the latter. SPs are individuals trained to act specific scenarios and display signs and symptoms of certain conditions within a fixed testing environment. ¹² , ¹³ Traditional OSCEs usually take place face to face and comprise a circuit of assessment stations whereby marshals and timekeepers guide students to the next station ensuring time is adhered to accurately. ¹⁴

Prior to embarking on developing an OSCE, it is important to have a proper structure in place to ensure that the process, once established, is self‐sustaining. A structure is defined as “an arrangement and organization of interrelated elements in a material object or system”. ¹⁵ In assessments, effective structuring requires outlining the regimen clearly, creating a well‐trained assessment team, and defining the appropriate settings in terms of resources, manpower and administrative support. ¹⁶ , ¹⁷ A process is a “series or set of activities that interact to produce a result”. ¹⁸ Within a dental examination, a process entails the course of action executed to achieve the outcome. This includes but not limited to identifying and codifying examination content, assigning students to their stations and observing their performance, following the scoring rubric appropriately and finally, post exam analyses. ¹⁹ , ²⁰

At the beginning of 2020, a novel coronavirus, COVID‐19 emerged. The pandemic took countless industries by surprise and resulted in unassailable changes to the way many things had previously taken place. In dental circles, there is a lot of discussion surrounding what may be the “new normal”. ²¹ It is generally accepted that things will be different for at least the near future, but perhaps even for the distant future. Dentistry is an industry that has had to undergo several changes in order to accommodate the new and advancing regulations owing to the COVID‐19 pandemic. The very nature of dentistry as an occupation is considered as high risk for the transmission of the coronavirus for obvious reasons. ²² Less documented, however, are the changes that undergraduate dentistry training programmes have had to endure.

Along with other university programmes, a great deal of dental educational activity has been moved online during the governmental lockdowns imposed. Moreover, even during periods outside of lockdowns, social distancing is still required to be adhered to. ²³ Dentistry along with other university courses moved lectures, tutorials and even assessments to online portals wherever possible. An obvious aperture left consequently unfilled is that of OSCEs.

Due to the nature of OSCE examinations, in their current form they would not comply with COVID‐19 regulations and as such may need to be either removed, delayed or replaced. In order to explore this topic, a scoping review was conducted to ascertain the current status of the published literature. The aim of this scoping review was to investigate structures, processes and outcomes of OSCEs in dental education during the COVID‐19 pandemic.

2. METHOD

A scoping review was used to establish the extent of the existing literature relating to the chosen topic. ²⁴ , ²⁵ Despite having a similar purpose to a systematic review, a scoping review captures a broader range of studies with greater variation in study designs and is appropriate when the search results fall in a body of literature that is diverse in nature. ²⁶ The strength of such method lies in its ability to summarise the existing evidence base and identify any gaps. However, the absence of assessing the quality of primary data presented is considered a limitation.

2.1. Design and search strategy

The review was performed systematically following the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses extension for Scoping Reviews (PRISMA‐ScR) checklist ²⁶ , ²⁷ and Joanna Briggs Institute Reviewers' Manual. ²⁸ A protocol was developed to pre‐define the objectives of the study and methods and allow for transparency of the process. The protocol detailed the criteria that the reviewers intend to use to include and exclude sources of evidence, identified what data were relevant and how the data will be extracted and presented. The search strategy was developed from the research question “what structures, processes and outcomes of OSCEs in dental education have been reported during the COVID‐19 pandemic?”. The following inclusion criteria were applied:

1. Articles from all countries, published in English, during the COVID‐19 pandemic, related to OSCE assessments in dentistry.

2. OSCEs performed during the undergraduate programme in dental institutions.

3. OSCEs cancelled or rescheduled and rational.

4. A method of assessment comparable to an OSCE in terms blueprinting, learning outcomes assessed and application method.

5. Qualitative, quantitative and mixed‐methods studies were included.

Any studies reporting an OSCE experience in healthcare professions other than dentistry were excluded.

Key words were identified through titles, abstracts, full text and their references. Full‐text manuscripts were retrieved using institutional access, Google Scholar, Research Gate or by contacting the authors. Discussion papers and letters to the editor were excluded. Databases were searched using the Boolean operators AND and OR combined with truncation (Table 1) and phrase searches. The review was undertaken to identify the published literature and the grey literature in relation to the topic between January 2020 and the end of January 2021 to cover the peak COVID‐19 pandemic period. The following scientific databases were searched: PubMed, Embase, Cumulative Index to Nursing and Allied Health Literature (CINAHL) and educational databases: Education Resources Information Center (Eric) and ProQuest. Additionally, a Google Scholar search was performed. The last date of search was executed on the 31st of January 2021

TABLE 1.

Search words used and truncation

Search word used (Keyword)	Truncation options
Dent*	Dental, Dentistry, Dentist
Objective Structured Clinical Exam*	OSCE, OSCEs
Coronavirus*	COVID‐19, COVID 19, SARS2, SARS 2, SARS‐COV‐2
Education	No truncation

For PubMed for example

the following strings were combined:

• CONCEPT 1: OSCE

Key Words

osce*[tw] OR “objective structured clinical exam*” [tw] OR exam*[tw] OR assessment*[tw]

• CONCEPT 2: Dental

Key Words

dental [tw] OR dentistry [tw] OR dentist*[tw]

• CONCEPT 3: Education

Key Words

education [tw] OR “Education

Distance”[Mesh] OR “Education

Dental

Graduate”[Mesh] OR “Education

Dental”[Mesh]

• CONCEPT 4: COVID‐19

Key Words

COVID‐19[tw] OR “COVID 19”[tw] OR SARS2[tw] OR “SARS 2”[tw] OR SARS‐COV‐2[tw] OR Coronavirus[tw] OR “severe acute respiratory syndrome coronavirus 2” [Supplementary Concept]

2.2. Screening process and inclusion criteria

All results obtained were exported to Endnote X9 and duplicates removed. Remaining articles were imported to Rayyan, a web application for screening scoping reviews. The titles and abstracts of 51 papers were scrutinised by AD and KS, who performed the screening independently. Titles were initially reviewed and filtered, excluding clearly irrelevant articles. Abstracts for the remaining articles were retrieved and inclusion/exclusion criteria were applied. Full text of the selected studies was considered for inclusion. The third author (DS) was involved when required to make a final decision. A reference check was conducted and included in the flowchart.

2.3. Data extraction and synthesis

Identified publications were independently scrutinised and summarised to present:

• Author and year of study

• Reported structures: What was the regime of the OSCE conducted (e.g. conventional or online), what settings took place (e.g. online systems, breakout rooms and IT support) and what training for staff and students was provided if any.

• Reported processes: How the OSCEs were carried out, number of students and staff involved, logistics and any methods used to prevent cheating and disruption.

• Reported outcomes: How it was determined if the OSCEs were successful, student and staff feedback and any suggestions for improvement.

• Reported cancelled, rescheduled or modified dental OSCEs. Reasons for decisions and any suggestions made.

• Reported methods of assessment equivalent/comparable to OSCEs in terms blueprinting, learning outcomes assessed and application method.

Analysis of the identified articles was performed based on the reported structures, processes and outcomes of OSCEs. Articles reporting cancellation or rescheduling of dental OSCEs were reviewed including the reason and any suggestions/recommendations made. This is to help explore why a dental institution was unable to run the OSCE; was it lockdown measures, unable to plan a safe, infection transmission‐free environment or even facility issues. In addition, if an institution reported rescheduling, was the reason explained, alternatives described, second attempt described?

3. RESULTS

3.1. Selection of sources of evidence

A search on OSCEs in dental education during the COVID‐19 pandemic yielded a total of 290 studies. As independent researchers searched various databases, a total of 239 sources were found to be duplicates, and hence removed, leaving 51 studies for title and abstract evaluation. A further 34 articles were excluded as they did not report on the topic of interest, leaving 17 for full‐text evaluation, of which nine were retained following application of the inclusion and exclusion criteria (Figure 1).

FIGURE 1.

Records selection in accordance with PRISMA for systematic scoping review

3.2. Characteristics of sources of evidence

A summary of the data extracted from each source is presented in Table 2. The table provides a deeper insight into the reasons behind decisions taken by institutions. Thematic analysis of the subject matter was conducted independently by AD and KS, to describe relevant data under each subheading, then a final agreement was achieved.

TABLE 2.

Table of results; evidence gathered from included articles

Reported OSCE Features during COVID‐19 pandemic
Study	Structures	Processes	Outcomes
Kakadia et al. 2020 37	Regime: Online OSCE Settings: Qualtrics For marking the exam Zoom Selected because of its ‘breakout’ room feature which allowed private sessions for each student. Training: The purpose of training was to allow standardisation through examiner calibration. This was done on two occasions, 1 month and 1 week prior to the OSCE. Secondly, there was a trial OSCE held for both the students and the examiners.	Execution: OSCE ran with two parallel groups, Group Red and Green. 34 students examined, split into Group Red or Green. 20 examiners; 2 examiners for each of the 10 disciplines examined within either Group Red or Green. One Zoom link with 21 pre‐assigned breakout rooms was created. Each breakout room had an examiner, 1 room used as a rest station. A clinical case was disseminated to each student 1 hour before the start time. Zoom host placed examiners in breakout rooms before students. Students had to sign in with their full names. Once all students had joined, they were placed into their first breakout rooms by the Zoom host. Six minutes were allowed for each examination. Students were then moved to the next room of the appropriate colour. One minute warnings were provided. Examiners graded students on Qualtrics or on hard copy. Prevention of cheating: Students were asked to keep cameras and microphones on at all times. Prevention of disruption: This institution held a rehearsal of the OSCE beforehand	Results: Not reported Feedback reported in article: Positive: Student: Time saving Student: Less anxious Student: More privacy Staff: The rehearsal was appreciated Staff: It went smoothly Negative: Student: Poor internet connectivity Student: Poor image quality Student: Poor sound quality Staff & Students: Errors in entering/leaving breakout rooms and difficulty re‐entering Staff: no hands‐on questions Staff: difficult to grade between the students. Suggestions made in article: An established form of communication between examiners and students at all times e.g. a phone line A supporting co‐host who is able to take over should the actual examiner become unavailable Examination case/question screen shared for the entire duration to maximise the amount of time examiner has with the student.
Khalaf et al. 2020 42	Regime: Online OSCE Settings: Blackboard Examination meeting Phone lines Communication if the Blackboard meeting suffered disturbance. Teams Meetings between faculty members in preparation for exam. Ran simultaneously with Blackboard as an extra means of communication during the exam and as a backup in case the examination meeting using Blackboard suffered disturbance. “Respondus” lockdown browser – for prevention of any other programme running on the computer Training: Online meetings were held on teams between staff. Training and mock exams were held for the students beforehand.	Execution: 71 students were examined in an online OSCE as part of an overall ‘exit’ examination for final year dental students in Sharjah, UAE. This examination consisted of MCQs, MEQs, oral examinations, and the OSCE. The OSCE formed 30% of the overall exit exam grade. Prevention of cheating: The “Respondus” lockdown browser. 10 min checking: a video recording of the student ID and the room they were in. Students allowed a maximum of 10 min late arrival to prevent sharing of information. Prevention of disruption: Teams meetings were held alongside Blackboard meetings in case the communication through blackboard was lost. They were held on mobile devices using mobile data, therefore if Wifi connectivity went down, the student could let their invigilator know this way. In the case of lost connectivity, the student was placed in a direct phone call with their invigilator to not disturb the other students, once connectivity was restored, the invigilator would arrange for the host to allow the student back into the examination.	Results: Outcome of the online Exit exam for the current cohort was similar to the conventional exam, however, specific OSCE results were not discussed. Feedback reported in article: Positive: 69% of the 29 responding staff agreed they were satisfied with the online OSCE. 60% of students felt the training and mock exams held beforehand were adequate to familiarise them with online format. Negative: Only 17% of staff agreed online OSCE assessed a different set of knowledge/skills than traditional OSCEs 33% of students felt online exams were unreliable and 44% felt online exam favoured some students over others. 37% of the 65 students agreed they had adequate previous knowledge of online systems (Blackboard, teams, etc.) whilst 36% of students felt less anxious after completing the practice exams. 37% of students were satisfied their final grade reflected their performance and only 32% of students were satisfied overall. Teams meeting in the background created unnecessary noise. Technical issues with internet connectivity. Suggestions made in article: More IT training needed
Donn et al. 2020 32	Regime: Online OSCE Settings: PowerPoint: How examination information, questions and cases were presented. Zoom Pre examination meetings for examiners For the exam meeting. Training: Practice sessions carried out for both students and examiners. Appropriate questions selected from the OSCE bank of questions that had previously been used. They were selected based on their appropriateness for this new style of examining. Zoom meetings were held to go through each question and arrive at a passmark.	Execution: A total of 10 questions for the three resitting students sitting the Virtual OSCE (VOSCE). 12 min stations, each with cases presented via PowerPoint, showcasing radiographs, photographs and special investigation results. Each breakout room featured two questions, two examiners and one reserve examiner. Examiner took on the role of simulated patient (SP) if required. Zoom meeting set up with breakout rooms. As many breakout rooms as number of exam questions. An additional room for the external examiner, and a student meeting room. Waiting room feature enabled to allow students physically into the examination by examiners. Five co‐hosts created allowing examiners to navigate freely between rooms without needing host's permission to enter. Briefing held before examiners were placed into their individual breakout rooms. Students briefed on what to expect & how to customise screen displays. Exams marked independently at the end of the OSCE by two examiners then a final mark for each student agreed. Examiners received an electronic copy of the mark sheet beforehand or a printed copy sent by recorded delivery. Mark sheets were then sent back electronically. Prevention of cheating: All students were asked to complete an academic integrity statement Prevention of disruption: Host emailed an emergency contact number for any IT issues. Initial meetings with each student online allowed evaluation of Wi‐Fi connectivity and IT device requirements. Students consented to OSCE being recorded for Quality Assurance purposes. To compensate for new format of questions and any IT disruption, original time allotted was doubled. Practice sessions held allowing identification of logistical and timing challenges.	Results: No information was provided on the results obtained for the OSCE. Feedback reported in article: Positive: For majority of staff and students, VOSCE progressed in the same manner as a conventional OSCE All staff and students agreed the VOSCE was a good substitute for the OSCE given the circumstances. Both groups felt it allowed students to give an accurate reflection of their abilities. They did not feel disadvantaged by the new format. Staff mentioned marking felt similar to marking the conventional OSCE. Training sessions considered vital. Staff happy to take on role of SP. Likewise, students felt at ease with it. Recording considered helpful for reviewing and staff calibration Cloud for storage useful Negative: Staff intensive. Extended time per station unnecessary. Inability to assess practical skills Suggestions made in article: Practical skills may be assessed through longitudinal evaluations and competency assessments. The use of WhatsApp or FaceTime to continue the exam in the event of IT failures. Increased time for moving students into correct rooms. Modification of software, making it more user friendly; improving communication to provide two‐way contact between the virtual breakout rooms and the host, increasing the size of text whilst in ‘Broadcast’ function to aid readability.
Alon et al. 2020 31	Regime: Online OSCE Settings: Canvas For information presenting prior to the exam. For the exam meeting. Training: A grading criteria table was formed as well as a content of the expected answers	Execution: A module was created on the university online portal publishing what's to be expected from the exam and studying materials. Exam integrated: open‐ended questions, multiple choice questions, and “fill in the blank”. Exam included questions about materials, diagnosis, treatment planning, clinical procedure, emergencies, complications, and communication. Pass grade decided after OSCEs were marked. Prevention of cheating: The assignment was only available on the portal for a limited time. Prevention of disruption: Not reported	Results: Majority of the students were deemed competent. Feedback reported in article: Positive: The evaluation process was rapid and successful Negative: This virtual format did not allow practical assessments. Suggestions made in article: Not reported
Hytonen et al 2020 30	Regime: Online OSCE Settings: Moodle For the exam. Training: Students had the choice to complete a practice test online with similar style questions to the new style OSCE	Execution: 179 dental students from all four dental schools in Finland examined simultaneously. Eight subject entities included up to 6 questions. The total exam was 90 min; 10 min per subject entity, and a 10‐min for changing between questions. An additional 10 min was provided for students with dyslexia. The questions had to be answered in the given order, students were unable to return to previous questions. Each subject entity had its own pass mark set between 50–70% and each passed individually. Students received model answers at the end. A re‐sit 2 weeks after the initial OSCE was allowed. Prevention of cheating: Not reported Prevention of disruption: Uniform setting for the online Moodle examination were created and edited nationally by the IT support personnel. A preferred browser was recommended. Use of a computer was advised instead of a phone or tablet. An emergency contact number was provided to the students	Results: The pass rate was lower than the traditional OSCE. Feedback reported in article: Positive: Students: received adequate information prior to exam Students: positive attitude to the exam Students: practice test useful and technical implementation of the OSCE was good. Students: sufficient teaching in relation to the OSCE The time frame was appropriate and the difficulty of the questions was adequate Negative: Before the exam, students worried about the possibility of technical issues, inability of OSCE to asses practical skills, professional behaviour and communication skills. Suggestions made in article: Include more patient cases and clinical photos. Split each subject entity into individual exams lasting 10 min each, instead of a single 80‐min exam. Increasing the number of subject entities Incorporating oral examinations as a new subject entity.
Wu et al. 2020 58	Decision: reschedule OSCE assessment. Reason: Challenging to ensure patient and student safety if use of live patients is required. Suggestions made in article: A new style of OSCE has been suggested; The Dental Licensure OSCE which is designed to examine clinical judgement and decision‐making. This style of OSCE does not require procedures to be performed on live patients
Iyer et al. 2020 23	Decision: Cancel or reschedule OSCE assessment. Reason: To satisfy quarantine requirements Suggestions made in article: A multidisciplinary OSCE combined with a written exam by sharing resources across different dental schools
Hattar et al. 2020 59	Decision: Cancel OSCE assessment. Reason: To satisfy quarantine requirements. Suggestions made in article: Online replacements for OSCE's were prepared. Students however, did not feel the online replacements were adequate
Assessment method comparable to OSCE
Stoopler et al. 2020 29	Regime: Online. Settings: BlueJeans For students and examiners meeting Canvas For reading the case presentation during exam. Training: Not reported	Execution: Split into two components. Component A for students to review a case presentation using Canvas, 30 min allocated. In B, students met examiners using BlueJeans answering five questions in 15 min. Predeveloped answer sheet with correct answers, and critical failure answers used. Students told if they had passed or failed at the completion of the examination. Prevention of cheating: Not reported. Prevention of disruption: Not reported	Results: all students successfully completing the Oral Cancer CCE on their first attempt. Feedback reported in article: Faculty members believed this method of examination was effective in assessing clinical competency related to oral cancer. Student and staff feedback was positive with particular focus on the efficiency of exam administration. Suggestions made in article: Not reported

One study demonstrated a virtual oral cancer clinical competency examination (CCE). ²⁹ The authors described the CCE as a “vehicle to ensure attainment of skill, knowledge, and value by dental students in a specific discipline”. The study was included in the current review as it was deemed similar to an OSCE in terms of level of competencies assessed, learning outcomes measured and methodology. Describing this assessment may inform the reader of how such assessment was approached in terms of structures, processes and outcomes.

3.3. Reported OSCE experience during the pandemic

Out of the nine included articles, six conducted the dental OSCE online, whilst the other three studies reported cancelling or rescheduling the OSCE, indicating that none of the studies ran their dental OSCEs face to face. All the studies demonstrating online OSCEs described the online setting utilised to run the OSCE; however, that varied from one institution to the other depending on many factors including familiarity with a particular portal, resources available and software specifications such as breakout rooms. The majority of articles described delivering online training for both staff and students to accustom with the new OSCE format, calibrate examiners, confirm appropriateness of questions and arrive at a pass mark. With regard to processes, conveying information on the execution of online OSCEs varied in depth amongst included articles, with some providing thorough informative information and others choosing to be brief. Most studies explored different ways of preventing cheating during the online OSCE, ranging from signing an academic integrity statement to using video recording and cameras open at all times.

Measuring the outcomes, all selected articles presented student and staff feedback to an extent, providing some idea regarding the perceptions and attitudes. No article clearly reported the results of the online OSCE compared to previously run face‐to‐face OSCEs. There was an apparent lack of consistency in approaches and data on effectiveness of methods reported.

4. DISCUSSION

4.1. Statement of key findings

This review was constrained to nine articles fulfilling the inclusion criteria. Notwithstanding, the reviewed articles were generally homogeneous in their themes, namely structures, processes and outcomes, which allowed for an in‐depth cross examination between them. Out of these, six dental OSCEs were performed online whilst the remaining were either cancelled or postponed. Although no reviewed article clearly mentioned why online methods were deemed preferable, intuitively they rigidly observed the quarantine protocols imposed during the pandemic. Moreover, this review observed that due to the longstanding opinion that OSCEs are an essential feature of the undergraduate dental assessment portfolio, most articles opted to run the OSCEs, clearly reinforcing the discernible efforts made in order to preserve them, even throughout the pandemic. Amongst the reviewed articles, the structures and processes of online OSCEs described diverse applications, however, not surprisingly, reported its inability to assess practical skills. ³⁰ , ³¹ , ³² Outcomes generally presented staff and student feedback, however, were scarce in revealing reliability and validity when running the assessment online.

4.2. Interpretation

The scoping review has identified that different dental institutions have responded contrary to one another in their handling of this dilemma. Structures and processes attributed to dental OSCEs must ensure validity, reliability and reproducibility of the assessment style, in addition to its ability to examine practical skills; this makes it an unparalleled form of assessment in producing desired outcomes. ³⁰ Exploration of OSCEs beyond the review revealed that in addition to its high fidelity, the strength of such an assessment tool lies in its ability to provide students with an opportunity to demonstrate essential competencies that are required of a graduating dentist to begin safe, independent, unsupervised dental practice. ³³ On the contrary, a study suggested that other methods of assessments such as workplace‐based assessments (WPBAs) should now take centre stage in clinical assessments of undergraduate medical students and advocated a reduced role for OSCEs. ³⁴ Another study exploring the perceptions of clinical assessment stakeholders (students, examiners, simulated patients and administrators) in relation to the future role of traditional OSCEs equally revealed WPBAs utilises a more holistic approach to ensuring undergraduates' work‐readiness, as it focuses on professional behaviours and identity in comparison to the OSCE. ³⁵ Programmatic assessments have also been advocated, as it allows for comprehensive information gathering about the leaner's competence and the accumulation of information from a variety of assessment tools. ³⁶

4.2.1. Structures

Based on the reviewed articles, the selection of a suitable online software is dependent on each institution with preference given to continuing with current online university portals. ²⁹ Additionally, it may be considered more important to use a software with previous examiner and student familiarity; or use due to specific features within the software such as breakout rooms, which may aid in the architectural implementation of the OSCE. ³⁷ Alternatively, Learning Management Systems such as “Moodle” and “Blackboard” have been suggested for a similar purpose. These are software applications allowing examiners to insert pictures and videos on which the student may then be examined. Further research within the medical field has disclosed their ability to use advance assessment settings, such as shuffling the order of question and answer options, which may be considered advantageous when designing an online OSCE. ³⁸

To date, the fundamental feature underpinning all published studies holding a dental OSCE during the pandemic was the fact they took place online. Contrarily, a number of medical institutions did attempt to hold face‐to‐face OSCEs during the pandemic. ³⁹ , ⁴⁰ Auxiliary research within the medical field suggests online is less effective than the face‐to‐face OSCE in assessing practical skills. ⁴⁰ Conversely, simulation‐based online OSCEs improve clinical outcomes, save time ⁴¹ and are preferred by the students. ⁴⁰

Provision of training for staff and students must be a pivotal aspect within the structure of online OSCEs. Five out of the six studies highlighted the importance of running training sessions which may allow for standardisation, examiner calibration and evaluation of students' internet connectivity, ³⁰ , ³¹ , ³² , ³⁷ , ⁴² with one study emphasising on selecting OSCE questions based on their appropriateness for this new style of examining. ³² Furthermore, training must focus on improving familiarity and reducing anxiety of students facing this new examination format. ⁴² A similar study in the medical field recommended the need for a specifically trained team in the technology utilised. ⁴³ In another medical study questioning alternative opportunities to enhance assessments during the pandemic, four structural ideas were suggested to accommodate the many changes which OSCEs may have to undergo: communication, flexibility, collegiality and compassion. ⁴⁴

4.2.2. Processes

The execution of online OSCEs varied significantly from one study to the other, providing rich data on how dental institutions may carry out such assessments tailored to their need. Undoubtedly, this may be considered a weakness too, as lacking consensus and evidence‐based approaches may jeopardise the validity and reliability of an OSCE. It would have been valuable to the reader to present the rationale behind reported methods of implementing OSCEs, as the assurance of evidence to support the validity of OSCE scores is an essential condition to guarantee a high‐quality and effective assessment. ⁴⁵ Each of the articles reviewed had varied in the depth of information provided relating to the execution of the OSCE. In terms of the timing per station, it ranged from 6 to 12 min each with no justification per se. Nevertheless, it was helpful to identify reported feedback on assigned timings for online OSCE stations. In one of the included studies, stations were allocated 6 min each, whereby students perceived the whole OSCE experience as “time saving” compared to their previous live OSCEs. ³⁷ Another study assigned 12 min for each OSCE scenario; however, feedback from students and staff regarded the extended time per station as “unnecessary”. ³² Allocating appropriate and realistic time for tasks at individual stations improves the extent to which the test's content is representative of the actual skill; in other words, enhances validity. ¹⁷ For practical purposes, a broader search of literature has generally recommended that sufficient reliability could be achieved with 14–18 stations of 5–10 min duration each, ⁴⁶ whilst the Cronbach's or Generalisability (G) value should fall between 0.7 and 0.8 for high‐stakes examinations. ⁴⁷ Coincidently, the feedback of one of the current studies has therefore deemed 12 min per station unnecessary and could be reduced. ³² Administrative support has also been praised in a few of the included studies. Institutions planning to conduct OSCEs must guarantee adequate administrative support. ¹⁷ For online purposes, including tasks such as trialling online platforms suitable for purpose, allocating students to different breakrooms, logistics of moving students between stations and distribution of examiner mark sheets, having the right administrative support will enable the smooth running of the exam. ²⁹

A repeatedly cited feature from our review was institutional efforts to ensure integrity of students' responses. Reflection on why cheating is wrong, its subsequent connotation of the student and their place within the profession is a topic of in‐depth ethical consideration. Outside of the review, the importance of this is emphasised by cases where cheating has led to a student's repetition of an academic year. The importance of academic integrity is ultimately due to its potential to echo into patient care. ⁴⁸ The online OSCEs have recognised and actioned this in many forms including the completion of an academic integrity statement ³² ; a recorded “check‐in” process of the surrounding environment ensuring no elicit materials were accessible to the students; and use of software such as “Respondus” to prevent access of any other materials on the device being used for the examination. ⁴²

Dependence on the unreliable nature of internet connectivity is an inherent constraint of online OSCEs. This review identified many attempts made to minimise the likelihood of such disruptions including simultaneous video calls alongside students' actual examination video calls in case it should fail ⁴² ; allowing more than usual time per OSCE station to countenance rectification of any connectivity matters, the latter considered unnecessary following later evaluation of the OSCE event as previously stated. ³² A discernible way of countering this would be the traditional face‐to‐face OSCE, which can be conducted in hospital wards, clinical skills laboratories or specially designed test canters. However, there is currently no evidence‐based method of conducting a COVID‐19 secure face‐to‐face dental undergraduate OSCE. A recent article concluded that both staff and students considered the virtual OSCE (VOSCE) a useful and fair assessment method, and a suitable alternative to the traditional OSCE, to overcome the difficulties inherent in traditional clinical examination, during pandemic restrictions, providing a number of improvements in the assessment process are undertaken. ⁴⁹

4.2.3. Outcomes

The majority of dental OSCE experiences explored to date failed to confirm comparability of online OSCE results with previous live OSCEs, indicating a gap when relaying outcomes, leaving the reader short of establishing evidence‐based conclusions on the reliability and validity of online dental OSCEs in testing practical skills when needed. A fundamental principle of any assessment is making certain it is designed to assess stated learning outcomes. ⁵⁰ In an article outside of our review outlining guidelines for online medical assessments in emergency remote teaching (ERT) during the COVID‐19 Pandemic, it was proposed that assessments may focus on the cognition involved in clinical skills instead of the practicality, such as history‐taking and patient counselling skills. ⁵¹ However, assessing the psychomotor domain has proven challenging in the scenario of university closures and social‐distancing orders. ⁵²

Dentistry is a practical profession; therefore, assessing these skills is an essential feature of the undergraduate training programme. ⁵³ Wider research suggests exploring methods through which this may be incorporated. These include development or modification of software devices or removal of the practical evaluation from the OSCE and into longitudinal observations throughout the year using clinical portfolios and practical competencies. ³⁰ , ³² Interestingly, one of the included studies described the use of OSCE as a final‐year exit assessment alongside MCQs, MEQs and an oral exam, all administered online. However, the authors did not elucidate the reason for conducting an online OSCE at this stage; though, they stated that these exams are part of a graduation comprehensive assessment that “does not exist in other medical/dental colleges” and has been developed and implemented several years prior to COVID‐19 pandemic. ⁴² A recent study in a UK dental institution provided a proof of concept for the validity of virtual reality (VR) dental simulator assessments. It revealed the ability of VR to provide reliable and clinically relevant qualitative feedback on basic operative dental procedures, ³³ which could in essence be explored as a validated method going forward to assess manual dexterity.

Additional study exposing elevated levels of anxiety and unpreparedness for the workforce of dental students as a result of the pandemic has been discussed in depth ⁵⁴ ; it is therefore of vital importance to contemplate the impact of changes to the OSCE, on the already wavering confidence of the premature dental professionals. In most studies retrieved in this scoping review, both staff and students agreed that the online OSCE was a good substitute for the conventional OSCE given the circumstances, allowing students to give an accurate reflection of their abilities. ²⁹ , ³² , ⁴² Students reported they felt less anxious, had more privacy and did not feel disadvantaged by the new online format. ³⁷ Nevertheless, nearly half of the students in one study felt the online exam favoured some over others, ⁴² with others showing concerns regarding internet connectivity and the possibility of technical issues occurring during the exam. ³⁰ , ³⁷ , ⁴²

Further review of literature outside of dentistry has revealed examples of live OSCEs held during the pandemic ³⁹ ; Bouriscot et al have documented an example of how to execute an OSCE during the pandemic that is comparable to those before the occurrence of COVID‐19, but also defensible in its compliance to new regulations. Their take‐home message was to ensure that sufficient measures are needed to conduct a clinical examination safely including strict infection control practices, minimising the mixing of students, staff and patients, social distancing and videoconferencing‐facilitated briefings for training and calibration purposes. The study reported no COVID‐19 infections amongst participants afterwards, and the defensibility of the examination results was affirmed by the external examiners. ³⁹

On the other hand, a paediatric medicine experience suggested that a “teleOSCE” is a comparable assessment to a live OSCE as no significant differences were found when compared to the previous 3 years' live OSCEs. ⁵⁵ A similar study found that a virtual objective structured clinical examination (VOSCE) for medical undergraduate students was not only feasible to deliver, but also acceptable to both students and examiners, with no complaints or concerns displayed. Nonetheless, they plan to return to the traditional “gold standard” live OSCE format once the COVID‐19 situation allows, acknowledging the need to implement elements of programmatic assessments into their curriculum. ⁵⁶

The expansive scope of the present review provides dental educators with a synopsis of key considerations to be aware of when designing a dental OSCE compliant within the COVID‐19 pandemic. Areas that lack depth of information modestly provide direction on precisely where further research in this area must be focussed.

4.3. Further research

Despite reporting that as many as 19% of dental institutions continued to hold face‐to‐face examinations, ⁵⁷ the authors of the present review have yet to identify details of any traditional dental face to face OSCE held during the pandemic. Examples of how this may have been done are essential if there is to be evidence‐based analysis of the possible structures, processes and outcomes rendering this a potential option. Additionally, further research is needed to better inform pedagogic and assessment strategies to the possibility of utilising VR haptics for assessing operative and other dental practical skills.

4.4. Strengths and weaknesses

This scoping review is strengthened by its adherence to the robust PRISMA‐ScR guidelines. The generic nature of a scoping review deprives itself of a qualitative assessment of the research articles. Nevertheless, the authors of the present article considered that this review identified useful existing features, as well as highlighting areas requiring further exploration. The small number of available articles identified and the search period potentially limit the generalisability of the findings and their likely impact. This number was considered few in comparison to other healthcare professional fields; wider research within the medical profession revealed there were many more studies published relating to OSCEs during the COVID‐19 pandemic. Despite these limitations, the authors were reassured that the quality of the articles in the current review presents an in‐depth conversation of a similar nature of findings, allowing comparability and discussion of the key features for this scoping review.

Restricting the search to articles published in English only could potentially mean an OSCE experience during the pandemic might have been left out; yet, the authors opted to select a common scientific language and were in the opinion that including articles in languages other than English may not add to the findings of the scoping review. As with any search strategy employed, articles could be missed out; however, conducting a pilot search independently by the authors, searching a variety of relevant databases and running multiple regular searches reinforced the search strategy and consolidated the data set retrieved.

5. CONCLUSION

It is suggested that adopted structures and processes for online OSCEs may be utilised during the COVID‐19 pandemic to measure cognitive skills. Regarding reported outcomes, little information has been provided on the specifics relating to the results of online dental OSCEs; therefore, its genuine capability to replace the traditional face‐to‐face style remains undetermined. The COVID‐19 pandemic is likely to change dental training and assessment approaches; accordingly, dental institutions might need to consider alternative ways such as longitudinal and competency evaluations to assess manual dexterity. Further research into structures, processes and outcomes beyond this study period maybe required.

CONFLICT OF INTEREST

The authors have no conflicts of interest to disclose.

Salawu YK, Stewart D, Daud A. Structures, processes and outcomes of objective structured clinical examinations in dental education during the COVID‐19 pandemic: A scoping review. Eur J Dent Educ. 2022;00:1‐13. doi: 10.1111/eje.12869

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available from the corresponding author upon reasonable request.