1. WHAT PROBLEMS WERE ADDRESSED?
During the coronavirus disease 2019 (COVID‐19) pandemic, the world witnessed varied responses from educational institutions to the lockdown. Within the next few years, we will see the consequences of these responses. Despite the disruption at this critical time, candidates in their final fellowship training year still need to be certified and it is arguable that they should be certified faster than ever after passing the written component.
The Saudi Commission for Health Specialties (SCFHS) is a national regulatory body that is responsible, amongst other functions, for postgraduate certification examinations. The SCFHS aimed to develop a feasible solution to the challenge of assessing candidates’ ability to demonstrate the depth of knowledge and understanding expected of an independent consultant despite the lockdown.
An online solution was needed in which multiple cases and scenarios could be presented in a structured clinical vignette format.
2. WHAT WAS TRIED?
After studying the impact of several possible solutions, the examination was blueprinted against the detailed learning outcomes of each programme curriculum and consisted of a series of four online virtual stations, each timed for a precise period of 15 minutes. Ideal answers were defined with critical features and some scenarios had enhanced reality features (eg, simulation) embedded. These included data, images or videos that could be shared with the candidate on a computer screen via the audiovisual platform.
Technical requirements included a built‐in or external webcam, microphone and speakers in a desktop computer or laptop. The candidate was required to be alone with a clean desk and to be able to rotate his or her camera 360 degrees when asked. A guide was shared with examiners and candidates to outline the process and its technical specifications.
After registration, identity verification and visual space check, the proctor admitted the candidate and an examiner to a virtual station where all discussions were recorded. When the encounter ended, examiners moved to the next station, but the candidate remained with the proctor present and audiovisual features turned on in the same virtual station. Time was allocated between stations for examiners to complete their performance marking.
ExamHD is a secure online portal that was designed to allow examiners to enter the performance ratings on preset checklists. The oral encounter components, including the audiovisual recording, candidate identification, performance rating and examiner details, are all linked together in this unique SCFHS e‐solution. In total, 433 candidates were enrolled in the first administration of this examination; they were supervised by 72 proctors and assessed by 232 examiners.
3. WHAT LESSONS WERE LEARNED?
This online solution was proposed to solve a certification emergency induced by the lockdown. It was challenging to plan assessments with many stakeholders involved. To prevent any setbacks, system tests and ‘dry‐runs’ were conducted many times before the actual examinations. The key to overcoming the multiple challenges was engaging key stakeholders and providing needed support with open communication channels.
Importantly, we witnessed a positive change of perspective regarding online examinations during this process. According to the post‐examination survey, 96% of examinees and 91% of examiners reported satisfaction with how the examination contents reflected real practice. As a result, SCFHS explored further adaptation to online oral assessment services for activities such as the assessment of ex‐patriot practitioners seeking employment in Saudi Arabia prior to their being granted work visas. This service is expected to help reduce the expenses of travelling and accommodation costs commonly associated with certification examinations.