Conducting orthopaedic practical examination during the Covid-19 pandemic

Abstract

Background

As the coronavirus disease (COVID-19) pandemic is expected to stay for a longer time, educational activities including residency training have gradually resumed with the aid of virtual tools. In addition to continuing the residency education during COVID-19, it is also important to conduct their examination so that the graduations of final year residents are not delayed. The conventional exam pattern involved clinical case presentations and required resident interaction with a number of patients. However, in view of the COVID-19 pandemic we conducted a “zero-patient contact virtual practical exit examination” for orthopaedic residents.

Methods

In order to replicate the conventional exam case-scenarios, clinical cases were prepared in a digital presentation format. The candidate used N-95 facemasks and gloves, and adequate social distancing was maintained in the examination area. We also designed a 10- item questionnaire aimed at assessing the quality and satisfaction with the exam pattern.

Results

The mean score for overall satisfaction with the virtual pattern was 4.5 (out of 5) in examiner group while it was 4.1 in examinee group. Higher scores were also reported for questions related to safety of the exam, relevance and quality of the virtual cases, etc. The mean total feedback score for the examiner and examinee group was 48 and 43.4 respectively (out of 50).

Conclusion

Orthopaedic residency end-of-training examinations can be successfully conducted during the COVID pandemic, and we hope our experience will be helpful to other residency programs.

1. Introduction

The coronavirus disease (COVID-19) pandemic has affected the lives of millions of individuals globally.¹ In India, a nationwide lockdown was implemented towards the end of March leading to disruption of routine hospital services and residency training curriculum. Despite continued efforts to stop the spread of COVID-19, the number of COVID-19 cases continue to rise in the country, as a result of which there has been a paradigm shift from “living after COVID” to “living with COVID-19”.² Since it is difficult to predict an end to the pandemic, many services including educational activities have gradually resumed taking into account the COVID-19 situation.^{3, 4, 5, 6, 7, 8, 9} Like many other residency programs, our department continued the majority of educational activities during the pandemic through online portals.^6,9,10 We also successfully conducted the end-training evaluation of the graduating residents. In this article, we would like to share our experience in conducting the orthopaedic residency exit examination during this pandemic.

The conventional examination pattern involved clinical case presentations and viva voce. Each candidate was traditionally allotted 4 clinical cases on the day of examination including one long case and 3 short clinical cases. The candidate was expected to elicit history, perform clinical examination and present the cases to the examiners following which there was a table viva.

However, the Covid-19 pandemic posed a few challenges that made conducting the practical exams in a conventional manner unseemly, difficult and impractical. A total of seven candidates completed the tenure of residency training and were appearing for the examination in June 2020 and therefore 28 clinical cases were needed. So, the first challenge was to arrange a relatively large number of patients with varied diagnoses who could be kept for examination. With the out-patient department shut since last two and a half months and with various travel restrictions in place, arranging cases was difficult. The second challenge was to ensure the safety of both the examinees and the examiners who would be coming in close contact with their patients. To overcome both these challenges, our department planned to conduct a “zero-patient contact virtual practical exit examination” for orthopaedic residents.

2. Methods

In order to replicate the conventional exam case-scenarios, clinical cases were prepared by the examiners and the faculty in a digital presentation format. The Faculty members contributed details of clinical cases and photographs showing examination findings, elicited clinical signs and Radiological imaging from their records and case archives. Selection of the cases with maximum details available including a detailed history, clinical photographs and videos was done by the Chairman of examination. The cases were categorised as shown in flow chart (Table 1) and a total of 14 sets (twice the number of needed sets) of presentations were prepared, each consisting for one major case and 3 short cases. An attempt was made to include at least one common type of short case in each group. The collection was mailed to the external examiners in a password protected file. The External examiners selected 7 sets out of the 14 available sets, to be kept for examination. Each set was given a numeric code and random allocation of the sets to the candidate was ensured by chit-pull system on the day of exam.

Table 1.

Flow chart showing collection of clinical cases and their division into groups of major and short cases.

A mock test was conducted two weeks before the date of the exam so that the candidates were familiar with the new virtual pattern. On the morning of the exam, the candidates were briefed regarding the pattern of exam. Random allocation of the cases to the candidate was ensured by chit-pull method. Arrangements were made in the waiting area of the candidates to ensure adequate social distancing. The Case discussion and viva was conducted in a 20 × 10 feet hall, with seating arrangements made as shown in Fig. 1, to ensure adequate social distancing. The candidate entered the hall wearing an N95 facemask and gloves after pre-sanitisation of hands with an alcohol-based sanitizer. The case presentations were shown on an 80” Globus multi-touch sensitive interactive display panel (Globus IFPD 80B, Globus Infocom Ltd., NOIDA, India), which is pre-installed at our institution. A time limit of 45 min for the long case and 30 min for 3 short cases was set as agreed upon by the examiners.

Fig. 1.

Representative diagram of the examination hall showing the seating arrangement for the examiners and the examinee.

Each case began with the first slide of the presentation showing a brief clinical history consisting of patient’s chief complaints and history of presenting illness. The candidate was given 2–4 min to read the slide following which the candidate was asked if he required any other details in history other than those mentioned. Gaps in history were filled by examiner whenever enquired by the examinee. The candidate was then asked for his differential diagnoses based on the history provided. A clinical photograph of the patient was shown in the subsequent slide and the candidate was asked to enumerate his findings from the photograph and other examination that he would like to perform to further consolidate his diagnosis and rule out the others. A slide showing a specific clinical test’s photograph/video was used to ask in detail regarding the intricacies of that particular test. Radiological images sought by the examinee were shown in the subsequent slides. Liberal use of touch screen allowed the examinee to draw and demonstrate some important surface markings, lines, and triangles on clinical pictures and draw radiographic lines and angles pertaining to the pathology on the radiographs (Fig. 2). Videos of important clinical tests, randomly demonstrating the correct or incorrect method of performing a test or clinical finding were played and examinee asked to confirm the same. Candidate was then asked to summarize the findings and was questioned regarding the final diagnosis and planned management. A sample long case scenario is depicted in Fig. 3 (a-e) An example of a Long case scenario used in the examination showing history (a), examination (b-d), and X-ray (e) along with the potential questions that can be asked by examiner (shown in the notes section which will only be visible to the examiner when the slide is in presenter mode). A video file was used to demonstrate telescopy test (c) and interactive tools were used to draw Bryant’s triangle (d) and describe X-ray findings (e).(A-E). The short case viva was conducted in a similar fashion, focussing primarily on the diagnosis and management of the condition shown (Fig. 4a, Fig. 4ba and 4b). The case discussions carried 320 marks and were followed by table viva for 80 marks on x-rays, osteology, orthopedic instruments, and, orthoses & protheses, which were conducted in a conventional manner but with use of hand sanitizers, gloves and adequate social distancing measures. After each candidate, the used instruments or materials were sanitized before the other candidate entered the examination room to avoid fomite transmission. If a candidate is from a hotspot or containment zone, the virtual examination can also be conducted using web applications like Zoom™ or WebEx™ so that candidates can take the exam from home (we have conducted the examination in similar fashion for the fellows at our institution in arthroplasty, arthroscopy and musculoskeletal oncology).

Fig. 2.

Photograph showing use of touch screen by the examinee to draw and demonstrate some important surface markings, lines, and triangles.

Fig. 3.

A long case used in the examination showing history (A), examination (B–D), and X-ray (E) along with the potential questions that can be asked by examiner (shown in the notes section which will only be visible to the examiner when the slide is in presenter mode). A video file was used to demonstrate telescopy test (C) and interactive tools were used to draw Bryant’s triangle (D) and describe X-ray findings (E).

Fig. 4a.

Photograph of the power point slide mentioning the brief history and clinical presentation of the patient to be read by the examinee in a typical short case scenario.

Fig. 4b.

Photograph of the power point slide containing suggested line of questions by the examiners and the relevant clinical signs and radiographs. The examinees are allowed to mark and draw lines over the photographs in the screen.

3. Results

To evaluate the quality of examination pattern and level of satisfaction amongst the examinees and the examiners, we designed a questionnaire aimed at assessing the feeling of safety, quality of digital case scenarios, and, overall satisfaction with the exam pattern. It comprised of 10 questions and each answer was graded on a Likert point scale of 1–5, producing a maximum score of 50 and minimum of 10 ^{11, 12} (Table 2). Mean scores for each question in both groups were calculated as shown in Fig. 5a, Fig. 5ba and 5b. The mean score for overall satisfaction for each question in examiner group was 4.5 while that in examinee group was 4.1. The mean total score for the examiner and examinee group was 48 and 43.4 respectively.

Table 2.

Questionnaire to assess the overall satisfaction with the exam pattern by the examiners as well as examinees.

QUESTIONS	SCORE (1–5)
1. Did you find the examination environment safe for yourself considering the ongoing Covid pandemic?
2. Were you comfortable with the seating arrangement made for you in the exam room?
3. Do you think that the case details given in the digital presentation was sufficient enough to make differential diagnoses?
4. Do you think that the clinical photographs were good enough to elicit the clinical findings?
5. Do you think that the radiological pictures were good enough to reach up to a provisional diagnosis?
6. Do you think that the interactive touch screen was helpful in compensating the absence of actual clinical examination?
7. Do you think that the examination pattern was relevant to the type of academics that you had received/given in your institute? ("Received" for the examinee; "Given" for the examiner)
8. Would you like to recommend such type of examination pattern in similar situations like pandemics?
9. Do you think a mock test helps/helped in preparing for this pattern of examination? ∗ ("helped" for examinee; "helps" for examiner)
10. Rate your overall satisfaction in (0–5) for the ‘Zero-patient contact virtual practical examination’
TOTAL SCORE

Fig. 5a.

Histogram showing mean score of satisfaction for individual questions by the examiners and examinees.

Fig. 5b.

Histogram showing total mean score of satisfaction by the examiners and examinees.

4. Discussion

Assessment of the capabilities of a resident is a continuous process, and the end of training examination is only a part of the overall assessment. There is great variation in the conduct of examinations among the different programs worldwide and within India.^{13, 14, 15} While residency programs in some countries like United States do not routinely conduct a practical examination, residency programs in India traditionally conduct a practical examination at the end of final year to evaluate the clinical skills of the resident.^{13, 14, 15} Even though the virtual examination conducted by our department has the potential to effectively test the clinical knowledge of the residents, it may not have been fully successful in testing certain skills such as orthopaedic physical examination, patient communication, etc.¹³ While the use of real patients replicates the real-world clinical settings, it can be extremely uncomfortable for the patients to be examined by a number of students.¹⁶ Moreover, there is often a lack of standardization and objectivity when real patients are used. Therefore, many exams prefer mannequins to test examination and motor skills and standardized patients/actors instead of real patients to simulate the doctor-patient interaction.¹⁶ The use of standardized patients provides students with an opportunity to clinically assess patients in a multi-dimensional manner which is not possible when images or videos are used. However, not only it was difficult to find standardized patients and get them to visit hospital during the pandemic but also the use of a large number of standardized patients could have compromised the safety of the residents as well as the patients, especially considering the fact that residents would have been forced to spend a considerable amount of time in close contact with them.

Although multiple centres have reported their experience on continuing residency education during the COVID, there is limited literature on the conduct of end of training examinations. Schwartz et al. reported that they conducted faculty-led online learning sessions to continue the didactics and used interactive online question-based learning in which questions were asked using an online question bank followed by discussion with faculty.¹⁷ Chick et al. suggested various innovative technologies like virtual classrooms, online question banks, virtual conferences, use of surgical videos, etc, for resident education.¹⁸ Many residency programs in India have already started using virtual tools to continue residency education, and the use of technology for medical education has been well received by medical students and residents.^{11,17, 18, 19, 20, 21} As virtual examinations are new to students, a mock test can be conducted when possible to ease the anxiety of the students. In our survey, candidates found mock test to be helpful and gave a rating of 4.8 for the usefulness of mock tests. Use of interactive screen was a great value addition in our examination and in future availability of the haptic devices coupled with surgical procedures videos may permit assessment of the surgical skills of examinees using virtual surgeries.

The study has a few limitations (Table 3). First, it is a pilot study and the number of participants from examiner as well as examinee group is small. A study with larger number of participants across the different states is needed to establish its reproducibility and value. We believe that the proforma designed by us could help capture the relevant data to that end. Secondly, good quality photographs and/or videos may not be available at other places. In such circumstances, the programs may have to rely upon freely accessible images/videos in the internet. However, usage of publicly available files might limit the choices of the examiner and may have already been familiar with the students if the online sources. More importantly, the images and videos available in the internet might be subject to copyright rules, and reproduction of these images might be considered as violation of intellectual property laws. Therefore, it is advisable to use clinical media files of individual institutions. Another option is to prepare videos of tests performed correctly or incorrectly on volunteers, and use this for exams to assess the response of the candidates Although the present study used a large interactive screen to conduct the exam, any other suitable devices like a projector, laptop or smart TV can be used in resource limited settings. The use of multiple devices might be suitable for institutions with large number of students so that exams can be conducted in a timely manner. Finally, the clinical case scenarios used in our exam were prepared by faculty with help of the examiners. Therefore, the approval ratings of the examiners might be biased.

Table 3.

Strengths and limitations of virtual examination.

Strengths
Can assess a wide range of clinical knowledge
Avoid direct patient contact
Helps in maintaining social distancing
Standardized evaluation possible
Limitations
Cannot assess doctor-patient interaction
Lack multi-dimensional assessment of patient
Requires repository of clinical files

As it is important for the final year residents to complete the course and move forward with their career, residency programs should find ways to conduct examination even during the COVID-19 pandemic. Unnecessary delay in the exams can prolong the residency tenure of the current graduating residents which may not be fair to them. Therefore, we encourage other residency programs to conduct the orthopaedic residency end-of-training examinations during the COVID pandemic with the necessary precautions and hope that our experience will guide them during this pandemic.