An exploratory analysis of factors contributing to resident pass rates on a national licensure OSCE in the United Arab Emirates

Abstract

Purpose:

To explore resident perceptions of factors contributing to pass rates on a high-stake licensing objective structured clinical exam (OSCE).

Materials and Methods:

A cross-sectional survey was administered to all 51 applicants of the April 2019 internal medicine Arab Board OSCE examination in Abu Dhabi, United Arab Emirates (UAE), and included questions on preparedness, stress level, and prior educational experiences. Exposures were evaluated for correlation against Arab Board pass rates using Pearson correlation and the two-tailed significance was recorded.

Results:

All 51 examinees completed the survey (100% response rate). Participants were primarily female n = 35 (67%) and all completed residency training in the UAE. Gender differences were noted, with higher pass rates for the male residents (13/35, 37% females vs. 8/16, 50% males, P < 0.05). Further, 65% (P < 0.001) of female examinees reported higher levels of anxiety than male residents. Examinees reported regular exposure during residency to clinical skills training (74%), standardized patients (71%), simulation (66%), and OSCEs (72%) but none of these educational modalities correlated with higher pass rates. Of multiple exam preparation modalities, only self-directed learning with deliberate practice, the intentional repetition of a task with feedback, was associated with higher pass rates.

Conclusion:

Clinical exam skills are vital for trainees to deliver high-quality primary healthcare services. National licensure OSCEs have become the norm in the Arab world for assessing resident clinical exam skills. Our results suggest that residency programs should encourage residents’ intentional deliberate practice and mastery learning in the acquisition and retention of physical examination techniques.

Background

An accurate and complete history and physical exam forms the foundation for clinical reasoning and is critical for safe and effective patient care. Primary care has become an important element of medical education and clinical exam skills are vital for trainees to deliver high-quality primary healthcare services. Yet, studies show that medical students and residents have significant deficiencies in their clinical examination skills.[1,2,3] How best to teach and assess these competencies has been a challenge for medical educators worldwide. The Objective Structured Clinical Exam (OSCE), since its introduction in 1975, has served as an important evaluation tool for assessing trainee clinical skills.[4] A national licensure OSCE was implemented by Canada in 1992, and has since been replicated by countries worldwide.[5] Within the past decade, many regulatory bodies in the Arab world, including the Arab Board of Medical Specialties and the Saudi Board, have adopted the OSCE as the final licensure examination.[6]

Studies have highlighted the noncognitive factors that affect OSCE performance, including confidence, anxiety levels, and preparedness.[7] The literature, however, has focused primarily on undergraduate students and in Western settings. As the OSCE has become a critical component of postgraduate licensure in the Arab world, it is important to understand the factors that influence resident performance during these clinical exams. This study is an exploratory analysis of factors that may contribute to improved pass rates on a high-stake OSCE for internal medicine residents.

Materials and Methods

Survey instrument

The survey instrument was developed through a review of the medical literature identifying factors that have been shown to contribute to improved clinical skills,[7,8] and piloted on a cohort of recent graduates, who did not participate in the study. The instrument collected basic demographic information and included questions on preparedness, stress level, and prior educational experiences.

Participants and setting

The Arab Board accredits residency programs throughout the Arab world. Residents must pass a high-stake OSCE as a prerequisite to residency completion and licensure. The 6-station OSCE includes a communication and a history-taking station and 4 physical exam stations, each focusing on a single organ system. Examinees are observed and graded through a checklist for a predetermined set of standardized observable techniques. In April 2019, the Arab Boards OSCE was delivered in Abu Dhabi, a hub for the country's residency programs and capital of the United Arab Emirates (UAE), to a cohort of general internal medicine residents. Using convenience sampling methodology, we conducted a cross-sectional survey of all applicants to the April 2019 internal medicine Arab Board OSCE. A written, self-administered survey was distributed and collected before the OSCE by a volunteer not involved in the exam process. Participation was anonymous and voluntary, and residents were informed that survey completion did not affect test scores. Examinees were assigned a unique identifier to match survey responses with examination scores.

Data analysis

The collected data was analyzed using the Statistical Package for Social Sciences, IBM, Chicago, IL, version 21 for descriptive statistics. Various exposures were evaluated for correlation against the Arab board pass rates using Pearson Correlation and two-tailed significance recorded. A value of P < 0.05 was considered to be significant. The manuscript adheres to STROBE guidelines for reporting observational research. The study was reviewed and approved by the Sheikh Khalifa Medical City Research Ethics Committee (RS-573).

Results

All 51 examinees signed written consents and completed the survey (100% response rate). Demographic characteristics of the residents are shown in Table 1. The majority of examinees reported regular exposure during residency to clinical skills training (40/51, 74%), standardized patients (37/51, 71%), simulation (34/51, 66%), and mock OSCEs (37/51, 72%); none of these educational modalities correlated with higher pass rates [Table 2]. The residents used several different preparation modalities. Only deliberate practice, defined as the mindful and intentional repetition of practicing physical exam skills with patients, ⁴ correlated with higher pass rates with 63% (P < 0.001) of those who passed reporting this as their primary study technique [Table 3]. Gender differences were noted, with higher pass rates for the male residents, using the Chi-square test (13/35, 37% females vs. 8/16, 50% males, P < 0.05). Further, 71% (25/35, P < 0.05) of female examinees reported higher levels of anxiety than male residents (44%, 7/16, P < 0.05) and higher anxiety correlated with higher pass rates (30/51, 59%, P < 0.05).

Table 1.

Demographics of the examinees taking the OSCE (n=51)

	n (%n)
Gender
Female	35 (68)
Male	16 (32)
Age Group
30 Years or Less	29 (56)
31+ Years	18 (35)
Marital Status
Single	28 (54)
Married	23 (45)
Number of Children
None	34 (66)
1-2 Children	15 (29)
3+ Children	2 (4)

Table 2.

Impact of Educational Activities on OSCE Pass rate (n=51)

Educational Activity	Number of Examinees Participating	% Total	Pearson Correlation Coefficient	Significance
Clinical Skills Training Course	40/51	74	-0.15	NS
Standardized Patient Training	37/51	71	-0.37	NS
Simulation	34/51	66	-0.19	NS
Mock OSCEs	37/51	72	-0.67	NS

OSCE=Objective Structured Clinical Exam. NS=not significant

Table 3.

Correlation Between Methods of Exam Preparation and OSCE Pass Rate (n=51)

Exam Prep Activity	Pearson Correlation Coefficient	Significance
Mock OSCE	-0.052	0.715
Online Clinical Skills Modules	-0.136	0.34
Deliberate Practice/Self-Directed Learning	0.074	0.05
Study Groups	-0.015	0.919
You Tube Videos	-0.254	0.73
Other Preparation	-0.118	0.408

OSCE=Objective Structured Clinical Exam

Discussion

Given the time and financial investment involved in training future physicians, optimizing licensure exam pass rates does not just affect the individual trainee, but has implications for education and healthcare systems worldwide. Further, as general internists provide primary healthcare services to the community, it is incumbent upon training programs to ensure that graduates are proficient in their clinical examination skills. In recent years, residency programs have included, at great manpower and financial expenditure, additional educational initiatives to improve resident clinical skills, including high-fidelity simulation, computer technologies, and standardized patients. Our results did not show a correlation between these modalities and improved OSCE pass rates. It has long been acknowledged that teaching clinical skills is time-intensive and is directly correlated to the amount of time trainees spent performing clinical exam techniques with their senior clinicians.[9] Although these technologies may be useful adjuncts to provide trainees with opportunities to practice and refine their physical exam techniques, our study suggests that they should not replace direct patient contact with observation and feedback from faculty.[10]

It has been suggested that the unfamiliarity of the OSCE format may contribute to increased test anxiety.[7] In our study, 72% of examinees had prior exposure to OSCEs during training, but these experiences did not translate into higher pass rates, even for those who participated in multiple OSCEs. Residents also reported several test preparation modalities; only deliberate practice, repetitive and mindful practice of physical examination techniques with patients, was associated with higher pass rates. Studies have shown that trainee retention of clinical skills is enhanced when they are allowed self-guided access to instructional material and set process goals.[11] Further, there is a large body of literature showing that mastery learning with deliberate practice is correlated with improved physician procedural skills and better patient outcomes.[12] Our data confirm the importance of deliberate practice and self-directed learning in physical examination skills attainment.

The gender differences in OSCE pass rates for our examinees varies from other studies, which found that female trainees generally score higher on standardized patient examinations and clinical skills tests.[13,14] Further studies are necessary to assess if these gender differences persist in larger scale or multinational studies or if gender biases contributed to these results. Our findings are consistent with other research that has shown higher test anxiety rates for female trainees.[15] It is notable that in this group of residents, higher anxiety correlated with higher pass rates. The literature is inconclusive on whether stress interferes with learning and leads to poor performance during assessments, or whether a high level of test anxiety is functional, perhaps leading to increased arousal and better scores.[16,17]

Limitations of this study include the small sample size and single specialty, which limits generalizability. The cross-sectional design can only determine correlation, not causality. Also, inherent to any self-report of complex issues, such as examination performance, are influencing factors that may not have been fully addressed, including trainee personality and previous examination performance. Nonetheless, this study offers important considerations for both trainees and educational program leaders.

Conclusions

A main objective of any training program is to graduate doctors who are competent in clinical examination skills. This holds especially true for primary care programs. As national licensure OSCEs have become the norm in the Arab world, our results suggest that residency programs should continue to encourage residents’ deliberate practice and self- directed learning in the acquisition and retention of physical examination techniques.

List of Abbreviations

OSCE: Objective Structured Clinical Examination.

Ethics approval and consent to participate

The study was approved by the Sheikh Khalifa Medical City Research Ethics Committee (RS-573). All participants signed written consent prior to survey completion.

Competing interests

The authors declare that they have no competing interests.

Financial support and sponsorship

The authors have not received any funding for this study.

Conflicts of interest

There are no conflicts of interest.