Objective Structured Clinical Examination: The Assessment of Choice

Abstract

The Objective Structured Clinical Examination is a versatile multipurpose evaluative tool that can be utilized to assess health care professionals in a clinical setting. It assesses competency, based on objective testing through direct observation. It is precise, objective, and reproducible allowing uniform testing of students for a wide range of clinical skills. Unlike the traditional clinical exam, the OSCE could evaluate areas most critical to performance of health care professionals such as communication skills and ability to handle unpredictable patient behavior.

Introduction

Since its introduction as a mode of students’ assessment in medical school in 1975, by Haden and Gleeson,¹ the objective structured clinical examination (OSCE) has become a standard method of assessment in both undergraduate and postgraduate students. Originally described as ‘a timed examination in which medical students interact with a series of simulated patients in stations that may involve history-taking, physical examination, counselling or patient management,¹ the OSCE examination has been broadened in its scope and has undergone a lot of modification to suit peculiar circumstances.^2-4 In the United Kingdom, United States, Canada and indeed most reputable colleges of medicine the OSCE is the standard mode of assessment of competency, clinical skills, and counselling sessions satisfactorily complementing cognitive knowledge testing in essay writing and objective examination.^3-6

The OSCE is a versatile multipurpose evaluative tool that can be utilized to evaluate health care professionals in a clinical setting. It assesses competency, based on objective testing through direct observation. It is comprised of several "stations" in which examinees are expected to perform a variety of clinical tasks within a specified time period against criteria formulated to the clinical skill, thus demonstrating competency of skills and/or attitudes. The OSCE has been used to evaluate those areas most critical to performance of health care professionals, such as the ability to obtain/interpret data, problem-solve, teach, communicate, and handle unpredictable patient behavior,^3-7 which are otherwise impossible in the traditional clinical examination. Any attempt to evaluate these critical areas in the old-fashioned clinical case examination will seem to be assessing theory rather than simulating practical performance.

Advantages and Disadvantages of OSCE

Written examinations (essays and multiple choices) test cognitive knowledge, which is only one aspect of the competency. Traditional clinical examination basically tests a narrow range of clinical skills under the observation of normally two examiners in a given clinical case. The scope of traditional clinical exam is basically patient histories, demonstration of physical examinations, and assessment of a narrow range of technical skills. It has been shown to be largely unreliable in testing students’ performance and has a wide margin of variability between one examiner and the other.^8,9 Data gathered by the National Board of Medical Examinations in the USA (1960–1963), involving over 10,000 medical students showed that the correlation of independent evaluations by two examiners was less than 0.25.⁸ It has also been demonstrated that the luck of the draw in selection of examiner and patient played a significant role in the outcome of postgraduate examinations in psychiatry using the traditional method.⁶

Published findings of researchers on OSCE from its inception in 1975 to 2004 has reported it to be reliable, valid and objective with cost as its only major drawback.⁹ The OSCE however, covers broader range like problem solving, communication skills, decision-making and patient management abilities.

The advantages of OSCE apart from its versatility and ever broadening scope are its objectivity, reproducibility, and easy recall. All students get examined on predetermined criteria on same or similar clinical scenario or tasks with marks written down against those criteria thus enabling recall, teaching audit and determination of standards. In a study from Harvard medical school, students in second year were found to perform better on interpersonal and technical skills than on interpretative or integrative skills.^10,11 This allows for review of teaching technique and curricula.

Performance is judged not by two or three examiners but by a team of many examiners in-charge of the various stations of the examination. This is to the advantage of both the examinee and the teaching standard of the institution as the outcome of the examination is not affected by prejudice and standards get determined by a lot more teachers each looking at a particular issue in the training. OSCE takes much shorter time to execute examining more students in any given time over a broader range of subjects.^10-12

However no examination method is flawless and the OSCE has been criticized for using unreal subjects even though actual patients can be used according to need.¹¹ OSCE is more difficult to organize and requires more materials and human resources.^9,13,14

Table 1. Advantages & Disadvantages of OSCE.

Advantages of OSCE Objectivity	Disadvantages
1. Uniform scenarios for all candidates 2. Availability 3. Safety, no danger of injury to patients 4. No risk of litigation 5. Feedback from Actors (simulators) 6. Allows for Recall 7. Stations can be tailored to level of skills to be assessed 8. Allows for teaching audit 9. Allows for demonstration of emergency skills	1. Organizational training 2. The idealized ‘textbook’ scenarios may not mimic real-life situations 3. Expensive

How is OSCE done?

OSCE’s basic structure is a circuit of assessment stations, where examiners, using previously determined criteria assess range of practical clinical skills on an objective-marking scheme.¹²

Such stations could involve several methods of testing, including use of multiple choice or short precise answers, history taking, demonstration of clinical signs, interpretation of clinical data, practical skills and counselling sessions among others.^13,14 Most OSCEs use "standardized patients (SP)" for accomplishing clinical history, examination and counselling sessions. Standardized patients are individuals who have been trained to exhibit certain signs and symptoms of specific conditions under certain testing conditions.^14,15

The basic steps in modelling an OSCE exam include:

1. Determination of the OSCE team.

2. Skills to be assessed (OSCE Stations).

3. Objective marking schemes

4. Recruitment and training of the standardized patients.

5. Logistics of the examination process.

The OSCE Team

Examiners, marshals and timekeepers are required. Some stations could be unmanned such as those for data or image interpretation but most require an examiner to objectively assess candidate performance based on the pre-set criteria.^16,17 A reserve examiner who can step in at the last time if required is a good practice. Examiners must be experienced and a standard agreed upon at the outset. Examiners must be prepared to dispense with personal preferences in the interests of objectivity and reproducibility and must assess students according to the marking scheme.¹² Marshals and timekeepers are required for correct movement of candidates and accurate time keeping. OSCE is expensive in terms of manpower requirement.

Skills Assessed in OSCEs

The tasks to be assessed should be of different types and of varying difficulties to provide a mixed assessment circuit. The tasks in OSCE depend on the level of students training. Early in undergraduate training correct technique of history taking and demonstration of physical signs to arrive at a conclusion may be all that is required.^10,12,13

At the end of the training however, testing a broader range of skills, may be required. This could include formulation of a working diagnosis, data and image interpretation, requesting and interpreting investigations, as well as communication skills. Postgraduate medicine may involve more advanced issues like decision taking, handling of complex management issues, counselling, breaking bad news and practical management of emergency situations. There is no hard or fast rules to the skills tested but are rather determined by the aim of assessment.^9,17 Complex stations for postgraduate student could test varying skills including management problems, administrative skills, handling unpredictable patient behaviour and data interpretation.^17-19 These assessments and many others are impossible in traditional clinical examination.

Objective marking scheme

The marking scheme for the OSCE is decided and objectively designed. It must be concise, well focused and unambiguous aiming to reward actions that discriminate good performance from poor one. The marking scheme must take cognizance of all possible performances and provide scores according to the level of the student’s performance. It may be necessary to read out clear instructions to the candidates on what is required of them in that station. Alternatively, a written instruction may be kept in the unmanned station.¹²

It is good practice to perform dummy run of the various stations, which enables exam designers to ensure that the tasks can be completed in the time allocated and modify the tasks if necessary. Candidates should be provided with answer booklets for the answers to tasks on the unmanned stations, which should be handed over and marked at the end of the examination.

Recruitment and Training of Standardized or Simulated Patient

Vu and Barrows defined standardized patients as "real" or "simulated" patients who have been coached to present a clinical problem.¹² Standardized patients may be professionally trained actors, volunteer simulators or even housewives who have no acting experience. Their use encompasses undergraduate and postgraduate learning, the monitoring of doctors’ performance and standardization of clinical examinations. Simulation has been used for instruction in industry and the military for much longer period,⁷ but the first known effective use of simulated patients was by Barrows and Abrahamson (1964),¹³ who used them to appraise students’ performance in clinical neurology examinations.

SP candidates must be intelligent, flexible, quick thinking, and reliable. Standardized patients’ understanding of the concept of the OSCE and the role given to them is critical to the overall process.^20,21

An advantage of simulated patients over real patients is that of allowing different candidates to be presented with a similar challenge, thereby reducing an important source of variability.^12,22 They also have reliable availability and adaptability, which enables the reproduction of a wide range of clinical phenomena tailored to the student’s level of skill. In addition, they can simulate scenarios that may be distressing for a real patient, such as bereavement or terminal illness.¹³ Their use also removes the risk of injury or litigation while using real patients for examination especially in sensitive area of medicine like obstetrics and gynecology.

The validity of the use of SP in clinical practice has been proved by both direct and indirect means. In a double-blind study, simulated patients were substituted for real patients in the individual patient assessment of mock clinical examinations in psychiatry. Neither the examiners nor the students could detect the presence of simulated patients among the real patients. Indirect indicators of validity might include the fact that simulators are rarely distinguished from real patients.^15,16

Simulated patients are however expensive in terms of the time it takes to train and coach them in performing and understanding concepts, this could be very difficult in some fields like pediatrics where problems in very young children need to be simulated.^17,18 The cost of paying professionals adds to the expense. However, the time efficiency of OSCE and its versatility makes the cost worthwhile.^18,22 Recruitment and training of the SP is critical to the success of the OSCE. SP could be used not only for history taking and counselling, but also for eliciting physical findings that can be simulated, including aphasia, facial paralysis, hemiparetic gait, and hyperactive deep tendon reflexes.^16,19,20

Logistics of the examination process

Enough space is required for circuit running and to accommodate the various stations, equipment and materials for the exam. The manned stations should accommodate an examiner, a student and possibly the standardised patient and also allow for enough privacy of discussion so that the students performing other tasks are not distracted or disturbed. A large clinic room completely cleared could be ideal and may have further advantage of having clinic staff that will volunteer towards the execution of the examination thereby reducing cost.

The stations should be clearly marked and the direction of flow should also be unambiguous. It is good practice to have test run involving all candidates for that circuit so that they acquaint themselves to the direction of movement and the sound of the bell.^14,18

Conclusion

The OSCE style of clinical assessment, given its obvious advantages, especially in terms of objectivity, uniformity and versatility of clinical scenarios that can be assessed, shows superiority over traditional clinical assessment. It allows evaluation of clinical students at varying levels of training within a relatively short period, over a broad range of skills and issues. OSCE removes prejudice in examining students and allows all to go through the same scope and criteria for assessment. This has made it a worthwhile method in medical practice.