An introduction to workplace-based assessments

Abstract

Recent trends in medical education are moving rapidly away from gaining a certain number of marks in high-stakes examinations and towards gathering evidence of clinical competence and professional behavior observed in clinical environments (workplace-based learning). In the Miller's framework for assessing clinical competence, workplace-based methods of assessment target the highest level of the pyramid and collect information about doctors’ performance in their everyday practice. Direct Observation of Procedural Skills (DOPS), Mini-Clinical Evaluation Exercise (mini-CEX) and Case-based discussion (CbD) are some of the most commonly used methods of workplace-based assessments. I explain these three methods of assessment and their advantages and discuss that if incorporated in a structured program of teaching for doctors in training, they can promote active, learner-centered learning and facilitate provision of developmental verbal feedback to the trainee immediately afterwards.

Introduction

Historically too much emphasis has been placed on determining whether students and trainees can pass exams, and insufficient emphasis on whether they can perform in the role expected of them as medical practitioners (1). Traditional clinical examinations such as Objective Structured Clinical Examinations (OSCEs) pioneered by Ronald Harden in Dundee (2) have been used widely across many education fields for several decades. However there are limitations with such assessments. Stations often require trainees to perform isolated aspects of the clinical encounter, which ‘deconstructs’ the doctor-patient encounter, and the type of cases that can be simulated constrain the type of patient problems can be used (3).

Recent trends in medical education are moving rapidly away from gaining a certain number of marks in high-stakes examinations and towards gathering evidence of clinical competence and professional behavior on a daily basis in the workplace. For this reason, on-the-job workplace-based assessments (WPBA) have been developed to assess workplace-based learning programs.

This paper aims to serve as an introduction of WPBA as an effective tool for evaluation of competence, complementing other more traditional and formal specialty examinations. We first describe the educational basis and background to WPBA and then discuss three of the most commonly used tools.

Educational basis of workplace-based assessments

In Miller's framework for assessing clinical competence, the lowest level of the pyramid is knowledge (knows), followed by competence (knows how), performance (shows how), and action (does) (4). “Action” focuses on what occurs in practice rather than what happens in an artificial testing situation. Workplace-based methods of assessment target this highest level of the pyramid and collect information about doctors’ performance in their everyday practice. Other common methods of assessment, such as multiple-choice questions target the lower levels of the pyramid (5).

Experts believe that assessments of actual practice are much better reflections of routine performance than assessments done under test conditions. A study was carried out to evaluate the use of comprehensive WPBA across the medical specialties in the United Kingdom between year 2003 and 2004, and it was recognised that these methods are feasible to conduct and can make reliable distinctions between doctors’ performances (6).

Direct Observation of Procedural Skills (DOPS)

DOPS is designed to provide feedback on procedural skills essential to the provision of good clinical care. Trainees are asked to undertake practical procedures with a different observer for each encounter. Each DOPS should represent a different procedure and will normally be completed opportunistically during everyday work. The trainee chooses the timing, procedure and the observer, which may be experienced Registrars, Consultants or appropriate nursing staff who are competent in the procedure assessed.

The assessment involves an assessor observing the trainee perform a practical procedure within the workplace; and a structured checklist is designed to give guidance for the assessors. Most procedures take no longer than 15-20 minutes. Feedback would normally take about 5 minutes. There are certain mandatory procedures to be covered for trainees at different stages of medical training, for example for newly qualified trainees (first year residents): venepuncture, arterial blood sampling, urinary catherterisation, etc.

Behaviours observed in a DOPS include:

• Demonstrating understanding of indications, relevant anatomy and technique

• Obtaining informed consent

• Demonstrating appropriate preparation pre-procedure

• Appropriate analgesia or safe sedation

• Technical ability

• Aseptic technique (if appropriate)

• Seeking help where appropriate

• Post procedure management

• Communication skills

• Consideration of patient/professionalism

• Overall ability to perform procedure

The following are the main advantages of DOPS as a valid assessment tool:

1. The trainee is assessed during everyday work performing procedures on real patients.

2. Not only the technical ability is observed, but also interaction with patients, colleagues and professional behaviors can be assessed.

3. A range of skills, from simple to very complex procedures can be assessed.

4. Many trainees will “need further development”, so after receiving feedback, the strengths and weaknesses can be highlighted and the trainee can work on them and be assessed at a later date.

5. There is a need to check that doctors’ procedural skills have been retained and are used appropriately within the context of everyday practice, DOPS is a suitable assessment tool for this purpose.

Mini-Clinical Evaluation Exercise (mini-CEX)

The Mini-CEX was developed by the American Board of Internal Medicine to assess medical residents in real life settings. Mini-CEX is a 15-minute snapshot of doctor-patient interaction, designed to assess the clinical skills, attitudes and behaviors essential to the provision of high quality care. The assessment involves observing the trainee interact with a patient in a clinical encounter. Each of these encounters should represent a different clinical problem and trainees should sample from a wide range of problem groups with each focusing on specific aspects of the clinical encounter. It permits evaluation based on a much broader set of clinical settings and patient problems, and is administered on site (7). Trainees are encouraged to choose a different assessor for each assessment. The estimated time required is 20 minutes (15 minutes for assessment, 5 minutes for feedback).

The areas of competence covered include:

• History taking

• Physical examination

• Professionalism

• Clinical judgment

• Communication skills

• Organisation

• Efficiency

• Overall clinical care

The main strengths of mini-CEX as an assessment tool are as follows:

1. It can be used in different clinical settings: on the ward, on ward rounds, during on-call shifts, or in outpatient clinics.

2. Skills such as history taking, communication skills, physical examination and the management of patient problems can be difficult to assess reliably and in the past such assessment has been sub-optimal. Mini-CEX provides a practical solution within the workplace.

3. Because the interaction is relatively short and each trainee can be evaluated on several occasions, in comparison to the traditional “long case examination”, mini-CEX assesses trainees in a much broader range of clinical situations, has better reproducibility, and offers trainees greater opportunity for instruction and feedback by “more than one” faculty member and with “more than one” patient.

4. Through being observed undertaking a number of cases, over a period of time, with a number of different assessors, these individual brief encounters add up to provide a reliable measure of a trainee's performance.

5. Mini-CEX format may produce less anxiety than the traditional formats, because the assessment is less formal and less dependent on a single, high-stakes encounter with one faculty member and one patient.

On the other hand, mini-CEX may be more difficult to administer because multiple encounters must be scheduled for each trainee. Exclusive use of mini-CEX also prevents trainees from being observed while doing a complete history and physical examination (8).

Case-based Discussion (CbD)

The CbD is a structured discussion between the trainee and educational supervisor about how a clinical case was managed by the trainee; talking through what occurred and reasons for actions. Normally before the discussion the trainee selects 2 (or more) cases and present copies of relevant clinical entries to the supervisor who selects one of them. The discussion should be framed around the actual case and should not explore hypothetical events. Most assessments take no longer than 15-20 minutes. Feedback would normally take about 5 minutes. The trainee and the trainer should ensure that throughout the placement, a balance of cases is represented across varying contexts.

The following are considered as the main advantages of CbD:

1. CbD is a structured, in-depth discussion between the trainee and educational supervisor about decision-making and application of medical knowledge in cases for which the trainee has been directly responsible, so it can be used to explore professional judgment. By using clinical cases that offer a challenge to the trainee, rather than routine cases, the trainee is able to explain the complexities and the reasoning behind choices made.

2. CbD can test higher order thinking and synthesis as it allows assessors to explore deeper understanding of how trainees prioritise and apply knowledge.

3. It enables the discussion of the ethical and legal framework of practice.

4. As actual patient records are the basis for dialogue, the assessor can also evaluate the quality of record keeping and the presentation of cases.

How to use workplace-based assessments

Workplace-based assessments should be part of a structured program of teaching that is designed for doctors in training – and in each clinical placement, the teaching program should constitute the following essential steps:

• Induction

• Systematic teaching, based on the curriculum

• Workplace-based learning and assessment

• On-going feedback

• Encouraging a holistic approach, reflective practice and life-long learning

Junior doctors should be asked to carry out a certain number of assessments (DOPS, Mini-CEX and CbD) in each placement. The trainees’ performance and progression can be reviewed at the end of each training year from a portfolio of on-going workplace based assessments.

Conclusion

Workplace-based assessments create a self-directive learning environment that is essential for continuing professional development. A broad discipline of everyday clinical encounters that is very relevant to trainees’ overall curriculum can be assessed at workplace and the interaction between trainees and their assessors provides an invaluable learning experience.

There are several common advantages that make WPBA a suitable and reliable method for assessment of doctors in training:

1. The trainee is responsible for selecting cases, requesting an assessment and proper completion of the paperwork, so it promotes active, learner-centered learning.

2. Assessment occurs as a natural part of the training environment, which minimises the artificiality of the task. In hospitals, there is plenty of opportunity to do WPBA.

3. Assessors do not need to have prior knowledge of the trainee.

4. The assessor's evaluation is recorded on a structured checklist that enables provision of developmental verbal feedback to the trainee immediately afterwards. Trainers and trainees can identify and agree strengths, areas for development and an action plan for each encounter.

5. All of the areas in Miller's pyramid which describes an overall assessment framework that is relevant to medicine both as a cognitive and skills-based discipline can be explored through WPBAs (6).

6. WPBA help identify trainees who are struggling and are in need of extra support early in training. This creates a supportive environment for trainees in difficulty.

Evidence collected will support the judgments made about the trainee at mid-placement and final reviews throughout the entire program of training.

(Please cite as: Liu C. An introduction to workplace-based assessments. Gastroenterol Hepatol Bed Bench 2012;5(1):24-28).