Toward competency-based curriculum: Application of workplace-based assessment tools in the National Saudi Arabian Anesthesia Training Program

Abstract

Background:

The anesthesia training program of the Saudi Commission for health specialties has introduced a developed competency-based anesthesia residency program starting from 2015 with the utilization of the workplace-based assessment (WBA) tools, namely mini-clinical exercises (mini-CEX), direct observation of procedural skills (DOPS), and case-based discussion (CBD).

Objectives:

This work aimed to describe the process of development of anesthesia-specific list of mini-CEX, DOPS, and CBD tools within the Saudi Arabian Anesthesia Training Programs.

Materials and Methods:

To introduce the main concepts of formative WBA tools and to develop anesthesia-specific applications for each of the selected WBA tools, four 1-day workshops were held at the level of major training committees at eastern (Dammam), western (Jeddah), and central (Riyadh) regions in the Kingdom were conducted. Sixty-seven faculties participated in these workshops.

Results:

After conduction of the four workshops, the anesthesia-specific applications setting of mini-CEX, DOPS, and CBD tools among the 5-year levels were fully described. The level of the appropriate consultation skills was divided according to the case complexity adopted from the American Society of Anesthesiologists physical classification for adult and obstetric and pediatric patient as well as the type of the targeted anesthetic procedure.

Conclusion:

WBA anesthesia-specific lists of mini-CEX, DOPS, and CBD forms were easily incorporated first into guidelines to help the first stage of implementation of formative assessment in the Saudi Arabian Anesthesia Residency Program, and this can be helpful to replicate such program within other various training programs in Saudi Arabia and abroad.

Introduction

The aim of the anesthesia training programs as well as all other postgraduate training programs is to prepare competent graduates with independent specialist practice. Unfortunately, the emphasis of most of the traditional assessment methods is how much the trainee gain of knowledge rather than his clinical ability.[1] Because of that, the workplace-based assessments (WBAs) have been introduced in many postgraduate programs and even become a compulsory component of many specialists training program.[2]

The anesthesia training program of the Saudi Commission for Health Specialties (SCFHS) has introduced a developed competency-based anesthesia residency program starting from 2015. It is provided to all anesthesia residents across the 26 accredited centers in the Kingdom of Saudi Arabia and the Kingdom of Bahrain. The developed anesthesia residency program included many changes in the processes for training, assessment, and evaluation as well as the process of certification. One of the main and important changes in the developed program is the utilization of the WBA tools namely mini-clinical exercises (mini-CEX), direct observation of procedural skills (DOPS), and case-based discussion (CBD). These tools are among the most frequently used tools in WBA.

The mini-CEX formalized the supervisory relationship and promoting educational interactions. The structured format broadened the scope of feedback and made it easier to address performance gaps.[3] Among the strengths of using mini-CEX in anesthesia are the very positive educational impact and its relative feasibility. On the other hand, large numbers of assessors are required to produce reliable scores.[4] Starting in 2013, DOPS assessment became an integral part of the revised training program of Australian and New Zealand College of Anaesthetists (ANZCA).[5] The ANZCA DOPS demonstrates construct validity in the assessment of ultrasound-guided regional anesthesia and is potentially feasible in daily practice.[6]

CBD involves the discussion of a case in a semi-structured way. The Academy of Medical Royal Colleges recommends that the CBD should be conducted as a face-to-face discussion of a case, to “probe the reasoning behind any decisions made” while offering constructive feedback to the trainee.[7] CBD is an example of formative assessment, which is an ongoing process in which learners take an active role, and includes interaction in the form of a teaching and learning conversation.[8] CBD was chosen as an example of a WBA because of its similarity to the informal case discussion, which has previously been widely used in teaching.[9] The use of each one of these forms requires prior arrangement to ensure optimum applications to various related clinical settings. These arrangements including developing a blueprint to use various forms at appropriate anesthesia care provision sites. The available data that help in preparing encounter that fits anesthesia training levels to match the needed structure of WBA forms is very limited. Therefore, this work aimed to describe the process of development of anesthesia-specific list of mini-CEX, DOPS, and CBD tools within the Saudi Arabian Anesthesia Training Programs.

Materials and Methods

The developed program was approved by the Scientific Board of Anesthesia and Critical Care at SCFHS. To introduce the main concepts of formative WBA tools and to develop anesthesia-specific applications for each of the selected WBA tools, four 1-day workshops were held at the level of major training committees at eastern (Dammam), western (Jeddah), and central (Riyadh) regions in the Kingdom were conducted. The total number of faculty participated in the four workshops was 67. Gender, specialty level, and hospital where the participants work all are shown in Table 1.

Table 1.

Demographics of participating faculty number (percentage) per regional training committee

Each workshop followed structured protocol and many focus group discussions were conducted. The last session of each workshop was dedicated to train the participants on the scoring scales of the tools and to discuss applications aspects of the upcoming implementation phase of the project.

After conduction of the four workshops, lists included the anesthesia-specific applications for each one of the selected WBA tools were generated to be included in the developed anesthesia program. These lists were further reviewed by experienced anesthesiologists at various training centers as well as by program residents at various levels of training to ensure its clarity and to be validated.

Results

The anesthesia-specific applications setting of mini-CEX, DOPS, and CBD tools among the 5 years levels are fully described in Tables 2–4, respectively. The level of the appropriate consultation skills were divided according to the case complexity adopted from the American Society of Anesthesiologists physical classification for adult and obstetric and pediatric patient as well as the type of the targeted anesthetic procedure [Table 2].

Table 2.

List of clinical encounters approved for mini-clinical exercises for each year of training

Table 4.

Case-based discussion suggested topics

For DOPS, the anesthetic procedures were categorized according to the subcategories of general, neuro-axial and regional anesthesia, airway, vascular access, and equipment. The residency level-specific procedural competency in each year of the residency years was shown in relation to overall training program structure [Table 3]. These guidelines provided full range of anesthesia training program required skills including simple anesthetic procedures with classical and landmarks based techniques to most up to date ultrasound guided interventions reflecting today's complex anesthesia training and practice requirements.

Table 3.

Direct observation of procedural skills

Table 4 shows CBDs applications divided according to the residency level and the subcategories of airway/breathing, cardiovascular/circulation, and other type of major anesthesia encountered complications and sentinel events.

Discussion

The developed competency-based anesthesia residency program launched by the SCFHS in 2015 was in response to the international trend toward competency-based anesthesia curricula, with defined criteria describing the knowledge, skill sets, and professional attributes of a specialist anesthetist.[10,11] WBAs are particularly important given the shift toward such competency-based medical education in which core competencies are articulated and subsequently measured.[12] Multiple formative assessments of trainees which inform evaluation and learning are fundamental to competency-based education.[13]

The development of WBA anesthesia-specific lists was an important step to help roll in this new significant change in the developed national curriculum of Saudi Anesthesia Training Program. This process involved the conduction of several workshops whereas both training of the faculty and focus group discussions with them took place to develop the required WBA specialty-specific applications of the assessment tools within the various anesthesia training environments. The used methods helped in obtaining the ownership of the development process by all the participants who included arrays of faculty, supervisors, and training directors of anesthesia residency training programs. Faculty participating in these workshops was made mandatory for participating at the first step of implementation of the program at various training centers accredited for anesthesia training in the Kingdom of Saudi Arabia. This improved and increased faculty role in the process of development and subsequent implementation of the findings and results in real practice locations across the training centers at national level. This is can be attributed to the sense of faculty ownership during the initial phases of mapping and planning of the program and their vital role in the developing the guidelines.

It was stated a long time ago that the technical skills should be assessed if they are either commonly performed or potentially lifesaving;[14] however, more recently Hopkins reported that the core skills change as medical technology and knowledge develops.[15] Among the benefits gained during the development of WBA anesthesia-specific lists are yielding level-appropriate tasks per each of the three tools for each level of the 5 years of the anesthesia residency training program. Such development led to clear articulation of expected level of performance for each trainee in terms of various clinical encounters (mini-CEX), anesthesia procedures (DOPS), and CBD. These lists help each faculty and program administrators to clearly identify level-specific competencies needed for successful progress within the 5-year program in a uniformed systematic manner. It was reported that comparing a resident's performance with that of residents at a similar training level is problematic if faculty are uncertain about what skills should be expected at a particular stage of training, or what constitutes competence.[16] Consequently, the aspects of performance that assessors regard as useful for determining quality are inconsistent between assessors.[17] When the foundation year one doctors’ experiences with mini-CEX were explored at Royal Derby Hospitals in the UK, the trainees recognized the potential of the mini-CEX; however, this was hindered by the inadequate knowledge about the assessment standard and guidelines that were generally lacking.[18]

The specific details of development of these anesthesia specialty-specific applications are important beyond anesthesia training program nationally and globally. Similar developments can help other medical and surgical residency training programs to develop similar specialties-specific lists for WBA tools within different training programs. As we see in our experience, such approach will have immediate benefits for all programs trainees, program administrators, and also practicing physicians. The developed lists of these forms lead to many immediate practical applications such as setting minimum level of competency for promotion from year to year during end-of-year examinations. Each resident is tested for competencies required at his/her level. This arrangement was used to set up examinations for all examinations in the program including the newly introduced final, especially Objective Structured Clinical Examination in the years followed the introduction of these tables. Further, many anesthesia departments used these level-specific competencies to set clinical privileges for anesthesiologists in various hospitals according to the years of practice and expertise. It is anticipated that other applications for such lists will help the process of certifications and licensing and revalidation for current and future anesthesia providers at the national level.

Conclusion

The Saudi Arabian Anesthesia Residency Training Program was used to deliberate plan to develop WBA anesthesia-specific list for each of mini-CEX, DOPS, and CBD forms. These lists were easily incorporated first into guidelines to help the first stage of implementation of formative assessment within the anesthesia residency program, and this can be helpful to replicate such program within other various training programs in Saudi Arabia and abroad. Coming studies will be conducted to investigate the effect of these changes developed in the Saudi Arabian Anesthesia Residency Program after the first cycle of implementation.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.