Teaching and Assessing Surgical Competence

Abstract

Surgical competence and its assessment is one of the most hotly debated topics engaging the profession. In the current climate of diminishing working hours and shorter training, the surgical profession is having to address the complex issue as to how surgery as a craft specialty should be taught, and how to assess when an individual is competent within their chosen sphere as well as how that competence should be maintained. Internationally, there is political pressure upon the professional to achieve contracted activity to comply with political imperatives and, at the same time, to achieve a greater degree of specialisation. Within Europe, the working time directive has led to a shift system of rotas and this, along with a shorter overall period of training, has led to reduced time available to surgical trainees in which to learn their craft.

There are four basic aims and objectives underpinning surgical education: (i) a sound knowledge base; (ii) good communication skills; (iii) proficient technical skills; and (iv) excellent clinical judgement. This latter characteristic cannot be understated in its importance. As Lord Smith, a past President of The Royal College of Surgeons of England once put it: ‘it would take me one year to teach a trainee how to do an operation, five years to teach them when to do the operation, but a lifetime to teach them when not to do an operation’.

Furthermore, when considering training methods in surgery, it is important to recognise the principles of adult education.¹ In the past when an apprenticeship style of training was the norm, trainees learned from watching and listening to the master. However, with current time constraints, training must be more pro-active and yet, at the same time, must recognise that adult education should be self-directed, experiential and reflective. Indeed, experience without any reflection – the danger of apprenticeship – is mere service, whilst reflection without any practical experience is simply pedantry. In an attempt to respond positively to many of these practical constraints, the surgical profession has sought to develop pro-actively an explicit curriculum with surgeon-led surgical skills' courses, an atmosphere where mentoring is encouraged, more accurate assessment tools and specific faculty development.

An explicit surgical curriculum

In the UK, the four surgical Royal Colleges have joined together, along with the Specialty Advisory Committees (SACs) to develop an explicit surgical curriculum.² This has been based on the CanMEDS roles³ which have indicated that the surgical expert has six strands to their professional role. This indicates that the surgeon is a professional, a communicator, a scholar, a collaborator, an advocate and a manager (Fig. 1). All these six roles have been brought into play within the curriculum which has been devised for each surgical specialty and is web-based and trainee-driven. This is due to come into formal use in August 2007 and, meanwhile, is being piloted in several centres to assess its credibility and sustainability.

Figure 1.

The CanMEDS key roles for the clinician.

For each speciality area in surgery, there are indicative years for which competency levels have been set. Using modern assessment tools,⁴ these competencies are clearly defined for each indicative year and each speciality. This means that a trainee will not advance until they have achieved those defined competencies, but will also allow a run-through training without any further major hurdles. There has long been a need for a more explicit curriculum within surgical training and it is to be hoped that the new Inter-collegiate Surgical Curriculum Project (ISCP) will deliver all that it promises. However, for such a curriculum to succeed, it needs to be supported by appropriately designed skills courses, assessment tools and faculty development.

Surgical skills' courses

Surgery always has been, and always will be, a craft speciality. The acquisition of psychomotor skills is, therefore, essential, but this must be done on the background foundation of adequate knowledge of other skills such as communication and clinical judgement. Lord Moynihan could point out ‘in surgery, the hand of the beginner is heavy!’ and yet mere experience may not simply be the answer. Charles Mayo reminded us that ‘experience can mean making the same mistake over and over again’. Therefore, the teaching of psychomotor skills is crucial; if one looks at other professions in which such skills are paramount, we find that talent often manifests itself at a young age.

The young Mozart was picking out chords at the age of 3 years and used to play regularly with his father Leopold, a talented violinist. By the time he was aged five, he was composing pieces, writing his first opera and symphony by the age of 12 years so that by the time he died under questionable circumstances at the age of 35 years he was one of the world's most prolific and talented composers. In more recent years, Yehudi Menuin started playing the violin at the age of three and by the time he was 10 years was an international concert performer. In more energetic pursuits such as gymnastics, the young Olga Korbut, who burst onto the Olympic arena in 1972 and walked away with all the gold medals, started her gymnastics at the age of four. Although these illustrations are not easily translatable to the realm of surgery, there are lessons that can be learned; indeed, it appears that experience in video games at an early age can facilitate the performance of virtual reality laparoscopic exercises, and repeated practice improves performance.⁵ It is, therefore, important for the profession to harness talent as early as possible and then to develop that talent as actively as possible.

In the UK, the Basic Surgical Skills (BSS) course was first envisaged by the Senate of Surgery in 1994 and became a mandatory course for all basic surgical trainees in 1996.^6,7 Its aim was to ‘teach, test and certify’ with regard to basic surgical techniques, safety and universal precautions. In this era when virally transmitted illness is prevalent, safety in surgery, not only for the patient but also for the surgeon and the surgical team, is paramount. The BSS course is now 10 years old and each year about 1500 UK surgical trainees have participated in the course, whilst internationally it is it is being run in Australia, South Africa, the Far East and many European and Scandinavian countries. Furthermore, it has spawned the Introduction to Surgical Skills (ISS) course,⁸ a teaching resource sponsored by the Commonwealth of Learning, which is now used in over 40 non-industrialised countries.

Such surgical skills courses are developing all the time and the arrival of more realistic virtual reality simulations is making such educational activities more realistic, more demanding and more profitable. The aerospace industry has long depended on flight simulation for safety in the air; as surgical simulations become more and more credible, they will provide a similar stimulus for safety in surgery. Models like SIM-man are becoming more relevant in surgical practice as well. As technology advances, increasing numbers of surgical procedures and situations will be simulated using virtual reality tools.

Mentoring

In the past, all too often the teaching of operative surgery followed the age-old aphorism of ‘see one, do one, teach one’. This is no longer acceptable, not only on the grounds of clinical governance and patient safety, but also on the basis of professionalism. A great deal can be learned by observing masters of their craft, watching videos, participating in live links between the operating room and conference centres, and assisting. However, a trainee that spends most of their time simply assisting or performing an operation unsupervised is not in an adequate training environment.⁹ The teaching of operative surgery requires the input of an experienced surgeon at the operating table, guiding, teaching and instructing the novice as they perform a given operative procedure. Many such procedures require staged operative exposure. This is particularly exemplified by a complex operation such as a Whipple's pancreatico-duodenectomy. It is not to be expected that the trainee will assist one day and then the next day perform the operation in its entirety under supervision. However, the operation lends itself to staged learning and the trainee surgeon will usually learn to perform each of these stages under supervision and once competent with them, will start to string these together until they are eventually able to undertake the full procedure.

For most routine operations, such an educational exercise will take place during the training years. However, in the future, reality dictates that even though an individual may have gained their Certificate of Completion of Training (CCT) and have developed the required competencies for their speciality, there will still be more advanced procedures that may need to be learned or perfected following the award of a CCT and this may require active mentoring even at consultant level. This for complex procedures is a most profitable activity and is to the advantage of both the patient and the surgical team. Similar models are seen in the aerospace industry where often a pilot and co-pilot will work together with the co-pilot flying the aircraft under the watchful eye of a more senior pilot. Although the comparisons between the aerospace industry and surgery have often been drawn,¹⁰ it is important not to take the comparison too far or too literally, but the disciplined way in which aircraft pilots are not only trained, but also have their skills maintained are a lesson to the surgical profession.

Assessment

All too often in the past, the medical profession has been wary of the term assessment, which has tended to raise the hackles of paranoia. However, the assessment of surgical competency¹¹ in the light of reduced training time is becoming even more important and should be a professional priority as well as being based on patient safety and political imperatives. If operative competency could be accurately measured, it may well be delineated by a normal distribution bell curve (Fig. 2). On the far left would be the mavericks and on the far right would be the surgical geniuses. What, therefore, should be the aim of assessment of surgical competency? No surgeon would like to be thought of as being below average, but by very definition, 50% always will be. The aim, therefore, of assessing surgical competency should not be to cut out those below average, because this would be unrealistic and immediately you would create a new population who were below average. However, the aim of good, sound, surgical teaching and competency assessment should be to eradicate the mavericks, to emulate the geniuses and thus move the whole bell curve to the right by the spreading of good practice (Fig. 3). Therefore, the profession should not fight shy from the concept of competency assessment, but should see it as a tool for improving surgical standards and providing evidence of good practice upon which annual appraisals and, ultimately, revalidation will be based.

Figure 2.

Potential normal distribution curve for surgical competence.

Figure 3.

The objective of assessing surgical competence – moving the competence distribution curve to the right, eliminating mavericks and emulating geniuses.

Several studies have been undertaken with regard to competency assessment;^12–14 although it is fully recognised that currently the tools are neither sophisticated nor accurate enough, they still provide a clear indication that maverick clinicians can be identified from a population of volunteer consultant surgeons. Similarly, simple Objective Structured Assessment of Technical Skills (OSATS)¹⁵ can demonstrate in a population of trainees a clear correlation between the duration of training, the number of index procedures performed and the quality of the operative procedure as assessed by both checklist scoring and global assessment. New assessment tools such as Procedure Based Assessments (PBAs) and Direct Observation of Practical Skills (DOPS) are being introduced into training programmes for the assessment of practical skills.⁴ Only time and longitudinal studies will be able to determine whether such assessments will truly be able to predict future clinical performance.

Faculty development

The delivery of any surgical curriculum, skills' course or competency assessment programme will only be possible with the development of a dedicated and trained faculty. This is going to require an active programme of faculty development utilising a pool of dedicated and committed trainers. The question, therefore, arises as to whether all consultant surgeons will indeed be formal trainers. In one capacity, any consultant can be a trainer by means of providing an environment whereby trainees can attend their operating lists, clinics or ward rounds, and yet at another level it is likely that other responsibilities will be taken on by formally trained and dedicated trainers. Programmes such as Training the Trainer¹ have been invaluable in developing an individual's ability to teach and train, particularly with regard to the understanding of the principles of adult learning, critiquing and teaching practical skills. With the arrival of a new surgical curriculum, these skills are going to be even more in demand and will require the trainer to understand fully the principles of adult education, utilisation of assessment tools and how a web-based curriculum might function.

For all this to be possible, time must be set aside for training within the job plans of all trainers. This will require explicit job descriptions and clearly defined roles. There must be clear incentives for trainers as well as being ascribed the appropriate kudos for being a fully accredited trainer.

To address these issues, specific development programmes for faculty development are being developed in order to equip faculty members, thus allowing them to deliver a new surgical curriculum, teach surgical skills on courses and in the operating theatre, test competency and to play the role of a mentor.¹⁶ Many may feel such programmes are unnecessary, but experience has indicated that where an active Training the Trainer programme is adopted, this not only leads to a higher quality of education and training, but also a greater confidence on the part of the trainer to be able to deliver what is a highly demanding agenda.

Conclusions

Professor Martin Allgöwer has been a pioneer in this field in so many respects. He led the way in multiple educational initiatives and has provided the stimulus for so many to follow in his footsteps. The political climate in which surgery is now being practiced, and the entirely appropriate demands for even safer surgery, both indicate that the profession can no longer afford to be reactive with regard to surgical education, but to respond pro-actively in developing these initiatives. Professional self-regulation is under threat as is indeed professionalism itself.¹⁷ The surgical fraternity should unite both nationally and internationally in delivering a highly demanding agenda upon which the future of our profession will be dependent.