The emerging role of screen based simulators in the training and assessment of colonoscopists

Abstract

Incorporation of screen based simulators into medical training has recently gained momentum, as advances in technology have coincided with a government led drive to increase the use of medical simulation training to improve patient safety with progressive reductions in working hours available for junior doctors to train. High fidelity screen based simulators hold great appeal for endoscopy training. Potentially, their incorporation into endoscopy training curricula could enhance speed of acquisition of skills and improve patient comfort and safety during the initial phase of learning. They could also be used to demonstrate competence as part of the future relicensing and revalidation of trained endoscopists. Two screen based simulators are widely available for lower gastrointestinal endoscopy training, with a third recently produced in prototype. The utility of these simulators in lower gastrointestinal endoscopy training has been investigated, and construct and expert validity has been shown. Novices demonstrate a learning curve with simulator training that appears to represent real learning of colonoscopy skills. This learning transfers well to the real patient environment, with improvements in performance and patient discomfort scores in subsequent initial live colonoscopy. The significant limitations of currently available screen based simulators include cost implications, and restrictions on a role in certification and revalidation. Many questions remain to be answered by future research, including how best to incorporate screen based simulators into a colonoscopy training programme, their role in training in therapeutic endoscopy and the impact of simulator training on patient safety.

Introduction

Thanks to advances in graphics and interactive technology, medical simulation has evolved from simple mechanical and animal models to haptic feedback based simulators, allowing operators to ‘feel’ resistance from tissue. Recently, in his annual report,¹ the Chief Medical Officer for England and Wales emphasised the need to improve patient safety by increasing use of simulator training in medical practice, the aviation industry being cited as an example of successful incorporation of advanced simulation technology into training.^{1 2}

Over the past decade, demand has increased for trained colonoscopists, particularly following the introduction of the national colon cancer screening programme. Currently, proficiency in colonoscopy is acquired by performing at least 200 supervised procedures, as recommended by the Joint Advisory Group on Gastrointestinal Endoscopy, followed by achievement of performance standards assessed by two trained colonoscopists.³ Given that time available for trainees has been limited by the introduction of both Modernising Medical Careers and the European Working Time Directive, the incorporation of screen based colonoscopy simulators into training programmes is appealing, as this may accelerate the learning curve while limiting patient risk.

The purpose of this review is to illustrate the simulators currently available in the UK and to evaluate the current evidence regarding their utility in lower gastrointestinal endoscopy training, specifically: validation, demonstration of a learning curve and transfer of skills to live colonoscopy.

Screen based simulators in lower gastrointestinal endoscopy

At present, two screen based simulators are widely available for endoscopy training. The AccuTouch (Immersion Medical, Gaithersburg, Maryland, USA) (figure 1) offers modules in upper and lower gastrointestinal endoscopy, including didactic teaching on endoscope handling, endoscopic, embryologic and gross anatomy of the gastrointestinal tract, as well as a pathology atlas. The clinical cases, derived from real patient anatomy, progress in difficulty.

Figure 1.

The AccuTouch simulator

The GI Mentor II (Simbionix, Ohio, USA) (figure 2) offers upper and lower gastrointestinal endoscopy and endoscopic ultrasound, with modules in colonoscopy and flexible sigmoidoscopy currently available, each containing 10 virtual cases, based on real anatomical data and progressing in difficulty (figure 3). It also features didactic teaching, and the supervisor can create custom made tutorials for trainees. In addition, non-clinical exercises designed to improve hand–eye coordination and endoscope handling (Endobubble and Endobasket) (figure 4) are available.

Figure 2.

The GI Mentor II simulator

Figure 3.

Snare polypectomy

Figure 4.

The Endobubble (left) and Endobasket (right) hand-eye coordination tasks

The Olympus Endo TS-1 Simulator (Olympus KeyMed, Southend, UK) is still under development and not yet commercially available. It has been designed for colonoscopy navigational skills training rather than recognition of mucosal abnormalities, offering a hand–eye coordination task (PolypExorcise) and allowing early correction of loop formation by incorporating a simulated three-dimensional endoscope imager view identical to that provided by ScopeGuide (Olympus Optical Company, Tokyo, Japan).

The realism and fidelity of these screen based simulators is enhanced by haptic feedback, the importance of which in virtual reality training environments being demonstrated in minimally invasive surgery, particularly for those in the early stages of training.⁴ In addition, feedback from the virtual patient, including pain and heart rate, is provided.

The operator can administer sedation, change the patient's position and apply abdominal pressure as desired. The suction and insufflation buttons will decompress and inflate the lumen, respectively. Complications, such as uncontrolled bleeding after polypectomy, perforation and vasovagal reactions, are also simulated. As the simulators are designed for use without senior supervision, they feature a ‘virtual attending physician’, who can advise on how to proceed if requested. Following each procedure, detailed feedback is available to the trainee in the form of performance metrics, such as time taken, scope length inserted, percentage of the mucosa visualised and identification of any abnormalities, time spent with no mucosal view, success in any therapeutic procedures and ability to complete the procedure.

Evaluation of screen based simulators for training in lower gastrointestinal endoscopy

Validation

To establish whether individuals training on screen based simulators demonstrate a learning curve, the metrics measured by the simulator must be able to distinguish between operators of different experience (construct validity). Furthermore, for simulator training to have a role in certification and revalidation of trained endoscopists, the simulated cases must closely resemble real patient colonoscopy (expert or face validity).

Several studies have looked at construct validity of the Immersion simulators (AccuTouch and its predecessor, the PreOp).^5–8 Participants were categorised by prior colonoscopy exposure into expert, intermediate and novice groups. Skills were evaluated using moderately challenging simulated cases, being complex enough for the expert but not too demanding for the novices. Although all novice groups had no prior endoscopy exposure, the arbitrary criteria chosen to define ‘expert’ endoscopists ranged from >101 to >750 colonoscopies, while those used for ‘intermediate’ experience varied from <15 to an average of 151 procedures.^5–8 Therefore, some intermediate endoscopists may have been more experienced than experts from a different study, limiting comparison of the results, particularly regarding the performance of those with intermediate experience.

Overall, the performance metrics showing the greatest construct validity are procedure time and percentage of mucosa visualised, particularly between the expert and novice groups. Differences between those with intermediate experience and the other two groups were less evident. The experts rated the AccuTouch substantially less difficult than real colonoscopy (mean score 2 on a 5 point Likert scale).⁸

Construct validity investigations of the Simbionix simulators show a similar variability in categorisation of participants.^9–12 Overall, experts performed significantly better on the EndoBubble and EndoBasket tasks than novices.^{10 11} When performance on simulated colonoscopy cases was assessed, the expert groups were again significantly better than novices, procedure time and mucosal visualisation once more appearing the most reliable parameters. Operators of intermediate experience were not reliably distinguished from the expert or novice groups. Experts felt the simulator would be useful for training but not for certification purposes.¹²

Two investigations have evaluated the construct and expert validity of the Olympus Endo TS-1.^{13 14} In the first one, studying just six outcome measures, it was found that only the ‘time to caecum’ metric discriminated between experts and novices.¹³ In the second, following the development of 40 performance metrics, the authors identified that 22 of them distinguished novices from experts and that far fewer parameters discriminated between intermediate and expert operators, mainly those related to time needed to reach various landmarks and loop formation.¹⁴ The expert groups in both studies rated the simulator less difficult than real colonoscopy.^{13 14}

Demonstration of a learning curve

Using the recorded performance metrics, a learning curve has been demonstrated on the simulators for novice endoscopists. A significant improvement in novices' performance was seen over time on both the EndoBubble exercise and simulated colonoscopy cases on the GI Mentor II. Experts, however, showed limited improvement on the same exercises.¹¹ After a period of training on the simulator, novices matched the performance metrics achieved by the experts when tested on a previously unseen colonoscopy case.¹⁰ While this again emphasises the lack of fine discriminatory power of the simulator between users of different experience, it suggests real learning occurred, rather than ‘learning the simulator’. Mahmood and Darzi reinforced this point, using the Immersion simulator.¹⁵ Novices were assessed over five repetitions of a colonoscopy case in the absence of any feedback; as a result, no performance improvement occurred, implying that feedback is required for learning.

Overall, the evidence to date suggests that the available screen based colonoscopy simulators can usefully distinguish between novice and fully trained operators. Novice endoscopists demonstrate a learning curve over a period of training. However, the ability to discriminate between intermediate levels of experience appears unreliable, and the simulators do not fully replicate the difficulty of real patient colonoscopy. The performance metrics measured by the simulators do not correlate well with an objective assessment of the endoscopist's performance made by a trained observer.¹⁶ For these reasons, their role in training is likely to be confined to the early phase of learning, and they are not yet perceived to support certification or revalidation of trained colonoscopists.

Effective transfer of skills to real patient colonoscopy

An early study on the PreOp simulator found that simulator trained novices performed worse on their first five flexible sigmoidoscopies with regard to patient comfort and completion rates than those receiving bedside teaching alone.¹⁷ However, the simulator based training (SBT) group was given free access to the simulator, without a structured training programme or faculty supervision, completing an average of 138 min. By contrast, the patient based training (PBT) group completed an average of 5 h of supervised bedside training. The different outcomes may reflect the training length, training structure or the presence of supervisor's feedback.

Several investigations have described effective transfer of skills learnt on the simulator to subsequent live colonoscopy.^18–22 Longer training periods on both the Immersion and Simbionix simulators have shown reduced patient discomfort in the SBT groups.^18–20 A prospective randomised trial conducted by Sedlack and colleagues¹⁸ demonstrated that SBT trainees compared with the PBT group showed superior performance over the first 15 live colonoscopies, following 6 h of training on the Immersion simulator. Some performance benefits persisted up to 30 procedures, but thereafter there were no differences from the PBT group, who had no prior training. However, assessors were not blinded to the training received and the number of participants was small. Findings were confirmed by a larger blinded trial with a longer training period on the Simbionix simulator showing this benefit may extend up to the first 80 colonoscopies although no difference in time to reach competency in colonoscopy was demonstrated.²¹

Ahlberg et al evaluated the effectiveness of goal directed training by deriving target criteria from performance metrics achieved by experienced colonoscopists on the AccuTouch simulator. Novices randomised to SBT were given structured training under supervision until criteria were reached. Better overall performance in live colonoscopy and lower patient discomfort scores were seen in the SBT group, who were considerably more likely to reach the caecum than the PBT group, who received no additional training.²⁰

Early data on the Olympus Endo TS-1 are also encouraging. Here, the SBT novice spent 16 h on a structured, minimally supervised training programme while the PBT group spent an equal time in bedside teaching. The two groups obtained equivalent performance results when assessed on three subsequent live colonoscopies, indicating effective transfer of early colonoscopy skills from the simulator to the real patient environment.²²

Discussion, limitations and future research

It is encouraging to see that real learning of colonoscopy skills occurs on screen based simulators, translating to improved performance in initial live colonoscopy and a better patient experience. A significant limitation of screen based simulators however is their cost, in both initial purchase and subsequent maintenance. A number of questions need to be addressed on how to incorporate screen based simulators into a training curriculum in the most effective manner, to fully justify this expenditure.

The optimum time length for a trainee to practice on the simulator before progressing to live colonoscopy has not been defined. Increasing the length of simulator training seems to lead to a longer lasting performance benefit in live colonoscopy, at least up to 10 h of training.^{19 21} An alternative approach,²² which has produced impressive outcomes, is to set goals which trainees must achieve on the simulator before progressing to live colonoscopy. Expert benchmarks have been described for Olympus Endo TS-1¹⁴ and the GI Mentor II,²³ although the impact of incorporating these into training has not yet been investigated.

Goal orientation, structured training, feedback and reflection are all known to be important components of adult learning.²⁴ The importance of feedback for learning on the simulator has been previously discussed.¹⁶ The user friendly nature of the simulators may tempt trainees to use them at will during any available free time. Furthermore, supervisors with a considerable workload may dispatch trainees to practice on simulators independently, overall resulting in a rather haphazard approach to training. It is possible that lack of structured training may have contributed to the poorer performance of the SBT group in subsequent live flexible sigmoidoscopy, in the only transfer of skills study where the SBT group was outperformed by the PBT group.¹⁷ It will be important to see if a goal directed approach, incorporating a structured training programme with supervision and feedback from a mentor, results in further improvements in performance or even accelerates acquisition of independent competence in colonoscopy.

Patient satisfaction with endoscopic procedures performed by trainees, particularly those in early training, is lower and an increased frequency of minor adverse events has been noted.²⁵ Simulator training appears to reduce patient discomfort in the early stages of live colonoscopy.^18–20 While it is tempting to speculate that this may also translate to fewer adverse events, this has not yet been demonstrated. It is possible that attaining a level of competence on a simulator may instil a degree of false confidence in trainees, particularly as the simulators are widely felt to be easier that real colonoscopy. As patient safety is a major force driving simulator incorporation into training, this represents an important area for future study.

Research to date has focused on learning and transfer of skills in diagnostic lower gastrointestinal endoscopy. The simulators also feature therapeutic modules, but the impact of training on these modules on performance in real life therapeutic endoscopy has not been investigated. As colonoscopy complication rates are higher when therapeutic procedures are undertaken,²⁶ it would be useful to see if simulator training can reduce the rate of adverse events associated with therapeutic colonoscopy.

Finally, although screen based simulators offer clear benefits to those at the earliest stages of endoscopy training, their role for intermediate levels of experience is currently limited. With further refinements in simulator technology this is likely to change. Furthermore, as realism and replication of the difficulty of live colonoscopy improves, it may be possible to utilise these simulators to develop objective parameters to define competence even in live colonoscopy. In fact, a more objective means of assessing proficiency would be desirable, particularly in the coming era of relicensing and revalidation of the endoscopists of the future.

Conclusions

Training on screen based simulators results in an acceleration of the initial stages of the colonoscopy learning curve with reduced patient discomfort in subsequent live procedures. A number of questions remain as to how best to incorporate simulators into a training programme. A structured and supervised programme of training on the simulator is likely to be required for learning to occur. Results may be better with a goal directed approach, with standards that must be achieved before the trainee can progress but this remains to be clearly proven. The amount of supervision and feedback required from senior staff for learning to occur also needs to be clarified. Once an ideal training curriculum is defined, further studies will be required to evaluate its effect on patient safety, as well as its impact on the learning curve in live colonoscopy and any acceleration of acquisition of competency in the procedure. Regarding therapeutic colonoscopy, the potential impact of simulator training remains to be elucidated. The role of simulators in certification of trained endoscopists or revalidation requires further refinement of the technology but remains a possibility for the future.