The Box Simulator Is Useful for Training Novice Endoscopists in Basic Endoscopic Techniques

Abstract

Purpose

The present study was aimed at evaluating the usefulness of box simulators for training novice endoscopists.

Materials and Methods

An explanation of the goals, contents, and features of the simulator was given to study participants. The participants then received "hands-on training" in gastrointestinal endoscopy techniques using a box simulator. Subsequently, they were asked to answer 19 structured questions about the simulator. Ratings were scored on a scale from 1 to 5 for questions concerning their first impression of the simulator. Questions on the usefulness of the simulator and the training course were answered as "agree", "disagree", or "no opinion".

Results

A total of 32 participants filled out the questionnaire. The mean scores on the simulator's usefulness, features, and realistic movements before the training were between 1.5 and 2.0. There were no significant differences between the mean values of the scores given by novice users compared to non-novice users. However, after receiving training on the simulator, 90.6% of the participants considered the box simulator a generally useful tool for learning basic endoscopic techniques, and 90.6% agreed that the simulator was useful for improving hand-eye coordination.

Conclusion

Box simulators may be useful for training novice endoscopists in basic gastrointestinal endoscopic techniques.

INTRODUCTION

There is a consensus that gastrointestinal endoscopic training should be structured to ensure safe and high-quality endoscopic performance. Proficiency in endoscopic techniques has previously been obtained via supervised, hands-on training on real patients.^1-3 However, training on real patients may reduce overall patient safety.

Simulator-based training may provide a safe and controlled environment for learning basic endoscopic skills without risk to patients.^4,5 Use of a simulator may be considered for developing knowledge and basic skills as an adjunct to more traditional training on real patients. However, before such an approach can be widely recommended, evaluation and validation of the simulators and their parameters are necessary.

Approaches for validation may include content and face validity. In general, content validity may be defined as "an estimate of the validity of a testing device based on a description of the contents of the test item" or judgment on what territories the device trains.^6,7 Therefore, content validation may be a summation of the function of the device rather than an actual study. Face validity may be referred to whether the simulator resembles the task it is based on and whether it is useful tor training.^6,7 To our knowledge, the usefulness of box simulators for the training of novice endoscopists have not yet been reported. The aim of this study was to establish the usefulness of box simulators for training novice endoscopists in basic endoscopic techniques.

MATERIALS AND METHODS

Participants

This was a prospective, single-center study at an endoscopy training center in Korea. A total of 32 participants from primary clinics and a training hospital were introduced to an esophagogastroduodenoscopy (EGD) box simulator on June 26-27, 2010. A "novice endoscopist" was defined as an endoscopist who had performed less than 10 EGDs, and a "non-novice endoscopist" was defined as an endoscopist who had performed more than 100 EGDs. Participants were excluded if they had previously attended an intensive endoscopy training course, or were previously participants in an endoscopy training or simulator training study. The Institutional Review Board of our hospital reviewed and approved our study.

The hardware system

The simulator (Model LM022, Koken Co., Ltd., Toshima-ku, Tokyo, Japan) comprised one opening for insertion on a box weighing 6.9 kg and measuring 51.8×28.8×18.8 cm (Fig. 1). This model was developed for the purpose of training endoscopists to use commercially available EGD scopes. All EGD examinations in this study were performed using a standard video endoscope (EG-590WR scope, Fujinon Inc., Saitama, Japan) used in most clinical fields.

Fig. 1.

The EGD simulator (Model LM022, Koken Co., Ltd., Toshima-ku, Tokyo, Japan). It consists of one opening for insertion on a box weighing 6.9 kg and measuring 51.8×28.8×18.8 cm. EGD, esophagogastroduodenoscopy.

Content validation

The goal of the program was to train novice endoscopists in the basic skills needed to perform EGDs. Participants underwent a 30-minute, standardized simulator training program, which began with a short explanation of the goals of the training course and a theoretical explanation of the basic skills and principles of EGD. During the explanation, subjects were asked to manipulate the simulator. The training program was designed to train hand-eye coordination and basic endoscopic skills, specifically insertion technique, in a sequential, hands-on format with feedback. Participants were provided an active tutoring and feedback from two specialized endoscopists (JM Cha and JJ Park) who each had performed more than 5000 EGDs. Subsequently, the participants were asked to give their opinion of the training system by completing a questionnaire.

Face validation: questionnaire

A questionnaire was completed by all participants. All participants were asked to submit their age, gender, specialty and previous EGD experience. The questionnaire was divided into two sections. The first section contained 6 questions concerning the participant's first impression of the simulator and the second section comprised 13 questions on the usefulness of the simulator and the training course (Table 1). Two questions of the second section were posed on the willingness to train with the box simulator. The questions were adapted and modified from a questionnaire previously used in a study on the validation of a laparoscopic surgery simulator.⁶ Questions in the first section were answered on a scale of 1 (very bad/useless) to 5 (excellent/very useful). Statements in the second section were to be answered with "agree", "disagree", or "no opinion."

Table 1.

The Questionnaire Regarding First Impression and Usefulness of the Simulator and Training Course

EGD, esophagogastroduodenoscopy.

Statistical analysis

Differences between the calculated mean scores of the novice and non-novice groups were analyzed by the Kolmongorov-Smirnov test (two-sided) for the 5-point ordinal scale. Fisher's exact test (two-sided) was used to compare differences between the groups on the responses of "agree" versus "disagree". For ordinal data, a 2-sample t-test was used to compare variables that were normally distributed, and the Mann-Whitney tests were used for variables that were not normally distributed. A p value of <0.05 was considered significant. Data were analyzed using SPSS software, version 13.0 (SPSS Inc., Chicago, IL, USA).

RESULTS

In total, 32 physicians participated in this study. Table 2 shows the characteristics of the participants including age, sex, previous EGD experience, and area of specialization. There were 29 men and 3 women with a median age of 46.5 years (SD 9.0). In total, 14 participants (43%) were classified as novice endoscopists. Twenty nine participants (90.6%) were medical specialists, and only 3 were general practitioners. The majority of the participants (81.2%) worked in internal or family medicine.

Table 2.

Demographic Data and Characteristics of the Participants

Table 3 shows the mean values of the scores for first impression and training capacities. Most responses tended to be "bad (2 point)", and the highest mean score of 2.0 was given for user friendliness of the box simulator. Opinions on the training of basic skills and hand-eye coordination received relatively low scores (1.5 point). No significant differences were found between the novice and non-novice users.

Table 3.

Participants' Responses to the First Six Questions of the Questionnaire

1=very bad/useless and 5=excellent/very useful.

Table 4 displays the results of the statements after simulator training. Ninety one percent of the participants considered the box simulator useful for the training of endoscopic techniques to novice endoscopists in general, and 90.6% agreed that the simulator was useful for training hand-eye coordination. Seventy-five percent of all of the participants believed that it was useful for training at home, and 90.6% believed that the simulator was useful for training within the hospital. Most novice endoscopists (92.9%) agreed that the simulator could be useful for measuring endoscopic procedure performance, but only 55.6% of the non-novice endoscopists agreed with this statement (p=0.018). Seventy-five percent of participants wanted to train with the box simulator, and 81.3% wanted to attend an endoscopy training course using the box simulator in the future. All participants (100%) agreed that the feedback from the trainer during the training course was very useful.

Table 4.

Participants' Responses to Questions Regarding Usefulness of the Simulator and Training Course

The percentage of "no opinion" is the gap between 100% and the sum of the percentages of "agree" and "disagree".

DISCUSSION

Structured training of endoscopic techniques is an important issue in endoscopic education, and simulator training may be considered a valuable training method for endoscopic education. The results of this study show that both novice endoscopists and non-novice endoscopists believe that box simulators are useful tools both for training basic endoscopic skills focused on insertion, and for learning hand-eye coordination. Although neither the novice endoscopists nor non-novice endoscopists gave the box simulator high scores before the training course, after the course, most participants agreed that the box simulator was useful. Interestingly, this positive effect was seen after a training course of only 30 minutes.

For gastrointestinal endoscopy, simulators may be utilized in a structured and gradual fashion before novice endoscopists take part in endoscopic evaluations of real patients. Previous studies have shown positive effects of virtual reality simulators on gastrointestinal endoscopic education^6,8,9 and laparoscopic surgical education.^10-12 Park, et al.¹³ showed that residents who practiced on a virtual visual colonoscopy simulator prior to their first patient encounter performed better in clinical settings than controls via a prospective, randomized, and blinded study. In our study, the box simulator also contributed to improvement in basic gastrointestinal endoscopy performance. It is the first study, to our knowledge, describing the usefulness of box simulators in endoscopy training.

Two studies of laparoscopic surgical training have indicated that simulator training was successful when the training schedule was intermittent rather than condensed into a short term of extensive training.^10,11 Such a training schedule may be easily implemented when the simulator is easily accessible and mobile (e.g., located in every teaching hospital or even at home). In this study, however, only 75% of participants thought that training at home would be useful. The reason for this is unclear, though it may be explained by the important role played by the trainers themselves. Indeed, 100% of the participants indicated that the trainer was very helpful in the training of basic endoscopic skills. Trainer feedback was not the focus of this study, however, and further studies are needed to understand its role and implications.

Participants were also asked whether the simulator could be useful for measuring endoscopic procedure performance. In contrast to most novice endoscopists, the non-novice endoscopists tended to respond negatively (p=0.018). This may be explained by the fact that non-novice endoscopists may not be familiar with the idea of measuring endoscopic performance using simulators, in general, or they may dislike the idea of accepting metrics for assessment of their performance. Although more studies are required to confirm our findings, the objective finding that simulator-based training improves novice performance in clinical settings may provide empirical justification for the use of simulators as training tools.

Both the box simulator and the virtual reality simulator may accelerate the acquisition of basic endoscopic skills, including hand-eye coordination. Box simulators and virtual reality simulators are similar in that they offer a good representation of real endoscopy, demonstrate good construct validity, and discriminate for time-to-landmark measurements and completion rate for novice endoscopists. Virtual reality simulators may have advantages in that they can assess the scoring of each task for each novice endoscopists, provide various case scenarios, and offer computerized feedback. Although the computerized system provides realistic simulation, it may be different from a real clinical environment using real endoscopic equipment. The box simulators have advantages in that they can incorporate customized real endoscopic equipment used in clinical settings and they are relatively inexpensive. The cost of the box simulator used in this study was approximately 60.0-86.0% lower than that of a virtual reality simulator. The high cost of most virtual reality simulators may limit their usefulness,^6,8,9 though whether training on box simulators is a cost-effective means of instruction deserves additional study. In contrast to virtual reality simulators, box simulators need a technical trainer to correct errors for endoscopic education, as they do not offer computerized feedback. In addition, box simulator training is unable test diagnostic abilities and various case scenarios.

One of the possible advantages of simulator training in gastrointestinal endoscopy compared to training on real patients is that it allows for unlimited practice with trainer feedback. The disadvantages of simulator-based training include the lack of a real clinical environment. The simulator does not train the endoscopist in non-technical skills such as patient communication and does not prepare them for the inherent unpredictability of real endoscopic procedures. More specifically, simulator-trained novices might be able to accomplish the simulator scenarios by using techniques that work on the simulator only but not on real patients. This inherent problem of simulators may be one of the reasons that simulation is unlikely to completely replace bedside training for endoscopic techniques. Other disadvantages of simulator-based training also include lack of training on how to recognize and handle complications and lack of training on therapeutic endoscopy. Further studies are needed to demonstrate the degree to which simulators can substitute for standard bedside training.

Our findings must be interpreted in the context of the study's limitations. First, our definition of novice and non-novice endoscopist as participants with "less than 10 EGDs" and "more than 100 EGDs" may be arbitrary. In studies of the learning curve of EGD, more than 90% of trainees successfully perform technically complete EGD after 100 cases.¹⁴ Therefore, endoscopists who performed more than 100 EGDs were classified as non-novice endoscopists in our study. Second, this study focused mainly on only one part of endoscopy, e.g., insertion technique, and other important endoscopic skills such as lesion recognition and complete visualization of the mucosa were not included in our training program. Third, our findings may be limited by the fact that the results are of a single-center study and based on a small number of participants. Despite this limitation, our study appears to have considerable potential for the further development of additional validation studies of gastrointestinal endoscopy training as there was, previously, no such study. Fourth, face validity has some weaknesses because it is based on the opinions of participants, even though the questionnaire was adapted from a previous study.^6,10 Furthermore, the questionnaire may still lead to systematic errors, including different interpretations of questions across subjects or answers biased by the enthusiasm of the presenter or the attractiveness of the training system. This bias may not be a strong factor in our study, however, because most participants did not give the box simulator high scores on their first impression. Finally, a control group without use of the box simulator was not enrolled and compared with the box simulator group in this study. As novices in the simulator group demonstrated superior technical skills in the randomized, blinded, controlled design study of virtual reality simulators for colonoscopy training,⁸ additional comparative studies with or without box simulators may be warranted.

Despite these shortcomings, our study is the first to describe the usefulness of box simulators in EGD training for novice endoscopists. Box simulator training may lead to better early clinical performance of novice endoscopists, as they were useful tools for both training basic endoscopic skills and learning hand-eye coordination. Therefore, our study appears to have considerable potential for the development of additional prospective studies with a larger number of participants.