Abstract
The aim of this study is to develop and assess the peg transfer training module face, content and construct validation use of the box, virtual reality (VR), cognitive virtual reality (CVR), augmented reality (AR), and mixed reality (MR) trainer, thereby to compare advantages and disadvantages of these simulators. Training system (VatsSim-XR) design includes customized haptic-enabled thoracoscopic instruments, virtual reality helmet set, endoscope kit with navigation, and the patient-specific corresponding training environment. A cohort of 32 trainees comprising 24 novices and 8 experts underwent the real and virtual simulators that were conducted in the department of thoracic surgery of Yunnan First People's Hospital. Both subjective and objective evaluations have been developed to explore the visual and haptic potential promotions in peg transfer education. Experiments and evaluation results conducted by both professional and novice thoracic surgeons show that the surgery skills from experts are better than novices overall, AR trainer is able to provide a more balanced training environments on visuohaptic fidelity and accuracy, box trainer and MR trainer demonstrated the best realism 3D perception and surgical immersive performance, respectively, and CVR trainer shows a better clinic effect that the traditional VR trainer. Combining these in a systematic approach, tuned with specific fidelity requirements, medical simulation systems would be able to provide a more immersive and effective training environment.
1. Introduction
Video-assisted thoracoscopic surgery (VATS), which is the most common minimal invasive surgery (MIS) therapy for lung carcinomas [1–3], is the most widespread cancer in the world with only approximately 16% five-year survival rate. Furthermore, hospitalization outcomes show the patients' quality of life (QoL) and follow-up adjuvant chemotherapy endurance that are significantly promoted compared with traditional thoracotomy, without interfering with survival outcomes [4, 5]. Peg transfer training, as one of the essential modules of the fundamental thoracoscopic surgery curriculum, is the compulsory test requirement before the surgeries take the American Board of Surgery examination [6, 7]. With two Maryland clamps, trainees need to pick up six tiny blocks with nondominate hand, transfer to the other hand, and place them stably on the other side peg, respectively, whereby to achieve the bimanual dexterity and eye-hand coordinative skills training [7, 8]. The surgical simulators can be divided into box trainer, VR trainer, AR trainer, and MR trainer, and each class has advantages and disadvantages. The Box trainer is the traditional surgical training framework based on a real physical model that is portable and easy to operate; however, the disadvantage is this kind of trainer cannot be reused for multitimes. VR-based surgical simulations have attracted many researchers' attentions over the years and gradually turned into a real-life medical training simulation solution, providing repeatable training experience, without ethical or hygienic issues [9–11], and the disadvantage of this simulator is lack of high immersive visual and haptic rendering algorithms. Due to the recent advances in the field of AR and MR, cognitive sense has brought into the next level of surgical simulator with enhanced immersion and interactivity [12–14]; however, there are also several defects of these simulators, such as the high price, visual uncomfortable, and lack of the real surgical environment fidelity. Maciel et al. developed a virtual reality laparoscopic skill trainer named VBLaST, with real-time evaluation function for the peg transfer [15], and the Lap Mentor™ simulator [16, 17]. Loukas et al. compared the AR-based peg transfer with the box simulator and VR simulator [6], and Huber et al. added a highly immersive 360° real operating room environment to construct an MR training environments [18, 19]. Nevertheless, there is no comparative study that has focused box, AR, VR, CVR, and MR simulators on the peg transfer training, in another word, which one is the most effective simulator to shorten the surgical learning curve [20, 21]. The aim of this research is to determine which simulator is better by the advantages and disadvantages of the five simulators that were compared through the evaluation results of the face and content and construct. There are three main innovative contributions in this study, and the overarching one is we compared the experimental data and simulation results among five different kinds of trainers of peg transfer training and summarized the conclusion which may benefit for the advanced research of the virtual surgery. The second one is we addressed a full immersive and accurate scenario to peg transfer training with a detailed assessment tool, and the last one is we addressed the detailed design of the first commercial available VATS VR simulator in both hardware and the training implementation.
The structure and content of this article are organized as follows: we briefly make an introduction on peg transfer training and challenges of virtual medical training in the abovementioned part, after that, we review the previous related works on the virtual peg transfer simulator. Thirdly, the simulator design, with virtual training evaluation experiments, is designed in the methodology part. Fourthly, the experimental data and evaluation results are demonstrated and discussed. Finally, we summarize the results and the potential contributions this paper makes.
2. Materials and Methods
2.1. Simulator Design
2.1.1. Hardware
In the light of the development of commercial VR-based laparoscopic surgical simulators, LapSim® (Surgical Science, Sweden) and LAP Mentor (3D System, USA) [1, 15], and after observed considerable VATS lobectomy operations in the thoracic surgery department of Yunnan First People's Hospital, we chose the three-port standardized anterior VATS surgery as the simulated content. As the commonest surgical approach in lung carcinomas lobectomy, the three-port VATS are described as opened 3 single diameters of 1.5 cm incisions in the side of the patient's thorax: one is for endoscopic camera recording the operation field and the other two are the operative tunnels for surgical instruments stretching in and out to dissect and staple the lesion [2, 16]. The surgical simulator is called VatsSim-XR, as shown in Figure 1, based on the procedures need to be simulated in the abovementioned, and hardware of VatsSim-XR (VR, AR, and MR) is 60 × 67 × 160 cm and principally composed of a 24-inch naked eye 3D display, a footswitch pedal, two VATS surgical devices connected with dual haptic devices with one held as the drag instrument and the another mimics the stapling device, an endoscope connected with a 3D mouse (3Dconnexion, Germany), an HMD, and a workstation. To improve the surgical immersion, we proposed a solution to incorporate multiple higher-fidelity factors towards a surgeon's sensations (vision, touch, and hearing) during practical surgery to achieve total immersion. We developed a versatile simulation platform VatsSim-XR that is able to implement 5 different simulation modes, specifically, the AR and MR modes. To ensure the same training block between the real and virtual peg transfer platform, the .STL file is exported from the blender firstly, and then, the 3D printer is employed to print all the training pegs for the box training simulator. For the AR peg transfer simulation, we utilized a web camera Logitech CC2900ep HD1080p as the detected sensor for the display, and for the CVR training and the MR training, HTC VIVE (HTC Corporation, Taiwan, China) is the first virtual reality helmet, and it is used to show the surgical environments [22]. Especially, for the real surgical environments rendering, we employed the Samsung Gear 360 (Samsung, South Korea) to record the 360° video in the operation room (OR) of an upper-right lobe VATS in Yunnan First People's Hospital [18, 19, 23, 24].
Figure 1.
The immersive training platform for virtual peg transfer consists of (a) guide video, (b) a laptop with i7 6700 (3.4 GHz) CPU, 4 GB memory, and 1070 NVIDIA graphics GPU, (c) surgical clamp, (d) endoscope, (e) AR tracking target, (f) AR camera, (g) camera tripod, and (h) haptic devices (PHANTOM Omni).
2.1.2. Software
We designed a framework for the implementation of peg transfer training simulation, as shown in Figure 2. The virtual simulator mainly includes two parts: visual (physics and graphic) and haptic (PHANTOM Omni hardware and OpenHaptic software). The corresponding OpenHaptic function will be invoked when pressing the physical button of PHANTOM Omni (Geomagic, USA) or there is contact between the virtual surgical instrument and the model with a rigid body, and then, there will have a corresponding force feedback. There are physics with rigid or deform and graphics with a shader in the visual aspect. We combine the visual plugin, such as Bullet, AR kit and SteamVR, and the haptic plugin OpenHaptic into the Unity3D (version 2017.3.1) for simulation training.
Figure 2.
Visual and haptic rendering pipelines of the virtual peg transfer simulator.
2.2. Participants
The evaluation cohort of 32 trainees comprising 24 novices and 8 experts underwent the peg transfer procedures on the five simulators conducted in the department of thoracic surgery of Yunnan First People's Hospital. Both expert and novice trainers firstly receive a didactic teaching from a developer of these simulators with the tasks and techniques of peg transfer. The medical experience of the novice group is 3 to 6 years and the expert group is 11 to 30 years. To evaluate if the virtual game experience may affect the training result, the experience of using both VR and HMD had been recored during the experiments. The demographic details of the trainees are shown in Table 1.
Table 1.
Demographic data of the thoracic surgery trainees.
| Group A (novices) | Group B (experts) | |
|---|---|---|
| Number | 24 | 8 |
| Age (years) | 25.5 (24–29) | 42.2 (39–56) |
| Postgraduate year of training | 4 (3–8) | 12 (11–30) |
| Male (%) | 92 | 83 |
| Right-handed (%) | 100 | 87 |
| Box trainer experienced | <10 | >50 |
| VR game experienced | 5/24 | 1/8 |
| HMD experienced | 3/24 | 1/8 |
2.3. Simulator Tasks
Both novice and expert groups after didactic training session are towards to evaluate the simulator validity. The order of training is the box, VR, CVR, AR, and MR, and the 30-minute break should be interspersed between each peg transfer simulator test. Firstly, the trainee adjusted the virtual endoscope to reach the proper position and angle to obtain the best viewing perspective of the operating scene; after that, two clamps are used to grasp and transfer the block model on the peg base. The left-hand clamp grabs the block on the left side and passes it to the right clamp; then, the right clamp places the block on the right side, which is demonstrated in Figure 3.
Figure 3.
Peg transfer training platform. (a) The manipulative platform. (b) The operation interface. CVR training and MR training need the HMD device, and the AR training needs the camera module.
After the aforementioned experiments, all 32 trainees' performance is recorded by the objective questionnaires to evaluate the detailed surgical skills and be compared with each other. Subjective questionnaires also need to be filled to evaluate the face and content validity of five simulators, and the detailed evaluation flow chart is demonstrated in Figure 4.
Figure 4.
Evaluation system procedures design underwent our immersive virtual peg transfer simulator.
2.4. Evaluation
A questionnaire consisting of 13 questions about the visual and haptic aspects of the simulators was created for face and content assessment validity. Due to the lack of experience of the novice, subjective judgment may have a large error in the understanding of the simulator. Face and content validities were established by 8 experts that standardized subjective questionnaires parameters. This questionnaire utilizes 5-point Likert-type scales to evaluate the visual and haptic of the simulators (1 = poor to 5 = excellent). The detail subjective questionnaire of the face and content validity is demonstrated in Table 2. The objective evaluation of the construct has six assessed items that are totally operation time (T), surgical clamps track length (CL), endoscope track length (EL), surgical clamps angle accumulation (CA), endoscope angle accumulation (EA), and the numbers of block drop (ND). First, verify whether each set of data obeys a normal distribution, and then compare the performance of multiple simulators under six parameters.
Table 2.
Subjective questionnaire of the face and content validity.
| Face and content validity questions (score: 1–5, 1 = poor to 5 = excellent) |
|---|
| Q1: Realism of peg model (visual) |
| Q2: Realism of endoscope model (visual) |
| Q3: Realism of surgical clamps (visual) |
| Q4: Realism of surgical environment (visual) |
| Q5: Comfortable of training content in HMD (visual) |
| Q6: Overall realism of visualization |
| Q7: Realism of peg manipulation (haptic) |
| Q8: Realism of endoscope manipulation (haptic) |
| Q9: Realism of surgical clamps manipulation (haptic) |
| Q10: Realism of interaction between the surgical clamps and peg (haptic) |
| Q11: Overall realism of manipulation |
| Q12: I would like to recommend this simulator to VATS surgical training for medical students |
| Q13: I would like to recommend this simulator as an assessment tool for VATS surgical skills |
2.5. Data Analysis
In terms of face and content validity, descriptive statistics were used to analyze the questionnaire data by calculating the mean value of subjective question and their standard errors. For the construct validity, assessing the test results of novice and expert on six parameters of five simulators with Shapiro–Wilk test, p > 0.05, is considered to be subject to normal distribution. Box plots are used for analysis of objective parameters. Each group of T of novices and experts on the five simulators follows a normal distribution and was compared between groups using an independent-sample t-test, and each group of T of five simulators on novices and experts was compared between groups using the one-way ANOVA test. But, most of the CL, EL, CA, EA, and ND were not a normal distribution, each group of those data of novices and experts on the five simulators was compared between groups using the two-tailed Mann–Whitney U test, and each group of those data of five simulators on novices and experts was compared between groups using the Kruskal–Wallis test. A p value < 0.05 was considered significant. Analyses were performed using the software package, IBM SPSS Statistics, version 20.0.
3. Results
3.1. Face and Content Validation
Aiming to the perceptions of visual and tactile sensations during the use of the five simulators, we set up a subjective questionnaire with 13 questions using the scoring method of the 5-point Likert-type scale. After the experiment of the simulation operation, the questionnaire was filled by 8 experts for the experience during the operation. Through the Shapiro–Wilk test, each set of score data is an approximately normal distribution. The thirteen subjective questionnaires were collated, and the score data of each simulator were assessed by the Shapiro–Wilk test. The average and standard deviation of each set of data are shown in Table 3, and the distribution of the average score is shown in Figure 5. It can be found that the box and MR scores demonstrated a higher score than the AR and VR scores in a visual sense. In terms of haptic sense, the box scores are higher than the AR, VR, CVR, and MR scores, which demonstrated the best haptic immersion among these virtual simulators.
Table 3.
Subjective questionnaire results of the face and content validity (experts group).
| Questionnaires | Box | AR | VR | CVR | MR | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| Mean | SD | Mean | SD | Mean | SD | Mean | SD | Mean | SD | |
| Score (1–5) | ||||||||||
| Q1 | 3.75 | 0.71 | 2.63 | 0.92 | 3.00 | 0.53 | 3.38 | 0.92 | 4.25 | 0.71 |
| Q2 | 4.13 | 0.64 | 2.63 | 0.92 | 2.63 | 0.91 | 3.38 | 0.92 | 3.63 | 0.92 |
| Q3 | 4.25 | 0.71 | 3.50 | 0.93 | 3.50 | 0.53 | 3.88 | 0.64 | 4.00 | 0.76 |
| Q4 | 1.50 | 0.76 | 1.63 | 0.74 | 1.63 | 0.74 | 2.75 | 0.71 | 4.25 | 0.71 |
| Q5 | — | — | — | 2.75 | 1.04 | 2.50 | 0.93 | |||
| Q6 | 3.13 | 0.64 | 2.50 | 0.93 | 3.13 | 0.64 | 3.13 | 0.64 | 4.13 | 0.64 |
| Q7 | 4.25 | 0.89 | 2.50 | 0.93 | 2.00 | 0.75 | 1.88 | 0.64 | 2.00 | 0.76 |
| Q8 | 3.75 | 0.71 | 2.00 | 0.76 | 3.13 | 0.83 | 2.00 | 0.76 | 2.50 | 0.93 |
| Q9 | 4.75 | 0.46 | 3.88 | 0.64 | 4.00 | 0.76 | 3.00 | 0.76 | 2.13 | 0.64 |
| Q10 | 4.63 | 0.52 | 3.03 | 0.64 | 2.00 | 0.76 | 1.63 | 0.74 | 1.25 | 0.46 |
| Q11 | 3.50 | 0.93 | 2.88 | 0.64 | 2.00 | 0.76 | 2.13 | 0.64 | 2.00 | 0.76 |
| Q12 | 3.13 | 0.64 | 2.00 | 0.76 | 2.88 | 0.64 | 2.00 | 0.76 | 3.25 | 1.04 |
| Q13 | 3.75 | 0.71 | 1.75 | 0.71 | 2.75 | 1.04 | 1.75 | 0.71 | 3.25 | 0.89 |
Figure 5.
Face and content validity score of the expert group on these five simulators.
3.2. Construct Validation
During the operation of 32 trainees, the VR, CVR, AR, and MR simulators automatically recorded the trajectory length and rotation angle of the surgical instruments and endoscopes in real time, as well as the total time spent and the numbers of block drop on each operation. However, the box simulator cannot record the length and angle of the motion track and the numbers of the block drop, and the total time of the operation can only be roughly recorded by the timer. Six box plots for the experimental data of six parameters T, CL, EL, CA, EA, and ND were made by SPSS, and those box plots which include mediums and means are shown in Figure 6. Similarly, after calculating these p values of the parameters of the different simulators for the novice and expert groups, the results are shown in Table 4. Most of the evaluation parameters of the expert group are lower than that of the novice group in each simulator. p < 0.05, which is only for the EL of VR simulator. EL and EA of CVR simulator data of the expert group demonstrated a higher score than that of the novice. Comparing VR, CVR, and MR scores of the expert group, those scores of the VR group demonstrate the best performance. The clamp moving trajectory of AR, VR, CVR, and MR groups during the simulation is demonstrated in Figure 7, and it shows that the constructive difference between these four training groups and the AR group demonstrated a better-concentrated trajectory than other groups.
Figure 6.
Performance comparison of six parameters in five simulators. (a) T: totally operation time. (b) ND: numbers of block drop. (c) CL: surgical clamps track length. (d) CA: surgical clamps angle accumulation. (e) EL: endoscope track length. (f) EA: endoscope angle accumulation.
Table 4.
The comparison of objective parameters for construct validity (experts and novices).
| p value | |||||||
|---|---|---|---|---|---|---|---|
| T | CL | EL | CA | EA | ND | ||
| Box | Experts | 0.002 | — | — | — | — | 0.781 |
| Novices | |||||||
|
| |||||||
| AR | Experts | 0.053 | 0.000 | — | 0.277 | — | 0.302 |
| Novices | |||||||
|
| |||||||
| VR | Experts | 0.000 | 0.761 | 0.001 | 0.024 | 0.037 | 0.084 |
| Novices | |||||||
|
| |||||||
| CVR | Experts | 0.744 | 0.041 | 0.177 | 0.030 | 0.486 | 0.405 |
| Novices | |||||||
|
| |||||||
| MR | Experts | 0.045 | 0.000 | 0.003 | 0.965 | 0.009 | 0.001 |
| Novices | |||||||
|
| |||||||
| p value | Experts | 0.000 | 0.000 | 0.000 | 0.005 | 0.012 | 0.014 |
| Novices | 0.000 | 0.000 | 0.000 | 0.013 | 0.000 | 0.000 | |
Figure 7.
Clamp moving trajectory comparison during the simulation. (a) VR group. (b) AR group. (c) CVR group. (d) MR group.
4. Discussion
There is an increasing use of simulators to learn, improve, and rehearse surgical skills in medical training. The VatsSim-XR is a versatile simulator that is able to perform multiple surgical training scenarios and implement 4 different simulation modes (VR, CVR, AR, and MR) on a single device. The Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) and the American College of Surgeons (ACS) have established a standard for fundamentals of laparoscopic surgery (FLS) training and surgical skills assessment [25–27]. The FLS consists of five basic foundations: peg transfer, pattern cutting, ligation loop, and suturing with either intracorporal or extracorporeal knot tying, and these tasks are designed to train the operational skills and assessment of medical interns and residents. Peg transfer is the basic training task to train the dual-hands coordination and the hand-eye coordination for the medical interns and residents [8]. FLS simulation training is mostly based on box and VR simulators in the market.
In terms of haptic, the box simulator is superior to the other four simulators, but an operation object is an object that is easily destroyed after repeated use. In terms of visual, the box and MR simulators are better. The AR simulator is the best in terms of interactive, and in the interactive environment part, where the simulator is more immersive: MR simulator > CVR simulator; however, the box, VR, and AR simulators have almost no surroundings and are less immersive.
Compared with novice groups of six parameters on five simulators, most of the expert groups show short operation time, short track length, small angle accumulation, and small drop number, but only these three groups, the EL of the VR simulator, EL, and EA of CVR the simulator, show the opposite result, and the main reason is that when adjusting the endoscope, the novice does not move and rotate the endoscope to an optimal viewing angle in strict accordance with the surgical standard, resulting in greater difference. For the VR, CVR, and MR simulators of six parameters, the data size of expert groups basically follows a rule VR < CVR < MR, but only the MR simulator data of the CA parameter are smaller than VR and MR simulators, because the experts have adapted to the 360° real operating room scene in the MR simulator in advance. Although the MR simulator is not perfect in terms of T, CL, EL, CA, EA, and ND, it provides a highly realistic operating room for hospital interns and residents. The environment, closer to the real surgery site, has great potential for development. Experiments and evaluation results conducted by both professional and novice thoracic surgeons show that the AR trainer is able to provide a more balanced training environment on visuohaptic fidelity and accuracy, the box trainer and MR trainer demonstrated the best realism 3D perception and surgical immersive performance, respectively, and the CVR trainer shows a better clinic effect that the traditional VR trainer.
The advantages and disadvantages of the five simulators are shown in Table 5. The haptic, visual, and surrounding environments are compared separately by levels (III means the highest and I means the lowest). The complex design of our integrated system will provide a more immense and effective training environment for the medical surgery simulation.
Table 5.
Comparisons of the five peg transfer simulators.
| Advantages | Disadvantages | Immersive | |||
|---|---|---|---|---|---|
| Haptic | Visual | Environments | |||
| Box | Realistic force feedback Realistic 3D perception Cost-effective Portable and easy to use |
Low integration Lack of objective assessment Low immersion Without OR environment |
III | I | I |
|
| |||||
| VR | Objective assessment High interactivity Unlimited use |
Low immersion Without OR environment Low 3D perception |
I | I | I |
|
| |||||
| CVR | Objective assessment High interactivity High immersion Motion track (HMD) |
Virtual OR environment Narrow FOV Low 3D perception |
I | II | II |
|
| |||||
| AR | Objective assessment High interactivity High immersion Realistic 3D perception |
Low immersion Without OR environment Environmental influence |
I | I | I |
|
| |||||
| MR | 100% OR environment Objective assessment High interactivity High immersion |
OR environment distortion Narrow FOV |
I | III | III |
5. Conclusions
In this paper, we have detailed a virtual surgical educative simulator with realistic performance in both visual and haptic sensations for the peg transfer procedures, build the face, content and construct validation on the peg transfer training by use of the box, VR, AR, and MR trainer, thereby to compare advantages and disadvantages of these simulators. However, during the interaction, the sense of touch is not immersive enough; furthermore, there is a cross between the gripper and the interacting virtual object. What is more, in the comparison experiments design, the box simulator is not able to automatically record the trajectory and rotation angle parameters of the instrument and endoscope, which means the comparison of the six parameters is not comprehensive enough in this manuscript. In the future works, we will improve the visual and haptic experiment design parts to achieve a high immersive and realistic simulation environment both visual and haptic perception.
Acknowledgments
We thank Professor Minghui, Xiao, Dr. Min Zou, Dr. Jie Shen, and Mr. Ran Cao of Yunnan First People's Hospital for the helpful surgical suggestions. This research was funded by the National Natural Science Foundation of China (61741516) and the National Science Foundation of Yunnan Province, China (ZD2014004), and Yunnan Key Laboratory of Opto-Electronic Information Technology, Kunming, China.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Supplementary Materials
The supplementary video demonstrated the peg transfer experimental procedures on the box, VR, CVR, AR, and MR trainer.
References
- 1.Solomon B., Bizekis C., Dellis S. L., et al. Simulating video-assisted thoracoscopic lobectomy: a virtual reality cognitive task simulation. Journal of Thoracic and Cardiovascular Surgery. 2011;141(1):249–255. doi: 10.1016/j.jtcvs.2010.09.014. [DOI] [PubMed] [Google Scholar]
- 2.Hansen H. J., Petersen R. H. Video-assisted thoracoscopic lobectomy using a standardized three-port anterior approach—the Copenhagen experience. Annals of Cardiothoracic Surgery. 2012;1(1):70–76. doi: 10.3978/j.issn.2225-319X.2012.04.15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Jensen K., Bjerrum F., Hansen H. J., Petersen R. H., Pedersen J. H., Konge L. Using virtual reality simulation to assess competence in video-assisted thoracoscopic surgery (VATS) lobectomy. Surgical Endoscopy. 2016;31(6):2520–2528. doi: 10.1007/s00464-016-5254-6. [DOI] [PubMed] [Google Scholar]
- 4.Gossot D., Lutz J., Grigoroiu M., Brian E., Seguin-Givelet A. Thoracoscopic anatomic segmentectomies for lung cancer: technical aspects. Journal of Visualized Surgery. 2016;2:p. 171. doi: 10.21037/jovs.2016.10.04. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.He J., Xu X. Thoracoscopic anatomic pulmonary resection. Journal of Thoracic Disease. 2012;4(5):520–547. doi: 10.3978/j.issn.2072-1439.2012.09.04. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Loukas C., Lahanas V., Georgiou E. Surgical simulation training systems: box trainers , virtual reality and augmented reality simulators. International Journal of Advanced Robotics and Automation. 2016;1(2):1–9. doi: 10.15226/2473-3032/1/2/00109. [DOI] [Google Scholar]
- 7.Ritter E. M., Kindelan T. W., Michael C., Pimentel E. A., Bowyer M. W. Concurrent validity of augmented reality metrics applied to the fundamentals of laparoscopic surgery (FLS) Surgical Endoscopy. 2007;21(8):1441–1445. doi: 10.1007/s00464-007-9261-5. [DOI] [PubMed] [Google Scholar]
- 8.Arikatla V. S., Ahn W., Sankaranarayanan G., De S. Towards virtual FLS: development of a peg transfer simulator. International Journal of Medical Robotics and Computer Assisted Surgery. 2013;10(3):344–355. doi: 10.1002/rcs.1534. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Suzuki N., Hattori A., Ieiri S., Tomikawa M., Kenmotsu H., Hashizume M. VR training system for endoscopic surgery robot. 6th International Workshop on Augmented Environments for Computer-Assisted Interventions (AE-CAI 2011); September 2012; Toronto, ON, Canada. pp. 117–129. [Google Scholar]
- 10.Boejen A., Grau C. Virtual reality in radiation therapy training. Surgical Oncology. 2011;20(3):185–188. doi: 10.1016/j.suronc.2010.07.004. [DOI] [PubMed] [Google Scholar]
- 11.Schijven M. P., Jakimowicz J. J. Validation of virtual reality simulators: key to the successful integration of a novel teaching technology into minimal access surgery. Minimally Invasive Therapy & Allied Technologies. 2009;14(4-5):244–246. doi: 10.1080/13645700500221881. [DOI] [PubMed] [Google Scholar]
- 12. Fida2018_Article_AugmentedRealityInOpenSurgery.pdf.
- 13.Moglia A., Ferrari V., Morelli L., Ferrari M., Mosca F., Cuschieri A. A systematic review of virtual reality simulators for robot-assisted surgery. European Urology. 2016;69(6):1065–1080. doi: 10.1016/j.eururo.2015.09.021. [DOI] [PubMed] [Google Scholar]
- 14.Meola A., Cutolo F., Carbone M., Cagnazzo F., Ferrari M., Ferrari V. Augmented reality in neurosurgery: a systematic review. Neurosurgical Review. 2017;40(4):537–548. doi: 10.1007/s10143-016-0732-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Maciel A., Liu Y., Ahn W., Singh T. P., Dunnican W., De S. Development of the VBLaST™ : a virtual basic laparoscopic skill trainer. International Journal of Medical Robotics and Computer Assisted Surgery. 2008;4(2):131–138. doi: 10.1002/rcs.185. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Muralha N., Oliveira M., Ferreira M. A., Costa-Maia J. Virtual reality simulation as a tool to monitor surgical performance indicators: VIRESI observational study. Acta Médica Portuguesa. 2017;30(5):p. 388. doi: 10.20344/amp.7983. [DOI] [PubMed] [Google Scholar]
- 17.Zhang A., Hünerbein M., Dai Y., Schlag P. M., Beller S. Construct validity testing of a laparoscopic surgery simulator (Lap Mentor®) Surgical Endoscopy. 2008;22(6):1440–1444. doi: 10.1007/s00464-007-9625-x. [DOI] [PubMed] [Google Scholar]
- 18.Huber T., Wunderling T., Paschold M., Lang H., Kneist W., Hansen C. Highly immersive virtual reality laparoscopy simulation: development and future aspects. International Journal of Computer Assisted Radiology and Surgery. 2018;13(2):281–290. doi: 10.1007/s11548-017-1686-2. [DOI] [PubMed] [Google Scholar]
- 19.Huber T., Paschold M., Hansen C., Lang H., Kneist W. Artificial versus video-based immersive virtual surroundings: analysis of performance and user’s preference. Surgical Innovation. 2018;25(3):280–285. doi: 10.1177/1553350618761756. [DOI] [PubMed] [Google Scholar]
- 20.Moustris G. P., Hiridis S. C., Deliparaschos K. M., Konstantinidis K. M. Evolution of autnomous and semi-autnomous robotic surgical systems: a review of the literature. International Journal of Medical Robotics and Computer Assisted Surgery. 2011;7(4):375–392. doi: 10.1002/rcs.408. [DOI] [PubMed] [Google Scholar]
- 21.Sihoe A. D. L. Uniportal video-assisted thoracoscopic lobectomy. Annals of Cardiothoracic Surgery. 2016;5(2):133–144. doi: 10.21037/acs.2016.03.14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Egger J., Gall M., Wallner J., et al. HTC Vive MeVisLab integration via OpenVR for medical applications. PLoS One. 2017;12(3) doi: 10.1371/journal.pone.0173972.e0173972 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Yoganathan S., Finch D. A., Parkin E., Pollard J. 360° virtual reality video for the acquisition of knot tying skills: a randomised controlled trial. International Journal of Surgery. 2018;54:24–27. doi: 10.1016/j.ijsu.2018.04.002. [DOI] [PubMed] [Google Scholar]
- 24.Mohiuddin S., Voronov P., Nishioka H., et al. Applying 360-degree videos to interdisciplinary training and simulation. Anesthesia & Analgesia. 2017;124(5):83–84. [Google Scholar]
- 25.Botden S. M. B. I., Buzink S. N., Schijven M. P., Jakimowicz J. J. Augmented versus virtual reality laparoscopic simulation: what is the difference? World Journal of Surgery. 2007;31(4):764–772. doi: 10.1007/s00268-006-0724-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Peters J. H., Fried G. M., Swanstrom L. L., et al. Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery. Surgery. 2004;135(1):21–27. doi: 10.1016/s0039-6060(03)00156-9. [DOI] [PubMed] [Google Scholar]
- 27.Fried G. M. FLS assessment of competency using simulated laparoscopic tasks. Journal of Gastrointestinal Surgery. 2008;12(2):210–212. doi: 10.1007/s11605-007-0355-0. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
The supplementary video demonstrated the peg transfer experimental procedures on the box, VR, CVR, AR, and MR trainer.
Data Availability Statement
The data used to support the findings of this study are available from the corresponding author upon request.