Abstract
Background:
Arthroscopy is one of the cornerstone skills learned during orthopaedic residency training. Previous studies have attempted to identify characteristics of arthroscopy naive individuals leading to superior initial arthroscopic performance with conflicting findings. Furthermore, other virtual reality simulator studies consisting of beginner trainees, have noted that certain individuals fail to progress at the rate of their peers despite rigorous training. Therefore the purpose of this study was to 1) identify trainee characteristics that may have an impact on initial arthroscopy skill and performance and 2) identify trainee characteristics affecting their ability to improve through training on an arthroscopy simulator.
Methods:
Forty-three medical students with no prior arthroscopy experience performed a diagnostic knee arthroscopy using an anatomic virtual reality simulator. Prior to the procedure, information was collected about each participant regarding various demographics, sports involvement, hand dominance, specialty interest, 3-D video game use and experience assisting in the operating room. Their baseline performance was measured using the following outcomes: time (seconds), camera path length (CPL) (centimeters), and an overall composite score. A subgroup consisting of 22 students underwent training with a non-anatomic virtual arthroscopy simulator consisting of a series of 5 self-guided modules after their initial knee scope. This group was retested using the same diagnostic knee scope one to two weeks later. Participant background characteristics were correlated with initial performance on the knee scope and the change from the first to second knee scope in the sub-group that completed the training.
Results:
At the time of the initial diagnostic knee arthroscopy, performance was most strongly correlated with how often an applicant currently plays video games and how often they have played video games in the past. However, this was only a weak correlation (r= 0.29 and 0.24, respectively). Interestingly, students pursuing a non-surgical residency outperformed those interested in a surgical specialty in all outcome measurements at baseline, although this did not reach a level of significance. Year of training in medical school, age, hand dominance, current or past participation in sports requiring hand-eye coordination, and number of surgical cases they have assisted in for did not influence initial performance. Participants that have operated another type of scope (e.g. bronchoscopy, laparoscopy) in the past showed a trend towards higher performance in composite score (21.6 vs 14.9, p = 0.07), although this did not reach a level of significance. Regarding the change in performance of those that completed the non-anatomic training prior to the second knee scope, change in time to complete the procedure was significantly different between participants in different years of medical school. Fourth year medical students improved by an average of 421 seconds compared to 98 seconds, 127, and 140 seconds for the other classes, p = 0.02. Those who have regularly participated in sports requiring hand eye coordination in the past improved their time (182 vs 78 seconds, p = 0.0245) and camera path length (96 vs 31 cm, p = 0.0372) significantly more than those were not involved in sports.
Discussion and Conclusions:
Our study demonstrates that baseline arthroscopy performance correlates most highly with 3D video game experience. The ability of trainees to learn arthroscopy using a virtual reality simulator appears to be influenced more by those who have regularly participated in sports requiring hand eye coordination rather than video game use.
Level of Evidence: III
Keywords: arthroscopy; simulator; knee, non-anatomic; skill; video games; sports; training, medical student
Introduction
Arthroscopy is one of the cornerstone surgical skills learned during orthopaedic surgery residency training. Orthopedic surgery residents that graduated in the class of 2012 performed an average of 340 arthroscopic surgeries over the course of their residency training; this accounted for approximately 15% of all cases during residency.1 Recent studies examining the trends in arthroscopic surgeries in the United States have demonstrated marked increases in the number of cases of hip arthroscopy, arthroscopic rotator cuff repair, and knee arthroscopy.2-4 Practice and development of arthroscopic surgical techniques is of paramount importance in modern orthopedic residency training. Anatomic arthroscopy simulators have been demonstrated to be effective training tools for the development of arthroscopic technique that translates to increased skill in the operating room.5-7
Previous studies have attempted to identify characteristics of novice trainees, including residents and medical students, which are associated with superior initial arthroscopic performance.5,8 These studies have demonstrated conflicting findings, with inconsistent associations between increased arthroscopic performance and activities such as video game experience and participation in sports.5,8,9 Furthermore, other virtual reality simulator studies using cohorts of novice trainees have noted that a portion of trainees fail to progress at the rate of their peers despite equal rigorous training.10 Identifying factors that may contribute to failure of progression would allow for identification of residents that may require additional intervention (i.e. focused training) to develop adequate arthroscopic skill.
The purpose of this study was twofold: (1) to identify trainee characteristics that may have an impact on initial arthroscopy skill and performance, and (2) to identify trainee characteristics affecting ability to demonstrate measured improvement through training on an arthroscopy simulator. We hypothesized that video game use, sports participation, and previous assistance in the operating room would correlate with improved initial performance, as well as a greater ability to improve with training.
Methods
After approval was obtained from the Institutional Review Board, 43 medical students were recruited from a single allopathic medical school for participation in this prospective study conducted from April 2016 to April 2017. Participants were excluded if they had previous simulated, live, or cadaveric arthroscopy experience or were unable to use an arthroscopy simulator. Study participant demographics including gender, year of medical school training, age, and hand dominance are seen in Table 1.
Table 1.
Study Participant Demographics
| Completed Simulation One | Completed Simulations 1 and 2 with training in between | P-value | |
|---|---|---|---|
| Age | 25.81 (2.94) | 25.13 (1.42) | 0.3513 |
| Year in Medical School (1/2/3/4) | 12/3/3/3 | 11/4/5/2 | 0.8789 |
| Males/Female | 14 / 7 | 15 / 7 | 0.9156 |
| Right handed/Left handed | 20 / 1 | 19 / 3 | 0.6069 |
| Listed Surgical/Non-surgi- cal specialty as #1 interest pre-test | 10 / 11 | 16 / 6 | 0.0923 |
| Time between knee scopes | 9.48 days | 8.86 days | 0.1969 |
| Total number of subjects | 21 | 22 |
Upon agreement to participate in the study, all 43 subjects initially completed a questionnaire regarding demographics, background characteristics, educational status, and career specialty interests. Pre-test questionnaire is shown in Figure 1. They then viewed a video demonstrating a simulated diagnostic knee arthroscopy performed by a sports medicine fellowship-trained orthopaedic surgeonon the same simulator used in this study (ArthroS knee). After viewing the video, they completed a self-guided PowerPoint module on the basics of arthroscopy and how to use the virtual reality simulator. Once these steps were complete, they were then given one minute to familiarize themselves with the scope and subsequently proceeded with the first diagnostic knee arthroscopy.
Figure 1.
The questionnaire administered to participants prior to beginning the first knee arthroscopy attempt.
Twenty-two subjects were randomized to undergo a series of five self-guided arthroscopic training modules on a non-anatomic simulator. These were performed on the Fundamentals of Arthroscopic Surgery Training (FAST) simulator. Self-guided modules included linear tracking, curved tracking, periscoping, palpation, and collecting stars using a grasping device all within a three dimensional virtual environment. In order to progress from one module to the next, subjects were required to obtain a 3 out of 3 star rating. Sample images from these modules are shown in Figures 2 through 5. After the training was complete, participants returned between 7 and 15 days after later to complete the post-test (second diagnostic knee arthroscopy).
Figure 2.
In this non-anatomic (FAST) module, the objective is to track along a line while keeping the scope centered at an appropriate distance.
Figure 5.
The objective of the periscoping module is to use the periscope to gain an appropriate view of the center of the green surface at various positions and angles within a 3D space.
Figure 3.
The palpation module uses a probe to touch the center of a freely moveable 3D ball in various spatial dimensions while under appropriate visualization from the arthroscope.
Figure 4.
This modules utilizes a grasper in one hand while controlling the scope in the contralateral hand to pick up and place stars floating in 3D area into a bucket.
Arthroscopy simulation was performed in a supervised, distraction free environment. The anatomic knee simulator used for the pre and post-tests was the ArthroS knee (Figure 6). The non-anatomic simulator used in this study was the FAST virtual reality simulator (Figure 7). Participants did not receive tips or feedback on performance during the simulation, however, to eliminate the need knowledge of anatomy during the procedure each area of interest to be visualized was highlighted in yellow and would turn green with appropriate visualization as seen in Figure 8. Total time, camera path length, and overall composite score were recorded for the diagnostic knee arthroscopy attempts.
Figure 6.
Demonstrates the anatomic knee virtual reality simulator set-up used for assessment in this study.
Figure 7.
Demonstrates the non-anatomic FAST simulator used for training in this study.
Figure 8.
Is a sample view of the knee joint during the simulated diagnostic arthroscopy procedure in this study. Objectives are visualized in yellow and begin to turn green once appropriately visualized with the scope.
Data analysis was performed by using SAS software, version 9.4 (SAS Institute, Inc. of Cary, North Carolina). Statistical methods include Pearson correlations and two-group t-test or paired t-test for continuous variables and Chi-Square test for categorical variables. P value < 0.05 was considered statistically significant.
Results
Twenty-nine males and fourteen females were enrolled in this study with a mean age of 25.1 years. Average length of time between the first and second knee arthroscopy was 8.9 days. Thirty-nine participants were right hand dominant and four were left hand dominant. Average time to complete the five FAST modules was 71 minutes. No subjects were lost to follow up in this study. Twenty-two of these students completed the self-guided non-anatomic arthroscopic fundamentals of arthroscopy training modules per randomization protocol.
Performance on the initial diagnostic knee arthroscopy most highly correlated with the frequency that an applicant played video games in the past followed by closely by how often an applicant currently plays video games, r = 0.29 and 0.24, respectively. Those who had operated another type of scope (e.g. bronchoscope, laparoscope) in the operating room setting tended to perform better compared to those who had not with an average composite score of 21.6 vs 14.9 p =0.07, time 273.4 vs 324.4 seconds p = 0.36, camera path length 121.1 vs 172.7 cm, p=0.13. Number of surgical cases assisted with as well as hand dominance, age, and year of medical training did not significantly influence initial performance. Interestingly, students planning to pursue a surgical residency tended to perform worse compared to those pursuing non-surgical residencies when comparing composite score (16.2 vs 19.2), time (325.1 vs 275.3 seconds), and camera path length (168.2 vs 131.1 cm), however these did not reach a level of significance.
Regarding the change in performance of those that completed the non-anatomic training prior to the second knee scope, change in time to complete the procedure was significantly different between participants in different years of medical school. Fourth year medical students improved by an average of 421 seconds compared to 98 seconds, 127, and 140 seconds for the other classes, p = 0.02. Those that reported they had regularly participated in sports requiring hand eye coordination in the past improved their time (182 vs 78 seconds, p = 0.0245) and camera path length (96 vs 31 cm, p = 0.0372) significantly more than those were not involved in sports. Specialty interest, video game use, hand dominance, experience in operating room assisting cases did not correlate with the ability to improve with training. Results are summarized in Tables 2 and 3.
Table 2.
Participant’s Year of Training Compared with Change in Performance
| Year of Training | Change in Composite Score (STD) | Change in Time, seconds (STD) | Change in Camera Path Length, cm (STD) |
|---|---|---|---|
| 1 (N=11) | 16.55 (9.8)) | -140.45 (127.29) | -89.82 (102.39) |
| 2 (N=4) | 11.00 (7.48) | -127.00 (55.47) | -34.73 (41.76) |
| 3 (N=5) | 13.20 (12.72) | -98.60 (130.73) | -93.74 (136.15) |
| 4 (N=2) | 11.00 (1.41) | -421.50 (127.99) | -68.75 (63.29) |
| P-value | 0.735 | 0.028 | 0.8 |
Table 3.
Pearson Correlation Coefficients Between Questionnaire Responses and Change in Performance
| Pearson Correlation Coefficient (p-value) | Age | Question 8 | Question 9 | Question 14 | Change in Composite Score | Change in Time (seconds) | Change in CPL (cm) |
|---|---|---|---|---|---|---|---|
| Age | 1 | -0.35 (0.12) | -0.14 (0.52) | -0.43 (0.05) | 0.16 (0.49) | -0.15 (0.5) | 0.03 (0.90) |
| Question 8 | -0.35 (0.12) | 1 | 0.41 (0.05) | 0.36 (0.10) | -0.20 (0.36) | -0.05 (0.8) | 0.05 (0.81) |
| Question 9 | -0.14 (0.52) | 0.42 (0.05) | 1 | 0.34 (0.12) | 0.06 (0.79( | -0.04 (0.87) | 0.024 (0.91) |
| Question 14 | -0.43 (0.05) | 0.36 (0.11) | 0.34 (0.12) | 1 | 0.15 (0.52) | 0.05 (0.84) | 0.05 (0.81) |
| Change in Composite Score | 0.16 (0.49) | -0.20 (0.36) | 0.06 (0.79) | 0.15 (0.52) | 1 | -0.32 (0.14) | -0.47 (0.02) |
| Change in Time (seconds) | -0.15 (0.50) | -0.05 (0.81) | -0.04 (0.87) | 0.05 (0.84) | -0.32 (0.14) | 1 | 0.7 (0.0002) |
| Change in CPL (cm) | 0.027 (0.90) | 0.05 (0.81) | 0.02 (0.91) | 0.05 (0.82) | -0.47 (0.03) | 0.71 (0.0002) | 1 |
Discussion
The results of this study suggest that of the variables examined, initial diagnostic knee arthroscopy performance was most highly associated with how often a student currently plays video games or how often they played video games in the past, i.e. cumulative video game experience. Participants that have operated another type of scope (e.g. bronchoscopy, laparoscopy) in the past showed a trend towards higher performance, although this did not reach a level of significance. A subject’s ability to train and improve through self-guided modules was most highly associated with participation in sports requiring hand-eye coordination. Additionally, fourth year medical students improved significantly more in time and camera path length compared to other years of medical education, suggesting that experience and knowledge may be a factor in skill acquisition.
Jentzsch et al. previously suggested individuals that are skilled at 3-D video games are better arthroscopists compared to inexperienced or poor 3-D gamers in a level II cross sectional study.5 Their study consisted of 30 volunteers who were asked to play five different types of video games with increasing visuospatial demands and their performance was then correlated with a score on the ArthroS knee arthroscopy simulator. They reported a positive correlation of 0.63, p < 0.001, between knee arthroscopy simulator performance and 3D video game performance. Cumulative video game experience showed slightly less correlation at 0.50 ( p = 0.005). Gomoll et al., however, did not find any correlation between shoulder arthroscopy simulator performance parameters and video game experience in a study consisting of 43 participants of varying surgical experience.11
Current literature examining laparoscopic surgical skill in novice medical students and junior residents suggests that novice laparoscopists with prior computer or console-based video game experience demonstrated better initial performance, greater overall proficiency, and faster time to achieve technical proficiency on laparoscopic training simulators.12-16 Prior and current video game experience was associated with faster task performance and fewer errors.12,14 Additionally, past video game experience was found to be predictive of laparoscopic surgical skill via regression analysis.14,15
The use of 3-D video games as a training tool aimed at improving spatial reasoning has been suggested within current laparoscopic simulation literature.16,17 Schlickum et al. demonstrated greater overall proficiency using a laparoscopic simulator in a cohort after spatial training using 3-D video games compared to cohorts that participated in 2-D video game training as and no video game training.16 The results of the current study echo the overall associations within laparoscopic simulation in part; current and past video game experience was associated with higher initial proficiency in the arthroscopy simulation. No association was found between video game experience and the ability to improve or progress in proficiency in this study. Hand dominance has also been found to be associated with higher initial proficiency in one laparoscopic simulation study; no such association was demonstrated in this study.12 Within the current study, participation in sports that require hand eye coordination was associated with an ability to improve through self-guided modules. Following a review of current literature, this appears to be a novel association. Logically, such an association makes sense – sports requiring hand-eye coordination demand participants to be adept at repeatable gross and fine motor tasks, as does arthroscopy.
In the present study, no subjects were lost to follow-up. Participants were enrolled from all four years of medical school, with varying interests in surgical and non-surgical specialties. Additionally, this study was conducted in a prospective design. The outcome measures were collected objectively by the virtual reality arthroscopy simulator, and the manner of testing was standardized for all students.
Despite the above strengths, there are several limitations to this study. First, some of the factors may have reached statistical significance with a greater study population. However, numbers were limited by the maximum amount we were able to recruit. In addition, participants were not blinded as to which group they were assigned to (FAST versus control group). Finally, while arthroscopy performance was assessed using objective measurements recorded by the simulator, it is unable to precisely replicate the arthroscopy experience in the operating room.
Based on the findings of this study, potential characteristics of novice orthopaedic trainees with innate arthroscopy ability, which are different from those that are indicative of individuals that respond best to training, are identified. Future studies are needed to validate these findings in either actual operating room situations or cadaveric models. Additionally, since arthroscopy is just one cornerstone skill learned during orthopaedic residency it would be useful to see how these characteristics apply to other areas of fracture management in the operating room.
Conclusions
Our study suggests that baseline arthroscopy performance correlates most highly with how often the trainee currently uses or has used video games in the past. The ability of trainees to learn arthroscopy using a virtual reality simulator appears to be influenced more by those who have regularly participated in sports requiring hand eye coordination rather than video game use.
References
- 1.Accreditation Council for Graduate Medical Education (ACGME) Data Systems. Case Log Statistical Reports, 2011-2012. 2018.
- 2.Colvin AC, Egorova N, Harrison AK, Moskowitz A, Flatow EL. National trends in rotator cuff repair. The Journal of bone and joint surgery. American volume. 2012;94(3):227–233. doi: 10.2106/JBJS.J.00739. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Kim S, Bosque J, Meehan JP, Jamali A, Marder R. Increase in outpatient knee arthroscopy in the United States: a comparison of National Surveys of Ambulatory Surgery, 1996 and 2006. The Journal of bone and joint surgery. American volume. 2011;93(11):994–1000. doi: 10.2106/JBJS.I.01618. [DOI] [PubMed] [Google Scholar]
- 4.Montgomery SR, Ngo SS, Hobson T, et al. Trends and demographics in hip arthroscopy in the United States. Arthroscopy : the journal of arthroscopic & related surgery : official publication of the Arthroscopy Association of North America and the International Arthroscopy Association. 2013;29(4):661–665. doi: 10.1016/j.arthro.2012.11.005. [DOI] [PubMed] [Google Scholar]
- 5.Jentzsch T, Rahm S, Seifert B, Farei-Campagna J, Werner CM, Bouaicha S. Correlation Between Arthroscopy Simulator and Video Game Performance: A Cross-Sectional Study of 30 Volunteers Comparing 2- and 3-Dimensional Video Games. Arthroscopy : the journal of arthroscopic & related surgery : official publication of the Arthroscopy Association of North America and the International Arthroscopy Association. 2016;32(7):1328–1334. doi: 10.1016/j.arthro.2015.12.047. [DOI] [PubMed] [Google Scholar]
- 6.Howells NR, Gill HS, Carr AJ, Price AJ, Rees JL. Transferring simulated arthroscopic skills to the operating theatre: a randomised blinded study. The Journal of bone and joint surgery. British volume. 2008;90(4):494–499. doi: 10.1302/0301-620X.90B4.20414. [DOI] [PubMed] [Google Scholar]
- 7.Waterman BR, Martin KD, Cameron KL, Owens BD, Belmont PJ. Simulation Training Improves Surgical Proficiency and Safety During Diagnostic Shoulder Arthroscopy Performed by Residents. Orthopedics. 2016;39(3):e479–485. doi: 10.3928/01477447-20160427-02. [DOI] [PubMed] [Google Scholar]
- 8.Rahm S, Wieser K, Wicki I, Holenstein L, Fucentese SF, Gerber C. Performance of medical students on a virtual reality simulator for knee arthroscopy: an analysis of learning curves and predictors of performance. BMC surgery. 2016;16:14. doi: 10.1186/s12893-016-0129-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Cannon WD, Garrett WE, Hunter RE, et al. Improving residency training in arthroscopic knee surgery with use of a virtual-reality simulator. A randomized blinded study. The Journal of bone and joint surgery. American volume. 2014;96(21):1798–1806. doi: 10.2106/JBJS.N.00058. [DOI] [PubMed] [Google Scholar]
- 10.Alvand A, Auplish S, Khan T, Gill HS, Rees JL. Identifying orthopaedic surgeons of the future: the inability of some medical students to achieve competence in basic arthroscopic tasks despite training: a randomised study. The Journal of bone and joint surgery. British volume. 2011;93(12):1586–1591. doi: 10.1302/0301-620X.93B12.27946. [DOI] [PubMed] [Google Scholar]
- 11.Gomoll AH, O’Toole RV, Czarnecki J, Warner JJ. Surgical experience correlates with performance on a virtual reality simulator for shoulder arthros-copy. The American journal of sports medicine. 2007;35(6):883–888. doi: 10.1177/0363546506296521. [DOI] [PubMed] [Google Scholar]
- 12.Grantcharov TP, Bardram L, Funch-Jensen P, Rosenberg J. Impact of hand dominance, gender, and experience with computer games on performance in virtual reality laparoscopy. Surgical endoscopy. 2003;17(7):1082–1085. doi: 10.1007/s00464-002-9176-0. [DOI] [PubMed] [Google Scholar]
- 13.Shane MD, Pettitt BJ, Morgenthal CB, Smith CD. Should surgical novices trade their retractors for joysticks? Videogame experience decreases the time needed to acquire surgical skills. Surgical endoscopy. 2008;22(5):1294–1297. doi: 10.1007/s00464-007-9614-0. [DOI] [PubMed] [Google Scholar]
- 14.Rosser JC, Lynch PJ, Cuddihy L, Gentile DA, Klonsky J, Merrell R. The impact of video games on training surgeons in the 21st century. Archives of surgery (Chicago, Ill. : 1960) 2007;142(2):181–186. doi: 10.1001/archsurg.142.2.181. discusssion 186. [DOI] [PubMed] [Google Scholar]
- 15.Badurdeen S, Abdul-Samad O, Story G, Wilson C, Down S, Harris A. Nintendo Wii video-gaming ability predicts laparoscopic skill. Surgical endoscopy. 2010;24(8):1824–1828. doi: 10.1007/s00464-009-0862-z. [DOI] [PubMed] [Google Scholar]
- 16.Schlickum MK, Hedman L, Enochsson L, Kjellin A, Fellander-Tsai L. Systematic video game training in surgical novices improves performance in virtual reality endoscopic surgical simulators: a prospective randomized study. World journal of surgery. 2009;33(11):2360–2367. doi: 10.1007/s00268-009-0151-y. [DOI] [PubMed] [Google Scholar]
- 17.Kolga Schlickum M, Hedman L, Enochsson L, Kjellin A, Fellander-Tsai L. Transfer of systematic computer game training in surgical novices on performance in virtual reality image guided surgical simulators. Studies in health technology and informatics. 2008;132:210–215. [PubMed] [Google Scholar]