Abstract
Hip arthroscopy (HA) is technically demanding and associated with a prolonged learning curve. Recently, arthroscopic simulators have been developed to anatomically model various joints including the knee, shoulder and hip. The purpose of this study is to validate a novel HA simulator. Twenty trainees and one sports medicine fellowship-trained orthopaedic surgeon at a single academic institution were recruited to perform a diagnostic HA procedure using the VirtaMed ArthroS hip simulator. Trainee characteristics, including level of training, general arthroscopy experience and hip specific arthroscopy experience, were gathered via questionnaire. For the purpose of this study, participants were categorized as novice (<25), intermediate (25–74) or experienced (≥75) based on the number of prior arthroscopies performed. Various performance metrics, including composite score, time and camera path length were recorded for each attempt. Metrics were analyzed categorically using ANOVA tests with significance set to P < 0.05. Composite performance score in the novice cohort was 114.5 compared with 146.4 and 151.5 in the intermediate and experienced cohorts (P = 0.0019), respectively. Novice arthroscopists performed the simulated diagnostic arthroscopy procedure in an average time of 321 s compared with 202 s and 181 s in the intermediate and experienced cohorts (P < 0.002), respectively. Cartilage damage and simulator safety score did not differ significantly between groups (P = 0.775). Simulator composite score and procedure time showed strong correlation with year of training (r = 0.65 and −0.70, respectively) and number of arthroscopies performed (r = 0.65 and −0.72). The ArthroS hip simulator shows good construct validity and performance correlates highly with total number of arthroscopic cases reported during training.
INTRODUCTION
Arthroscopic surgery is one of the most rapidly growing areas within orthopaedics, with surgical utilization reflecting this growth [1–3]. Hip arthroscopy (HA) has most recently seen a marked increase in utilization. HA offers surgeon a minimally invasive approach for the diagnosis and treatment of a myriad of hip pathology, including femoroacetabular impingement syndrome. Despite the proposed advantages of the minimally invasive technique, HA is technically challenging and associated with a prolonged learning curve before excellent outcomes and minimal morbidity can be consistently achieved [4].
Given the technical challenges associated with HA, there has been increasing focus on how to best prepare orthopaedic trainees interested in HA [5, 6]. Recent changes in resident work hour restrictions in the United States have resulted in a perceived decline in operative experience [7–9]. This is particularly concerning when considering technically demanding procedures with a prolonged learning curve, such as HA. To this point, Mehta et al. found that surgeons needed to perform at least 388 HA for their patients to have ≤10% chance of requiring subsequent hip surgery within 5 years of the index HA [10]. However, Westermann et al. found that residents in the United States graduate with, on average, only five HA surgical experiences during their 5 years of training (podium presentation at International Society for Hip Arthroscopy, 2018). As a result, arthroscopic simulators have been increasingly utilized and studied as an alternative modality to aid trainees in recent years given growing concerns with limited operative experience for these technically demanding procedures [11–13]. Virtual reality (VR)-based simulators provide several advantages for trainees, including a risk-free environment and time flexibility for skill development. Prior studies have validated the use of simulators for both knee and shoulder arthroscopy skill acquisition along with transfer validity from a simulated environment to the operating room [14–16]. However, there has been a lack of evidence regarding the validity of simulated HA and its potential use for skill development [17, 18].
The purpose of this study was to establish the construct validity of a novel VR simulation system, the VirtaMed ArthroS for HA by comparing various parameters between groups of novice, intermediate and experienced arthroscopists. We hypothesized that more experienced arthroscopists would outperform less experienced arthroscopists for all measured metrics, thus supporting the novel simulator’s construct validity.
MATERIALS AND METHODS
Sixteen orthopaedic surgery residents, four senior medical students and one orthopaedic sports medicine fellowship-trained attending at a single academic tertiary institution participated in this prospective study in June 2018. For the purpose of this study, three cohorts were identified a priori by level of experience with respect to prior arthroscopy volume. The three cohorts were stratified as novice (<25 arthroscopic procedures performed), intermediate (25–74 arthroscopic procedures performed) and experienced (≥75 arthroscopic procedures performed). The novice cohort was comprised of four post-graduate year 1 (PGY 1) orthopaedic surgery residents and four medical students interested in orthopaedic surgery. The intermediate group comprised of eight junior residents (PGY 2/3) and one senior resident (PGY 4/5). The experienced cohort comprised of three senior residents and one fellowship-trained orthopaedic sports medicine attending. Participants were excluded if they had previous experience with the VR HA simulator. Demographic data for study participants are shown in Table I. Subjects filled out a questionnaire prior to participating in the surgical tasks, including hand dominance, sub-specialty interest, total number of previous arthroscopic procedures performed and total number of previous HA procedures performed. In addition to categorizing trainees based on number of procedures performed, participants were also categorized level of training (PGY 1 through 5, fellow, staff) for Pearson correlation analysis.
Table I.
Participant demographics
| Participant characteristics | Novice (<25 arthroscopies) | Intermediate (25–74 arthroscopies) | Experienced (>74 arthroscopies) |
|---|---|---|---|
| Participants (n) | 8 | 9 | 4 |
| Mean number of arthroscopies (n, ±SD) | 0.25 ± 0.66 | 36 ± 9.6 | 395 ± 465.64 |
| Mean number of hip arthroscopies (n, ±SD) | 0 | 4 ± 5.68 | 52 ± 85.74 |
| Handedness (right:left) | 7:1 | 7:2 | 4:0 |
| Year in traininga | 4:4 (MS4: PGY-1) | 8:1 (PGY2/3: PGY4/5) | 3:1 (PGY4/5: Prof.) |
MS4, fourth-year medical student; PGY-1, orthopaedic intern; PGY2/3, junior orthopaedic resident; PGY4/5, senior orthopaedic resident; Prof., fellowship-trained sports medicine surgeon.
SD, standard deviation.
Simulation testing was performed in a supervised environment using the ArthroS Hip module by VirtaMed (Zurich, Switzerland). A standardized introduction and orientation to the simulator was given to all participants by the same individual not affiliated with the current study. This simulation exercise used pre-established mid-anterior and anterolateral portals. Five anatomic structures were visualized through the anterolateral portal and six through an anterior portal (Table II). Composite score, procedure time, camera path length, iatrogenic femoral head cartilage injury, iatrogenic acetabular cartilage injury and number of fluoroscopic images taken were recorded by the simulator software. Composite score was calculated using a proprietary scoring formula by VirtaMed.
Table II.
Anatomic objectives for diagnostic HA simulation
| Anterolateral portal | Anterior portal |
|---|---|
| Acetabular fossa | Acetabular fossa |
| Ligamentum teres | Ligamentum teres |
| Posterior medial acetabulum and labrum | Posterior transverse ligament |
| Anterior acetabulum and labrum | Anterior transverse ligament |
| Anterolateral acetabulum and labrum | Superior articular cartilage |
| Lateral labrum |
Statistical analysis was performed using Microsoft Excel (Redmond, WA, USA). Descriptive statistics included participant factors such as level of experience with arthroscopy, hand dominance and level of training. Categorical analysis using the one-way ANOVA test was used to compare composite performance score, time to simulation completion, camera path length and safety score between the three cohorts, with statistical significance set at P < 0.05.
RESULTS
On an average, the novice cohort performed 0.25 arthroscopic procedures (standard deviation [SD]: 0.66, range: 0–2), the intermediate cohort 36 (SD: 9.6, range: 28–50) and the experienced cohort performed 395 (SD: 465.64, range 80–1200) prior to participation in the study. On an average, the intermediate cohort performed four hip arthroscopies (SD: 5.68, range: 0–15) and the experienced cohort performed 52 hip arthroscopies (SD: 85.74, range: 1–200), whereas the novice cohort did not have any prior experience with HA.
Composite performance score was significantly lower in the novice cohort (114.5 ± 14.91) compared with the intermediate (146.4 ± 17.17) and experienced cohort (151.5 ± 17.18) (P < 0.002). Novice arthroscopists performed the module in an average time of 321 s (SD: 67.04) compared with 202 s (SD: 36.26) and 181 s (SD: 24.52) in the intermediate and experienced cohort, respectively (P < 0.002). Camera path length did not differ significantly between groups with an average of 202 ± 42.2 cm, 172 ± 74.5 cm, and 147 ± 59.4 cm in the novice, intermediate and experienced cohorts (P = 0.3804), respectively. Additionally, cartilage damage and safety score did not differ significantly between groups with a mean safety score of 78.75 ± 10.44, 82.11 ± 9.81 and 82 ± 6.52 in the novice, intermediate and experienced cohorts (P = 0.775), respectively (Table III).
Table III.
Comparison of objective measures of hip arthroscopy simulator
| Novice (<25 arthroscopies) | Intermediate (25–74 arthroscopies) | Experienced (>74 arthroscopies) | P-value | |
|---|---|---|---|---|
| Composite performance score (score, ±SD) | 114.5 ± 14.91 | 146.44 ± 17.17 | 151.5 ± 17.18 | P < 0.002 |
| Procedure time (s, ±SD) | 321.75 ± 67 | 202.44 ± 36.3 | 181.69 ± 24.5 | P < 0.002 |
| Camera path length (cm, ±SD) | 202.63 ± 42.21 | 172.26 ± 74.51 | 147.36 ± 59.41 | P = 0.3804 |
| Safety score (score, ±SD) | 78.75 ± 10.44 | 82.11 ± 9.81 | 82 ± 6.52 | P = 0.7749 |
Significant findings are indicated in bold (P < 0.05).
Pearson correlation coefficients were utilized to assess for significant correlations between different testing variables (Table IV). Simulator composite score and time to task completion showed strong correlation with year of training (r = 0.65 and −0.70, respectively). With respect to prior arthroscopic experience (novice, intermediate or experienced), there was a strong correlation with total score (r = 0.65) and time (r = −0.72), but a poor correlation with path length (r = −0.32) and acetabulum or femoral head cartilage damage (r = 0.01 and 0.18, respectively).
Table IV.
Pearson’s correlation (r) between participant characteristics and simulator metrics
| Participant characteristics | Total score | Procedure time | Camera path length | Safety score | Scratching of the femoral cartilage | Scratching of acetabulum |
|---|---|---|---|---|---|---|
| HA | 0.3815 | −0.2759 | −0.3918 | 0.2075 | −0.1178 | −0.0813 |
| Total arthroscopy | 0.4008 | −0.3296 | −0.3857 | 0.1775 | −0.1052 | −0.0468 |
| Year of training | 0.6496 | −0.7036 | −0.3806 | 0.1490 | −0.2372 | 0.1444 |
| Simulator use | 0.4604 | −0.4776 | −0.3037 | 0.1608 | −0.1052 | 0.2420 |
| Experience (novice, intermediate, experienced) | 0.6497 | −0.7220 | −0.3185 | 0.1430 | 0.1778 | 0.0141 |
DISCUSSION
Virtual reality HA simulators provide a risk-free environment for orthopaedic trainees to improve their skills for a technically demanding surgical procedure. In this study, we found that the VirtaMed ArthoS HA simulator performance correlates highly with previous arthroscopic experience and level of training, with composite performance score, procedure time and camera path length showing the highest correlation with prior arthroscopic experience. However, procedural safety, including iatrogenic femoral head and acetabular cartilage injury, did not differ between the various skill levels.
Currently, there is a paucity of data on HA simulators. A 2018 cross-sectional study by Erturan et al. recruited 52 participants to perform a standardized bench-top simulated HA task using a HA simulator (Sawbones Europe) [19]. Participants were divided into expert (four fellowship-trained staff), trainee (28 residents and fellows) or novice (20 interns and medical students). The study noted a significant difference between the arthroscopic ability of all groups when analyzing performance based on time and motion analysis as well as the Basic Arthroscopic Knee Skill Scoring System (BAKSSS) Global Rating Scale [20]. Recently, Bartlett et al. assessed the face validity of a VR HA simulator, the Simbionix ArthoMentor [21] (Littleton, CO, USA). They recruited 18 surgical residents and 7 faculty hip arthroscopists from a HA training course without previous experience using VR simulators. All participants in their study performed a basic diagnostic HA locating 12 anatomical targets within the joint. Upon completion, they completed a questionnaire using a 10-point Likert scale evaluating the simulator on realism as well as its use in a learning environment. The simulator used in that study was found to have a high degree of realism with respect to visual representation, instrumentation and procedure itself. However, less than 50% reported tactile feedback received from the soft tissues to be realistic. They found that 84% of participants reported the simulator to be a useful tool for intern level trainees, 88% reported useful for resident level and 80% said it would be useful for attending level hip arthroscopists.
The present study showed a greater difference in nearly all outcome measures except iatrogenic cartilage injury when comparing novice to intermediate skill cohorts rather than intermediate to experienced cohorts. The findings of the study may suggest that participants in the intermediate cohort performed comparably to those in the experienced arthroscopists cohort. Therefore, the findings of this study should be understood in the context of total arthroscopic experience as opposed to HA experience. Intuitively, simulators are likely to provide the most benefit for novice or intermediate trainees as opposed to experienced, fellowship-trained hip arthroscopists. We found that composite performance score and time to complete simulation correlated strongly with experience. This is in agreement with a recent Level II randomized trial using an arthroscopic knee simulator which showed shorter time to completion and a trend toward improved performance with training compared with an untrained cohort [22].
Previous literature has shown training on VR simulators to translate into improved arthroscopic proficiency on both cadaveric specimens as well as in vivo. In 2015, Rebolledo et al. showed that training junior orthopaedic residents with knee and shoulder arthroscopic simulators resulted in improved performance in cadaveric models when compared with those that received didactic training only [23]. Watermann et al. demonstrated significant improvement in task time, probe distance and ASSET score in vivo in a cohort of orthopaedic trainees who underwent training using a VR shoulder simulator compared with a control group [24]. The present study corroborates these findings; the Arthro-S simulator, especially when using the composite score, may be useful for evaluating the progress of mid-level trainees interested in learning HA. However, further research on how improvements in these measured and scored tasks translated to an in vivo operative scenario are warranted.
LIMITATIONS
There are several limitations to this study. First, there is no control group in this study, and therefore, we are unable to speculate whether or not the simulator training is superior to other modalities, including didactic or cadaver-based training. In addition, the number of previous total arthroscopic procedures and hip arthroscopies were obtained using a questionnaire, relying on accurate procedure reporting prior to enrolment in the study. The simulated procedure was not comprised of certain key components of in vivo HA such as hip distraction, bleeding/visibility issues, joint access or portal establishment. Lastly, the study included one fellowship-trained hip arthroscopists. Therefore, conclusions regarding construct validity of the VirtaMed ArthroS hip simulator in relation to experience with HA could not be established in this study. Therefore, the findings of this study should be understood in the context of total arthroscopic experience as opposed to HA experience.
CONCLUSIONS
The ArthroS hip simulator shows good construct validity and performance correlates highly with total number of arthroscopic cases reported during training. Certain metrics, such as simulated iatrogenic cartilage injury, cannot distinguish between different levels of surgical experience. The ArthoS composite score may be a useful metric to assess progress for trainees learning HA.
ACKNOWLEDGEMENTS
We acknowledge Catherine Fruehling for her help with setting the simulation laboratory for this study.
FUNDING
No outside financial funding was utilized for this study.
CONFLICT OF INTEREST STATEMENT
Dr. Karam has stocks or stock options with Iowa Simulation Solutions LLC. Dr. Westermann receives research support from Smith and Nephew. Dr. Willey receives research support from Biomet. All other authors declare no conflict of interest.
REFERENCES
- 1. Griffiths EJ, Khanduja V.. Hip arthroscopy: evolution, current practice and future developments. Int Orthop 2012; 36: 1115–21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Glick JM, Valone F 3rd, Safran MR.. Hip arthroscopy: from the beginning to the future – an innovator’s perspective. Knee Surg Sports Traumatol Arthrosc 2014; 22: 714–21. [DOI] [PubMed] [Google Scholar]
- 3. Colvin AC, Harrast J, Harner C.. Trends in hip arthroscopy. J Bone Joint Surg Am Vol 2012; 94: e23.. [DOI] [PubMed] [Google Scholar]
- 4. Lee YK, Ha YC, Hwang DS. et al. Learning curve of basic hip arthroscopy technique: cUSUM analysis. Knee Surg Sports Traumatol Arthrosc 2013; 21: 1940–4. [DOI] [PubMed] [Google Scholar]
- 5. Duchman KR, Westermann RW, Glass NA. et al. Who is performing hip arthroscopy?: an analysis of the American Board of Orthopaedic Surgery part-II database. J Bone Joint Surg Am Vol 2017; 99: 2103–9. [DOI] [PubMed] [Google Scholar]
- 6. Buyukdogan K, Utsunomiya H, Bolia I. et al. Right versus left hip arthroscopy for surgeons on the learning curve. Arthrosc Tech 2017; 6: e1837–44. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Mir HR, Cannada LK, Murray JN. et al. Orthopaedic resident and program director opinions of resident duty hours: a national survey. J Bone Joint Surg Am Vol 2011; 93: e1421–29. [DOI] [PubMed] [Google Scholar]
- 8. Peabody T. The effect of work hour restrictions on the education of orthopaedic surgery residents. Clin Orthop Relat Res 2006; 449: 128–33. [DOI] [PubMed] [Google Scholar]
- 9. Peabody T, Nestler S, Marx C. et al. Resident duty-hour restrictions - who are we protecting? AOA critical issues. J Bone Joint Surg Am Vol 2012; 94: e131.. [DOI] [PubMed] [Google Scholar]
- 10. Mehta N, Chamberlin P, Marx RG. et al. Defining the learning curve for hip arthroscopy: a threshold analysis of the volume-outcomes relationship. Am J Sports Med 2018; 46: 1284–93. [DOI] [PubMed] [Google Scholar]
- 11. Karam MD, Pedowitz RA, Natividad H. et al. Current and future use of surgical skills training laboratories in orthopaedic resident education: a national survey. J Bone Joint Surg Am Vol 2013; 95: e4.. [DOI] [PubMed] [Google Scholar]
- 12. Frank RM, Erickson B, Frank JM. et al. Utility of modern arthroscopic simulator training models. Arthroscopy 2014; 30: 121–33. [DOI] [PubMed] [Google Scholar]
- 13. Pedowitz RA, Marsh JL.. Motor skills training in orthopaedic surgery: a paradigm shift toward a simulation-based educational curriculum. J Am Acad Orthop Surg 2012; 20: 407–9. [DOI] [PubMed] [Google Scholar]
- 14. Howells NR, Gill HS, Carr AJ. et al. Transferring simulated arthroscopic skills to the operating theatre: a randomised blinded study. J Bone Joint Surg Br Vol 2008; 90: 494–9. [DOI] [PubMed] [Google Scholar]
- 15. Dunn JC, Belmont PJ, Lanzi J. et al. Arthroscopic shoulder surgical simulation training curriculum: transfer reliability and maintenance of skill over time. J Surg Educ 2015; 72: 1118–23. [DOI] [PubMed] [Google Scholar]
- 16. Cannon WD, Garrett WE Jr, Hunter RE. et al. Improving residency training in arthroscopic knee surgery with use of a virtual-reality simulator. A randomized blinded study. J Bone Joint Surg Am Vol 2014; 96: 1798–806. [DOI] [PubMed] [Google Scholar]
- 17. Pollard TC, Khan T, Price AJ. et al. Simulated hip arthroscopy skills: learning curves with the lateral and supine patient positions: a randomized trial. J Bone Joint Surg Am Vol 2012; 94: e68.. [DOI] [PubMed] [Google Scholar]
- 18. Phillips L, Cheung JJH, Whelan DB. et al. Validation of a dry model for assessing the performance of arthroscopic hip labral repair. Am J Sports Med 2017; 45: 2125–30. [DOI] [PubMed] [Google Scholar]
- 19. Erturan G, Alvand A, Judge A. et al. Prior generic arthroscopic volume correlates with hip arthroscopic proficiency: a simulator study. J Bone Joint Surg Am 2018; 100: e3.. [DOI] [PubMed] [Google Scholar]
- 20. Insel A, Carofino B, Leger R. et al. The development of an objective model to assess arthroscopic performance. J Bone Joint Surg Am Vol 2009; 91: 2287–95. [DOI] [PubMed] [Google Scholar]
- 21. Bartlett JD, Lawrence JE, Khanduja V.. Virtual reality hip arthroscopy simulator demonstrates sufficient face validity. Knee Surg Sports Traumatol Arthrosc 2018; 27: 3162–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Cychosz CC, Tofte JN, Johnson A. et al. Fundamentals of arthroscopic surgery training program improves knee arthroscopy simulator performance in arthroscopic trainees. Arthroscopy 2018; 34: 1543–9. [DOI] [PubMed] [Google Scholar]
- 23. Rebolledo BJ, Hammann-Scala J, Leali A. et al. Arthroscopy skills development with a surgical simulator: a comparative study in orthopaedic surgery residents. Am J Sports Med 2015; 43: 1526–9. [DOI] [PubMed] [Google Scholar]
- 24. Waterman BR, Martin KD, Cameron KL. et al. Simulation training improves surgical proficiency and safety during diagnostic shoulder arthroscopy performed by residents. Orthopedics 2016; 39:e479–485. [DOI] [PubMed] [Google Scholar]