SURGICAL SIMULATION TRAINING REDUCES INTRAOPERATIVE CATARACT SURGERY COMPLICATIONS AMONG RESIDENTS

Abstract

Introduction

This retrospective consecutive case series examined whether training on a surgical simulator reduces intraoperative complication rates among novice ophthalmology residents learning cataract surgery.

Methods

Beginning July 2014, training on the Eyesi simulator became mandatory for novice PGY-3 ophthalmology residents prior to live cataract surgery at our institution. Complication rates of the 11 simulator-trained residents (study group) were compared to their immediate 11 simulator-naïve predecessors (comparison group). Only straightforward cataract cases (according to standardized preoperative criteria) where PGY-3 residents served as the primary surgeon were included. Complication data were obtained from Morbidity & Mortality records and compared using Fisher Exact test. A survey was administered to the residents to gauge the perceived utility of simulation training.

Results

The simulator-trained group (n=501 cataract cases) and the simulator-naive comparison group (n= 454 cases) were analyzed. The complication rate in the simulator group was 2.4% compared to 5.1% in the comparison group (p=0.037, Fisher Exact test). Both the mean posterior capsule tear (PCT) rate and vitreous prolapse rate in the simulator group were 2.2% compared to 4.8% in the comparison group (p=0.032, Fisher Exact test). The survey had a response rate of 100% (11/11), and 91% (10/11) of respondents felt the training was “extremely worthwhile” and should be mandatory.

Conclusions

The addition of surgical simulation training was associated with a significantly reduced rate of complications, including PCTs and vitreous prolapse, among novice PGY-3 residents. There is a perceived utility among residents to incorporate virtual simulation into surgical training.

Introduction

Almost 2 million cataract surgeries are performed in the United States each year, making it the most frequently performed ophthalmic surgery.¹ Outcomes are generally positive,² however less experienced surgeons tend to have higher complication rates, ranging from 3.8%³ to 10.2%.⁴ The most common intraoperative complications among residents include posterior capsular tears (PCTs) at a rate of 3.4%⁵ – 9.6%⁴, as well as vitreous prolapse (1.8%⁶ – 10.2%⁴). In fact, resident-performed cataract surgery has been identified as an independent risk factor for PCTs.⁷

Cataract surgery requires mastery of complex visuospatial techniques with a substantial learning curve. Residency programs have recently begun utilizing virtual reality in addition to formal lectures, wet-lab, and live surgical experiences to train their residents.⁸. One of the leading devices is the Eyesi surgical simulator (VRMagic Holding AG, Mannheim, Germany), composed of a computer system linking a mannequin head with a virtual eye, two foot pedals (controlling microscope and phacoemulsification machine), and an operating microscope providing a 3D stereoscopic image.⁹ Construct validity has been demonstrated in multiple studies.^{10,11,12,13,14,15} Researchers have begun to examine the effect of simulation training on surgical complications, with several concluding that simulation training does not significantly alter complication rates among residents.^16,17

There are a few studies that support decreased complications but have inherent limitations such as small sample size, lack of internal control groups, and using proxies such as errant curvilinear capsulorhexis rather than directly measuring surgical complications.^18,19 Based on our clinical observations, we hypothesize that simulation training performed prior to first cataract surgery lowers complication rates during the initial cataract rotation in the second year of ophthalmology residency for novice postgraduate year 3 (PGY-3) residents.

Methods

This retrospective consecutive case series was approved by the Miami Veterans Affairs Medical Center (VAMC) Institutional Review Board (IRB) and was compliant with the Health Insurance Portability and Accountability Act. The IRB granted waiver of informed consent for this retrospective study. This study was conducted at a single hospital setting, namely the Miami VAMC, where the Bascom Palmer Eye Institute (Miami, FL) residents are introduced to cataract surgery during their second year of ophthalmology training as PGY-3 residents. Prior to July 2014, pre-surgical training consisted of didactic sessions on phacoemulsification techniques conducted immediately prior to the first cataract rotation and a wet lab at the start of each academic year. Beginning July 2014, the pre-surgical curriculum was augmented by mandatory virtual training on the Eyesi surgical simulator just prior to the first cataract surgery rotation at the Miami VAMC. All PGY-3 ophthalmology residents, who never performed cataract surgery before, are now required to complete CAT-A (Introduction to Microsurgery) and CAT-B (Introduction to Cataract Surgery) modules. CAT-A emphasizes safe instrument handling with extensive anti-tremor training and bimanual navigation, as well as introduces beginner capsulorhexis work and phacoemulsification concepts. CAT-B provides more advanced capsulorhexis work and phacoemulsification concepts, including divide-and-conquer and chopping techniques. Each resident must complete the same modules on a personal account. The software requires a passing score on standard built-in milestones before allowing the trainee to advance to the next step. Once all milestones are passed, a completion certificate is generated. All novice residents are then cleared to begin live surgery, having met the required simulator benchmarks.

PGY-3 ophthalmology residents then perform 30–50 live cataract surgeries as a primary surgeon. The residents are allowed to perform all steps of the procedure from the first case, while the teaching surgeon intervenes as needed to correct surgical steps. The teaching surgeon will routinely switch back with the resident once the correction has been made, allowing the resident to complete the case. Three surgical preceptors, each with a decade of teaching experience at the Miami VA, staff all of the cataract extractions utilizing their standard teaching style. The distribution of cases was similar over the study period, with instructor AKJ typically staffing 6 out of 8 cases per week, while instructors AG and NG each staff 1 case per week.

The surgical cases are selected by the same attending physicians in clinic (RG and BEG) according to well-established criteria to include only surgically straightforward cases. Cases with a pupil ≤ 6 mm, high myopia >6 D, presence of pseudoexfoliation, phacodonesis, extremely dense 4+ nucleus, monocular patients, history of trauma, intravitreal injections, or vitreoretinal surgery are strictly excluded. All patients are consented to have surgery performed by resident-attending teams as part of their routine clinical care. Residents’ cases are booked into a standard operating room schedule. The same operating room, scrub technician, and INFINITI phacoemulsification machine (Alcon, Fort Worth, TX) are utilized. All cases are performed with the divide-and-conquer technique, in which the nucleus of the lens is divided into 4 quadrants.²⁰ There were no changes in the surgical technique, preoperative process, equipment, surgical preceptors, or teaching technique during the study period.

All PGY-3 resident-performed phacoemulsifications were included between December 3, 2012 and January 31, 2016. Surgical cases were excluded if they were not phacoemulsifications, not performed during the study period, or if the primary surgeon was not a second-year resident on their first cataract surgery rotation. The comparison group consisted of the 11 PGY-3 residents trained just prior to the simulator acquisition, spanning 19 months from 12/3/12 - 6/30/14. The study group consisted of the 11 PGY-3 residents trained since installation of the simulator device, spanning 19 months from 7/1/14 - 1/31/16.

One instructor (AKJ) actively collects surgical complication data from the postoperative reports, and the three instructors conduct Morbidity and Mortality meetings every 6 weeks. The following intraoperative and postoperative (within 30 days after surgery) complications were derived from the Morbidity and Mortality data: any complication, posterior capsule tear, vitreous prolapse, retained lens fragment, zonular dehiscence, endophthalmitis, intraocular lens dislocation, and return to operating room within 30 days. Any complication meant that one case could have multiple complication types, but the case would only be counted once.

In order to elicit the residents’ impression of the surgical simulator, residents who completed simulation training were asked to complete a questionnaire about their experience, including rating how closely the training modules resembled live surgery (scale of 1–5; 1= “not at all”; 2= “poorly”; 3= “moderately”; 4= “well”; 5= “extremely well”), whether they felt prepared for live surgery (scale of 1–4; 1= “not at all”; 2= “somewhat”; 3= “well prepared”; 4= “extremely well prepared”), and whether the training modules were a worthwhile part of pre-surgical training (scale 1–3; 1= “not at all”; 2= “slightly worthwhile”; 3= “extremely worthwhile”).

Statistical Analysis

A Fisher Exact test was used for statistical comparison with p-value significance set to p<0.05. This test was chosen because the complication rates do not follow a normal distribution, are relatively rare, and are divided between two categorical variables (simulator and non-simulator). A two sample t-test was also used to confirm the pooled analysis of the Fisher Exact test by comparing complication rates averaged over each study group. Randomization of residents was not possible given the retrospective nature of the data and ethical considerations.

Results

Eleven residents with simulation training performed a total of 501 phacoemulsification cases, and the 11 non-simulator trained comparison group residents performed a total of 454 cases. The simulator group performed an average of 41.3 surgeries per resident (SD=8.6, range: 27–52), while the comparison group also performed an average of 41.3 surgeries (SD=4.9, range: 39–52). The simulator group had an average complication number of 1.1±0.7 (range, 0–2) per resident, while the comparison group had an average complication number of 2.1±0.9 (range, 0–3) per resident. Any complication rate in the simulator group was 2.4%, significantly lower than the 5.1% any complication rate in the comparison group (p=0.037, Fisher Exact Test). Significantly lower rates compared to comparison group were also found with posterior capsule tears (2.2% versus 4.8%; p=0.032, Fisher) and vitreous prolapse (2.2% versus 4.8%; p=0.032, Fisher). No significant difference was found in other complications including retained lens fragments, zonular dehiscence, endophthalmitis, intraocular lens dislocation, and return to operating room within 30 days (Table 1). A two sample t-test was used to compare the complication rates averaged over the eleven residents in each group, and this analysis supported the pooled analysis of rates by the Fisher Exact test. Any complication, PCTs, and vitreous prolapse once again achieved significance (p=0.007, 0.007, 0.005, respectively). A scatterplot comparing the number of surgeries with one or more complications versus total surgeries performed per resident did not qualitatively demonstrate that any individual resident is an outlier compared to their group (Figure 1).

Table 1.

Comparison of Intraoperative and Early Post-operative Complications in the Simulator and Comparison Groups

Total	Comparison Group		Simulator Group		Fisher Exact P-value
	N* = 454		N* = 501
	n	Rate	n	Rate
Any Complication*	23	5.1%	12	2.4%	0.037
Posterior Capsular Tears	22	4.8%	11	2.2%	0.032
Vitreous Prolapse	22	4.8%	11	2.2%	0.032
Retained Lens Fragment	9	2.0%	4	0.8%	0.162
Zonular Dehiscence	3	0.7%	0	0.0%	0.107
Endophthalmitis	1	0.2%	1	0.2%	>0.999
IOL Dislocation	0	0.0%	0.0	0.0%	n/a
Return to OR 30 days	1	0.2%	0.0	0.0%	0.475

The simulator-trained group had statistically significant reductions in any complication, posterior capsular tears, and vitreous prolapse rates.

^*Any complication = number of surgeries with one or more complications; *N = total number of cataract surgeries performed by the group of residents

Figure 1. Total number of surgeries with one or more complications versus total number of cataract surgeries performed by each resident.

The total number of surgeries with one or more complications for each resident was plotted against the total number of cataract surgeries performed by each resident in the simulation and comparison groups. The lower-right quadrant represents lower complication rates, while the upper-left quadrant represents higher complication rates. The simulator group is concentrated in the lower-right quandrant and the comparison group is uniformly distributed across the upper quandrants. In the simulator group, 18% had 0 complications, 55 % had 1 complication, 27% had 2 complications, and none had 3 complications. While in the comparison group, 9%, 9%, 45%, 36% had 0, 1, 2, and 3 complications respectively. 100% of the simulator group had 0–2 complications, while 81% of the comparison group had 2–3 complications. Qualitatively, no individual outliers can be appreciated.

The survey had a response rate of 100% (11/11), and showed 91% (10/11) of the residents felt the training was “extremely worthwhile” and 91% (10/11) believed it should be mandatory. In regards to how closely the modules resembled live surgery (scale of 1–5, with 5=”extremely well”) the residents rated the capsulorhexis module 3.45, sculpt experience 3.18, and quadrant removal 3.1. This resulted in an average score of 3.25 (“resembles well to moderately-well”). Following simulation training, 63% (7/11) of the residents felt “somewhat prepared” and 36% (4/11) felt “well-prepared” for live surgery.

Discussion

Our results are noteworthy in that they represent the first study, to our knowledge, to find a significant reduction in live cataract surgery complications associated with the addition of surgical simulation training. Any complication, PCT, and vitreous prolapse rates were lower after simulation training (p=0.037, p=0.032, p=0.032, respectively). PCTs and vitreous prolapse drive the significance of the any complication analysis and have been reported to be the most common complications in resident-performed cataract surgey.^5,22,23 The other complication types occurred at rates too small to demonstrate a difference between groups. Table 2 illustrates how our results compare to the current literature. Our measured complication rates, especially in the simulator-trained group, are noticeably lower than the rates reported for resident-performed cataract surgeries.

Table 2.

Complication Rates in the Simulator and Comparison Groups Compared to the Literature

Complication Type	Simulator	Comparison	Literature (all physicians)	Literature (residents only)
Any Complication*	2.4%	5.1%	1–10%	3.8%–10.2%3, 4
Posterior Capsular Tear	2.2%	4.8%	1.6–9%23	3.4–9.6%4, 5
Vitreous Prolapse	2.2%	4.8%	1.1% 22	1.8–10.2% 4,6
Endophthalmitis	0.2%	0.2%	0.04–0.49%24	0.11%25
Zonular dehiscence	0.0%	0.7%	0.46%26	2.0%27
Retained lens fragment	0.8%	2.0%	0.18–0.28%2	0.7–1.0%28, 29
Dislocated IOL	0.0%	0.0%	0.1–0.13%30	0.39%29
Return to OR 30 days	0.0%	0.2%	N/A*	2.11%29

Complication rates for simulator and comparison groups are compared with complication rates for all physicians and for residents-only as reported in the literature

^*Any complication = number of surgeries with one or more complications; *N/A = Not available; “Return to OR” is highly dependent on whether a retina specialist is available to assist immediately, therefore it is not commonly reported; IOL = Intraocular lens; OR = operating room

The studies that support decreased complications after simulation training do so indirectly. For example, Bergqvist et al. (2014) and McCannel et al. (2013) found decreased “virtual” complications and decreased errant continuous curvilinear capsulorhexis (respectively), but did not collect data from live surgeries.^{19, 21} An important caveat, noted by Feudener et al. (2009), is that the simulator works best as an adjunct to a pre-established curriculum. They showed that residents who received adjunctive Eyesi training performed better in wet-lab.⁸ Similarly, Baxter et al. (2013) showed that using the Eyesi as an adjunct to a pre-established curriculum yielded lower complication rates in live surgery, however the study lacked sample size and a comparison group.¹⁸ Meanwhile, Belyea et al. (2011) and Pokroy et al. (2013) showed that Eyesi training did not decrease complication rates in live surgery compared to a simulator-naive comparison group (p=0.44, p=0.63, respectively).^16,17 This could have been due to a relatively smaller sample size,¹⁶ and lenient exclusion criteria for advanced cataract cases.^16,17 Our study addresses each of the above concerns, tracking complications in 955 carefully selected, straightforward phacoemulsification cases performed by 11 new-to-cataract-surgery residents before and after simulator acquisition, while utilizing the simulator as an adjunctive training modality.

In regards to the survey, the majority of ophthalmology residents felt their exposure to the simulator was beneficial. While virtual simulation alone was not sufficient to feel prepared for surgery (only 36% felt “well-prepared” for live surgery), the majority believed that simulation training was “extremely worthwhile,” represented live surgery “well” to “moderately well,” and 91% believed it should be a “mandatory” part of the pre-surgical curriculum.

Some inherent limitations of the study should be addressed. Randomization of the residents was not possible due to the retrospective nature of the data, ethical considerations of withholding available training tools, and hospital policy that requires surgical simulation prior to live cataract surgery. Nonetheless, assigning residents based on when they trained in relation to Eyesi acquisition and balancing the groups with an equal number of residents is congruent with the literature; this method was also used by Belyea et al. (2011) and Pokroy et al. (2013).^16,17 Another expected criticism of longitudinal studies is that operating room factors may have changed during the study period. However, the strength of our study is that the same hospital setting and the same three highly experienced instructors (staffing surgeries in the same proportion throughout the study period) were utilized for both groups. No other changes occurred during the study period. In addition, the same attending physicians evaluated each cataract surgery candidate in clinic to ensure that all cases met the standardized criteria for the PGY-3 resident curriculum in order to have uniform case difficulty across both groups. Notably, there was no change in the wet lab or didactic aspects of the training during the study period, including the number of lectures or the instructors. The study was not conceptualized until January 2016, thus all data were collected retrospectively without introducing a bias to the surgical cases and instructors’ teaching approaches.

It is possible that additional training, including more time in the wet lab, may have similarly affected complication rates. While that would be interesting to examine in the future, this study was not specifically designed to compare simulation training with additional wet lab training. Rather, our design attempted to reflect the current nationwide trend of ophthalmology residency programs adopting virtual simulation as an adjunct to their curriculums, and how this addition will affect complications. More importantly, our study design was based on previous literature published on the topic,^16,17 and yet we ultimately reached a different conclusion by being the first to show decreased live surgery complications.

In our opinion, training on a virtual simulator is remarkably realistic. The ability to perform surgical steps countless times without additional cost per attempt makes this a unique training opportunity. While there is a high initial investment, there is no further cost (software updates are free), making this modality cost-effective in the long term. An additional advantage of the simulator is that skills can be measured and evaluated in a standardized fashion, which is not possible in a traditional wet-lab. For example, trainees must practice until certain milestones are reached and only then are they allowed to advance to the next step. While this may take some residents more time than others, what is important is that they pass the required modules before being allowed to enter the operating room. This ensures a uniform training experience.

Meanwhile, wet labs offer less uniformity and are unfavorable due to high recurring costs that prevent how many times residents can practice surgical steps. In addition, the various modalities are less than ideal - pig eyes have thick, flexible lens capsules and deep, soft lens nuclei which make capsulorhexis and quadrant removal difficult to teach; human eyes provided by the eye bank often have cloudy corneas that are only suitable for wound construction and suturing.

It should be noted that our study focused on intraoperative and immediate postoperative complications, as these are most dependent on surgical technique and therefore most likely related to simulation training. Unless occurring shortly after surgery, the Morbidity and Mortality committee does not collect data on retinal detachments, posterior capsular opacifications, cystic macular edema, and bullous keratopathy; thus, they were not included in this study. Other performance measures such as phacoemulsification time and power used, surgical time, or amount of balanced salt solution were not available for analysis as they are not routinely collected. Since patient selection was based on comorbidities affecting surgical challenges but not visual acuity prognosis, visual acuity outcomes between the groups were not analyzed.

Another limitation is the fact that each individual resident enters training with a different skill set that will influence surgical performance, a bias that cannot be corrected for in this type of study. However, no outliers in terms of complication rate were identified by scatterplot. Therefore, it is unlikely that an individual resident’s complication rate drove the difference in rate between the groups. Future research may address intragroup variability, possibly by using performance scores on the simulator to ascertain baseline skill. This could be achieved by recording scores on the initial attempts at simulator modules as well as the total time or number of attempts needed to achieve first passing score on a module. More stringent metrics could help address whether simulator performance correlates with complication rates or other surgical endpoints such as best-corrected visual acuity.

In conclusion, surgical simulation training prior to the first live cataract surgery was associated with a significantly reduced rate of intraoperative complications, especially PCT’s and vitreous prolapse, among novice ophthalmology residents in this study. In addition, there was a perceived utility and desire among residents to incorporate surgical simulation into clinical training.