Table 3.
Published reference, context of final assessment, source of assessment, skills assessed post-training, and results from studies
| Citation | Contextual setting for final assessment | Source of final assessment ratings | Skills assessed post training | Results from research studies |
|---|---|---|---|---|
| Aggarwal et al., 2007 | Porcine Model (pre on box trainer) | 2 observers (OSATS global rating and a motion tracking device) | · Change in operative performance: 1) time taken 2) total path length 3) total number of movements · The OSATS global rating scale. |
· Trained group performed significantly better on time (p=.038), total path length (p=.001), total number of movements (p=.009) and overall rating scores (p=.001). · Trained group demonstrated dexterity scores equivalent to expert levels. |
| Ahlberg et al., 2007 | Porcine Model | · The performance with MIST-VR correlated with surgery skills. · MIST-VR did not improve surgical skills · MIST-VR did predict surgical outcomes. |
||
| Ahlberg et al., 2002 | Patients in OR (pre on a simulator) | 2 observers (reliability greater .98) | · Exposure errors, clipping and tissue division errors, and dissection errors · Performance was broken up into phases: 1) exposure of the cystic duct and artery 2) clip placement followed by division of the cystic duct and artery; and 3) gallbladder excision. · Total time, path length, angular path, tissue damage, and max damage |
· Intervention group made significantly fewer errors. · The trained group made significantly fewer objectively assessed, intraoperative errors during the exposure portion of the procedure (p<.04), clipping and tissue division (p<.008), and dissection (p<.03). · The control group made 3 times as many errors and used 58% longer surgical time |
| Andreatta et al., 2006 | Porcine Model | 2 surgeons (.99 reliability) | · 30-degree Camera navigation: 1) Time 2) Accuracy 3) Efficiency of motion 4) Instrumentation use · Eye-hand coordination: two handed transfer of ski needle: 1) Time 2) Accuracy 3) Efficiency of motion 4) Instrument handling · Eye-hand coordination: 0-degree camera navigation and one-handed object transfer: 1) Time 2) Accuracy 3) 0-degree camera navigation skills 4) perceptual ability · Safe placement of clips and application of electrocautery: 1) Clipping 2) Electrocautery performance |
· Intervention group outperformed the control group in: camera navigation skills (p<.05), efficiency of motion (p<.001), optimal instrument handling (p<.001), perceptual ability (p<.001), and performance of safe electrocautery (p<.01). · Time and accuracy ratings on 30-degree navigation (p<.05), and eye-hand coordination two-handed transfer of ski needle (p<.001) was better in the trained group. · Prior training with LapMentor leads to improved resident performance of basic skills in the animate operation room. |
| Banks et al., 2007 | Patients in OR (post only. Pre assess was done on simulator and then the training group performed on the simulator again before being evaluated in the OR) | Observers | · Task specific checklist: assessed 4 categories of skills: 1) preoperative skills 2) surgical technique 3) laparoscopic technique 4) laparoscopic BTL-specific skills · Global rating scale: 1) respires for tissue 2) time and motion 3) instrumental handling 4) knowledge of instruments 5) flow of operation 6) use of assistants 7) knowledge of the specific procedure · Pass/fail |
· Intervention group performed significantly better than control group on all 3 surgical assessment tools (p=.002, checklist; p=.003, global score; p=.003, pass rate; p=.003, posttest) and scored significantly better on the knowledge posttest (p=.009) |
| Bennet et al., 2011 | Patients in OR (post only) | Observers | · Identification of all 4 target numbers and the ability to maintain correct orientation of the camera at each target and to properly position the post at each target for a maximum total score of 12 points. · Max time was 120 seconds. |
· No difference in learning between groups (p=.40). |
| Gala et al., 2013 | Patients in OR | Observers | · Time · Competence levels of participants pre and post intervention · Technical skills for both groups |
· Time the intervention group improved significantly higher (p<.01) · Intervention group was significantly higher with competence levels (p<.01) · The intervention group also had higher technical skills in the operating room (p<.03) |
| Ganai et al., 2007 | Porcine Model (pre and post) | 3 External observers (90%) and from Endo Tower simulator | · 12 structured scope navigation tasks in 3 phases: 1) Navigation within the peritoneal cavity 2) Navigation around the retracted gallbladder 3) Navigation around a suspended small intestinal loop |
· Intervention group was significantly better in object visualization (p<.05), scope orientation (p<.05), and horizon errors (p<.05) |
| Grantcharov et al., 2004 | Patients in OR | 2 Senior surgeons rated 1 surgery (cohen’s kappa .71) | · Economy of movement: 1) Unnecessary movements 2) Confidence of movements · Errors: 1) Respect for tissue 2) Precision of operative technique |
· Intervention group showed greater improvement in error (p=.003) and economy of movement (p=.003). · Intervention group was significantly faster than the control group when performing cholecystectomy (p=.021). |
| Hogleet al., 2009 | Study 1: OR Patients Study 3: Porcine Model (pre and post) | Study 1: Attending surgeon Study 3: Observer | · Study 1 and 3: GOALS rating: 1) Depth perception 2) Bimanual dexterity 3) Efficiency 4) Tissue handling and autonomy |
· Study 1 and 3: No significant differences were found between groups. |
| Hung et al., 2012 | Porcine Model | 3 expert robotic surgeons blinded | · GOALS: 1) Depth perception 2) Bimanual dexterity 3) Efficiency 4) Tissue handling 5) Participant autonomy to accomplish task |
· Groups 1 and 2 were comparable in pre-study surgical experience and had similar baseline scores on simulator and tissue exercises (p > 0.05). · Overall baseline simulator performance significantly correlated with baseline and final tissue performance (p <0.0001) · Simulator training significantly improved tissue performance on key metrics for group 1 subjects with lower baseline tissue scores than their group 2 counterparts (p < 0.05) · Group 1 tended to outperform group 2 on final tissue performance, although the difference was not significant. |
| Korndorffer, et al., 2007 | Porcine Model (pre and post) | Observers | · Time · Accuracy errors · Knot security |
· The training group and the control group demonstrated significant improvement in completion time, and overall score. · The training group also demonstrated significant improvement in accuracy errors. · The trained group performed significantly better in completion time and overall score when comparing posttest scores to the control group. · Intervention group performed significantly better than control group |
| Larsen et al., 2009 | Patients in OR (post only, pre was on a VR Simulator) | Observers | · Primary outcome measure: 1) technical performance using the objective structured assessment of laparoscopic salpingectomy 2) 5-items general rating scale and five-item task specific rating scale. · Time |
· Intervention group gained experience equivalent to 20-50 procedures. · The median score on general and task specific scale reached 33 points for the trained group and 23 in the control group (p<.001). · The median score for time was 12 minutes for the trained group and 24 minutes for the control group (p<.001). |
| Seymour et al., 2002 | Patients in OR (post only, pre was only ability tests) | Observers | · Operative errors 1) lack of progress 2) gallbladder injury 3) liver injury 4) incorrect plan of dissection 5) burn nontarget tissue 6) tearing tissue 7) instrument out of view 8) attending takeover |
· Intervention group was faster for gallbladder dissection (29% faster), and control group was more likely to fail to make progress (Z=-2.677, p<.008) and more likely to injure the gallbladder or burn non-target tissue (5times more likely, Chi square=4.27, p<.039). · The mean number of scored errors per procedure was significantly greater in the control group than the trained group (p=-2.76, p<.006). |
| Stefanidis et al., 2008 | Porcine Model (pre and post) | Objective scores based on time and errors using a published formula | · Time · Errors |
· Intervention group performed substantially better than control group (p<.001). · Proficiency-based simulator training results in improved operative performance. |
| Stefanidis et al., 2007 | Porcine Model (pre and post) A posttest was taken right after training was done, and then a retention test was taken after 5 months | Observers | · Errors · Time |
· Intervention group outperformed control group (p<.001). · Proficiency-based simulator training results in durable improvement in operative skill of trainees even in the absence of practice for 5 months. |
| Sroka et al., 2010 | MISTELS and Box Trainer on Patients in the OR | Attending surgeon or external evaluator | · FLS ratings and GOALS ratings: 1) Depth perception 2) Bimanual dexterity 3) Tissue handling 4) Efficiency 5) Autonomy |
FLS scores · Scores increased and SD decreased in the trained group as compared to the non-trained group (p=.004). At baseline no participant had reached the required FLS scores. · Post training 100% of the trained group reached required scores and 37.5% of the non-trained reached required passing scores. GOALS scores · The trained group improved significantly and clinically by a mean of 6.1 +/- 1.3 (p = .0005 vs. control, and p <.0001 vs. baseline)Gender was examined as a covariate and results remained the same, trained group scores were significantly better than the control group (p=.001)Of the 5 individual domains evaluated by the GOALS rating structure greater improvements were shown in the specific domains than the generic domains for the trained group (bimanual dexterity, p=.04; depth perception p=.08; tissue handling p=.04) |
| Van Sickle et al., 2008 | Patients in OR (post only) | 2 surgeons (agreement > .80) | · Suturing operative errors | · Intervention group performed significantly faster (p<.003), made fewer errors (p<.01), and fewer excess needle manipulation (p<.05). |
| Verdaasdonk et al., 2008 | Porcine Model (post only) | 2 Expert laparoscopic surgeons | · Observer rated error assessments · Global ratings of knot tying economy of movements · Error assessments |
· Intervention group tied knots faster (30%, p=.034) and made fewer errors (33%) as compared to control group. · Experimental group dropped the needle fewer times and made less frequent unnecessary contact with the tip of the needle against the tissue tan the control group (p<.05). · No significant differences in the scores assigned to the groups by the two experts (economy of movement p=.114; error assessment p=.148). |
| Zendejas et al., 2011 | OR (pre and post) | Observers and medical records | · Operative performance by using a global rating using: 1) GOALS 2) operating time 3) proportion of procedure performed by the trainee 4) need for overnight stay 5) recurrence of inguinal hernia and chronic groin pain and complications. |
· The trained group were on average 6.5 minutes faster than the control group (p<.0001). · Resident participation was also different between the groups with the trained group performing more of the procedure than the control group (88% vs. 73%). · After correcting time to account for varying participation rates, the trained group performed the procedure 13.1 minutes faster. · The trained group had higher performance scores than the trained group (p=.001). · Intraoperative and postoperative complicates and overnight stay were less likely in the trained group than the control group p<.05. · When follow ups with patients were conducted the number of patients who experienced a hernia recurrence or were evaluated for groin pain at least 3 month post repair there was no difference between the groups. |
Note: * Indicates articles that are unclear or do not supply an explanation of information.