Table 3.

Assessment type and associated results.

Author	Year	Title	CTA Method	Type of Assessment	Results	CTA Improved training?	Positive Subjective Outcome?	CTA Research Ranking
Amer et al.	2017	A Mobile-Based Surgical Simulation Application: A Comparative Analysis of Efficacy Using a Carpal Tunnel Release Module	Carpal Tunnel Surgery steps on ‘Touch Surgery™' vs Traditional Methods.	1. 21 question post-study standardised test	1. Test group average - 89.3% (±6.0%). Control group average 75.6% (±8.7%)	Yes	Yes	10
				2. Likert Scale for subjective rating	2. Overall content validity, quality of graphics, ease of use, and usefulness to surgery preparation rated as very high (4.8 of 5)
Bhattacharyya et al.	2017	Knee Arthroscopy Simulation A Randomized Controlled Trial Evaluating the Effectiveness of the Imperial Knee Arthroscopy Cognitive Task Analysis (IKACTA) Tool	Imperial Knee Arthroscopy Cognitive Task Analysis (IKACTA) tool used to describe each phase of a diagnostic knee arthroscopy vs No additional learning material.	1. Validated Arthroscopic Surgical Skill Evaluation Tool [ASSET] global rating scale.	1. Mean ASSET score (and standard deviation) →IKACTA group = 19.5 ± 3.7 points. Control Group = 10.6 ± 2.3	Yes	Yes	11
				2. Likert Scale for subjective rating	2. All participants agreed that the cognitive task analysis learning tool was a useful training adjunct to learning in the operating room.
Bhattacharyya et al.	2018	Trauma simulation training: a randomized controlled trial evaluating the effectiveness of the Imperial Femoral Intramedullary Nailing Cognitive Task Analysis (IFINCTA) tool	Imperial Femoral Intramedullary Nailing Cognitive Task Analysis (IFINCTA) tool used to describe each phase of antegrade femoral intramedullary nailing vs Standard operative technique manual.	1. Validated “Touch Surgery™” application assessment tool on femoral intramedullary nailing.	1. Post-test Median Score improvement (Intervention group over control group): Patient positioning and preparation- 20%, Femoral Preparation- 21%, Proximal locking- 10% and Distal Locking- 19%	Yes	Yes	9
				2. Likert Scale for subjective rating	2. All participants agreed the tool made the procedure easy to understand. The multi-modality approach was beneficial and that it was beneficial to use the tool prior to operating. 10/11 participants agreed that the tool was easy to use and 9/11 enjoyed using the tool.
Logishetty et al.	2019	A Multicentre Randomized Controlled Trial Evaluating the Effectiveness of Cognitive Training for Anterior Approach Total Hip Arthroplasty	Anterior approach Total Hip Arthroplasty- Participants cognitively trained vs Training with a standard operation manual with surgical video.	1. Assessment of time taken, errors made, prompts required and acetabular cup orientation.	1. Cognitive trained- 35% faster, 69% fewer errors in instrument selection, 92% fewer prompts, Reduced inclination and anteversion errors.	Yes	Yes	11
				2. Training survey assessing the usability and applicability of the Cognitive Training Tool for Learning Total Hip Replacement.	2. 34 of 35 residents agreed that the CTT was useful for understanding technical skills, decision making, and common errors related to AA-THA, was easy and enjoyable to use, and contributed to a marked improvement over standard preoperative preparation
Levin et al.	2018	Pre-course cognitive training using a smartphone application in orthopaedic intern surgical skills “boot camps”	Ankle open reduction + internal fixation and lag screw fixation using ‘Touch Surgery™'	1. Feedback from participants via post course survey	1. 10/14 participants believed using CTA improved baseline understanding, 9/14 believe learning was accelerated and 8/14 felt the application made the procedure easier to learn.	Yes	Yes	7
Sugand et al.	2015	Training effect of using Touch Surgery™ for intramedullary femoral nailing	Intra-medullary femoral nailing on ‘Touch Surgery™'- Consisting of four modules, participants completed a pre-module MCQ, had 6 attempts at each module and then completed a post module MCQ.	1. Pre- and Post- module MCQs, Comparison of Post module MCQ with Experts	1. Module- (i) patient positioning and preparation- 83% improvement (ii) femoral canal preparation- 94% improvement (iii) proximal locking 90% improvement (iv) distal locking and closure- 89% improvement. P- Value < 0.001 Similar post module scores between novices and experts.	Yes	N/A	6
Sugand et al.	2015	Validating Touch Surgery™: A cognitive task simulation and rehearsal app for intramedullary femoral nailing	Intra-medullary femoral nailing on ‘Touch Surgery™'- Consisting of four modules, done by experts and novices.	1. Construct validity using objective metrics.	1. Experts outperformed novices to demonstrate construct validity. Module - (i) Patient positioning and preparation- 32.5% higher, (ii) Femoral canal preparation- 31.5% higher, (iii) Proximal locking- 22.5% higher, (iv) Distal Locking and closure - 17% higher	Yes	Yes	7
				2. Face and content validity using a subjective questionnaire.	2. Both cohorts rated the face validity, quality of graphics, willingness to use the app, usefulness for preoperative rehearsal as good or very good. Experts rated the content validity as good too.