Table 5.
Summary of included studies on real-time assistance and safety analysis.
| Study | Year | Data modality | Data source | Sample size | MLa method | Data availability | |
| Real-time feedback/guidance | |||||||
|
|
[89] | 2025 | Video | Clinical (robotic mastectomy) | 10 surgeries | Dissection plane segmentation model | Private |
|
|
[90] | 2025 | Video | Clinical (colorectal) | N/Ab | Nerve recognition model | Private |
|
|
[91] | 2024 | Video | Clinical (pituitary surgery) | >200 videos | Mask R-CNNc | Private |
|
|
[3] | 2024 | Video | Clinical (robotic urology) | 2 surgeries | Video transformer network | Private |
|
|
[92] | 2023 | Video (multicamera) | Laparoscopic box-trainer (Fundamentals of Laparoscopic Surgery peg transfer) | 55 videos; 5000+ images (9 participants) | SSDd (ResNet50e V1 FPNf); rule-based sequential assessment | Public |
|
|
[57] | 2022 | Video (top-down cameras) | Laparoscopic box-trainer | 55 videos (9 residents) | SSD; ResNet50; rule-based | Public |
|
|
[29] | 2022 | Video; kinematics | dVRKg simulator (SurRoL) | >1000 trials | Reinforcement learning (RLh) | Private |
|
|
[93] | 2022 | Video | Clinical (cholecystectomy) | 290 surgeries | Semantic segmentation (ResNet50+PSPNeti) | Private |
| Tool tracking/pose estimation | |||||||
|
|
[94] | 2025 | Video | Laparoscopic simulator | 10 surgeons | DeepLabCut (Pose Estimation) | Private |
|
|
[15] | 2025 | Video | 3D printed model | 15 procedures | YOLOv8j; BoTk-SORTl | Private |
|
|
[95] | 2024 | Video; IMU | SutureCoach Simulator | 81 participants | CNNm (ResNet-50) | Private |
|
|
[23] | 2023 | Video; kinematics | Custom Multiprocedure Dataset | 950 frames (5 surgeries) | GCNn; ResNet; Transformer | Private |
|
|
[96] | 2022 | Video (simulated) | da Vinci simulator | 9 videos | 6-DoFo Motion Planning | Private |
|
|
[97] | 2022 | Video (synthetic) | Synthetic MICCAIp; JIGSAWSq | 8 videos | PnPr; optical flow; time series forest | Public |
|
|
[98] | 2022 | Video | ATLAS Diones; Endovist; Cholec80u | 99 clips + 1083 frames | DSCNet-CLSTMv; CNN regression | Public |
|
|
[31] | 2021 | Video | Clinical (cataract surgery); CaDISw; Cataract-101 | 629 images | Faster R-CNN; SSD; RetinaNet; Scaled-YOLOv4 | Public, Private |
| Error detection/safety | |||||||
|
|
[99] | 2024 | Video | JIGSAWS | N/A | Transformer | Public |
|
|
[100] | 2024 | Video | Microsurgery simulator | 24 trials (12 surgeons) | Semantic segmentation (ResNet-50) | Private |
aML: machine learning.
bN/A: not available.
cR-CNN: region-based convolutional neural network.
dSSD: single shot detector.
eResNet: residual network.
fFPN: feature pyramid network.
gdVRK: da Vinci research kit.
hRL: reinforcement learning.
iPSPNet: pyramid scene parsing network.
jYOLO: You Only Look Once.
kBoT: bag of tricks.
lSORT: simple online and realtime tracking.
mCNN: convolutional neural network.
nGCN: graph convolutional network.
o6-DoF: 6 degrees of freedom.
pMICCAI: Medical Image Computing and Computer Assisted Intervention.
qJIGSAWS: Johns Hopkins University and Intuitive Surgical Inc Gesture and Skill Assessment Working Set.
rPnP: perspective-n-point.
sATLAS Dione: Applied Technology Laboratory for Advanced Surgery Dione dataset.
tEndoVis: Endoscopic Vision Challenge.
uCholec80: a dataset of 80 cholecystectomy procedure videos.
vDSCNet-CLSTM: depth-wise separable convolutional network with convolutional long short-term memory.
wCaDIS: cataract dataset for image segmentation.