TABLE IV.
SUMMARY OF THE PAPERS THAT USED MACHINE LEARNING IN EYE TRACKING IN FUNDUS PHOTOGRAPHY, HISTOPATHOLOGY, SURGICAL VIDEO, ENDOSCOPY, COMPUTED TOMOGRAPHY (CT), MAGNETIC RESONANCE (MR) AND OPTICAL COHERENCE TOMOGRAPHY (OCT) ANALYSIS
| Reference | Year | Tracker | Anatomy | Image modality | # cases | # readers | Gaze data form | ML method | Objective |
|---|---|---|---|---|---|---|---|---|---|
| Jiang et al. [48] | 2023 | Tobii Pro Spectrum | eye | fundus photography | 1097 | 1 | paths | CNN | Diagnosing diabetic retinopathy and age-related macular degeneration |
| Akerman et al. [107] | 2023 | Pupil Labs Core | eye | fundus photography | 20 | 13 | paths | ID CNN | Classifying the expertise of clinicians whiel reading OCT using statistical features from eye movements |
| Jiang et al. [48] | 2023 | Tobii Pro Spectrum | eye | fundus photography | 1020 | 1 | heatmaps | Inception-V3, ResNet | Improving automated diagnosis by using gaze heatmaps as additional network input |
| Mariam et al. [5] | 2022 | Gazepoint GP3 | oral cavity | histopathology | 4 | 1 | paths | Fast R-CNN, YOLOv3, YOLOv5 | Annotating histopathological image analysis with gaze assistance |
| Hosp et al. [75] | 2021 | Tobii Glasses 2 | shoulder | videos | 150 | 15 | path statistics | SVM | Recognizing the level of expertise from gaze data |
| Xin et al. [46] | 2021 | Tobii X-60 | colon | endoscopic video | 1 | 10 | paths | GAN, LSTM | Predicting loss of navigation during colonoscopy using gaze paths over endoscopic videos |
| Sharma et al. [52] | 2020 | SMI RED | gallbladder | endoscopic videos | 2 | 29 | paths | HMM | Predicting if experts can recognize surgical error using their gaze data |
| Zimmermann et al. [74] | 2020 | SMI Glass 2 | vasculature | videos | 33 | 5 | fixations | Mask R-CNN | Optimizing fluorography use by capturing the surgeon attention on the fluoroscopic screen |
| Pedrosa et al. [26] | 2020 | Tobii 4C | lungs | 3D CT | 1 | 2 | fixations | YOLO | Annotating 3D lung image with gaze assistance |
| Aresta et al. [25] | 2020 | Tobii 4C | lungs | 3D CT | 20 | 4 | fixations | YOLO | Using gaze fixations for lung nodule detection |
| Dmitriev et al. [96] | 2019 | pancreas | 2D CT | 134 | 4 | heatmaps | CNN | Comparing activation maps of tumor classification CNN with gaze maps | |
| Stember et al. [60] | 2019 | Fovio | brain | 2D MR | 356 | 1 | gaze paths | U-Net | Investigating if tumors segmented with gaze can be used for CNN training |
| Dimas et al. [57] | 2019 | EyeTribe | bowel | endoscopic video | 226 | 1 | heatmaps | VGG-16 | Automatically generating gaze heatmap |
| Khosravan et al. [59] | 2018 | Fovio | lung | 3D CT | 6960 nodules | 3 | gaze paths | CNN | Using 3D gaze paths to find regions of attention and applying CNN to detect nodules in such regions |
| Lejeune et al. [80, 83] | 2018 | eyeTribe | endoscopy, brain, eye, cochlea | video, 2D MR, OCT, 2D CT | 4/4/4/4 | 1 | fixations | U-Net | Separating an image into superpixels and classifying superpixels that received the maximum attention |
| Ahmidi et al. [50] | 2012 | RED | sinuses | videos | 1 | 20 | fixations | HMM | Recognizing the level of experience from surgical tool motion and gaze data |
| Thiemjarus et al. [19] | 2012 | Tobii 1750 | gallbladder | endoscopic video | 1 | 3 | fixations | SVM, RF | Recognizing surgical steps from gaze data over endovideos |
| Ahmidi et al. [49] | 2010 | RED | sinuses | videos | 2 | 11 | fixations | HMM | Recognizing surgical tasks from tool motion and gaze data |
| James et al. [17] | 2007 | Tobii 1750 | gallbladder | endoscopic video | 5 | 3 | fixations | PLP | Recognizing surgical steps from gaze data over endovideos |
| Vilariño et al. [71] | 2007 | EyeLink2 | colon | endoscopic video | 6 | 1 | fixations | SVM, SoM | Using gaze data as seed and then classify the resulting image patches to represent polyps |
Abbreviations: ML (machine learning), CNN (convolutional neural network), SVM (support vector machine), GAN (generative adversarial network), HMM (hidden Markov model), RF (random forest), PLP (parallel level perception), SoM (self-organized maps);