Table 3.
Technical and methodological details of AI models presented in the reviewed studies
| Authors | Data set numerosity | Training/validation/testing split | Type of input | Type of output | Type of AI model | AI technology employed | Brief description of applications | Reliability of model | Code availability |
|---|---|---|---|---|---|---|---|---|---|
| Aggelides et al., 2020 [10] | 662 gestures | Variable T:T splits | COM (data from wristband device with accelerometer and gyroscope) | CAT (interpretation of gesture type according to own classification) | 4 KNN, SVM, RF, DT | DL | Recognizing gestures associated with allergic rhinitis | ACC 0.93 | No |
| Arfiani et al., 2019 [6] | 200 CT scans | Variable T:T splits | COM (patient data and maxillary sinus Hounsfield unit) | BIN (diagnosis of acute sinusitis) | KSPKM, SVM | ML | Diagnosing acute sinusitis from data compiled from CT scans | ACC 0.97 (KSPKM), 0.90 (SVM) | No |
| Bieck et al., 2020 [11] | 3850 navigation workflow sentences | 9:1T:T split | VER (descriptions of the endoscope state) | CAT (prediction of next action and landmark in ESS) | S2S, TRF, HMM, LSTM | ML | Predicting next steps or landmarks in ESS from surgical annotations | ACC 0.53 (TRF), 0.35 (LSTM), 0.32 (S2S), 0.83 at sentence-level | No |
| Borsting et al., 2020 [12] | 18,148 pre- and post-rhinoplasty images | 8:1:1T:V:T split | N2D (pre- and post-rhinoplasty photos) | BIN (identification of patients who underwent rhinoplasty) | CNN | DL | Recognizing patients who underwent rhinoplasty | CCR on true rhinoplasty status 0.67 (95% CI 0.59, 0.74) | Yes |
| Chowdhury et al., 2020 [13] | 147 patients | n/a | COM (biochemical data, SNOT-22 and polyp status from single patients) | CON (reduction of SNOT-22 after surgery) | RF, DT | ML | Selecting main factors for SNOT-22 reduction after ESS | Mean squared errors are reported for each variable | Freeware built-in feature (R) |
| Chowdhury et al., 2019 [14] | Pre- and post-treatment single slides from CT scans from 239 patients | 8:1:1T:V:T split | N2D (single images from head CT scan) | BIN (open or closed ostiomeatal complex) | CNN | DL | Recognizing ostiomeatal complex patency from CT scans | AUC 0.87 (95% CI 0.78–0.92) | Open source framework available |
| Hasid et al., 1997 [15] | 49 patients | n/a | COM (clinical data, morphonuclear features, glycohistochemical features) | CAT (polyps classification according to own classification) | DT | ML | Evaluating features characterizing each class of polyps | ACC 0.959 (95% CI 0.86–0.995) | Open source framework available |
| Dimauro et al., 2019 [16] | 4429 NCS cells | 3,4:1T:T split | E2D (single cells from NCS) | BIN (recognition of cells), CAT (identification of cell type) | CNN | DL | Identifying and classifying cells in NCS | RV 0.99 on the test set and 0.94 on the validation set for cell categorization, 0.977 for single cells identification | No |
| Dimauro et al., 2020a [17] | 326 NCS fields tiles | 2.3:1T:T split | E2D (tiles from NCS) | BIN (identification of biofilm-producing bacterial colonies) | CNN | DL | Identifying biofilm-producing bacterial colonies | ACC 0.98 | No |
| Dimauro et al., 2020b [18] | 87 cytology fields (cells identification); 1990 cells (cells classification) | n/a (pre-trained model see art. 100) | N2D (fields from cytological centrifugation) and non-native 2d graphical (single cells) | BIN (recognition of cells), CAT (identification of cell type) | CNN | DL | Identifying and classifying cells in NCS | RV 0.523 (cell identification), 0.00–0.90 (cell classification, different for each cell type) | No |
| Dorfman et al., 2020 [19] | 100 patients (pre- and post-operative images) | n/a (pre-trained commercial algorithm) | N2D (pre- and post-operative photos from open rhinoplasty patients) | CON (estimated patient age) | CNN | DL | Estimating patients age | Correlation coefficient with real age r = 0.91 on pre-operatory images | Commercial code |
| Kannan et al., 2020 [20] | 872 patients | 8:2T:T | COM (92 items from clinical data and allergy test results) | BIN (diagnosis of allergic rhinitis) | Genetic algorithm for selecting features and extreme learning machine for classification purposes | DL | Recognizing allergic rhinitis patients starting from allergy tests and clinical data | ACC 0.977 | No |
| Farhidzadeh et al., 2016 [21] | 25 nasopharyngeal carcinoma MRIs | n/a | E2D (contoured neoplasms from contrast-enhanced T1 MR images) | BIN (disease progression estimate) | SVM | ML | Estimating disease progression from radiomics features | ACC 0.76–0.80 AUC0.6–0.76 | No |
| Fujima et al., 2019 [22] | 36 sinonasal squamous cells carcinoma MRIs | 8:1T:V | E2D (contoured neoplasms from MR images, with different parameters) | BIN (disease control estimate) | SVM | DL | Predicting local disease control | ACC 0.92, SEN 1, SPE 0.82, PPV 0.86, NPV 1 | No |
| Girdler et al., 2021 [23] | 222 endoscopy frames | 15:3:2T:V:T | N2D (frames from endoscopy in normal patients, nasal polyps or inverted papilloma) | CAT (normal endoscopy, inverted papilloma or nasal polyp) | CNN | DL | Distinguishing normal endoscopy, nasal polyp and inverted papilloma images | ACC 0.742 ± .058 | Yes |
| Huang et al., 2020 [24] | 1063 single images from CT scans | 3.4:1T:T | E2D (cropped single coronal CT slice at the anterior ethmoidal foramen) | BIN (anterior ethmoidal artery adherent or suspended in mesentery | CNN | DL | Identifying suspended anterior ethmoid arteries | ACC 0.827 (95% CI 0.777–0.878) | Open source framework available |
| Humphries et al., 2020 [25] | 700 CT scans | 14:4:51T:V:T | NVI (whole head CT scan) | CON (percentage of sinus opacification) | CNN | DL | Quantifying overall sinus opacification in CT scans | DSC mean 0.93; range 0.86–0.97 | Open source framework available |
| Jeon et al., 2021 [26] | 1535 patients | 10:1:1T:V:T | N2D (Waters' and Caldwell's projection radiographs) | BIN (presence of maxillary, ethmoidal, or frontal sinusitis) | CCN on two networks | DL | Diagnosing sinusitis from Waters- or Caldwell projections | AUC 0.71 (95% CI 0.62–0.80) for Waters' view and 0.78 (95% CI 0.84–0.92) for Caldwell's view | No |
| Jung et al., 2021 [27] | 123 cone beam CT scans | 4:1:1T:V:T | NVI (whole head cone beam CT) | CON (maxillary sinus volume and air/lesion ratio inside) | CNN | DL | Identifying, segmenting, and defining in air/lesion level the maxillary sinus | Best DSC for air 0.93 ± .16, best DSC for lesions 0.77 ± 0.18 | No |
| Kim et al., 2019a [28] | 9340 radiographs | 80:10:3.4T:V:T | E2D (maxillary sinus images from Waters' view radiographs) | BIN (diagnose maxillary sinusitis) | CNN | DL | Diagnosing maxillary sinusitis from Waters' radiograph | AUC 0.93 and 0.88 for the two different test sets | No |
| Kim et al., 2019b [4] | 5020 radiographs | 30:1T:T | E2D (trimmed Waters' view radiographs) | BIN (identify the maxillary sinus and diagnose maxillary sinusitis) | Majority decision algorithm on 3 CNN models | DL | Identifying the maxillary sinus and diagnosing maxillary sinusitis | ACC 0.941 and 0.9412, AUC 0.948 and 0.942 for internal and external test sets, respectively | No |
| Kim et al., 2021 [2] | 129 patients | N/A | COM (clinical and histology features) | BIN (satisfactory surgical outcome) | DT and RF | ML | Predicting surgery outcomes from patient- and histology-specific variables | ACC 0.8404 | Free add-on for free software |
| Kuwana et al., 2021 [29] | 1168 ortopantomographs | 3:1T:T | E2D (labeled images from orthopantomograph) | BIN (maxillary sinusitis diagnosis, presence of maxillary sinus cysts) | DetectNet neural network | DL | Locating the maxillary sinus on ortopantomographs and identifying healthy sinus, sinusitis and cysts | ACC 0.9–0.91, SEN 0.88–0.85 SPE 0.91–0.96 for maxillary sinusitis; ACC 0.97–1, SEN 0.8–1 SPE 1–1 for maxillary sinus cysts, over 2 test sets | No |
| Lamassoure et al., 2021 [3] | 531 mallet impacts from osteotomies on anatomic models | N/A | COM (impacts kinetics from a receiver in the surgical mallet) | BIN (identification of fractured state of bone after impact) | SVM | ML | Evaluating the state of bone after impact in osteotomies | ACC 0.83, 0.91, and 0.93 with a tolerance of 0, 1, and 2 impacts, respectively | No |
| Laura et al., 2019 [30] | 513 CT scan slices | 17:3T:V | E2D (cropped slices from CT scans) | CAT (sinuses and nasal cavity identification) | Darknet-19 deep neural network combined with the You Only Look Once method (YOLO) | DL | Identifying paranasal sinuses and nasal cavities in CT scans | Variable precision and recall rates according to sensitivity of evaluation methods and specific structure, reported graphically | No |
| Lötsch et al., 2021 [31] | 90 patients | N/A | COM (37 nasal anatomy and pathology, olfactory function, quality of life, sociodemographic and clinical parameters) | CON (influence of each criterion on outcomes of ESS) | RF, KNN, SVM, and binary logistic regression | DL | Identifying factors contributing to outcomes of ESS | N/A (weight and role of different features are reported, but the overall model it's not tested) | Built-in feature in freeware software (R) |
| Murata et al., 2019 [32] | 6000 ortopantomographs regions of interest | 7:1T:T | E2D (regions of interest from single sinuses on ortopantomographs) | BIN (detection of inflammation) | CNN | DL | Diagnosing maxillary sinus inflammatory conditions on orthopantomographs | ACC 0.875, SEN 0.867, SPE 0.883, AUC 0.875 | Open source framework available |
| Neves et al., 2021 [33] | 150 CT scans | 13:2T:T | NVI (whole CT scans for testing, manually segmented CT scans for traning) | CON (internal carotid artery, optic nerve and sella turcica auto-segmentation) | Clara SDK-based AHNet7 algorithm | DL | Autosegmenting internal carotid artery, optic nerve, and sella turcica | DSC 0.76 ± 0.12 for the internal carotid artery, 0.81 ± 0.10 for optic nerve, and 0.84 ± 0.08 for sella turcica | Commercial code |
| Parmar et al., 2020 [34] | 447 images | 3.5:1T:T | E2D (cropped single side single images from CT scans) | BIN (presence of concha bullosa) | CNN | DL | Identifying conchae bullosae | ACC 0.81 (95% CI 0.73–0.89), AUC 0.93 | Open source framework available |
| Parsel et al., 2021 [35] | 545 patients | N/A | COM (22 items from demographic, quality of life, and clinical data) | CAT (association with one of 7 specific clusters associated with specific rhinological diagnoses) | Non-hierarchical cluster analysis performed with partitioning around medoids method | AI | Clustering patients for diagnosis and disease behavior according to their baseline characteristics | N/A (weight and role of different features are reported, but the overall model it's not tested) | Commercial code |
| Putri et al., 2021 [36] | 200 patients | Variable T:T splits | COM (gender, age, air cavity, and Hounsfield unit in CT scan) | BIN (diagnosis of acute sinusitis) | SVM | DL | Identifying patients with maxillary sinusitis from compiled data | ACC up to 1.00 | No |
| Quinn et al., 2015 [37] | 331 and 262 regions of interest from 2 cohorts of patients | N/A | COM (autoregressive models from optical flow in regions of interest in videos from optical microscopy from NCS) | CAT (Type of ciliary movement alteration, if any) | SVM | DL | Identifying anomalies in ciliary movement | Best ACC 0.938 and 0.867 for the two tested cohorts | Yes, upon request; open source license software used |
| Ramkumar et al., 2017 [38] | 46 MRIs | 3:1T:T | COM (texture analysis from regions of interest in MRI) | BIN (distinguishing inverted papillomas from squamocellular carcinomas) | Diagonal Linear Discriminate Analysis, SVM, and Diagonal Quadratic Discriminate Analysis | ML | Distinguishing between SCC and IP | ACC 0.909 in training and 0.846 in testing for SVM, 0.87% concordance with radiology review | No |
| Soloviev et al., 2020 [39] | 201 optical coherence tomography images | Variable T:T splits | COM (depth-resolved histogram matrix from optical coherence tomography images) | CAT (classification of normal or diseased nasal mucosa, either atrophic or hypertrophic) | KNN, RF, gradient boosting decision trees, support vector clustering, and logistic regression | DL | Classifying normal, atrophic and hypertrophic nasal mucosa from optical coherence tomography images | ACC > 0.94 for all methods for binary classification of normal and pathological tissues; ACC > 0.91 for diagnostic classification of normal, hypertrophic and atrophic tissues | No |
| Staartjes et al., 2021 [40] | 549 images | 2:1T:T | N2D (frames from surgical videos) | CAT (identification of septum, inferior turbinate, middle turbinate) | U-Net17 neural network | DL | Identifying anatomical structures in surgical frames from video | 36.1% cases correct recognition, 19.2% correct recognition with overshoot, 44.7% incorrect recognition or recognition including 2 or more structures | No |
| Thorwarth et al., 2021 [41] | 80 patients | 3:1T:T | COM (peripheral eosinophil count, urinary leukotriene E4 level, and polyp status) | BIN (diagnosis of eosinophil chronic rhinosinusitis) | Artificial neural network | ML | Predicting eosinophilic chronic rhinosinusitis based on preoperative data | AUC of 918 (0.756–0.975) and 0.956 (0.828–0.999) using random and surgeon specific data sets | No |
| Wirasati et al., 2020 [42] | 200 CT scans | 7:3T:T | COM (gender, age, air cavity, Hounsfield units) | BIN (diagnosis of acute or chronic rhinosinusitis) | CNN and LSTM | DL | Identifying patients with acute or chronic sinusitis | ACC 0.9833 | No |
| Wu et al., 2020 [43] | 26,589 histology slides | 14:1:1,3T:V:T | N2D (extracted patches from histology slides) | BIN (diagnosis of eosinophil chronic rhinosinusitis) | CNN | DL | Identifying eosinophil chronic rhinosinusitis on whole histology slides | best AUC 0.974 and 0.957 on validation and testing data sets with InceptionV3 | No |
| Wu et al., 2021 [44] | 24,625 patches from histology slides | 14:1T:T | E2D (regions of interest from extracted patches from histology slides) | CON (number of eosinophils, lymphocytes, neutrophils, and plasma cells) | CNN | DL | Classifying different subtypes of nasal polyps | Mean absolute errors of the ratios of eosinophils, lymphocytes, neutrophils, and plasma 0.164, 0.213, 0.106, and 0.122% | No |
CT computed tomography, NCS nasal cytology smear, ESS endoscopic sinus surgery, MRI magnetic resonance imaging; T:T train:test, T:V:T train:validate:test, T:V train:validate, COM compiled, VER verbal, N2D Native 2D graphical, E2D elaborated 2D graphical, NVI Native volumetric information, CAT categorical outcome, BIN binary outcome, CON continuous outcome, (AI artificial intelligence, DL deep learning, KNN K-nearest neighbors, SVM support vector machine, RF random forest, DT decision tree, KSPKM Kernel Spherical K-Means, S2S encoder–decoder model, TRF transformer model, HMM hidden markov model, LSTM long–short-term model, ACC accuracy, AUC area under curve, CCR concordance, CI confidence interval, RV recall value, SEN sensitivity, SPE specificity, PPV positive predictive value, NPV negative predictive value, DSC Dice similarity coefficient