A novel AI system for preliminary triage support in single-tooth edentulous spaces using intraoral images

Abstract

This study aimed to demonstrate the integration of deep learning (DL) and machine learning (ML) using only occlusal photographs to provide preliminary triage support between orthodontic and prosthodontic treatments in cases requiring alternatives to implant-based interventions. Occlusal photographs (n = 2,962) were collected under routine conditions using smartphones and digital cameras. Two groups of dental specialists independently annotated the dataset. A YOLOv8m model was used for the localization of single-tooth edentulous spaces (S-TES), which were defined as localized areas within the dental arch where a single tooth is absent while adjacent teeth remain present, as well as their mesial and distal adjacent teeth. ResNet-50/ResNet-101 and VGG-16/VGG-19 models were employed to classify the clinical conditions and anatomical categories of teeth adjacent to S-TES. A deterministic function converted the mesiodistal width of the S-TES from pixels to millimeters using mean central incisor widths. Logistic Regression (LR) and XGBoost (XGB) models were used to predict the preliminary triage outputs. Among the classifiers for the clinical tooth condition classification task, VGG-19 showed the highest macro F1 score (0.928) and ResNet-101 yielded the highest macro AUC (0.961) and weighted kappa (0.927), with the narrowest 95% confidence intervals (95% CI) for both metrics. For the anatomical categorization task, ResNet-101 showed the highest F1 score, recall, and precision. For the triage support models, LR showed the highest AUC (0.896) with an expected calibration error (ECE) of 0.041, and XGB demonstrated the highest accuracy (0.869), sensitivity (0.874), and specificity (0.862) with an ECE of 0.024. The proof-of-concept demonstrates that the integration of DL and ML can achieve acceptable performance for preliminary triage support between orthodontic and prosthodontic treatment from occlusal images, within the acknowledged limitations. Further research is needed to improve model generalizability. However, the findings indicate a promising direction for future work. While in the long term, such technology has the potential to support preliminary triage between treatment options and become a clinically applicable tool, our current findings show that external validation and improved generalizability of the proposed AI framework are necessary before such clinical applications can be realized.

Supplementary Information

The online version contains supplementary material available at 10.1038/s41598-026-58704-7.

Introduction

Previous studies have shown that treatment decision-making in dentistry extends beyond simply selecting among treatment options and plays a critical role in reducing overall treatment time, minimizing unnecessary clinical visits, optimizing human and financial resources, and ultimately improving patient satisfaction^1–3. Many patients initially present to general dental practitioners. At this stage, accurate treatment decision-making may be challenged by limited access to specialist evaluation. As a result, patients may be referred to inappropriate dental specialists or experience considerable delays before reaching the appropriate specialist, which can prolong treatment timelines and increase unnecessary follow-up visits^4,5. Such delays may allow the underlying dental conditions to progress, thereby increasing treatment complexity, cost, and duration of treatment^6,7.

Single-tooth edentulous spaces (S-TES) are localized areas within the dental arch where a single tooth is absent while adjacent teeth remain present. S-TES are highly prevalent and most commonly result from advanced dental caries, failed endodontic treatment, periodontal breakdown, or traumatic injury. Clinicians manage these spaces using implant treatment, prosthodontic methods, or orthodontic interventions⁸. Implant-supported methods are often considered the treatment of choice for S-TES management. However, many patients either cannot receive implants or decline implant treatment due to financial limitations, systemic conditions such as uncontrolled diabetes and other medical concerns, or personal preferences and anxiety related to the treatment process. In such cases, orthodontic treatments (e.g., orthodontic space closure) and prosthodontic treatments (e.g., fixed dental bridge placement and removable partial denture placement) are commonly proposed as alternative treatment options. From a clinical standpoint, these cases are often challenging at the initial visit, as the dentist may face uncertainty in selecting and recommending the most appropriate treatment^9–13.

On the other hand, recent advances in artificial intelligence (AI) have significantly expanded its applications in dentistry, providing powerful tools that can simplify diagnostic workflows and reduce clinical workload^14–16. AI frameworks have shown high performance in detecting and analyzing clinical conditions from dental images, which can reduce patient waiting times and improve the overall efficiency of access to treatment^17–19. These advances show that AI can play a crucial role in minimizing delays that commonly arise during the initial dental examination^20,21. Among current AI methods in dentistry, deep learning (DL) has achieved high accuracy in dental image analysis. Specifically, convolutional neural networks (CNNs) automatically extract clinically relevant features directly from dental images and assist in more efficient and objective clinical assessment^22–24. In parallel, machine learning (ML) methods can effectively process clinical variables and integrate categorical clinical findings into predictive models to support diagnostic assessment and treatment decision-making in dental practice^25,26.

Intraoral photographs have become a routine component of clinical documentation in dental practice. These images are captured at various stages of the treatment process (e.g., initial examinations and follow-up visits)²⁷, which makes them ideal input data for AI frameworks^28–30. S-TES can be identified from occlusal photographs due to the clear visualization of interdental spacing. However, relying solely on occlusal images for implant treatment is insufficient, as implant placement requires evaluation of additional clinical and radiographic records that cannot be assessed from such images. In contrast, occlusal photographs can provide valuable clinical information to support preliminary triage between prosthodontic and orthodontic treatment for S-TES management.

Given these considerations and focusing on S-TES management without implant placement, a feasibility investigation is warranted to address a fundamental clinical question: Can an AI framework be designed to assist clinicians in the preliminary clinical assessment of prosthodontic and orthodontic treatment options for S-TES management using only occlusal images? In response, this proof-of-concept study aimed to integrate DL and ML methods using only occlusal images to provide preliminary triage support in cases requiring alternatives to implant-based interventions. In other words, this exploratory study was designed to model the clinical circumstances that challenge a general dentist during the initial examination, when the dentist must make an initial treatment decision solely based on a visual examination of the dental arch, without access to additional clinical information. Additionally, this study aimed to statistically evaluate the proposed AI framework against dental specialists. Although several studies have explored CNN and ML methods using intraoral photographs for the automated detection, segmentation, and treatment planning across various dental conditions, to our knowledge, an AI framework that directly identifies S-TES from occlusal images while also supporting preliminary triage has not been reported.

Materials and methods

Study design

This study investigated the feasibility of developing and evaluating an AI framework to provide preliminary triage support using only occlusal images in S-TES cases without considering implant-based interventions. As shown in Fig. 1, a CNN object detection model (YOLOv8m) was used to detect the S-TES and its adjacent teeth. To classify the clinical condition and anatomical category of the adjacent teeth, CNN classifiers (ResNet-50, ResNet-101, VGG-16, and VGG-19) were employed. A deterministic function was used to convert the S-TES mesiodistal width measurements from pixels into millimeters. Finally, all extracted features were integrated into ML models (Logistic Regression [LR] and XGBoost [XGB]) to assist clinicians in preliminary triage between orthodontic and prosthodontic treatment.

Fig. 1.

Overview of the proposed AI framework. Single-tooth edentulous spaces (S-TES) refer to localized interruptions in the dental arch characterized by the absence of a single tooth with preservation of the neighboring teeth.

Dataset

The Ethics Committee of Sechenov First Moscow State Medical University approved the study protocol (Protocol No. 17–25). All procedures were performed in accordance with the Declaration of Helsinki and institutional ethical guidelines. For minors under 18 years of age, a parent or legal guardian was present in the examination room during all photographic procedures. Written informed consent was obtained from all patients or from their parents or legal guardians for research purposes. All participants were verbally informed, and the consent form explicitly stated that their occlusal photographs would be used for AI development and analysis. No exclusion criteria were applied except for patient refusal or the presence of retained primary teeth. Clinical trial registration was not required because this study was observational and did not involve any therapeutic intervention.

A total of 2,962 occlusal photographs with at least one S-TES were collected under routine clinical conditions during dental examinations using common mobile devices such as Apple iPhone models (iPhone 16 Pro, iPhone 15, iPhone 13), Samsung smartphones (Galaxy S23 Ultra, Galaxy S22), and Xiaomi devices (Xiaomi 14 Ultra, Xiaomi 13 Pro), as well as digital cameras including the Canon EOS 800D and Nikon COOLPIX P950. Dataset partitioning was performed at the patient level, and it was ensured that all test images remained separate from the learning process and were used solely for final evaluation (Table 1).

Table 1.

Dataset distribution and partitioning.

Dataset characteristic	Learning set		Test set
Subset number	Subset 1 (Used for training and validation of CNN models, 90:10 split)	Subset 2 (Used for training and validation of ML models, 90:10 split)	Subset 3 (Used for evaluation of CNN models and the AI framework)
Orthodontic treatment n (%)	Maxillary: 294 (20.2%) Mandibular: 377 (26.0%) Total: 671 (46.2%)	Maxillary: 240 (19.4%) Mandibular: 307 (24.9%) Total: 547 (44.3%)	Maxillary: 42 (15.2%) Mandibular: 67 (24.3%) Total: 109 (39.5%)
Prosthodontic treatment n (%)	Maxillary: 322 (22.2%) Mandibular: 459 (31.6%) Total: 781 (53.8%)	Maxillary: 301 (24.4%) Mandibular: 386 (31.3%) Total: 687 (55.7%)	Maxillary: 70 (25.4%) Mandibular: 97 (35.1%) Total: 167 (60.5%)
Images with 1 S-TES n (%)	Maxillary: 434 (29.9%) Mandibular: 557 (38.4%) Total: 991 (68.3%)	Maxillary: 327 (26.5%) Mandibular: 410 (33.2%) Total: 737 (59.7%)	Maxillary: 71 (25.7%) Mandibular: 102 (37.0%) Total: 173 (62.7%)
Images with 2 S-TES n (%)	Maxillary: 182 (12.5%) Mandibular: 279 (19.2%) Total: 461 (31.7%)	Maxillary: 214 (17.3%) Mandibular: 283 (22.9%) Total: 497 (40.3%)	Maxillary: 41 (14.9%) Mandibular: 62 (22.4%) Total: 103 (37.3%)
Total images n	1452	1234	276
Total S-TES instances n	Maxillary: 798 Mandibular: 1115 Total: 1913	Maxillary: 755 Mandibular: 976 Total: 1731	Maxillary: 153 Mandibular: 226 Total: 379

Percentages were calculated within each subset based on the total number of images in that subset. Notably, the rows “Images with 1 S-TES” and “Images with 2 S-TES” refer to image-level counts, not the total number of S-TES instances. Thus, the total number of S-TES instances exceeds the total number of images in subsets containing images with two S-TESs.

S-TES pixel measurement function

A deterministic function was used to convert pixel measurements to millimeters. For this purpose, the mesiodistal width of the right maxillary and the right mandibular central incisors was measured for each patient using digital intraoral scanners and recorded in Microsoft Excel. For patients missing the right central incisor, the left central incisor was used as an alternative reference. After data collection, the mean mesiodistal width was calculated from the learning set only (maxillary: 8.44 mm; mandibular: 5.28 mm) and used as a reference value for pixel to millimeter conversion in the proposed AI framework. For each image, the mesiodistal width of the central incisor was manually annotated as a pixel distance by four independent dental examiners. This annotated pixel distance was supplied to the function, which used a reference value of 8.44 mm for maxillary images and 5.28 mm for mandibular images to compute an image-specific scaling factor. Thus, any pixel distance measured within the same image could be converted to millimeters. In other words, the function generated an image-specific calibration for each image. These mean reference values were used instead of exact patient-specific measurements to reduce reliance on intraoral scanners and to improve methodological reproducibility.

Additionally, a random sample of 100 occlusal images (50 maxillary and 50 mandibular) was used to evaluate inter- and intra-examiner agreement for manual pixel annotation. Inter-examiner reliability was assessed using the first round of measurements from four independent dental examiners and was calculated using the intraclass correlation coefficient (ICC), based on a two-way random-effects model with absolute agreement for single measurements. For intra-examiner reliability, the same four examiners repeated the annotation of the same 100 images after a one-month interval while blinded to their previous annotation. Intra-examiner reliability was calculated separately for each examiner using the ICC, based on a two-way mixed-effects model with absolute agreement for single measurements.

Data annotation

All photographs were independently annotated (labeled) by two groups of experienced dental specialists (Group 1: one orthodontist and two prosthodontists; Group 2: one orthodontist and one prosthodontist). Each subset was labeled for a predefined purpose (Table 1).

Subset 1 (training and validation of CNN models): For the S-TES detection stage, annotators labeled the S-TES with its two adjacent teeth (mesial and distal). For the adjacent teeth classification, the teeth were independently annotated into five clinical conditions (intact tooth, restored tooth, fixed prosthetic crown, extensive caries involvement, and partial crown loss) and four anatomical categories (incisor, canine, premolar, molar). This categorization was designed to reflect the clinical relevance of the adjacent teeth condition, as their structural integrity and restorative condition play a critical role in treatment decision-making for S-TES management and are clinically important in both orthodontic and prosthodontic treatment^31–33. Each tooth was assigned to a single clinical condition category. Teeth with both restoration and carious involvement were annotated as extensive caries involvement, and teeth with both restoration and partial crown loss were annotated as partial crown loss. The presence of an additional condition in a restored tooth was considered to indicate greater structural loss, as these teeth had already lost part of their original integrity. This prioritization was applied because carious destruction and partial crown loss more directly reflect the current structural condition of the tooth and are therefore more clinically relevant to treatment planning than restoration alone^34–36.

Subset 2 (training and validation of ML models): Since each image was considered a patient case and the annotators assigned one preliminary treatment decision per case, the preliminary triage task became similar to a true or false problem. Thus, for each image, the preliminary treatment decision (orthodontic or prosthodontic) was recorded in a structured dataset and used as the ground truth for ML models.

Subset 3 (final evaluation of CNN models and proposed AI framework): All labeling procedures applied to Subsets 1 and 2 were also performed for this Subset.

Cross evaluation was performed between the two annotator groups, and inter-annotator agreement was assessed using Cohen’s kappa. All cases with disagreements between groups were discussed, and the final decision was recorded as the ground truth. It is important to note that the annotators determined the preliminary treatment decision solely from the occlusal images, without access to patient history or additional clinical information such as radiographs. This protocol was designed to reflect the circumstances of the initial examination, when a general dentist often relies on direct oral inspection before additional diagnostic information becomes available, determines the most appropriate preliminary treatment for the patient, and refers the patient to the relevant dental specialist. Thus, this annotation process was designed to reflect real-world preliminary clinical assessment using initial information rather than fully informed treatment planning.

Model training and performance evaluation

Supplementary Table S1 presents a summary of the training configurations and key hyperparameters for all models trained using the learning set. All configurations were optimized through validation to improve performance and ensure reliable predictions. Subset 1 was used to train the CNN models. Input images were resized to 640 × 640 for YOLOv8m and to 240 × 240 for the CNN classifiers, with batch sizes of 16 and 32, respectively. A random seed of 42 was used for all models, and AdamW was employed as the optimizer, with initial learning rates of 0.001 for YOLOv8m and 0.0001 for the classification models. The ML models were trained on Subset 2 using features extracted from the YOLOv8m, the CNN classifier showing the most reliable performance after validation, and the pixel measurement function, including the clinical condition and anatomical category of adjacent teeth, as well as the S-TES measurements in millimeters. For model training, the LBFGS solver with L2 regularization and the gbtree booster with a learning rate of 0.05 were used for LR and XGB, respectively. A random seed of 42 was applied to all ML models. Images that contained two S-TES were included in the learning set for all models, enabling the models to process both S-TES simultaneously within the same image. Specifically, for the ML models, features extracted for each detected S-TES were processed separately, automatically sorted according to S-TES horizontal position (x-center), and assigned to a predefined feature slot. This structure was designed to ensure that the features of each S-TES remained separated and consistently ordered.

For performance evaluation, the YOLOv8m model was evaluated using the F1 score at an intersection over union (IoU) threshold of 0.8 or higher to ensure accurate detection. For the CNN classifiers used in the clinical tooth condition classification task, performance was evaluated using macro F1 score, macro AUC (area under the receiver operating characteristic [ROC] curve), and weighted kappa, with 95% confidence intervals (95% CI) reported for both macro AUC and weighted kappa, estimated using bootstrap resampling. The macro F1 score was evaluated to address potential class imbalance by assigning equal importance to each of the five clinical tooth conditions. The macro AUC was used to evaluate the overall discriminative ability, and the weighted kappa was evaluated to assess agreement between predicted and ground truth labels. In addition, confusion matrices, ROC curves, and t-distributed stochastic neighbor embedding (t-SNE) visualizations were used to provide a complementary evaluation of model behavior. For the anatomical tooth categorization task, performance was evaluated using F1 score, recall, and precision, as this task represented a simpler classification problem compared to the clinical condition classification task. Thus, standard metrics were considered sufficient to evaluate the performance of classifiers.

The ML models were evaluated using accuracy, sensitivity, specificity, expected calibration error (ECE), and AUC, with a 95% CI for AUC, estimated using bootstrap resampling. Sensitivity and specificity were important for the safe clinical assessment, as evaluating them showed the model’s ability to identify cases that required a specific treatment. ECE was a key metric in this study, as the primary aim was a preliminary assessment of treatment options, and this metric was used to evaluate the calibration of the model’s confidence. In other words, ECE demonstrated model calibration by evaluating the correspondence between predicted confidence and observed accuracy. In addition, ROC curves and reliability diagrams were used for visualization, and decision curve analysis (DCA) was employed to support clinical interpretation.

It is important to note that the unit of analysis differed across stages of the evaluation. The CNN models were evaluated for each S-TES (n = 379), and the ML models were evaluated for each image (n = 276) that represented a single patient case. To assess the adequacy of the test set for performance evaluation, a precision-based justification was used by evaluating the width of the 95% CIs around the primary performance metrics. All evaluation processes were performed using Subset 3. YOLOv8m and the CNN classifiers were evaluated independently as standalone models. The ML models were evaluated following an end-to-end pipeline. In other words, the CNN models analyzed the images, predicted the required outputs, and quantitative features were then extracted and provided as inputs to the ML models for the final prediction. No ground truth features were used during evaluation, and ground truth annotations were only used for reference and performance assessment.

Dental clinician comparison

We performed an additional validation step to evaluate the agreement between the proposed AI framework and dental specialists. Three groups of junior specialists (each consisting of one orthodontist and one prosthodontist) independently reviewed occlusal images from Subset 3 and recorded their own preliminary treatment decisions. These specialists were independent of the ground truth annotation team and were blinded to the model predictions and to each other’s evaluations. Each image was considered as a single patient case during evaluation, and agreement between model predictions and clinician decisions was evaluated using case-level 2 × 2 contingency tables. To evaluate the level of agreement beyond chance, Cohen’s kappa statistic was used, and odds ratios (OR) were calculated from contingency tables to assess the association between the proposed framework outputs and the specialists’ preliminary decisions.

Results

In this section, results are presented on Subset 3 using the evaluation metrics and visualizations described in Subsection 2.5. Inter-annotator agreement between two groups of annotators was evaluated using Cohen’s kappa (κ = 0.907), showing an acceptable level of consistency in the annotation process. Agreement analysis for manual pixel annotation showed an ICC of 0.946 (95% CI, 0.913 to 0.976) for inter-examiner reliability. For intra-examiner reliability, the ICC values for examiners 1 through 4 were 0.983, 0.974, 0.991, and 0.988, respectively.

CNN models

The YOLOv8m model yielded an F1 score of 0.903 at an IoU threshold of 0.8 or higher. As shown in Table 2, for the clinical tooth condition classification task, ResNet-50 and VGG-19 showed the lowest and highest macro F1 scores, respectively. ResNet-101 had the highest macro AUC and weighted kappa, along with the narrowest 95% CIs for both metrics. Although VGG-19 had a similar performance to ResNet-101, it showed wider 95% CIs compared with ResNet-101. In other words, the model with narrower 95% CIs was more reliable, which was crucial given the clinical nature of the dataset and the direct impact of model outputs on preliminary treatment assessment³⁷. As summarized in Table 2, for the anatomical tooth categorization task, ResNet-101 showed the highest F1 score, recall, and precision. A higher recall demonstrated fewer missed anatomical tooth classifications, and a higher precision indicated greater reliability with fewer incorrect categorizations.

Table 2.

Performance of CNN classifiers across the two classification tasks.

Task	Metric	ResNet-50	ResNet-101	VGG-16	VGG-19
Clinical tooth condition	Macro F1 score	0.893	0.924	0.910	0.928
	Macro AUC (95% CI)	0.933 0.918–0.939	0.961 0.958–0.967	0.946 0.931–0.951	0.948 0.933–0.949
	Weighted Kappa (95% CI)	0.898 0.873–0.901	0.927 0.917–0.931	0.903 0.888–0.910	0.924 0.913–0.929
Anatomical tooth categorization	F1 score	0.918	0.940	0.917	0.937
	Recall	0.898	0.921	0.910	0.918
	Precision	0.940	0.961	0.926	0.957

Different evaluation metrics were defined for each task according to its specific classification objective. For the clinical tooth condition task, model performance was assessed using macro F1 score, macro AUC, and weighted kappa, with 95% confidence intervals provided for macro AUC and weighted kappa. For the anatomical tooth categorization task, performance was evaluated using F1 score, recall, and precision.

As shown in Fig. 2, VGG-19 had overlapping clusters in the t-SNE visualization, despite an acceptable diagonal pattern in the confusion matrix. Since each class represented a clinical tooth condition, the confusion matrix summarized the final classification outcomes across classes, and t-SNE showed the separability of the learned features. In contrast, ResNet-101 showed more distinct clusters in the t-SNE visualization together with an acceptable diagonal pattern in the confusion matrix, indicating more reliable performance on this clinical dataset.

Fig. 2.

Visual comparison of ResNet and VGG models using row-normalized confusion matrices, ROC curves, and t-SNE plots for five-class tooth condition classification. Rows represent different models (ResNet-50, ResNet-101, VGG-16, and VGG-19), and columns show confusion matrices, ROC curves, and t-SNE visualizations.

ML models

As presented in Table 3, LR showed a higher AUC, while XGB showed higher accuracy, sensitivity, and specificity. Confidence calibration analysis demonstrated that XGB had a lower ECE than LR, indicating that its observed accuracy was more aligned with its confidence. The reliability diagrams in Fig. 3 supported this finding, as a model with points lying closer to the diagonal line was better calibrated^38,39.

Table 3.

Performance of ML models.

Model	LR	XGB
Accuracy	0.862	0.869
Sensitivity	0.868	0.874
Specificity	0.853	0.862
AUC (95% CI)	0.896 0.851–0.939	0.885 0.847–0.925
ECE	0.041	0.024

Fig. 3.

Performance evaluation of the Logistic Regression and XGBoost models. The left column shows the reliability diagrams, the middle column presents the ROC curves, and the right column illustrates the decision curve analyses. The top row corresponds to the Logistic Regression model, and the bottom row corresponds to the XGBoost model. The reliability diagrams assess calibration performance, the ROC curves evaluate discriminative ability, and the decision curve analyses demonstrate the clinical net benefit across threshold probabilities.

As shown in Fig. 3, models with ROC curves closer to the upper left corner had better discrimination, as reflected by the AUC (ranging from 0 to 1). The DCA in Fig. 3 compared the clinical net benefit of the models across a range of threshold probabilities, where higher curves indicated a better expected benefit in preliminary triage^40,41.

Dental clinician comparison

As shown in Tables 4 and 5, a total of 276 cases were included in this validation. Given the higher reliability and overall performance of the ResNet-101 and XGB models, the comparison with dental specialists was performed using these models. The observed ORs with 95% CIs excluding 1.0 showed an acceptable level of association between model predictions and dental specialists’ preliminary decisions. Cohen’s kappa analysis showed moderate agreement beyond chance.

Table 4.

Agreement contingency tables between the AI framework and three dental specialist groups.

Model prediction	Group 1		Group 2		Group 3
	Yes	No	Yes	No	Yes	No
Yes	140	21	142	19	139	22
No	27	88	24	91	30	85

Yes = prosthodontic treatment, No = orthodontic treatment.

Table 5.

Association and agreement between AI framework predictions and decisions made by dental specialists.

Group 1		Group 2		Group 3
OR (95% CI)	Kappa (95% CI)	OR (95% CI)	Kappa (95% CI)	OR (95% CI)	Kappa (95% CI)
21.72 11.57–40.77	0.63 0.54–0.73	28.33 14.69–54.65	0.67 0.58–0.76	17.90 9.69–33.03	0.60 0.51–0.70

Discussion

This proof-of-concept study aimed to develop an AI framework integrating CNN object detection and classification models with ML methods to support preliminary triage between orthodontic and prosthodontic treatment options for S-TES management, without considering implant-based treatment. The primary significance of our study lies in moving beyond classification or segmentation toward automated preliminary clinical triage.

Among all evaluated CNN classifiers, ResNet-101 showed the highest macro AUC and weighted kappa with the narrowest CIs for both metrics. The stronger diagonal pattern in the confusion matrix and more separated clusters in the t-SNE visualization indicated that ResNet-101 not only showed accurate classification but also learned meaningful anatomical and restorative features of the teeth. In addition, it exhibited the lowest interclass AUC variability and showed the highest AUC for each clinical tooth condition independently compared with other classifiers. The results supported its selection as the most reliable model for classifying the clinical condition and anatomical category of teeth adjacent to the S-TES, and this balanced performance was critical for reliable preliminary triage between orthodontic and prosthodontic treatment. Models with the VGG architecture yielded acceptable macro F1 scores and AUC values; however, the t-SNE visualizations showed substantial overlap between clinical tooth conditions. This observation, together with the confusion patterns in the confusion matrix, raised the possibility that the VGG models may have relied on superficial visual features rather than clinically meaningful tooth morphology. In contrast, the residual architecture of ResNet enabled more effective learning of complex anatomical and restorative features. The final output with gradient-weighted class activation mapping (Grad-CAM) visualization showed that regions corresponding to S-TES were among the most influential areas contributing to the model’s predictions (Fig. 4).

Fig. 4.

Final output of the AI framework with Grad-CAM visualization. Activation intensity indicates the relative strength of neural activation across image regions, where warmer colors denote higher activation and greater contribution to the model’s prediction, while cooler colors indicate lower activation.

The XGB model demonstrated more reliable performance on this dataset. Analysis of the ROC curves indicated that both ML models showed high performance in separating cases that required orthodontic or prosthodontic treatment. Given the study’s clinically oriented aim of supporting preliminary triage, model reliability and calibration were key considerations. XGB showed lower ECE and better calibration compared to LR. Analysis of the reliability diagrams supported this finding, as the XGB model showed smaller deviations and less dispersion around the diagonal line, especially in the confidence range of 0.7 to 0.95, which was most important for preliminary clinical triage. In this range, higher confidence predictions were more aligned with higher true positive rates. Since DCA visualizations directly depend on probability (confidence) thresholds, these calibration differences translated into meaningful differences in benefit for preliminary triage. Analysis of the DCA visualizations showed that the XGB model provided a higher clinical net benefit than both the treat all (assigning all patients to prosthodontic treatment) and treat none (assigning all patients to orthodontic treatment) strategies across a wide range of threshold probabilities between orthodontic and prosthodontic treatment. This showed that the model balanced cases requiring orthodontic or prosthodontic treatment. In addition, the relatively stable net benefit across a broad threshold range demonstrated robustness to variations in threshold probabilities and consistent performance of the proposed AI framework across different clinical risk preferences. In comparison, LR exhibited a narrower range of clinical benefit, indicating lower reliability for the triage task. In light of these findings, the comparison to dental specialists was interesting and demonstrated that an AI framework with moderate performance may provide treatment recommendations comparable to those of dental practitioners. The OR values alone suggested a strong association between the proposed AI framework’s predictions and the specialists’ preliminary decisions. For example, in Group 1, the odds of specialists making a prosthodontic decision were about 21.7 times higher when the model also predicted a prosthodontic option than when it predicted an orthodontic option. However, the ORs should not be interpreted without considering their 95% CIs. The relatively wide 95% CIs indicated limited precision in the exact magnitude of the association, although the direction of the association remained clearly positive. Thus, interpreting the ORs together with their 95% CIs showed an acceptable level of association. Notably, this supported the potential clinical value of the framework within the clinical workflow and did not imply that the AI framework could replace specialist judgment. Simply put, the OR showed the degree of alignment between the AI framework’s treatment recommendation and the specialists’ decisions, rather than clinical interchangeability. Although the agreement between the proposed AI framework and the three groups of junior specialists was lower than the inter-annotator agreement within the experienced ground truth team, the level of agreement in this comparison remained moderate, encouraging, and clinically acceptable. This observation may suggest differences in clinical experience as well as differences in decision-making priorities and judgment patterns between experienced and less experienced clinicians^42,43. The observed agreement supports the clinical value of the framework; however, this may inadvertently increase the risk of automation bias in clinical practice, especially when a model demonstrates both high accuracy and high confidence. In clinical practice, automation bias refers to the tendency to rely excessively on AI systems, resulting in a reduced independent evaluation of the model’s predictions and insufficient consideration of conflicting clinical evidence. This risk may be mitigated by ensuring meaningful human oversight of the proposed framework’s predictions. For this purpose, several safeguards should be emphasized. First, clinicians should not accept the framework’s recommendations independently or uncritically. Instead, they should first make their own clinical judgment and then interpret the model’s prediction in relation to that judgment. An interesting study by Yin et al. compared three diagnostic workflows: ex post AI advice, in which AI recommendations were provided after an initial physician diagnosis; ex ante AI advice, in which AI recommendations were presented alongside the clinical information; and a control condition without AI. The authors found that the ex post workflow showed the best performance, and the control condition showed the worst⁴⁴. Second, the prediction should not be treated as a final model decision; instead, clinicians should evaluate the image regions that contributed to the prediction⁴⁵. Third, clinicians should be familiar with the framework’s documentation, including the clinical aims for which the model was developed, its limitations, the precise meaning of its inputs and outputs, and the conditions that fall outside the boundaries of its intended use. Without these three fundamental safeguards, the blind clinical use of the proposed framework or any guidance provided to the patient solely on the basis of a single prediction would be inconsistent with the aims of the present study and would not be clinically valid.

Currently, no research in the literature allows for direct comparison of these results; however, our findings can be indirectly compared with studies that used AI or ML methods to address related clinical workflow objectives. A study by Volovic et al.⁴⁶ employed multiple ML models to predict orthodontic treatment duration using structured pretreatment clinical variables, demonstrating the feasibility of ML applications for orthodontic workflow optimization. Their approach relied on manually recorded variables without incorporating direct and automated visual variables from clinical images, and focused solely on treatment duration. Similarly, a study by Mason et al.⁴⁷ developed ML models to predict orthodontic extraction and non-extraction decisions using pretreatment records. They reported AUC values ranging from 0.910 to 0.925, providing a meaningful benchmark and showing that ML models could support clinically relevant orthodontic decision-making within an acceptable performance range. A study by Prasad et al.⁴⁸ also used seven ML models on 700 orthodontic case records. For each patient, structured pretreatment variables were used as input data, and the models were designed to predict clinically relevant orthodontic diagnoses and treatment decision-making tasks. They reported an overall average accuracy of 0.84, with random forest and XGB achieving the highest accuracies. A recent study by Hoang et al. evaluated an AI-assisted digital workflow for designing 3D-printed lingual indirect bonding trays and assessed bracket transfer accuracy in vivo. Although their work did not address image-based diagnosis or triage support, it demonstrated that AI-assisted tray design achieved clinically acceptable transfer accuracy in most linear and angular dimensions. This supported the broader relevance of AI-assisted workflows in clinical practice and suggested that AI-based systems should be evaluated using task-specific clinical endpoints rather than relying only on overall performance measures⁴⁹. In contrast, our study exhibited several key strengths. The present study integrated CNN and ML models to support dentists in the preliminary assessment of two distinct treatment options at an early and fundamental stage of clinical evaluation. Instead of relying on manually recorded variables, our framework supported preliminary triage using clinically meaningful features automatically extracted from occlusal images, reducing reliance on manual feature input and making the clinical workflow more scalable and less operator-dependent. Rather than addressing a single clinical task, the proposed framework integrated S-TES detection, adjacent tooth characterization, S-TES mesiodistal width estimation, and preliminary triage support. This multitask design extended beyond approaches limited solely to detection, segmentation, or diagnosis. The framework was also compared with independent dental specialist groups, demonstrating agreement in a clinically meaningful preliminary assessment context, an approach that has been infrequently described in the literature. It is important to note that, in addition to traditional performance metrics, this study also evaluated confidence calibration using ECE and reliability diagrams. This aspect is highly significant for clinical evaluation, as preliminary clinical triage is often guided by predicted probabilities, and this dimension has been overlooked in prior studies^38,39. In addition, potential clinical utility was also evaluated using DCA, which extended the assessment to the clinical level by evaluating the model’s net benefit across a range of thresholds relevant to preliminary triage support in practice.

Our study has several limitations that warrant consideration. First, as a proof-of-concept, our dataset was annotated by dental specialists from a single dental center, and these defined ground truth labels were used for model training. A related study by Etemad et al.⁵⁰ used ML models to predict orthodontic tooth extraction and showed that models trained using ground truth labels from a single dental center may experience a reduction in performance by up to 20% when evaluated on a dataset from another dental center. Ground truth labels used in this study may partially reflect center-specific clinical practice patterns. As a result, the model’s performance may not fully generalize to other dental centers with different patient populations or clinical decision preferences. In other words, differences in preliminary decisions and available resources among dental specialists may lead to variations in judgment, which can affect model generalizability. Thus, the current findings should not be interpreted as generalizable to other clinical centers without external validation using independent multicenter datasets. Notably, this study was not designed to evaluate long-term beneficial patient outcomes. Thus, after multicenter external validation, prospective evaluation of patient outcomes would be necessary to determine the extent to which the proposed framework can improve clinical workflow, referral accuracy, and patient satisfaction. Second, converting pixel measurements of the S-TES mesiodistal width into millimeters may introduce a potential source of bias. Although the conversion relies on an average anatomical reference for the central incisors, this reference may be influenced by inter-individual and population-level anatomical variability, as well as by measurement and data entry errors. Third, our dataset did not include cases requiring the simultaneous initiation of both orthodontic and prosthodontic treatment. Although such cases may be uncommon, their clinical importance should not be overlooked. In contrast, our dataset included cases requiring both treatments with staggered initiation. In dental practice, such cases are considered for one of the two treatments at the initial stage, with the second treatment initiated subsequently after suitable clinical conditions have been established. Thus, although our framework could support preliminary triage for assessing the initial treatment priority in these cases, it was not designed to provide recommendations for subsequent treatment initiation or broader staged treatment planning. Fourth, treatment planning for orthodontic and prosthodontic methods cannot be determined solely from S-TES width and its two adjacent teeth. From a clinical perspective, treatment planning in these cases is influenced by other clinical factors, such as occlusal relationships, periodontal condition, and root morphology using radiographic imaging. Including these factors required larger and more representative datasets, and it was beyond the scope of this proof-of-concept study. Thus, the findings of this exploratory study should not be interpreted as supporting a comprehensive clinical tool for treatment planning, because the proposed framework was designed to support general dentists in preliminary assessment using limited initial clinical information before referral to an appropriate specialist, rather than to provide fully informed treatment planning or replace dental specialist judgment. Fifth, although the widths of the evaluated 95% CIs for the CNN and ML models were narrow and relatively narrow, the corresponding test sample sizes should not be interpreted as definitively sufficient, but rather as acceptable for this study. A more accurate assessment of sample size adequacy would require an a priori power calculation with a clearly defined endpoint. However, the proof-of-concept nature of the present study limited the feasibility of performing such a formal calculation. Sixth, multiple models were compared across several performance metrics without a predefined statistical correction. Thus, inflation of Type I error cannot be excluded, and these comparative findings should be considered exploratory. These limitations should be understood in the context of the current role of AI in clinical practice. Given existing challenges in the clinical implementation of AI in dentistry, such as limited data availability, low data quality, and limited population diversity, AI approaches should be expected to assist dentists with specific clinical tasks, complementing comprehensive clinician assessment rather than replacing it⁵¹.

A priority for future work is multicenter data collection and external validation. This should include collecting occlusal images and preliminary treatment decision labels from multiple dental centers representing different patient populations, imaging conditions, and clinical practice patterns. To improve consistency across centers, standardized case inclusion criteria and annotation protocols should be defined before model validation. In future multicenter collaborations, data sharing should be performed under ethical approvals and formal data sharing agreements. To protect patient privacy, shared datasets should contain no personally identifiable information and should be stored in encrypted systems with controlled access. Access should be restricted to authorized project supervisors, and any data used for annotation should be made available only to trusted dental specialists directly involved in the study. As a proof-of-concept, the present study demonstrated the feasibility of the proposed framework. Future work may extend its development using multicenter data and validate it on fully independent external cohorts to further assess robustness, generalizability, and reliability across diverse clinical settings. Future research should develop the proposed framework into a clinical tool with greater practical applicability and clinical benefit. In this context, additional CNN and ML models that have demonstrated strong performance in related studies can be investigated and systematically compared. The framework should also be further developed toward a comprehensive clinical decision support system by incorporating jaw anomalies and occlusal conditions, as these factors can significantly influence the assessment of orthodontic and prosthodontic treatment options. The current preliminary triage support model could be expanded through the integration of structured clinical variables obtained beyond occlusal photographs, such as prior treatment history. Future work could also develop a multimodal framework that processes both intraoral images and radiographs to incorporate periodontal condition, root morphology, and other important factors relevant to comprehensive treatment assessment. Such extensions would allow the proposed framework to move beyond preliminary triage support toward a broader clinician-supervised decision support tool. Also, extending the framework to provide recommendations for subsequent treatment initiation and broader staged treatment planning may represent a valuable direction for future research. From a translational and regulatory perspective, the proposed framework should be used as a support tool for the dentist’s initial assessment, with a defined intended use, intended user, and operational setting. Any pathway toward regulatory approval would require a locked final version for deployment on mobile or embedded devices using optimized runtimes such as TensorRT, Core ML, or ONNX Runtime. This would involve a formal clinical evaluation including multicenter external validation and prospective testing under routine practice conditions, as well as continued post-deployment monitoring to ensure calibration stability and performance drift. The most appropriate workflow integration would be at the initial dental examination. At this stage, the framework could analyze the image immediately after capture and provide a recommendation to support referral decisions regarding orthodontic or prosthodontic management. The recommendation should be reviewed by the dentist during the initial assessment, and the final decision should be made by the dentist. Thus, the most realistic implementation pathway is as a preliminary triage tool integrated into the routine workflow of the initial examination. The proposed framework may have potential economic relevance, as occlusal photographs are already obtained as part of routine clinical documentation. In principle, a clinically reliable preliminary triage support tool used at the first examination could help reduce inappropriate referrals, avoid unnecessary consultations, and support the more efficient use of clinical resources. However, these potential benefits remain hypothetical at the proof-of-concept stage and should be assessed in future prospective implementation studies using formal health economic evaluation. Finally, given that the current proof-of-concept primarily aimed to investigate the feasibility of the proposed AI framework, a more promising future direction involves developing an integrated, clinically applicable system to assist dentists in clinical assessment.

Conclusion

To the best of our knowledge, this is the first study to demonstrate the feasibility of developing an AI framework that solely uses occlusal photographs to support preliminary triage between orthodontic and prosthodontic treatment in cases requiring alternatives to implant-based interventions. Within the limitations of this proof-of-concept study, CNN and ML models, trained on a small and limited dataset in a single dental center, showed encouraging preliminary performance. These findings not only align with our predefined study objectives but also underscore the technical and clinical challenges of the proposed AI framework in its current form, highlighting critical areas for future refinement.

Supplementary Information

Below is the link to the electronic supplementary material.

Author contributions

E.S.: Conceptualization, Methodology, Software, Formal analysis, Visualization, Writing—original draft, Resources, Writing—review and editing, Data curation, Model development, Investigation. S.A.: Writing—review and editing, Data curation, Methodology, Visualization. M.M.: Validation, Writing—review and editing, Supervision. I.A.: Validation, Writing—review and editing, Project administration.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Data availability

The datasets used and analyzed during the current study are available from the corresponding author upon reasonable request.

Declarations

Competing interests

The authors declare no competing interests.