Effect of AI-assisted caries annotation on dental students’ performance in caries detection on panoramic radiographs

Abstract

Background

Dental caries remains one of the most prevalent oral diseases globally. The integration of artificial intelligence (AI) into dental radiographic interpretation, particularly for caries detection, has expanded rapidly. AI-assisted caries annotation may help dental students identify carious lesions on panoramic radiographs—a commonly used diagnostic tool for evaluating teeth and surrounding structures—thereby improving diagnostic accuracy, efficiency, and confidence. This study aimed to assess the effect of AI-assisted caries annotation on dental students’ diagnostic performance, confidence, and time efficiency in detecting caries on panoramic radiographs.

Methods

Fifty panoramic radiographs with multistage carious lesions, verified by bite-wing radiographs as the gold standard, were randomly selected. Caries were annotated using recently developed AI-assisted software. Forty fourth-year dental students participated after calibration with ten sets of unannotated and AI-annotated radiographs. In Session 1, participants identified carious lesions on 40 unannotated panoramic radiographs. One month later (Session 2), the same radiographs were re-evaluated with AI-assisted caries annotation, alongside an unannotated radiograph. For each radiograph, the location and depth of detected caries, diagnostic time, and self-reported confidence (0–10 scale) were recorded. Diagnostic accuracy, sensitivity, specificity, balanced accuracy, precision, negative predictive value, and miss rate between the sessions were compared using paired t-tests or Wilcoxon tests.

Results

AI-assisted caries annotation significantly enhanced diagnostic performance compared with conventional interpretation. Accuracy increased from 0.91 to 0.96, sensitivity from 0.35 to 0.67, specificity from 0.96 to 0.99, balanced accuracy from 0.65 to 0.83, precision from 0.33 to 0.77, negative predictive value from 0.95 to 0.98, while the miss rate decreased from 0.65 to 0.33 (p < 0.001). Students’ confidence improved notably for enamel caries (4.0 to 6.0), dentin caries (5.0 to 7.0), and pulp-involved caries (8.0 to 8.5), as well as overall detection (5.0 to 7.0) (p < 0.001). Moreover, the mean diagnostic time per radiograph significantly decreased from 67.89 to 53.92 s (p < 0.001).

Conclusions

AI-assisted caries annotation substantially enhanced diagnostic efficiency, reduced interpretation time, and improved dental students’ confidence in detecting dental caries across all depths. These findings highlight the potential of AI-assisted annotation as an effective educational adjunct for developing diagnostic competence in dental radiology.

Background

Dental caries is one of the most prevalent diseases globally [1] and has a major impact on the quality of life. Radiographic diagnosis of dental caries is a critical responsibility of dental practitioners. Undetected caries may progress deeper, leading to dental pain, increased treatment complexity, and higher healthcare costs. Therefore, diagnosing dental caries is an essential task for dentists. In terms of dental education, training undergraduate students to accurately diagnose caries is a crucial skill.

Nowadays, several studies on artificial intelligence (AI) in many aspects related to dentistry, e.g., medical charting, orthodontics, prosthodontics, and radiology, including the diagnosis of dental caries, are continuously increasing. Several studies using AI-based caries detection on bitewing radiographs have been reported [2–12]. Among these, three studies have examined deep learning (DL) methods for training dental students in caries detection on bitewing radiographs [5, 8, 12]. In dentistry, panoramic radiograph is the method of choice for evaluation and treatment planning of various conditions for dental treatment. It demonstrated teeth and adjacent structures, including the maxilla and mandible, with low radiation exposure. However, studies of caries detection using AI on panoramic radiographs remain limited [13–16]. The use of AI-assisted caries detection in panoramic radiographs has the potential to enhance diagnostic capabilities, support educational outcomes, and increase efficiency in diagnostic processes among dental students. Additionally, since students represent future dental professionals, assessing their interaction with AI aids in understanding its potential integration into clinical practice.

As the authors of the current study previously reported [17], the DL system using You Only Look Once version 5 (YOLOv5) for tooth detection and Attention U-Net for caries segmentation on panoramic radiographs showed strong agreement with oral and maxillofacial radiologists for caries detection and classification, minimizing missed lesions. Therefore, this study aimed to evaluate caries detection efficiency on panoramic radiographs by the fourth-year dental students using AI annotation. Additionally, the confidence level and time efficiency of dental students in detecting dental caries with and without AI annotation were also evaluated. We hypothesized that AI-assisted annotation would significantly improve diagnostic accuracy, confidence, and efficiency among dental students.

Methods

This study was approved by the institutional ethic committees (MU-MOU CoA 2024/048.1508). All methods were performed in accordance with the ethical principles of the Declaration of Helsinki. Informed consent was obtained from all participants.

Sample size calculation

To evaluate the effectiveness of AI annotation in dental caries detection by dental students, the sample size calculation was based on Schropp et al., who studied the AI software in proximal caries assessment on bitewing radiograph [12]. The sample size, calculated with a 95% confidence level, 21.6% standard deviation, and +/-6% margin of error, was 49.33. As this study had a finite population of 107 fourth-year students, the sample size was adjusted to 33.78. Therefore, 40 dental students in the fourth year of their undergraduate course (which has a duration of six years) were recruited for this study.

AI-assisted caries annotation

The recently developed dental AI-based software, designed to label caries on panoramic radiographs, was utilized in this study [17]. The YOLOv5 DL model was utilized for tooth detection, and the Attention U-Net architecture was applied for caries segmentation. During AI model development, the original panoramic radiographs were first processed into cropped single-tooth images, and these cropped tooth images—not the full panoramic radiographs—were used as the input data for model training. The cropped-tooth dataset was randomly divided into training and test subsets at a ratio of 75% and 25%, respectively. The images used for AI model training, validation, and testing were managed separately from the observer-performance dataset used in the present study.

Unannotated and AI-annotated panoramic radiographs

To prevent data leakage, the 50 panoramic radiographs used in the present observer-performance study were selected independently from the images used to train the AI model. These radiographs were used only for evaluating dental students’ caries detection performance with and without AI-assisted annotation. Within the present study, the 10 radiographs used for student calibration were also separated from the 40 radiographs used for the main evaluation. All selections and dataset splits were performed randomly. Panoramic digital radiographs were taken with a Kodak 9000 C system (Carestream Health, Rochester, NY, USA). Selection criteria included the presence of multistage dental caries—encompassing enamel, dentine, and pulp involvement— on at least one tooth surface, with minimal overlapping of the proximal surfaces of the posterior teeth. Examples of the studied panoramic radiographs are illustrated in Fig. 1.

Fig. 1.

Unannotated and AI-annotated panoramic radiographs. The original unannotated radiograph (a) and the radiograph with AI-assisted caries annotation displayed in pink color (b)

Across all panoramic radiographs, a total of 701 teeth were identified. Each tooth was divided into three sites—mesial, distal, and occlusal—resulting in a total of 2,103 sites. Among these, 151 sites exhibited dental caries (7.2%), comprising 60 sites with enamel caries (all proximal), 64 sites with dentine caries (61 proximal and 3 occlusal), and 27 sites with caries involving the pulp (13 proximal and 14 occlusal).

Dental students’ caries detection performance

All 40 observers provided informed consent prior to participation. Before radiographic evaluation, 10 sets of unannotated and annotated panoramic radiographs were used for calibration. This process was conducted by an experienced oral and maxillofacial radiologist (S.P.D.) to standardize the observers’ knowledge and skills in caries detection. Initially, 10 unannotated panoramic radiographs were presented, during which each observer was trained to classify caries as enamel (radiolucent area limited to the enamel), dentine (a radiolucent area extending beyond dentoenamel junction to dentine), and pulp involvement (a large radiolucent area reaching the pulp chamber resulting in widening apical periodontal ligament space and/or thickening apical lamina dura). Bitewing radiographs of the same patients, used as a gold standard for caries verification, were also provided for reference. Subsequently, the same panoramic radiographs with AI-assisted caries annotation were displayed. All observers were instructed regarding the potential errors in caries detection on panoramic radiographs both with and without AI annotations.

Following calibration, observers assessed the panoramic radiographs individually using their personal tablets. Image adjustments—including brightness, contrast, and zoom—were permitted. In Session 1, each observer reviewed 40 unannotated panoramic radiographs and was instructed to identify caries by tooth number, surface, and depth. One month later, in Session 2, the same radiographs were randomly re-evaluated alongside versions containing AI annotations highlighting carious regions. Observers were asked to assess both the unannotated and annotated images and record their caries detections in the same manner as in Session 1.

Observers were asked to rate their confidence levels in detecting caries at each depth— enamel, dentine, and pulp involvement— as well as their overall confidence, using a 10-point scale ranging from 1 (least confident) to 10 (most confident) in each session. In addition, the diagnostic time on caries detection for each panoramic radiograph was recorded in seconds.

Statistical analyses

In this study, correctly identified caries in terms of tooth number, surface, and depth were classified as “True Positive (TP)”, while the caries that were not detected were classified as “False Negative (FN)”. Sound surfaces incorrectly marked as caries were categorized as “False Positive (FP)”, and sound surfaces correctly identified as no caries were categorized as “True Negative (TN)”. Accuracy, sensitivity, specificity, balanced accuracy, precision, negative predictive value (NPV), and miss rate were calculated to evaluate observer performance and compare diagnostic outcomes between Session 1 and Session 2. The metrics were calculated as follows:

Accuracy =

Sensitivity =

Specificity =

Balanced accuracy =   

Precision =

Negative predictive value =

Miss rate =

Statistical analyses were performed using SPSS version 27.0 (IBM Corp., Armonk, NY, USA). The normality of the data distributions was assessed using the Kolmogorov-Smirnov test with Lilliefors significance correction. A paired t-test was applied to normally distributed variables, whereas the Wilcoxon signed-rank test was used for non-normally distributed variables. A p-value less than 0.05 was considered statistically significant. Effect sizes (Cohen’s d) were also calculated. In addition, confidence levels (rated 1–10) for detecting caries at various depths, along with evaluation time, were compared between the two sessions.

Results

The means or medians of accuracy, sensitivity, specificity, balanced accuracy, precision, negative predictive value, miss rate, confidence level at each caries depth, and diagnostic time during radiographic evaluation in Sessions 1 and 2 are summarized in Table 1. The integration of AI-assisted annotation in Session 2 led to significant improvements in all diagnostic validity measures compared to Session 1 (p < 0.001). Large effect sizes were observed across all diagnostic performance measures between Session 1 and Session 2. Cohen’s d values demonstrated substantial improvements in sensitivity, balanced accuracy, NPV, accuracy, specificity, and precision, while miss rate showed a large reduction. These findings indicate marked enhancement in diagnostic performance following the training.

Table 1.

Means or medians of accuracy, sensitivity, specificity, balanced accuracy, precision, negative predictive value, miss rate, confidence level, and diagnostic time

Test performance	Session 1	Session 2		p value
			Cohen’s d	Paired t-test	Wilcoxon test
Mean (95% CI)
Sensitivity	0.35 (0.31,0.38)	0.67 (0.62,0.72)	2.32	< 0.001*
Balanced accuracy	0.65 (0.63,0.66)	0.83 (0.80,0.85)	2.64	< 0.001*
NPV	0.95 (0.95,0.96)	0.98 (0.98,0.98)	1.88	< 0.001*
Miss rate	0.65 (0.62,0.69)	0.33 (0.28,0.38)	-2.32	< 0.001*
Median (IQR)
Accuracy	0.91 (0.02)	0.96 (0.01)	4.27		< 0.001**
Specificity	0.96 (0.02)	0.99 (0.01)	2.56		< 0.001**
Precision	0.33 (0.15)	0.77 (0.20)	3.36		< 0.001**
Confidence level
Enamel	4	6			< 0.001**
Dentine	5	7			< 0.001**
Pulp-involvement	8	8.5			< 0.001**
Overall	5	7			< 0.001**
Diagnostic time	67.89 s	53.92 s		< 0.001*

CI confidence interval, IQR interquartile range, NPV negative predictive value, s second

* Paired t-test, significant difference at p < 0.001; ** Wilcoxon test, significant difference at p < 0.001

Figure 2 illustrates the results for each observer in Sessions 1 and 2. Nearly all observers demonstrated higher accuracy, sensitivity, specificity, balanced accuracy, precision, and NPV accompanied by a lower miss rate in Session 2.

Fig. 2.

Dental students’ caries detection performance. Accuracy (a), sensitivity (b), specificity (c), balanced accuracy (d), precision (e), NPV (f), and miss rate (g) of each observer. The results in Session 1 are represented with grey line and in Session 2 with a pink line

The number of observers reporting increased confidence levels in Session 2 rose across all caries depths (Fig. 3). Participants also demonstrated significantly higher confidence levels in identifying caries at all depths (p < 0.001), as shown in Table 1.

Fig. 3.

Confidence levels at each caries depth in Session 1 (a), Session 2 (b), and across all depths (c)

The diagnostic time for caries detection by each observer in both sessions is presented in Fig. 4. The mean diagnostic time per radiograph was 67.89 s in Session 1 and 53.92 s in Session 2. The implementation of AI annotations significantly reduced the mean diagnostic time by approximately 20.58%.

Fig. 4.

Diagnostic time for caries detection by each observer in both sessions

Discussion

The findings of the present study support our hypothesis that AI-assisted annotation would significantly improve diagnostic accuracy, confidence, and efficiency among dental students. Specifically, participants demonstrated higher diagnostic performance, greater confidence in their interpretations, and reduced assessment time when using AI-assisted annotation compared with conventional assessment. These findings suggest that AI-assisted annotation can serve as a valuable tool for enhancing dental radiographic interpretation in an educational setting.

AI is increasingly being integrated into dentistry, enhancing diagnostic accuracy, treatment planning, and clinical decision-making. AI Applications in oral and maxillofacial radiology—such as the detection of dental caries, periodontal disease, and oral pathologies—have significantly improved diagnostic performance by automating disease detection and segmentation across various AI platforms. As AI continues to evolve, it holds great potential to improve patient outcomes, streamlining clinical workflows, and strengthening dental education through advanced diagnostic support systems.

The detection of pathologies in dental radiographs is essential for clinical decision-making and represents a routine diagnostic practice in dentistry. Consequently, training in radiographic interpretation is a fundamental part of undergraduate dental education. All fourth-year dental students who participated in this study had already completed preclinical courses in cariology and dental radiography, including the principles of radiology, panoramic radiography, and radiographic interpretation. In clinical practice, dental caries can be found at various depths. The radiographs selected for this study had a higher distribution of enamel and dentin caries than pulp-exposed caries. This may be beneficial to the students because enamel and dentin caries are more difficult to detect, whereas pulp-exposed caries typically present as larger lesions on radiographs and are therefore easier to identify. Nevertheless, the higher proportion of enamel and dentin caries compared with pulp-exposed caries may influence the overall diagnostic performance in this study. Since enamel and dentin lesions are generally more difficult to detect on radiographs, this imbalance may lead to a reduction in overall accuracy and related performance metrics. However, it may better highlight the potential benefit of AI-assisted annotation, particularly in improving detection of subtle lesions.

For caries detection, bitewing radiographs are generally recommended, particularly during a patient’s first dental visit. However, in certain cases, bitewing radiographs may not be obtained due to cost-related concerns. Panoramic radiographs are widely used for a comprehensive assessment of the maxilla and mandible, complementing clinical examinations by providing additional diagnostic information that is valuable for treatment planning. To date, studies investigating the application of various AI models for caries detection on panoramic radiographs remain limited [13–16]. One study evaluated the performance of an AI application for caries detection on panoramic radiographs in comparison with three junior dentists, and found that the AI software outperformed the dentists, who had one to two years of experience [13]. The authors recommended that junior dentists should receive support from AI until they gain experience, both for the speed of diagnosis and in the correct diagnosis of diseases. Another study employed the nnU-Net architecture to detect and classify the extent of caries on panoramic radiographs, comparing its performance with that of expert dentists with three to fifteen years of experience [15]. They concluded that the AI model’s performance did not differ significantly from that of experienced dentists in caries lesion segmentation.

Zhou et al. proposed a context-aware convolutional neural network for caries diagnosis on panoramic radiographs and found that the overall performance of the AI model was slightly lower than that of professional dentists with five years of experience. However, the AI demonstrated a significantly faster diagnostic speed. Therefore, the authors suggested that such AI systems could be effectively used to assist caries diagnosis, particularly for clinicians or students with limited experience [16].

In the context of dental education, only three studies have reported on the use of AI software for caries detection by dental students [5, 8, 12]. All of these studies evaluated the impact of AI on students’ ability to detect proximal caries using bitewing radiographs. However, the findings regarding AI-assisted caries detection in previous studies remain inconsistent. Ayan et al. evaluated the diagnostic performance of dental students trained with the YOLO algorithm for detecting enamel and dentine caries, compared to a control group without AI-based training. Their results demonstrated that students trained with AI achieved superior diagnostic performance, suggesting that AI intervention may enhance clinical education in dentistry [5]. Similarly, Devlin et al. conducted a pilot comparative study assessing dental students’ ability to detect enamel-only proximal caries on bitewing radiographs with and without the use of AssistDent AI software. They reported that the AI-assisted group showed significantly higher diagnostic accuracy compared with the control group without AI support [8]. The authors concluded that such AI tools could promote minimally invasive and preventive dentistry in both educational and clinical settings. In contrast, Schropp et al. evaluated the use of AssistDent^® for training third-year dental students and found that the AI software did not significantly improve the students’ ability to detect proximal enamel caries on bitewing radiographs [12].

In the aforementioned studies, students were divided into two groups. One group received training to identify caries using an AI application, and their ability to detect caries was subsequently compared with a control group that did not receive AI training. In contrast, our study employed a different methodology. Each student first evaluated a panoramic radiograph without AI annotation. The radiograph of the same case, now with AI annotation, was then presented alongside the original image, and the students evaluated both radiographs, recording caries according to various depths. This approach was chosen to reflect real clinical practice, where students may compare AI-annotated areas with the original radiograph of the same case when utilizing such software.

In this study, beyond assessing accuracy, we also evaluated the balanced accuracy to address the class imbalance caused by the large number of negative findings associated with normal surfaces. Based on the results of our study, AI-based caries annotation on panoramic radiographs could assist dental students in identifying carious lesions with improved accuracy, sensitivity, specificity, balanced accuracy, precision, NPV, and reduced miss rate. Most students demonstrated shorter diagnostic times for caries detection in Session 2. However, six students required longer time, as they needed to compare the original radiographs with AI-assisted annotated images. Regarding confidence levels, students reported increased confidence when using AI-assisted annotation, particularly in detecting caries in the enamel and dentine, which is crucial for preventing progression into the pulp. Nevertheless, these confidence levels were self-reported and may be subject to bias, as students might overestimate their diagnostic abilities.

With respect to radiographic assessment for proximal caries detection, bitewing radiographs are superior to panoramic radiographs owing to their higher spatial resolution and their ability to minimize image distortion and tooth overlap. In the present study, however, the design was intended to assess the effect of AI-assisted caries annotation on preliminary caries detection by dental students using panoramic radiographs, as this modality is widely utilized for initial assessment of the dentition and jaws. Nevertheless, due to certain limitations and potential errors in the AI software, as illustrated in Fig. 5, bitewing radiographs are still recommended in some cases to confirm the presence and extent of dental caries under the diagnostic expertise of experienced clinicians.

Fig. 5.

The original unannotated cropped radiographs (a, b) and the cropped radiographs with AI-annotation (c, d) illustrating a false positive at the lower left third molar (white arrow) and a false negative at the upper right second molar (black arrow)

The developed AI-assisted annotation has the potential to be systematically integrated into undergraduate dental training, particularly within radiographic interpretation curricula. It can be introduced as a supplementary learning tool in preclinical and clinical courses to support the development of diagnostic skills, especially in identifying early carious lesions in enamel and dentine. It may enhance diagnostic accuracy, confidence, and efficiency by providing immediate visual feedback and supporting pattern recognition. However, its use should be guided to ensure students continue to develop independent diagnostic reasoning and avoid overreliance on AI.

This study has several limitations. First, observers used personal tablets for radiographic evaluation to simulate real clinical conditions; however, variability in devices may have influenced the results. Second, the same observers evaluated the same radiographs in Session 1 and then again one month later in Session 2 with AI assistance. This design may introduce a substantial risk that part of the observed improvement is due to prior exposure rather than the AI annotations themselves. Moreover, there is a potential risk of overreliance among inexperienced users—specifically fourth-year dental students—which may hinder the development of independent diagnostic reasoning. Uncritical acceptance of AI outputs, particularly in cases of false positives or false negatives, may reinforce incorrect interpretations and limit exposure to radiographic variability. Consequently, AI should be used as a supervised adjunct to, rather than a replacement for, traditional training. In addition, performance metrics in this study were evaluated based on the accuracy of identifying tooth number, surface, and lesion depth. Only confidence levels were analyzed using depth-specific stratification. Extending depth-specific analyses to the other performance metrics would yield more comprehensive insights into students’ diagnostic capabilities and limitations in interpreting dental caries on radiographs across varying lesion depths. Accordingly, future research should incorporate depth-specific stratification for all relevant performance metrics to enhance the rigor and interpretability of findings.

The study was also limited to a single cohort from one institution, with the sample size constrained by time. Furthermore, the observed improvements may reflect a cueing effect (i.e., visual guidance) rather than a true enhancement of independent diagnostic proficiency. Therefore, the long-term educational impact on autonomous diagnostic ability remains unclear. Future research should include a broader range of participants, such as senior dental students and junior clinicians across multiple institutions, as well as a more diverse dataset of carious lesions. Additionally, to better assess the true educational impact of AI, a post-training evaluation without AI assistance—ideally using a separate set of radiographs—is warranted to determine whether improvements are retained independently of AI support.

Conclusions

The implementation of AI-assisted annotation on panoramic radiographs significantly enhanced diagnostic accuracy, increased observer confidence, and improved time efficiency for dental caries detection across all depths by the fourth-year dental students. We propose that the integration of AI annotation can contribute to the clinical education of dental students by supporting routine panoramic radiographic interpretation for caries detection as a supportive tool, with appropriate supervision. However, future study of post-training evaluations without AI assistance are essential to determine whether students can effectively retain and transfer these skills to an unaided clinical context.

Abbreviations

AI

Artificial intelligence

CI

Confidence interval

DL

Deep learning

FN

False negative

FP

False positive

IQR

Interquartile range

NPV

Negative predictive value

TN

True negative

TP

True positive

YOLO

You Only Look Once

YOLOv5

You Only Look Once version 5

Authors' contributions

SPD contributed to the conceptualization, resources, methodology, analyzation, discussion, and critical reviewing the manuscript. JK contributed to the conceptualization, data analyzation, writing, reviewing, editing, and was the major contributor in writing the manuscript. SV contributed to writing the AI model algorithm, data analysis. PM contributed to the conceptualization, design, and critically reviewing the manuscript. PS, PW, PC, SB performed data collection and analyzation, writing original draft preparation, and discussion. All authors read and approved the final manuscript.

Funding

This study was supported by the Faculty of Dentistry, Mahidol University.

Data availability

The data that support the findings of this study are available from the corresponding author on reasonable request.

Declarations

Ethics approval and consent to participate

The ethical approval for the study was approved by the Ethics Committee of Mahidol University Multi-Faculty Cooperative IRB Review (MU-MOU CoA 2024/048.1508). This study involved human participants and was conducted in accordance with the ethical principles of the Declaration of Helsinki. Informed consent was obtained from all participants prior to their participation in the study.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.