A phantom study on the evaluation of the YOLOv8 deep learning model in the detection of tooth ankylosis on cone-beam computed tomography images

Abstract

This study aimed to assess the performance of a deep learning algorithm for detecting tooth ankylosis on cone-beam computed tomography (CBCT) scans. A dataset of CBCT scans taken from phantoms simulating tooth ankylosis, including 60 teeth and a total of 4971 axial sections, was used to design an ankylosis detection model using the YOLOv8 algorithm. The first dataset (42 teeth, 70%) with 3822 axial sections comprised the training dataset, while the second dataset (18 teeth, 30%) with 1149 axial sections comprised the test dataset. The YOLOv8 algorithm was used to optimize the precise and efficient detection of the ankylotic area. Training loss was monitored during the learning process. Also, precise optimizations such as adaptive learning rate were used to ensure model convergence. The dice, precision, recall, and F1 score were calculated to assess the model’s performance. Training loss of the designed model reached < 10% after 100 training epochs. The model showed 80% precision, 82% dice, 90.9% recall, and 83.9% F1 score. The designed model using the YOLOv8 algorithm performed optimally in efficient and precise detection of tooth ankylosis on CBCT scans and could be integrated into the clinical workflow, aiding clinicians and radiologists. Future studies are recommended to focus on further refinement of this model using larger datasets, architectural optimization, combining in-vivo and in-vitro datasets for model generalizability, and comparing radiologists’ and the model’s performance.

Introduction

Tooth ankylosis is defined as the fusion of the root cementum or superficial root dentin with the adjacent alveolar bone that occurs in the absence of a normal periodontal ligament (PDL). PDL includes the connective tissue, collagen fibers, and vasculature [1]. In tooth ankylosis, a tight connection between the alveolar bone and tooth structure replaces the PDL structure following the proliferation of bone tissue into the PDL space [2]. Resultantly, tooth eruption may be impaired due to the impossibility of physiological movement [3]. Ankylosis can be classified into pre- and post-eruption subtypes based on occurrence time. Pre-eruption ankylosis may be related to genetic factors, endocrine disorders, craniofacial malformations, and skeletal dysplasia. Post-eruption ankylosis may occur following dentoalveolar trauma, traumatic surgical procedures, orthodontic treatment, or idiopathic factors [2, 4].

Early detection of ankylosis is imperative for comprehensive treatment planning to prevent malocclusion. Since ankylosis prevents alveolar bone growth and development, it may alter the eruption path of the adjacent teeth and lead to progressive changes in tooth position and the subsequent need for unnecessary therapeutic procedures [3, 4].

Tooth ankylosis should be suspected when a tooth does not respond to orthodontic force application, or when problems such as infra-occlusion of the involved tooth, supra-eruption of the opposite tooth, or inclination of the adjacent teeth are present. Nonetheless, the clinical diagnosis of ankylosis is prone to errors because several factors may impair tooth eruption [2, 3, 5]. Since direct observation of intra-bony structures is not feasible in clinical examination, imaging serves as a valuable diagnostic aid in assessing the possibility of tooth ankylosis [6]. Periapical radiography and panoramic radiography are commonly requested for the detection of ankylosis. Nonetheless, two-dimensional (2D) imaging modalities have limitations that may hinder an accurate diagnosis, because the ankylosed areas in the buccal, palatal, and lingual root surfaces are not visualized on 2D radiographs. Also, identifying areas of local ankylosis is usually difficult on 2D radiographs [2, 7].

Currently, cone-beam computed tomography (CBCT) serves as a precise and reliable three-dimensional (3D) imaging modality that can be used to evaluate teeth with eruption problems. The 3D nature of CBCT images enables better assessment of the dentoalveolar complex by eliminating adjacent overlapping structures, and providing multiplanar slices of the respective area, enabling 3D observation of volumes and objects [8–10]. The superiority of CBCT to the conventional imaging modalities for detecting periodontal problems has also been reported [11]. Nonetheless, detecting tooth ankylosis on CBCT scans is time-consuming and challenging even for experienced dental clinicians [6, 12]. Furthermore, limited information is available regarding the protocols of ankylosis detection on CBCT scans [6, 13].

Deep learning (DL) is increasingly used for medical imaging diagnostic purposes. It is a subset of artificial intelligence (AI) that can design a model according to a dataset by creating a connection among several internal nodes. A convolutional neural network (CNN) is a subset of DL algorithms that employs a convolutional operation in the neural network layers. CNN significantly improves performance in many fields of medical image analysis and has applications in many dental fields as well [14].

The present study was undertaken to assess the performance of a DL model for detecting tooth ankylosis on CBCT scans, which was designed using the Ultralytics YOLOv8 DL algorithm.

Materials and methods

This study used CBCT scans taken from phantoms simulating tooth ankylosis, including 60 teeth and a total of 4971 axial sections. The sample size was calculated according to a previous study [15]. This study was approved by the ethics committee of Shahid Beheshti University of Medical Science (IR.SBMU.DRC.REC.1402.078) and was conducted according to the latest version of Declaration of Helsinki.

Simulation of tooth ankylosis

Sixty single-rooted mandibular canines and premolar teeth extracted for reasons not related to this study were used. The teeth were immersed in 5.25% sodium hypochlorite for 30 min, the residual attachments on them were scraped off with a toothbrush and then stored in saline.

The teeth were subsequently mounted individually in type I dental stone to provide support during the vacuuming process and coated with thermoplastic sheets (Forplast, Roko dental systems, Czestochowa, Poland) with a thickness of 1 mm using a vacuum former (Easy-Vac2, 3 A MEDES, Gyeonggi-do, Republic of Korea) to simulate the PDL space (Fig. 1).

Fig. 1.

a The mounted tooth before covering with a thermoplastic sheet. b uncovered samples arrangement on the vacuum former (c) samples are covered with a thermoplastic sheet

To ensure a uniform thickness along the roots, the drop of the thermoplastic sheet after the onset of heating and before vacuuming was considered 4 cm. The mean thickness of the covering layer at different parts of the root (measured at one point in the coronal, one point at the center, and one point in the apical) was 125 ± 25 μm, as measured by a digital caliper (Song Young Int CO, Taipei, Taiwan).

To simulate tooth ankylosis, a small part of the applied coating measuring 2 × 2 mm was carefully removed by a #15 C scalpel (Moris, Hamburg, Germany) from a random location in either of the cervical, middle, or apical third of the root from either of the buccal, lingual, mesial, or distal root surfaces from all samples.

A negative imprint of a 3D-printed human model of the mandible was made using additional silicone impression material (Betasil vario putty soft, Mueller-Omicron, Lindlar, Germany). The impression was poured 10 times with type III dental stone and ground rice (to simulate bone trabeculae) in a 2:1 ratio [16]. Coated teeth were randomly assigned to 10 groups (n = 6, 6 teeth in each mandible model) and mounted in the unset mixture of stone-rice after recording the precise location of ankylosis and model number for each tooth in a table. A slurry mixture with a lower concentration was prepared and applied to the uncoated areas to ensure complete penetration of the stone-rice mixture into these root areas. Fig. 2 shows the process in brief.

Fig. 2.

a A 3D-printed model of the mandible; (b) making an impression from the model; (c) pouring the impression and mounting the teeth in the cast; (d) final simulated model

CBCT and data collection

The phantoms were placed in a cylindrical polypropylene container measuring 16 × 16 × 17.7 cm filled with water. The water thickness around the model ranged from 20 to 50 mm [17]. The CBCT scans of the first dataset (42 teeth, 70%) were obtained by the NewTom VGi Evo system (Newtom, Verona, Italy) with the exposure settings of 8 × 8 cm field of view, 0.150 mm voxel size, 110 kVp potential, 11.86 mA, and 5.4 s exposure time. The CBCT scans of the second dataset (18 teeth, 30%) were obtained with a CS 8100SC 3D scanner (Carestream Dental, Atlanta, USA) with the exposure settings of 8 × 9 cm field of view, 0.150 mm voxel size, 90 kVp potential, 4 mA current, and 15 s exposure time. Fig. 3 represents axial, coronal, and sagittal CBCT images of a simulated ankylotic tooth.

Fig. 3.

Cone beam computed tomography images of a phantom. The circles in a, and b indicate the simulated ankylotic areas. a axial, (b) coronal, and (c) sagittal images of a first premolar with simulated ankylosis in the mesial middle third of the root. (images created with a VGi Evo (Newtom, Verona, Italy))

All CBCT scans were saved in DICOM format and exported.

Data processing

The first dataset (7 volumes, 42 teeth, 3822 axial sections) comprised the training dataset, while the second dataset (3 volumes, 18 teeth, 1149 axial sections) comprised the test dataset. Accordingly, the data were assigned to the train and test groups in a 7:3 ratio. The area of ankylosis was manually marked in all sequential slices of each volume according to the respective table (ground truth) by a senior oral and maxillofacial radiology resident under the supervision of an expert radiologist using ITK-SNAP 4.2 software.

In brief, the process of marking included the following two steps:

• (I)The image was first rotated by the 3D cursor in the required amount for reorientation of the respective tooth
• (II)Next, all ankylotic areas in each volume were marked in the axial window. To increase the accuracy, they were reassessed in the sagittal and coronal windows as well.

Due to the small size of the respective area, the 3D-brush option was disabled during marking to decrease the possibility of false positive marks. The segmentation process was performed with high precision by the paintbrush tool, size 3–5, depending on the size of the simulated PDL.

Developing the DL model using the YOLOv8 algorithm

Ultralytic’s YOLOv8 algorithm is an advanced DL model used for numerous tasks, including pose estimation, classification, object detection, and segmentation. In the present study, it was used for identifying tooth ankylosis and its delineation. It is composed of a CNN with three primary building blocks: head, neck, and backbone. The backbone is responsible for data extraction from the model. The neck serves as a connecting organ and fuses the background information through feature fusion. The head is responsible for designing the final output. Its subgroups include 8n (nano), 8s (small), 8 m (medium), 8 L (large), and 8x (X-large), which vary depending on the number of parameters and size of the network. The depth, width, and number of network parameters increase from 8n to 8x. The simpler and the fewer the layers, the lighter and faster the trained model would be, but with a lower accuracy. More complex models have a heavy structure, high precision, and slow performance [18]. In the present study, to select the most suitable configuration, a benchmark concerning different YOLOv8 subtypes was performed on the training dataset. After assessing and comparing different subtypes, considering precision, speed, and parameters, the medium model was chosen. A graphical representation of the YOLOv8 model is shown in Fig. 4.

Fig. 4.

Overview of the YOLOV8 model. The main structural components are the head, neck, and backbone. The backbone extracts important information from the input. The neck functions as a connector, and the head detects ankylosis as output

The intensity window for all CBCT scans and corresponding labels was set from 600 to 1600 HU, so the data was normalized into this range. To improve model performance, different data augmentation techniques, including horizontal flip, contrast change, and rotation, were used. The model was trained with 100 training epochs using CBCT images and manual masks.

In this study, an independent validation set was not partitioned from the primary dataset to maintain the number of samples that can be used for model training. Instead, we implemented a fivefold cross-validation procedure on the training dataset to validate the data, and the performance measure was the average of the values computed in the fivefold loop.

In the training process, the level of training loss was monitored to assess the learning capacity of the model. For this purpose, the batch loss function was enabled to detect false predictions and guide the model towards more accurate results. Also, precise optimization methods, such as adaptive learning rate, were used to ensure model convergence.

The hyperparameters of the network, including batch size, learning rate, and training epoch, were 16, 0.001, and 100, respectively, and Adam’s optimizer was used for training. Early stopping was used to prevent overfitting, and the corresponding parameter was set to 100. Coding was performed in Python 3.12.3 and implemented on an NVIDIA 3060Ti GPU.

Data analysis

Considering the detection of ankylosis as a binary classification with ankylosis areas as a positive class, and other areas as a negative class, the pixels predicted by the model as ankylosis were compared with the ground truth (marked areas) and subsequently, true positive (TP), false positive (FP), true negative (TN), and false negative (FN) variables were calculated. These variables were then used to calculate the dice coefficient, precision, recall, and F1 score to evaluate the model’s performance.

Dice is a commonly used index for the assessment of the performance of models. It measures the agreement between the predicted data and the reference data, and was calculated as follows:

Precision indicates the ability of the model to correctly detect true positives and reduce false positives when differentiating tooth ankylosis patterns. It was calculated using the following formula:

The recall and IoU was also calculated as follows:

Since precision and recall are inversely correlated, the F1-score can serve as a reliable performance index by mixing them through the calculation of their harmonic mean:

All statistical analyses were carried out using SPSS version 24 (SPSS Inc., IL, USA).

Results

In the process of 100 training epochs, a significant reduction occurred in the training loss such that it reached less than 10% (Fig. 5), pointing to the high performance of the YOLOv8 model in learning tooth ankylosis patterns on CBCT scans.

Fig. 5.

Training loss curve. The horizontal axis represents the training epochs, and the vertical axis represents the loss value. It can be seen that as training progresses, the overall loss decreases to less than 10% after 100 epochs

The highest precision of the designed YOLOv8 model reached approximately 80%. Fig. 6 shows the performance of the model in terms of precision during the training and validation phases. As shown, precision gradually improved and reached nearly 80% after 80 training epochs.

Fig. 6.

Precision curve. The horizontal axis is training epochs, and the Vertical axis is precision level. Precision approaches approximately 0.8 after 80 epochs, indicating good precision

The dice index of the YOLOv8-based model was calculated to be 82%, indicating a high agreement between the prediction and ground truth. Recall was calculated to be 90.9%, IoU of 74% and the F1-score was found to be 83.9%.

Figure 7 illustrates representative outputs of the detection model and compares manually contoured (mask) images with automatically diagnosed images using the YOLOv8 model. High agreement between the two image sets indicated the optimal performance of the model in the precise detection of ankylosed areas. High visual agreement between the reference data and automated output confirmed model strength.

Fig. 7.

Representative outputs of the model: a comparison between the original images (first column), manually contoured images (second column), and automatically diagnosed images (third column) by using the YOLOv8 algorithm

Discussion

This study assessed the performance of a DL model for detecting tooth ankylosis on CBCT scans, which was designed using the Ultralytics YOLOv8 DL algorithm. The results showed that the training loss of the developed model reached < 10% after 100 training epochs. The model showed 80% precision, 82% dice, 90.9% recall, and 83.9% F1 score.

Martins et al. [16] evaluated the diagnostic sensitivity and specificity of CBCT for detecting tooth ankylosis by oral radiologists using different imaging protocols and voxel sizes. They reported low mean sensitivity (0.32 to 0.49) and relatively high specificity (0.71 to 0.83). Low sensitivity values in their study confirmed the difficult detection of tooth ankylosis on CBCT scans. Difficult detection of ankylosis and a high rate of false negative results may be due to the small size of ankylosed areas. When the size of the ankylosed area is smaller than the selected voxel size for imaging, the risk of missing the ankylosed area increases due to the generation of a partial volume-averaging artifact [19]. Ducommun et al. [6] also reported low specificity of CBCT for detecting tooth ankylosis, indicating difficult confirmation of the absence of ankylosis. They added that although CBCT is a beneficial diagnostic modality, it cannot be used as the sole diagnostic tool for detecting tooth ankylosis due to its high false-positive rate [6]. The high rate of false positive results in their study may be attributed to the possible occurrence of blooming artifacts due to lamina dura, making it appear thicker on the images, and causing false positive results [20].

Evidence shows that AI algorithms have high potential in automated aided diagnostics. In general, AI algorithms can be used for several purposes, including classification, segmentation, detection, and tracking, among others [20]. Image classification is used in medicine and dentistry for screening and diagnosis of disease conditions. For instance, Hu et al. [20] used the ResNet50 classification algorithm to design a model for detecting vertical root fractures. It should be noted that the classification algorithms cannot find the location of an object in an image. Thus, the designed classification models cannot identify the location of vertical root fractures or detect multiple vertical root fractures on an image. Therefore, detection algorithms were used in the present study, considering the clinical significance of detecting the location of ankylosed areas on the images for orthodontic and surgical purposes, and the possible presence of several ankylosed areas in one tooth.

The DL algorithms used for detection purposes (referred to as detectors) can be divided into two general groups: one-stage detectors, such as YOLO (you only look once), and two-stage detectors, such as RCNN (regional-based CNN) [21, 22]. The YOLO algorithms, compared with the RCNN, acquire information directly without requiring a region proposal network. The YOLO algorithms were developed from the YOLOv1 algorithm, and YOLOv8 is the optimized version based on YOLOv5, offering unique properties in precise and fast detection of objects [22]. The YOLOv8 algorithm emphasizes a balance between precision and speed and is therefore suitable for real-time assessments. In fact, a major part of the occupational power required by this algorithm is related to the training phase, and the model outcome can be presented even in personal computers commonly used in office settings and hospitals [23]. Xie et al. [24] evaluated the possible use of the YOLOv8 DL algorithm to detect tooth cracks in optical images. The precision, recall, and F1-score in the YOLOv8 algorithm were 83.9%, 79.4% and 81.6% respectively, which were higher than the corresponding values in other YOLO versions (YOLO3, 4, 5, 7, and X), the Faster RCNN, and the Centernet al.gorithm [24]. Similar studies on the detection of bone fractures also confirmed the optimal detection performance of this algorithm [23, 25]. Nonetheless, the detection of tooth ankylosis with the help of DL algorithms has not been previously investigated.

DL algorithms usually require a large pile of data for optimization of parameters. The number of images with a confirmed definite diagnosis of tooth ankylosis is highly limited compared with other dental problems, such as caries and periodontal disease, or anatomical variations. Additionally, sectioning and histological analysis of the suspected tooth is the only reliable method (gold standard) for observation of PDL loss and confirmation of the diagnosis of tooth ankylosis, which is clinically feasible only after tooth extraction, and is therefore unacceptable for research purposes (as in the present study) or even diagnostic purposes [13]. Therefore, phantoms simulating tooth ankylosis were used in the present study and underwent CBCT.

PDL is usually simulated for in vitro assessment of radiographic detection of ankylosis. Ozmeric et al. [12] designed a phantom simulating teeth and PDL space with 0.1-, 0.2-, 0.3-, and 0.4-mm widths to compare the performance of CBCT and conventional radiography to evaluate the PDL space. However, in the relevant study, the use of artificial composite teeth instead of real human teeth constitutes a limitation in simulating the clinical setting. Also, Martins et al. [16] simulated tooth ankylosis to assess the diagnostic efficacy of CBCT with different exposure protocols for detecting ankylosis. The PDL width in their study was 0.25 mm, which was larger than the normal mean width of PDL, i.e., 0.15 to 0.21 mm in the clinical setting. PDL width is an important parameter affecting the detection of ankylosis. The phantoms designed in the present study simulated a PDL with 0.1–0.15 mm width.

Since tooth ankylosis may occur along the entire PDL space, and the ankylosed area is observed continuously in consecutive sections, 2D CBCT sections were used to train the model in the present study. Yeshua et al. [26] used axial CBCT sections to train a model for detecting benign lesions in the maxillofacial region and reported its optimal performance using the test data. This method was also used in some recent studies to design a classification model to assess the correlation between the inferior alveolar nerve and mandibular third molar position and detection of root fracture on CBCT scans [27, 28].

The model designed in the present study with the YOLOv8 algorithm showed high precision in detecting tooth ankylosis. However, this study had some limitations. Possible patient movements during CBCT scanning (motion artifact) and other possible sources of artifact in the clinical setting can affect the image quality and resultantly the model performance in the detection of ankylosed areas, which were not simulated in the present study. Also, only single-rooted teeth were assessed.

Future studies are required to design variable simulations, such as simulation of ankylosis of multi-rooted teeth, especially in the furcation area, modeling of maxillary teeth, and simulation of ankylosis of fully impacted maxillary and mandibular teeth, to improve the performance of the model. Also, further studies are recommended on CBCT images of ankylosed teeth with a definite diagnosis of ankylosis by radiologists to assess the model performance after training with a combination of in vitro and in vivo datasets. Furthermore, the precision and time to detect ankylosis should be compared between radiologists and the designed model. The YOLOv8 algorithm still has room for further refinement, using larger datasets and advanced techniques, as well as its architectural optimization to maximize its performance. Other DL algorithms may be used to train ankylosis detection models, and their performance may be assessed and compared with YOLOv8.

The designed model using the YOLOv8 algorithm performed optimally in efficient and precise detection of tooth ankylosis on CBCT scans and could be used as a diagnostic tool, aiding clinicians and radiologists in detecting tooth ankylosis with increased reliability. Also, integrating this model into the clinical workflow could increase its dicoverability as an incidental finding on CBCT images ordered for other reasons.

Authors’ contributions

The concept suggested by M.M. S.V design the study. Formal analysis provided by S.S. Methodology, Writing and editing provided by N.N and P.P. All authors reviewed the manuscript.

Data availability

The datasets analyzed during the current study are available from the corresponding author upon request.

Declarations

Ethics approval and consent to participate

Not applicable.

Competing interests

The authors declare no competing interests.