Deep learning-based identification of mesiodens using automatic maxillary anterior region estimation in panoramic radiography of children

Abstract

Objectives:

The purpose of this study is to develop and evaluate the performance of a model that automatically sets a region of interest (ROI) and diagnoses mesiodens in panoramic radiographs of growing children using deep learning technology.

Methods:

Out of 988 panoramic radiographs, 489 patients with mesiodens were classified as an experimental group, and 499 patients without mesiodens were classified as a control group. This study consists of two networks. The first network (DeeplabV3plus) is a segmentation model that uses the posterior molar space to set the ROI in the maxillary anterior region with the mesiodens in the panoramic radiograph. The second network (Inception-resnet-v2) is a classification model that uses cropped maxillary anterior teeth to determine the presence of mesiodens. The data were divided into five groups and cross-validated. Deep learning model were created and trained using Inception-ResNet-v2. The performance of the segmentation network was evaluated using accuracy, Intersection over Union (IoU), and MeanBFscore. The overall network performance was evaluated using accuracy, precision, recall, and F1-score.

Results:

Segmentation performance using posterior molar space in panoramic radiographs was 0.839, IoU 0.762, and MeanBFscore 0.907. The mean values of accuracy, precision, recall, F1-score, and area under the curve for the diagnosis of mesiodens using automatic segmentation were 0.971, 0.971, 0.971, 0.971, and 0.971, respectively.

Conclusions:

The diagnostic performance of the deep learning system using posterior molar space on the panoramic radiograph was sufficiently useful. The results of the deep learning system confirmed the possibility of complete automation of the classification of mesiodens.

Introduction

Supernumerary teeth are defined as teeth that exceed the normal number of teeth. When a supernumerary tooth exists between maxillary central incisors, it is called mesiodens. Mesiodens are presumed to be caused by a combination of genetic and environmental factors. Although the exact cause of mesiodens has not been explained, recent studies have shown the cause as genetic factors. ^1,2 The prevalence of mesiodens varies by population, ranging from 0.1 to 3.6% in permanent dentition and 0.3–0.8% in mixed dentition. Mesiodens is twice as common in males as in Asian females, and the prevalence varies by race. ^3–5

Mesiodens vary in shape and size, and most of them are impacted on the palatal side. If they are not extracted at the appropriate time, they may cause complications such as displacement of the maxillary incisor, diastema, delayed eruption of adjacent permanent teeth, resorption of the roots of adjacent permanent teeth, and crowding in the anterior area. ^6,7 Therefore, an early detection of mesiodens is important to prevent such complications, and if not removed at the appropriate time, it can grow into an abnormal dentition. ^8–10

To extract an impacted mesiodens, its exact shape and positional relationship with adjacent teeth should be identified. Recently, cone beam computed tomography (CBCT) has been used to obtain important diagnostic information such as the three-dimensional position, shape, impact direction, and relationship of the mesiodens with the adjacent teeth. ¹¹ However, as children grow and develop, they should be treated with more caution than adults when it comes to radiation exposure, and CBCT should only be performed when essential. ^12,13 Conversely, panoramic X-ray can obtain substantial information in the maxillofacial area with a single exposure. Images obtained with the panoramic X-ray can provide information about not only the cause of the patients' chief complaints, but also their other problems. ¹⁴ In the case of growing children, it is common in clinical practice to find mesiodens in panoramic images obtained for examination purposes. ¹⁵ However, depending on the clinician’s skill level, it may be difficult to accurately read panoramic radiographs owing to overlapping of various anatomical structures and distortion, enlargement, and reduction of images due to various problems occurring during imaging. ¹⁶

With the recent development of artificial intelligence (AI), a deep learning-based convolutional neural network model has been used in the medical field to diagnose images in various areas, and its accuracy and efficiency are often excellent. ¹⁷ In recent years, deep convolutional neural network (DCNN) technology has been used in various fields in dentistry, including radiographic imaging diagnosis. ¹⁸ It was reported to perform exceptionally well in detecting dental caries, periodontal diseases, maxillary sinusitis, radiolucent lesions, osteoporosis, and the presence of mesiodens on panoramic radiographs. ^19–25

As research on AI is actively progressing, diagnostic methods using AI are being further developed. Previous studies on the classification of mesiodens using deep learning technology have exhibited more than 80% accuracy in both permanent and mixed dentitions. ^25,26 However, to obtain high accuracy from the deep learning model, an image preprocessing process in which a person manually sets the region of interest (ROI) predicted to have mesiodens was required. In particular, in the anterior region of the panoramic radiograph of the mixed dentition, there are unerupted permanent teeth and many anatomical overlaps, so accurate interpretation can be difficult. ²⁵ Therefore, it is difficult to set the ROI in the anterior region of the panoramic radiograph. In diagnosis using medical images, ROI setting is an important process that affects the performance of deep learning; therefore to improve accuracy, adequate preprocessing is needed to obtain as small a ROI as possible that provides enough context. If this manual segmentation process can also be replaced by AI, the diagnosis will be more time-saving and efficient.

Because the ROI was set up manually, few previous studies can be regarded as having performed fully automated processes using AI. In this study, we used a two-step AI system that automatically segmented the anterior ROI and classified the mesiodens of the panoramic radiograph. Therefore, the purpose of this study is to develop a model that fully automates the segmentation process for ROI setting and mesiodens diagnosis based on panoramic X-ray image of growing children and evaluate its performance.

Methods and materials

This study was conducted with the approval of the Research Ethics Review Committee (Institutional Review Board, IRB) of Pusan National University Dental Hospital (IRB No.: N-2020–002-IIT).

Subjects

This study was conducted on patients who had their panoramic and CBCT radiographs taken at Pusan National University Dental Hospital between January 2010 and January 2020. Patients who were diagnosed with one or more mesiodens by CBCT were selected as the experimental group, and those without mesiodens were selected as the control group. All patients were in the early stage of primary dentition or mixed dentition. Some panoramic radiographs, difficult to read due to severe image distortion, were excluded from the study. Finally, panoramic radiographs of 489 and 499 people in the experimental and control groups, respectively, were extracted and used in the study. The age and gender distribution of the subjects of this study are shown in Table 1.

Table 1.

Age and gender distributions of the subjects in this study

Characteristic	Patients without mesiodens (control group) (n = 499)	Patients with mesiodens (experiment group) (n = 489)	Total (n = 988)
Mean age (SD)	7.29 (1.33)	6.95 (0.98)	7.12 (1.19)
Sex
Female	230	113	343
Male	269	376	645
Dentition
Primary	74	157	231
Mixed	425	332	757

SD, standard deviation.

Methods

This study consisted of two networks: (1) DeeplabV3plus, ²⁷ a segmentation model that acquired the posterior molar space region to limit the ROI to the anterior region and increase the detection accuracy for mesiodens, and (2) Inception-resnet-v2, ^28,29 the process of classifying the existence of mesiodens from the cropped maxillary anterior teeth using segmented posterior molar regions. The flowchart summarizing the overall process is shown in Figure 1. All the data processing including ROI determination, deep learning training and validation, and diagnostic performance evaluation were performed using MATLAB 2020a (MathWorks, Natick, MA)

Figure 1.

Blue arrow: training phase, yellow and green arrows: validation phase using trained network.

Data processing

All panoramic radiographs were obtained using a Veraviewepocs unit (J. Morita Mfg. Corp., Kyoto, Japan) or an AUTO III NTR unit (Asahi Roentgen Industry Co., Ltd., Kyoto, Japan) and downloaded as JPEG files (2560 × 1500 pixels).

To limit the maxillary anterior ROI, the position of the posterior molar space was lined in red on the panoramic radiograph by a dentist using the paint software in Windows 10. The posterior molar space was as follows: the anterior lining is the distal surface of the most posterior molar, and the superior and inferior lining are the maxillary and mandibular alveolar ridges, respectively. The posterior lining is the anterior part of the ramus of the mandible that contacts the maxillary alveolar bone (Figure 2A). The previous study set the area directly in the anterior area for adult panoramas with a mean age of 39.2 years. ²⁵ However, the panorama of children with mixed dentition had difficulty in establishing a clear reference point for setting the anterior area owing to the overlap of various anatomical structures such as tooth germ compared to that of adults. Therefore, the anterior area was set based on the posterior molar space in this study, the region observed clearly in all panorama, with relatively little overlap of anatomical structures.

Figure 2.

Process of finding the ROI in the maxillary anterior region including mesiodens. (a) Posterior molar space on panoramic radiograph (red line), (b) horizontal limit setting, (c) vertical limit setting, (d) ROI is outlined by the blue rectangle in the maxillary anterior region, (e) segmentation result using the original panoramic image, (f) cropped image by posterior molar space. ROI, region of interest.

After the posterior molar space is determined manually, this space is recognized using RGB values, and horizontal and vertical limits of the ROI were set automatically. For the horizontal limit of the ROI, vertical lines passing through the horizontally furthest point of bilateral posterior molar spaces were drawn first. Then, length corresponding to half of the selected point was extended from the midline to the left and right to set the horizontal limit (Figure 2B). The vertical limit was set at the highest and lowest points among those divided into thirds by determining the highest and lowest points among the left and right posterior molar space region. Then, one-third from each highest and lowest end was extended to set the vertical limit (Figure 2C).

Network architecture

Inception-ResNet-v2, which has exhibited high accuracy in many dental and medical deep learning studies, was used for classification. ^28,29 Because this network was pre-trained using over a million images from the ImageNet database, transfer learning using pre-trained features was possible. Inception-resnet-v2 is composed of 164 layers, has 55.9 million parameters, and receives 299 × 299 RGB images as input. Deeplabv3plus has shown high segmentation accuracy in many dental and medical applications with encoder–decoder structure using atrous separable convolution. ²⁷ Inception-ResNet-V2 also used as a backbone of deeplabv3plus network in this study.

Fivefold cross-validation and data augmentation

Fivefold cross-validation was performed as the next step. This method is used to obtain reliable validation accuracy by reducing data deviation when the amount of data used to train a deep learning model for image classification is small (Figure 3). The images of the control and experimental groups were randomly divided into five groups each. Among them, four groups were used as training data, and one group was used as validation data. As each group was used only once as validation data, fivefold cross-validation was possible.

Figure 3.

Fivefold cross-validation.

To prevent overfitting due to a small amount of data, the amount of training data was increased through the data augmentation process. Data augmentation was performed by flipping left and right, rotating from −10° to +10°, translating horizontally or vertically by five pixels, and resizing 0.8 to 1.2 times.

Network training options

The Windows 10 operating system was used for network training. Deep learning and parallel computing toolboxes of MATLAB 2020a (MathWorks, Natick, MA) were installed, and NVIDIA Titan RTX was used to support it. The first model was trained for up to 500 epochs using the stochastic gradient descent with momentum optimizer. The size of the mini batch was 14, and the initial learning efficiency was 3e-4. The training process was stopped prematurely when the validation patience value reached 5. The second model was trained for up to 500 epochs using the Adam optimizer. The size of the mini batch was 16, and the initial learning efficiency was e-4. The training process was stopped prematurely when the validation patience value reached 30.

Diagnostic performance evaluation

The diagnostic performance of the model that performed best in the validation data set was evaluated. ³⁰ Accuracy, Intersection over Union (IoU), and mean boundary F1 (BF) score were calculated in the first network, and accuracy, precision, recall, and F1 scores were calculated in the second network. Moreover, receiver operating characteristics (ROC) and the area under the curve (AUC) were evaluated using perfcurve function of MATLAB 2020a.

• Accuracy = $\frac{\mathrm{T}\mathrm{P}\mathrm{}+\mathrm{}\mathrm{T}\mathrm{N}}{\mathrm{T}\mathrm{P}\mathrm{}+\mathrm{}\mathrm{F}\mathrm{P}\mathrm{}+\mathrm{}\mathrm{F}\mathrm{N}\mathrm{}+\mathrm{}\mathrm{T}\mathrm{N}}$

• Precision = $\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}\mathrm{}+\mathrm{}\mathrm{F}\mathrm{P}}$

• Recall = $\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}\mathrm{}+\mathrm{}\mathrm{F}\mathrm{N}}$

• F1 Score = $\frac{2\mathrm{}\times \mathrm{}(\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{}\times \mathrm{}\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n})\mathrm{}}{\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{}+\mathrm{}\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}}$

• Mean BF score* = $\frac{2\mathrm{}\times \mathrm{}(\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{}\times \mathrm{}\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n})\mathrm{}}{\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{}+\mathrm{}\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}}$

• IoU = $\frac{\mathrm{A}\mathrm{r}\mathrm{e}\mathrm{a}\mathrm{}\mathrm{o}\mathrm{f}\mathrm{}\mathrm{O}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{l}\mathrm{a}\mathrm{p}}{\mathrm{A}\mathrm{r}\mathrm{e}\mathrm{a}\mathrm{}\mathrm{o}\mathrm{f}\mathrm{}\mathrm{U}\mathrm{n}\mathrm{i}\mathrm{o}\mathrm{n}}$

• TP: true positive, FP: false positive, FN: false negative, TN: true negative

IoU: Intersection over Union

*The BF score is defined as the harmonic mean (F1-measure) of the precision and recall values with a distance error tolerance to decide whether a point on the predicted boundary has a match on the ground truth boundary. ³⁰

The value of accuracy, precision, recall, F1 score, and AUC of manual and automatic segmented ROI was compared using K-fold paired t-test procedure. ³¹ We also examined the classification performance using entire panorama for comparison.

Visualizing model decision

Gradient-weighted class activation mapping (Grad-CAM) was used to compare and analyze the area that AI is mainly interested in and the actual location of the mesiodens.

Results

Classification performance using entire panorama is listed in Supplementary Table 1 of appendix, which shows lower performance than using cropped images with the same parameter setting.

Segmentation performance of posterior molar space

Table 2 shows the results of posterior molar space segmentation through fivefold cross-validation. The mean value was 0.839 for accuracy, 0.762 for IoU, and 0.907 for mean BF score.

Table 2.

Performance of posterior molar space segmentation network on panoramic radiograph using five fold cross-validation

Cross-validation	Accuracy (background)	IoU (background)	Mean BF score (background)
1cv	0.853 (0.998)	0.770 (0.996)	0.917 (0.990)
2cv	0.828 (0.998)	0.751 (0.996)	0.898 (0.988)
3cv	0.839 (0.998)	0.758 (0.996)	0.902 (0.988)
4cv	0.846 (0.998)	0.769 (0.996)	0.910 (0.989)
5cv	0.831 (0.999)	0.763 (0.996)	0.909 (0.989)
Mean	0.839	0.762	0.907

IoU: Intersection over Union, BF score: boundary F1 score

Classification performance using manually segmented ROIs

Table 3 shows the performance of mesiodens classification model using manually segmented ROIs. The mean value of accuracy, precision, recall, F1 score, and AUC value were 0.892, 0.892, 0.893, 0.892, and 0.892, respectively.

Table 3.

Performance of mesiodens classification network using manually segmented panoramic anterior regions through fivefold cross-validation

Pre-trained network	Accuracy	Precision	Recall	F1-score	AUC
1cv	0.895	0.895	0.896	0.896	0.895
2cv	0.875	0.875	0.875	0.875	0.875
3cv	0.867	0.867	0.872	0.869	0.867
4cv	0.904	0.904	0.904	0.904	0.904
5cv	0.917	0.917	0.917	0.917	0.917
Mean	0.892	0.892	0.893	0.892	0.892

AUC: area under the curve

Classification performance using automatically segmented ROIs

Table 4 shows the overall performance of the two-step network for classifying mesiodens through the network learned using the automatically segmented posterior molar space region. The mean value of accuracy, precision, recall, F1 score, and AUC value were 0.971, 0.971, 0.971, 0.971, and 0.971, respectively, which were higher than manual segmented ROIs (p < 0.05). ³¹

Table 4.

Performance of mesiodens classification network using automatically segmented panoramic anterior regions through fivefold cross-validation TWO STEP

Cross-validation	Accuracy	Precision	Recall	F1-score	AUC
1cv	0.970	0.969	0.970	0.970	0.969
2cv	0.985	0.985	0.985	0.985	0.985
3cv	0.960	0.959	0.962	0.960	0.959
4cv	0.970	0.970	0.970	0.970	0.970
5cv	0.970	0.969	0.970	0.970	0.969
Mean	0.971	0.971	0.971	0.971	0.971

AUC: area under the curve.

ROC curve for classification of mesiodens

The AUC value from the ROC curve of automatically segmented ROI is higher than that of manually segmented ROI (Figure 4).

Figure 4.

ROC curves for mesiodens classification accuracy on the (a) manually segmented ROI model (5cv) and (b) automatically segmented ROI model (2cv). ROC, receiver operating characteristic; ROI, region of interest

Visualization of model classification

To identify the image regions that affect the classification result, we represented the regions with heat maps using Grad-CAM (Figure 5). In each Grad-CAM image, bright red indicates the area with the greatest influence on screening patients with mesiodens. In most cases, visual identification of the network focusing on the correct area of mesiodens existence was possible. For cases without mesiodens, red portions also focus on area including permanent incisors.

Figure 5.

An example of Grad-CAM* to find the position of mesiodens after region of interest segmentation using posterior molar space. *Grad-CAM: Gradient-weighted class activation mapping

Workflow of posterior molar space segmentation process

Figure 6 summarizes this study. The segmentation process using the posterior molar space and the classification process of mesiodens using the set ROI are shown.

Figure 6.

Summary of the entire process. CAM, class activation mapping

Discussion

Mesiodens cause many complications, therefore surgical intervention is often required. ⁸ Therefore, when mesiodens exist or are suspected, CBCT is essential to determine their number and exact location. ¹¹ However, because CBCT has a higher radiation dose than the panorama, it cannot be the first X-ray of choice when pediatric patients are concerned. ¹³ Nevertheless, panorama is commonly used in clinical practice to obtain basic information about teeth or intraosseous lesions but not to check mesiodens. ¹⁴ In particular, the panoramic images, obtained for comprehensive evaluation of oral conditions of growing children, have many items to be evaluated clinically, such as caries, the presence of intraosseous lesions, and evaluation of the position of permanent teeth. Therefore, accurate readings may be difficult for dentists with little dental experience. ²⁶ Therefore, the AI system of high accuracy applied to the panoramic analysis will help clinicians diagnose the presence or absence of mesiodens.

In this study, a fully automatic system for diagnosing mesiodens in panoramic radiographs was developed, and its accuracy was evaluated. According to previous studies, the ROI is manually set for diagnosing mesiodens, making it difficult to view them as a fully automatic diagnosis system. ^25,26 Therefore, in this study, the area around the maxillary anterior region was automatically set in the panoramic picture using a segmentation model of deep learning. Thereafter, the presence of mesiodens was classified based on the automatically set ROI. DeepLabv3+ was used as the backbone of the segmentation network.

In the panoramic radiograph, various anatomical structures (teeth, cervical vertebrae, bones, etc.) overlap in the maxillary anterior region. According to Anthonappa et al, the accuracy of panoramic images with complex anatomical structures was relatively low. ¹⁶ Therefore, in this study, segmentation was performed using the posterior molar space on the panoramic radiograph. The posterior molar space was defined as an area surrounded by the distal surface of the posterior molar, the superior and inferior surfaces of the maxillary and mandibular alveolar bones, and the posterior surface of the mandibular ramus. In the panoramic image, the posterior molar space has no hard tissue, and it does not overlap with other anatomical structures. Hence, it is thought that accurate segmentation is possible in this region because the color and border are relatively clear compared to other regions. ³²

The maxillary anterior region, where mesiodens were expected, was set using the posterior molar space. In this study, the mean performance accuracy of posterior molar space segmentation was 0.839. In a previous study of alveolar bone loss detection using panoramic images, ²³ with N = 12,179,, training data = 11,198,, validation = 190, and test = 800 sets, the ROI segmentation accuracy was 0.68. Furthermore, in a study on the detection of the third molar and mandibular canal using a panoramic radiograph, ³³ the Dice coefficients were compared to quantify the similarity between manually and automatically segmented images, and their values were 0.947 and 0.847, respectively. Compared with previous studies, the posterior molar space segmentation performed in this study was relatively accurate even with a small amount of data.

The mean value accuracy of the two-step process of classifying the presence of mesiodens after segmentation using AI was 0.971. It was relatively higher than 0.892, which is the performance accuracy of the process of classifying the presence of mesiodens after manual segmentation of the posterior molar space. During segmentation using AI, a relatively smaller posterior molar space was recognized, and the maxillary anterior region was often reduced accordingly, without prominent location differences. Due to this effect, it is thought that the accuracy of mesiodens classification using AI is higher.

In this study, Inception-ResNet-v2 was used to classify mesiodens. It was the best when evaluating the performance using various networks in existing research on mesiodens data, and this is because the CAM area had a high rate of expressing the actual mesiodens. ²⁶

In a study comparing the accuracy of detecting mesiodens on panoramic radiographs, in pediatric dentistry, dentists with little experience (less than 1 year) were compared to dentists with 1–2 years of experience. ^16,26 The sensitivities were 0.39 and 0.6, respectively, suggesting that interpretation errors may occur if the proficiency in finding supernumerary teeth through only panoramic radiographs is low.

In a previous study on the classification of mesiodens using panoramic radiographs, ^26,34 the anterior ROI was based on the distal and uppermost points of both permanent canine tooth germs and the mandibular anterior alveolar bone level. The mean value results of the model set manually based on the posterior molar space in this study and the model using two steps showed high accuracy, with AUCs of 0.892 and 0.971, respectively. This is thought to be because the ROI setting based on the posterior molar space is similar to the ROI set in previous studies using the maxillary canine tooth placement and mandibular alveolar bone.

If the mesiodens are included in the center of the image layer of the panorama, they will have a clear outline in the panorama. In this case, the exact number and location of mesiodens can be found using deep learning networks for segmentation or detection. However, if the mesiodens are far away from the center of the image layer and it is difficult to find a clear shape in the panorama, finding the exact number and location of mesiodens will be difficult. ³⁵ This study is a good means to detect the presence of mesiodens and inform clinicians even when the outline of mesiodens is not clear. If this study shows the same high accuracy even with more data, we believe that CBCT imaging for determining the number and exact location of mesiodens in children can be justified because the benefit is greater than the harm. CBCT obtained in these cases is expected to reduce unnecessary surgical invasions and undesirable accidents.

This study has some limitations. First, the possibility of complete automation was confirmed for the detection of mesiodens, however, the number of mesiodens and their exact location could not be determined. Second, as the system was evaluated using panoramic images from a single institution, it may be difficult to accurately diagnose when using panoramic images obtained with other machines. Therefore, using large-scale panoramic images collected from various machines at various institutions will be helpful for accurate diagnosis. In a follow-up study, it is necessary to develop a deep learning network that detects the location and number of mesiodens using panoramic radiographs. Furthermore, it is expected that automatic diagnosis of various oral diseases will be possible if various ROIs that can be evaluated on a panoramic radiograph are set based on the posterior molar space used as the ROI in this study.

Conclusion

The posterior molar space was segmented with high accuracy in this study, which may lead to successful ROI confinement and high classification accuracy of mesiodens. The results of this study confirmed the possibility of a fully automated process for diagnosing mesiodens using only panoramic radiographs. Furthermore, it is possible that this will be the basis for automatically identifying other diseases using only panoramic radiographs.