Machine Learning Models in the Detection of MB2 Canal Orifice in CBCT Images

Abstract

Objectives

The objective of the present study was to determine the accuracy of machine learning (ML) models in the detection of mesiobuccal (MB2) canals in axial cone-beam computed tomography (CBCT) sections.

Methods

A total of 2500 CBCT scans from the oral radiology department of University Dental Hospital, Sharjah were screened to obtain 277 high-resolution, small field-of-view CBCT scans with maxillary molars. Among the 277 scans, 160 of them showed the presence of MB2 orifice and the rest (117) did not. Two-dimensional axial images of these scans were then cropped. The images were classified and labelled as N (absence of MB2) and M (presence of MB2) by 2 examiners. The images were embedded using Google's Inception V3 and transferred to the ML classification model. Six different ML models (logistic regression [LR], naïve Bayes [NB], support vector machine [SVM], K-nearest neighbours [Knn], random forest [RF], neural network [NN]) were then tested on their ability to classify the images into M and N. The classification metrics (area under curve [AUC], accuracy, F1-score, precision) of the models were assessed in 3 steps.

Results

NN (0.896), LR (0.893), and SVM (0.886) showed the highest values of AUC with specified target variables (steps 2 and 3). The highest accuracy was exhibited by LR (0.849) and NN (0.848) with specified target variables. The highest precision (86.8%) and recall (92.5%) was observed with the SVM model.

Conclusion

The success rates (AUC, precision, recall) of ML algorithms in the detection of MB2 were remarkable in our study. It was also observed that when the target variable was specified, significant success rates such as 86.8% in precision and 92.5% in recall were achieved. The present study showed promising results in the ML-based detection of MB2 canal using axial CBCT slices.

Introduction

One of the main factors for the success of any endodontic treatment is the accuracy with which the root canals are identified during the treatment procedure.¹ The chances of failure of endodontic treatment increases approximately 4-fold when canals are missed during root canal treatment.² The maxillary first molar exhibits many variations of root canal anatomy.³ The presence of a second mesiobuccal canal (MB2) is one of the variations with an incidence ranging from 33% to 96% as reported by several studies.4, 5, 6, 7 It has been found that the mesiopalatine tilt of the MB2 canal orifice causes difficulty in locating the canal during endodontic treatment.⁸ Conventionally, intraoperative examination and periapical radiographs were used to detect the presence of MB2. However, several recent studies have shown that cone-beam computed tomography (CBCT) has been more effective in the detection of MB2 compared to conventional radiographs.^9,10

There has been a recent surge in the research related to the application of artificial intelligence (AI) in dentistry.¹¹ Machine learning (ML) which is a subgroup of AI, is being explored as an adjunct in maxillofacial and dental imaging.¹² Further, unsupervised ML is a subset of ML that learns from data sets with no human supervision, labelling, or structuring.¹³ This subcategory of ML works on specific data without needing instructions.¹³ In this study, we have applied unsupervised ML algorithms and measured the success of ML algorithms in classifying images obtained without any human intervention.

AI-based detection of MB2 in maxillary first molar remains unexplored.¹⁴ To the best of our knowledge, there is only one research paper published in this specific area of research, which mainly focuses on the application of a deep learning model to primarily detect and further segment unobturated (missed) MB2 canals in maxillary molars which have been endodontically obturated.¹⁴ To further explore this area, we conducted a study to determine the accuracy of ML models in the detection of MB2 canals in axial CBCT sections.

Methodology

The study protocol was approved by the research ethics committee of the University of Sharjah (REC-23-10-12-01-F). Two examiners obtained 2500 CBCT scans from the dental radiology files of the University Dental Hospital, Sharjah, between June 2022 and June 2024, using a Planmeca Viso 7 CBCT machine. The scans belonged to male and female patients within the age group of 18 and 60 years.

After screening the CBCT scan, 277 scans with maxillary first molars were eligible for the study. The scans with the following criteria were included in the study. Only the scans that were obtained using high-resolution settings (150 μm voxel size, 100 kVp, and 12.5 mA, 5-second exposure time) small-volume scans (Ø 6.0 × 6.0 cm) were included in the study. The scans showed complete coverage of the crown and root apex of maxillary first molar.

Scans not covering the region of interest (ROI) (mesial side of maxillary second premolar to distal side of maxillary second molar) were excluded from the research. Scans with caries, restoration, fracture, and root canal treatment of maxillary first molar were not included in the study.

Two examiners with 10 years of clinical experience screened the high-resolution CBCT scans for the presence/absence of MB2 canals. In case of disagreement between the 2 examiners, a third examiner with similar experience was called in to assist with determining the classification.

Later, 10% of the scans were reevaluated by each of the examiners after 2 weeks to obtain intrarater reliability.

Once confirmed the images were cropped uniformly (200 × 400-pixel square), labelled (N = MB2 not present, M = MB2 present) and saved in Joint Photographic Expert Group (JPEG) format. The first axial section at the level of canal orifice was considered as mentioned in a study by Normando et al.¹⁵ (Figures 1A and B). The images were preprocessed using ‘ImageFilter’, a package of Python Imaging Library (PIL) available in Python (Figures 2 A–C).

Fig. 1.

A, Sagittal cone-beam computed tomorgraphy (CBCT) section showing the level at which axial images were obtained. Using the technique by Normando et al. B, Cropped axial CBCT section with maxillary first molar with mesiobuccal orifice (red arrow).

Fig. 2.

Original image (A) and preprocessed image (B) with increased sharpness. Snap of the code for image filter in Python Imaging Library.

The examiners then used a pretrained deep learning model (Google's Inception v3) for image embedding (image reading and uploading to the server). The activations of the penultimate layer in the neural network (NN) using vector representations of images were used for embedding. The feature vector of each image was then subjected to classification by ML models (logistic regression [LR], naïve Bayes [NB], support vector machine [SVM], K-nearest neighbours [Knn], random forest [RF], NN) using Orange Data Mining version 3.38.1 (Figures 3A and B). The 6 ML models were selected based on their performance in recently published studies on medical image classification.^16,17 The ML models calculated a feature vector for each image. The image descriptors were returned as an advanced data table

Fig. 3.

A, Workflow of the model analysis. (B) Screenshot of the program used in the study showing details of the classification model. (Orange data mining version 3.38.1)

A set of images in classified folders were selected for image clustering. These folder names were taken as classification names. Cross-validation was applied (number of crosses: 10), and the accuracy matrices of each ML algorithm were examined.

The study consisted of 3 stages. In the first stage, the success criteria of ML algorithms in classification were examined without selecting the target variable. The target variable is the outcome that the model aims to predict. In the present study, M (presence of MB2) and N (absence of MB2) were the target variables. In the second and third stages, the success rates of ML algorithms in classification were analysed by specifying the target variables M and N.

Results

Among the 277 scans, 160 of them showed the presence of MB2 orifice (M) and the rest 117 did not show MB2 (N). The interrater reliability between the 2 examiners for the detection of MB2 was 0.85 indicating almost perfect agreement. The intrarater reliability for examiners 1 and 2 were 0.92 and 0.96, respectively. Classification success rates of ML algorithms were evaluated under 5 performance criteria: area under the curve (AUC), classification accuracy (CA), F1 score, precision, and recall.

Table 1 presents the classification metrics of ML algorithms without specifying any target variable (step 1). NN showed the highest AUC value (0.903), whereas kNN showed the lowest (0.649). When the classification accuracy was evaluated, LR showed the highest value (0.84) (Figure 4). Similarly, highest F1-score (0.841), precision (0.841), and recall (0.841) were also shown by LR. kNN showed the lowest performance metrics among the 6 ML models.

Table 1.

Performance metrics without specifying target variable (cross-validation number of folds: 10).

	Classifier	AUC	CA	F1	Precision	Recall
Image data set (277)	LR	0.885	0.841	0.841	0.841	0.841
	NN	0.903	0.838	0.837	0.837	0.838
	SVM	0.886	0.819	0.815	0.826	0.819
	RF	0.812	0.744	0.736	0.746	0.744
	NB	0.800	0.726	0.727	0.735	0.726
	kNN	0.649	0.599	0.601	0.607	0.599

AUC, area under the curve; CA, classification accuracy; kNN, K-nearest neighbours; LR, logistic regression; NB, Naïve Bayes; NN, neural network; RF, random forest; SVM, support vector machine.

Fig. 4.

Confusion matrix for logistic regression (showing number of instances)

In the second step, M category was selected as the target variable and the classification success of ML algorithms (Table 2). When AUC metric was evaluated, NN (0.896) showed the best performance, closely followed by LR (0.893) and SVM (0.886) (Figure 5). SVM showed the best recall metrics (0.925)

Table 2.

Performance metrics with specification of target variable target M and target N (cross-validation number of folds: 10).

Target M (step 2)	Classifier	AUC	CA	F1	Precision	Recall
Image data set (277)	LR	0.893	0.841	0.865	0.849	0.881
	NN	0.896	0.838	0.862	0.848	0.875
	SVM	0.886	0.819	0.855	0.796	0.925
	RF	0.809	0.744	0.797	0.735	0.869
	NB	0.810	0.726	0.748	0.796	0.706
	kNN	0.649	0.599	0.636	0.669	0.606
Target N (Step 3)	Classifier	AUC	CA	F1	Precision	Recall
Image data set (277)	LR	0.893	0.841	0.807	0.829	0.786
	NN	0.896	0.838	0.803	0.821	0.786
	SVM	0.886	0.819	0.760	0.868	0.675
	RF	0.808	0.744	0.654	0.761	0.573
	NB	0.798	0.726	0.698	0.652	0.752
	kNN	0.649	0.599	0.554	0.523	0.590

AUC, area under the curve; CA, classification accuracy; kNN, K-nearest neighbours; LR, logistic regression; NB, naïve Bayes; NN, neural network; RF, random forest; SVM, support vector machine.

Fig. 5.

A, Area under the curve–receiver operating characteristic (AUC-ROC) of the 6 machine learning (ML) models with specified target variable M. (B) AUC-ROC of the 6 ML models with specified target variable N.

In the last step, the N category was selected as the target variable and the classification success rates of ML algorithms are given in Table 2. The highest AUC value was observed with NN (0.896), followed by LR (0.886) and SVM (0.886) (Figure 5). SVM showed best (0.868) precision metrics.

When the AUC values (without specifying target variable) of the ML models were compared using the vassarstats.net program, NN, LR, and SVM showed significantly higher AUC values compared to RF, NB, and kNN (Table 3).

Table 3.

Significance of difference among the AUC values (without specifying target variable) of the ML models used in the study.

Model	Model	Standard error of difference	Z value	P value
LR	NN	0.0268	-0.6714	.250983 (NS)
	SVM	0.0279	-0.0359	.485681 (NS)
	RF	0.0321	2.2769	.011396*
	NB	0.0327	2.6029	.004622*
	kNN	0.0382	6.1741	<.000001*
NN	SVM	0.0267	0.6357	.262486 (NS)
	RF	0.0311	2.929	.0017*
	NB	0.0317	3.2511	.000575*
	kNN	0.0374	6.7923	<.000001*
SVM	RF	0.032	2.312	.010389*
	NB	0.0326	2.6379	.004171*
	kNN	0.0382	6.2076	<.000001*
RF	NB	0.0362	0.3312	.370247(NS)
	kNN	0.0413	3.9445	.00004*
NB	kNN	0.0418	3.6136	.000151*

LR, logistic regression; ML, machine learning; NB, naïve Bayes; NN, neural network; SVM, support vector machine; RF, random forest.

Level of significance <0.05, NS, not significant,

^⁎Significant.

Discussion

There has been a recent surge in AI-based research and publication in dentistry.¹⁸ Accordingly, focus groups have also developed guidelines for the reporting of the AI studies in dentistry.^19,20 Some of these research papers have focused on the application of AI in endodontics.²¹ A wide array of areas has been explored for a possible scope of endodontics, which includes tooth segmentation, detection of caries, periapical pathologies, obturation, and C-shaped canals.^22,23 AI-based detection of C-shaped canals in mandibular molars is a good comparator for our study. The C-shaped canals have been detected in periapical radiographs, panoramic radiographs, and CBCT.^24,25

A large percentage of root canal failures related to maxillary first molars have been attributed to undetected MB2.^26,27 Studies have revealed that conventional 2-dimensional radiographs were not reliable in detecting multiple canals as in case of MB2, thus additional diagnostic modalities were required.²⁸ Over the past few years, detection of MB2 canals has been aided by CBCT imaging to a significant extent.²⁹

Recently, various ML models have been employed in the identification and classification of dental anatomical variations and pathologies.^30,31 In the present study, 6 different ML algorithms were used to classify the images.

Numerical imbalance between data sets would have influenced the performance metrics in the present study. Therefore, both variables (M and N) were defined as target variables in our model classification. The performance of the model was evaluated without specifying target variables in the first step. This was followed by the evaluation of the model by specifying target variables (M and N) in the second and third steps. There was no major difference in the success rates of the model in the 3 steps. The impact of different data set sizes on model performance and the solution were discussed by Althnian et al.³²

Prior to ML-based classification, image embedding was carried out using Inception-v3, which is a third genre of Google's Inception convolutional neural network (CNN).^33,34 The Inception-v3 can be used as a base architecture and transfer learning.^35,36 The activations of the penultimate layer in the NN using vector representations of images are used for embedding.³⁷ Google Image Embedding reads images and uploads them to a remote server or evaluates them locally. Deep learning/ML models are used to calculate a feature vector for each image. The image descriptors return an advanced data table. Image Embedding offers several embedders, each trained for a specific task. Images are sent to a server or evaluated locally on the user's computer, where vector representations are calculated. No internet connection is required for the process, and images sent to the server are not stored anywhere.³⁸ Google's Inception V3 network is the latest advanced deep architecture widely used in medical image processing. It introduced an innovative block known as the Inception V3 block. This block allows determining convolutions and pooling operations simultaneously. It enables multi-level feature extraction for medical image processing.^39,40 A recent study using transfer learning approach using Inception V3 with LR and SVM classifier showed high specificity and sensitivity for classification of chest X-rays.⁴¹

In our study, NN (0.896) and LR (0.893) showed the highest values of AUC with specified target variables. A study reporting CNN-based detection of C-shaped canals in a mandibular second molar on panoramic radiograph reported AUC of 0.98.²⁴ Another study by Yang et al.²⁵ also reported AUC of 0.98 in periapical radiographs for detecting C-shaped canals. A recent study revealed that LR and NN showed high training and testing accuracy when used for classification of binary images.⁴¹ Another study conducted using LR for the histopathological detection of breast cancer revealed an AUC value of 0.83, which is marginally lower than the to the AUC values observed in the present study.⁴² The same study also used SVM for histopathological classification and achieved an AUC of 0.84, which was also marginally lower than the AUC of the SVM model in the present study.

In the present study, the highest precision (86.8%) and recall (92.5%) were observed with the SVM model. A recent study employed an AI architecture like our research for the classification of lung ultrasound images for the detection of coronavirus disease 2019 (COVID-19).⁴³ They used a combination of pretrained deep learning CNN models with SVM as a supervised classifier.⁴⁴ Higher precision (99%) and recall (99%) were observed in their study compared to the present study. Similar outcomes were observed in studies using SVM to detect dental caries on radiographs.⁴⁵ In that study, the SVM classifier model showed an accuracy of 95.7%. In the same study, researchers also used another classifier, kNN, which showed an accuracy compared to our study. Another study using SVM using binary classification of 105 dental radiographic images into carious and noncarious teeth showed an accuracy of 97.3%.⁴⁵

In the present study, the SVM model showed precision of 86.8%. The SVM model showed a precision of 90% when it was used for classification and numbering of posterior teeth on dental radiographs.⁴⁶ Similar accuracy outcomes were observed in the detection of caries on panoramic radiographs using multiclass SVM.⁴⁷

In the first stage of the present study, the success criteria of ML algorithms in classification were examined without selecting the target variable. In the second and third stages, the success rates of ML algorithms in classification were analysed without making any changes to the rules by specifying the target variable. The target variable is a variable that the researcher is looking to predict during the classification process.⁴⁸ In the present study, the presence or absence of the MB2 was the target variable.

RF and NB were the other ML models used for the binary classification of axial CBCT images of canal orifices in the present study. The classification accuracies of RF and NB were 74.4% and 72.6%, respectively. When RF was used for tooth identification on panoramic radiographs, the accuracy was 98%.⁴⁹ Another study using RF for the detection of periodontal bone loss in dental radiographs of 116 patients reported an accuracy of 83%.⁵⁰ The study concluded that RF showed higher accuracy with small sample size, very similar to the present study. Modified RF models have also been used for dental radiographic image segmentation and cephalometric evaluation.⁵¹

In the study, the classification of images was done with unsupervised ML algorithms. To alleviate the existing difficulty caused by the scarcity of annotated examples and studies and the laborious annotation process, a large number of studies based on unsupervised learning have been developed. Unsupervised classification aims to create groups or clusters with similar values ​​in large data sets and then determine users’ classes by taking samples from these clusters.^52,53

One of the unique features of our study is that we have used axial CBCT images for ML-based classification for the presence of MB2. Most of the previous dental radiographic studies were based on intraoral and panoramic radiographs for detection of C-shaped canals.^24,25

One of the limitations of our study is that the smaller data set of high-resolution small CBCT scans. A future study involving multiple teaching hospitals can be planned. This would help us to obtain a larger data set of high-resolution, small field-of-view CBCT scans with and without MB2 canal orifice. The present study can provide a base model for developing deep learning models that can detect MB2 canals in 3D CBCT scans.

Conclusion

The success rate (AUC, precision, recall) of ML algorithms in the detection of MB2 was remarkable in our study. It was also observed that when the target variable was specified, significant success rates such as 86.8% in precision and 92.5% in recall were achieved. The present study showed promising results in the ML-based detection of MB2 canals using axial CBCT slices. The methodology of the study can serve as a foundation for creating an ML model for the detection of MB2 in 3D CBCT scans, thereby aiding the dental practitioner in the accurate identification of MB2 canals and better treatment outcomes.

Conflict of interest

None disclosed.