Skip to main content
NCBI home page
Springer Nature - PMC COVID-19 Collection logoLink to Springer Nature - PMC COVID-19 Collection
. 2022 Oct 14;36(1):257–272. doi: 10.1007/s11424-022-2057-9

Transformer-Based Deep Learning Network for Tooth Segmentation on Panoramic Radiographs

Chen Sheng 1,2, Lin Wang 1,2,3,✉, Zhenhuan Huang 1,2,3, Tian Wang 1,2,3, Yalin Guo 1,2,3, Wenjie Hou 1,2,3, Laiqing Xu 1,2,3, Jiazhu Wang 1,2,3, Xue Yan 1,2,3
PMCID: PMC9561331  PMID: 36258771

Abstract

Panoramic radiographs can assist dentist to quickly evaluate patients’ overall oral health status. The accurate detection and localization of tooth tissue on panoramic radiographs is the first step to identify pathology, and also plays a key role in an automatic diagnosis system. However, the evaluation of panoramic radiographs depends on the clinical experience and knowledge of dentist, while the interpretation of panoramic radiographs might lead misdiagnosis. Therefore, it is of great significance to use artificial intelligence to segment teeth on panoramic radiographs. In this study, SWin-Unet, the transformer-based Ushaped encoder-decoder architecture with skip-connections, is introduced to perform panoramic radiograph segmentation. To well evaluate the tooth segmentation performance of SWin-Unet, the PLAGH-BH dataset is introduced for the research purpose. The performance is evaluated by F1 score, mean intersection and Union (IoU) and Acc, Compared with U-Net, Link-Net and FPN baselines, SWin-Unet performs much better in PLAGH-BH tooth segmentation dataset. These results indicate that SWin-Unet is more feasible on panoramic radiograph segmentation, and is valuable for the potential clinical application.

Keywords: Deep convolutional neural network, panoramic radiograph, SWin-Unet, Tooth segmentation

Footnotes

This paper was recommended for publication by Editor QI Hongsheng.

Contributor Information

Chen Sheng, Email: shengchen301@163.com.

Lin Wang, Email: 13581891907@163.com.

Zhenhuan Huang, Email: zhenhuan@buaa.edu.cn.

Tian Wang, Email: wangtian@buaa.edu.cn.

Yalin Guo, Email: guoyalin301@163.com.

Wenjie Hou, Email: houwenjie2022@163.com.

Laiqing Xu, Email: xulaiqing@163.com.

Jiazhu Wang, Email: yuiyuan654@163.com.

Xue Yan, Email: a137872648@163.com.

References

  • [1].Perschbacher S. Interpretation of panoramic radiographs. Australian Dental Journal. 2012;57:40–45. doi: 10.1111/j.1834-7819.2011.01655.x. [DOI] [PubMed] [Google Scholar]
  • [2].Kim J, Kim H, and Ro Y, Iterative deep convolutional encoder-decoder network for medical image segmentation, Proceedings of Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2017, 685–688. [DOI] [PubMed]
  • [3].Zhao J, Ma Y, Pan Z, et al. Research on image signal identification based on adaptive array stochastic resonance. Journal of Systems Science and Complexity. 2022;35(1):179–193. doi: 10.1007/s11424-021-0133-1. [DOI] [Google Scholar]
  • [4].Wu C, Tsai W, Chen Y, et al. Model-based orthodontic assessments for dental panoramic radiographs. IEEE Journal of Biomedical and Health Informatics. 2017;22(2):545–551. doi: 10.1109/JBHI.2017.2660527. [DOI] [PubMed] [Google Scholar]
  • [5].Ammar H, Ngan P, Crout R, et al. Three-dimensional modeling and finite element analysis in treatment planning for orthodontic tooth movement. American Journal of Orthodontics and Dentofacial Orthopedics. 2011;139(1):59–71. doi: 10.1016/j.ajodo.2010.09.020. [DOI] [PubMed] [Google Scholar]
  • [6].Jiang Y, Qian J, Lu S, et al. LRVRG: A local region-based variational region growing algorithm for fast mandible segmentation from cbct images. Oral Radiology. 2021;37(4):631–640. doi: 10.1007/s11282-020-00503-5. [DOI] [PubMed] [Google Scholar]
  • [7].Wang T, Qiao M, Zhang M, et al. Data-driven prognostic method based on self-supervised learning approaches for fault detection. Journal of Intelligent Manufacturing. 2020;31(7):1611–1619. doi: 10.1007/s10845-018-1431-x. [DOI] [Google Scholar]
  • [8].Razali M, Ahmad N, Hassan R, et al., Sobel and canny edges segmentations for the dental age assessment, Proceedings of International Conference on Computer Assisted System in Health, 2014, 62–66.
  • [9].Pérez-Benito F, Signol F, Perez-Cortes J, et al. A deep learning system to obtain the optimal parameters for a threshold-based breast and dense tissue segmentation. Computer Methods and Programs in Biomedicine. 2020;195:105668.1–36. doi: 10.1016/j.cmpb.2020.105668. [DOI] [PubMed] [Google Scholar]
  • [10].Bergeest J, Rohr K. Efficient globally optimal segmentation of cells in fluorescence microscopy images using level sets and convex energy functionals. Medical Image Analysis. 2012;16(7):1436–1444. doi: 10.1016/j.media.2012.05.012. [DOI] [PubMed] [Google Scholar]
  • [11].Gong X, Chen S, Zhang B, et al., Style consistent image generation for nuclei instance segmentation, Proceedings of the IEEE Winter Conference on Applications of Computer Vision, 2021, 3994–4003.
  • [12].Mao M, Gao P, Zhang R, et al., Dual-stream network for visual recognition, Proceeings of Advances in Neural Information Processing Systems, 2021, 34–46.
  • [13].Esteva A, Kuprel B, Novoa R, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542(7639):115–118. doi: 10.1038/nature21056. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [14].Wang T, Qiao M, Lin Z, et al. Generative neural networks for anomaly detection in crowded scenes. IEEE Transactions on Information Forensics and Security. 2018;14(5):1390–1399. doi: 10.1109/TIFS.2018.2878538. [DOI] [Google Scholar]
  • [15].Leite A F, Van Gerven A, Willems H, et al. Artificial intelligence-driven novel tool for tooth detection and segmentation on panoramic radiographs. Clinical Oral Investigations. 2021;25(4):2257–2267. doi: 10.1007/s00784-020-03544-6. [DOI] [PubMed] [Google Scholar]
  • [16].Vinayahalingam S, Xi T, Bergé S, et al. Automated detection of third molars and mandibular nerve by deep learning. Scientific Reports. 2019;9(1):1–7. doi: 10.1038/s41598-019-45487-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [17].Xu X, Liu C, Zheng Y. 3D tooth segmentation and labeling using deep convolutional neural networks. IEEE Transactions on Visualization and Computer Graphics. 2018;25(7):2336–2348. doi: 10.1109/TVCG.2018.2839685. [DOI] [PubMed] [Google Scholar]
  • [18].Van Eycke Y, Foucart A, Decaestecker C. Strategies to reduce the expert supervision required for deep learning-based segmentation of histopathological images. Frontiers in Medicine. 2019;6:222–231. doi: 10.3389/fmed.2019.00222. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [19].Miotto R, Wang F, Wang S, et al. Deep learning for healthcare: Review, opportunities and challenges. Briefings in Bioinformatics. 2018;19(6):1236–1246. doi: 10.1093/bib/bbx044. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [20].Liu P, Song Y, Chai M, et al. Swin—unet++: A nested swin transformer architecture for location identification and morphology segmentation of dimples on 2.25 cr1mo0. 25v fractured surface. Materials. 2021;14(24):7504.1–15. doi: 10.3390/ma14247504. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [21].Luo C, Zhang J, Chen X, et al. UCATR: Based on CNN and transformer encoding and cross-attention decoding for lesion segmentation of acute ischemic stroke in non-contrast computed tomography images. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 2021;2021:3565–3568. doi: 10.1109/EMBC46164.2021.9630336. [DOI] [PubMed] [Google Scholar]
  • [22].Wang C, Huang C, Lee J, et al. A benchmark for comparison of dental radiography analysis algorithms. Medical Image Analysis. 2016;31(24):63–76. doi: 10.1016/j.media.2016.02.004. [DOI] [PubMed] [Google Scholar]
  • [23].Wirtz A, Mirashi S G, and Wesarg S, Automatic teeth segmentation in panoramic x-ray images using a coupled shape model in combination with a neural network, Proceedings of International Conference on Medical Image Computing and Computer-assisted Intervention, 2018, 712–719.
  • [24].Chan H, Samala R, Hadjiiski L, et al. Deep learning in medical image analysis. Deep Learning in Medical Image Analysis. 2020;1213:3–21. doi: 10.1007/978-3-030-33128-3_1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [25].Schwendicke F, Golla T, Dreher M, et al. Convolutional neural networks for dental image diagnostics: A scoping review. Journal of Dentistry. 2019;91:103226.1–8. doi: 10.1016/j.jdent.2019.103226. [DOI] [PubMed] [Google Scholar]
  • [26].Goodfellow I, Bengio Y, Courville A. Deep Learning. Cambridge: MIT Press; 2016. [Google Scholar]
  • [27].Zhang Y, Zhang S, Li Y, et al. Single- and cross-modality near duplicate image pairs detection via spatial transformer comparing CNN. Sensors (Basel) 2021;21(1):255. doi: 10.3390/s21010255. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [28].Zhou Z, Siddiquee M M R, Tajbakhsh N, et al. Unet++: A nested u-net architecture for medical image segmentation. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. 2018;11045:3–11. doi: 10.1007/978-3-030-00889-5_1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [29].Krois J, Ekert T, Meinhold L, et al. Deep learning for the radiographic detection of periodontal bone loss. Scientific Reports. 2019;9(1):1–6. doi: 10.1038/s41598-019-44839-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [30].Chaurasia A and Culurciello E, Linknet: Exploiting encoder representations for efficient semantic segmentation, Proceedings of IEEE Visual Communications and Image Processing, 2017, 1–4.
  • [31].Arora R, Saini I, Sood N. Multi-label segmentation and detection of covid-19 abnormalities from chest radiographs using deep learning. Optik. 2021;246:167780.1–18. doi: 10.1016/j.ijleo.2021.167780. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [32].Lin T, Dollár P, Girshick R, et al., Feature pyramid networks for object detection, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, 2117–2125.
  • [33].Lahoud P, EzEldeen M, Beznik T, et al. Artificial intelligence for fast and accurate 3-dimensional tooth segmentation on cone-beam computed tomography. Journal of Endodontics. 2021;47(5):827–835. doi: 10.1016/j.joen.2020.12.020. [DOI] [PubMed] [Google Scholar]
  • [34].Nishitani Y, Nakayama R, Hayashi D, et al. Segmentation of teeth in panoramic dental X-ray images using U-Net with a loss function weighted on the tooth edge. Radiol Phys. Technol. 2021;14(1):64–69. doi: 10.1007/s12194-020-00603-1. [DOI] [PubMed] [Google Scholar]
  • [35].Silva G, Oliveira L, Pithon M. Automatic segmenting teeth in x-ray images: Trends, a novel data set, benchmarking and future perspectives. Expert Systems with Applications. 2018;107:15–31. doi: 10.1016/j.eswa.2018.04.001. [DOI] [Google Scholar]

Articles from Journal of Systems Science and Complexity are provided here courtesy of Nature Publishing Group

RESOURCES