Abstract
Objectives:
It is unclear whether computer-aided detection (CAD) systems for panoramic radiography can help inexperienced dentists to diagnose maxillary sinusitis. The aim of this study was to clarify whether a CAD system for panoramic radiography can contribute to improved diagnostic performance for maxillary sinusitis by inexperienced dentists.
Methods:
The panoramic radiographs of 49 patients with maxillary sinusitis and 49 patients with healthy sinuses were evaluated in this study. The diagnostic performance of the CAD system was determined. 12 inexperienced dentists and 4 expert oral and maxillofacial radiologists observed the total of 98 panoramic radiographs and judged the presence or absence of maxillary sinusitis, under conditions with and without the support of the CAD system. The receiver operating characteristic curves of the two groups were compared.
Results:
The CAD system provided sensitivity of 77.6%, specificity of 69.4% and accuracy of 73.5%. The diagnostic performance of the inexperienced dentists increased with the support of the CAD system. When the inexperienced dentists diagnosed maxillary sinusitis with CAD support, the area under the curve (AUC) was significantly higher than that without CAD support. When the focus was only on panoramic radiographs in which CAD support led to a correct diagnosis, the AUC of the inexperienced dentists increased to an equivalent level to that of the experienced radiologists.
Conclusions:
The CAD system supported the inexperienced dentists in diagnosing maxillary sinusitis on the panoramic radiographs. If the accuracy of the CAD system can be increased, the benefits of CAD support will be further enhanced.
Keywords: computer-aided detection, panoramic radiography, maxillary sinusitis
Introduction
Maxillary sinusitis has been found in association with allergic rhinitis and the common cold.1 Most cases of unilateral maxillary sinusitis have an odontogenic origin caused by periapical lesions and extraction of maxillary teeth.2 Changes in the maxillary sinus were reported in 71.3% patients with dental infections.3
Panoramic radiography provides a variety of information for dentists, not only for caries and periodontal diseases but also for jaw tumours and temporomandibular joint arthritis,4 together with systemic diseases, such as osteoporosis.5–8 In addition, for diagnosis of diseases affecting the maxillary sinus, panoramic radiography can provide effective information, such as the presence of a globular radio-opaque mass on the posterior wall or floor.9 However, there are some limitations to the detection of defects in the posterior wall10 arising from artefacts or superimposition caused by the hard palate and inferior turbinate.11 Therefore, diagnosis of diseases affecting the maxillary sinus is often difficult, especially for inexperienced observers.
One of the aims of computer-aided detection (CAD) systems is to assist inexperienced observers in correctly diagnosing lesions that are difficult to interpret. Such a CAD system has been developed for the detection of abnormalities on chest radiographs, in which the contralateral subtraction technique can be used for detection of asymmetric abnormalities, such as lung nodules and pneumothorax.12 A CAD system has the potential to assist human readers or identify subjects who require further testing.13 CAD systems have already been applied to panoramic radiographs.14,15 The contralateral subtraction technique has been applied for diagnosis of diseases of the maxillary sinus, similar to the case for chest radiographs.14,15 However, it remains unclear whether CAD systems can support interpretation by inexperienced dentists.
The purpose of this study was to clarify whether a CAD system can contribute to improved diagnostic performance for maxillary sinusitis by inexperienced dentists.
Methods and materials
Subjects
Patients who underwent panoramic radiography and CT imaging were consecutively selected from the image database of the Department of Radiology and Diagnostic Imaging of Aichi Gakuin University Dental Hospital, Nagoya, Japan, between April 2010 and December 2014.
Panoramic radiographs were used to determine the diagnostic performance of a CAD system and to evaluate the interpretation of individual observers. The diagnosis of maxillary sinusitis was determined based on the clinical symptoms and CT images. In all subjects, the panoramic radiography preceded the CT examination. When the two examinations were not performed on the same day, only subjects with symptoms of sinusitis at the CT examination were selected for the patient group. The median interval between the two examinations was 8 days.
First, patients with unilateral maxillary sinusitis were selected as the disease group. These patients had clinical symptoms, including buccal swelling and pain, and pus discharge around the maxillary teeth. Their CT images displayed mucosal thickening and/or fluid collection in more than one-third of the unilateral maxillary sinus, including the sinus floor.16 Patients with a mucus retention cyst were also included in this group. Patients with bilateral maxillary sinusitis, and those with carcinoma and other diseases of the maxillary sinus were excluded. Finally, 49 patients (21 males, 28 females; age range: 15–86 years; mean age: 47.1 ± 17.1 years) were enrolled in this group.
Second, patients without pathology in the maxillary sinus were selected as the healthy group, and matched for age and sex distribution with the disease group. These patients had no clinical symptoms in the maxillary sinus and underwent a CT examination for other diseases, such as jaw deformities, temporomandibular joint disorders and impacted teeth. The range of their CT scans included the maxillary sinus, and their CT images displayed either no mucosal thickening or mucosal thickening of <4 mm. Finally, 49 patients (23 males, 26 females; age range: 17–87 years; mean age: 42.2 ± 16.0 years) were enrolled in this group.
Computer-aided detection system
The CAD system evaluated was developed in a joint research project between Gifu University, Gifu, Japan, and Asahi University, Mizuho, Japan.14,15 The system can detect pathology in the maxillary sinus based on differences in the radio-opacities between the left and right maxillary sinuses. The contralateral subtraction technique consisted of three steps:14,15 (1) image filtering of the smoothing and Sobel operation for noise reduction and edge extraction; (2) image registration of the mirrored image using mutual information; and (3) image display method for subtracted pixel data.
The CAD system was placed on a cloud computer, which was protected by a password. We logged into the CAD system using the password and uploaded the digital imaging and communications in medicine or JPEG-formatted data of the panoramic radiographs. Thereafter, the results of the CAD system were displayed as shown in Figure 1. When the CAD system detected maxillary sinusitis, the sinus was displayed in red (left maxillary sinus). If disease was absent, the sinus was displayed in green (right maxillary sinus).
Figure 1.
Results of the computer-aided detection system. The left maxillary sinus was judged as disease (displayed in red), whereas the right sinus was judged as healthy (displayed in green). For colour image see online.
Interpretation of panoramic radiographs
The observers of the panoramic radiographs consisted of two groups: an inexperienced group and experienced group. The inexperienced group was defined as the dentists with experience within 2 years. It included 12 dentists (9 males, 3 females; age range: 25–28 years; mean age: 26.2 ± 1.0 years), comprising two dentists with 1-year experience of oral and maxillofacial radiology, six dentists with 1-year experience of oral and maxillofacial surgery, one dentist with 2-year experience of orthodontics and three dental residents. The experienced group consisted of four specialists in oral and maxillofacial radiology (3 males, 1 female; age range: 51–59 years).
Observations of the panoramic radiographs were performed on a personal computer monitor. The used personal computer monitor was RadiForce G20 (Eizo Nanao Corp., Ishikawa, Japan), with a size of 20.1 inches and resolution of 1600 × 1200 dpi. The following points were explained before interpretation: only unilateral maxillary sinusitis and mucus retention cysts were included; and bilateral maxillary sinusitis and other diseases were not included. The results of the CAD system were not necessarily correct. The observers evaluated the presence or absence of disease according to the following five scores: 1, absence of disease; 2, probable absence of disease; 3, unable to determine presence or absence of disease; 4, probable presence of disease; and 5, presence of disease. If the observers judged that disease was present, they were instructed to write down whether the disease existed in the left or right sinus.
At the first interpretation, the panoramic radiographs were randomly presented on the monitor. The observers viewed the images and assessed the presence or absence of disease (interpretation without CAD). At the second interpretation, the panoramic radiographs were presented together with the CAD results. The observers assessed the images, with reference to the CAD results (interpretation with CAD). The interval between the first and second interpretations was longer than 2 weeks. The sensitivity, specificity and accuracy in diagnosing maxillary sinusitis were calculated based on the results of the five-score evaluation. Receiver operating characteristic curves were drawn, and the area under the curve (AUC) values were obtained.
We preliminarily examined whether or not there were differences in diagnostic performance between the first and second interpretations. Five observers performed interpretations of 30 panoramic radiographs twice, with an interval of longer than 2 weeks. As a result, no differences in sensitivity, specificity and accuracy were observed between the two interpretations. We considered that improvement of diagnostic performance from experience was not observed with an interval of longer than 2 weeks.
If the diagnostic performance of CAD system increases, the system would be useful for the diagnosis especially in inexperienced dentists. To show it, we focused on 72 panoramic radiographs showing 100% correct answer by CAD system and investigated the diagnostic performance. The images consisted of 38 radiographs in the disease group and 34 radiographs in the healthy group. Based on the results of the second interpretation, the above-mentioned diagnostic performances were calculated (interpretation with CAD100).
The present study was performed with approval from the ethics committee of our university (approval number: 324).
Statistical analysis
For the statistical analysis, software Ekuseru-Toukei 2015 (Social Survey Research Information Co., Ltd., Tokyo, Japan), which can plot the receiver operating characteristic and test the difference of AUC values, was used. For comparisons of the AUC values according to the different observation conditions or between the two observer groups, the χ2 test was performed. Values of p < 0.05 were considered to indicate statistical significance.
Results
Diagnostic performance using the computer-aided detection system
The CAD system for diagnosis of maxillary sinusitis on panoramic radiographs showed sensitivity of 77.6%, specificity of 69.4% and accuracy of 73.5%.
Diagnostic performance in two observer groups
The results for diagnostic performance in the inexperienced observer group are shown in Table 1. The interpretation without CAD showed sensitivity of 63.4 ± 12.3%, specificity of 68.6 ± 21.0% and accuracy of 66.0 ± 6.5%. The interpretation with CAD showed sensitivity of 71.6 ± 9.6%, specificity of 75.3 ± 14.0% and accuracy of 73.4 ± 4.8%. The interpretation with CAD100 showed sensitivity of 82.7 ± 13.1%, specificity of 90.4 ± 11.7% and accuracy of 86.3 ± 6.5%.
Table 1.
Diagnostic performance of the maxillary sinusitis
| Conditions of interpretation | Sensitivity | Specificity | Accuracy | AUC |
|---|---|---|---|---|
| Inexperienced observers | ||||
| Interpretation without CAD | 63.4 ± 12.3 | 68.6 ± 21.0 | 66.0 ± 6.5 | 0.728 |
| Interpretation with CAD | 71.6 ± 9.6 | 75.3 ± 14.0 | 73.4 ± 4.8 | 0.780 |
| Interpretation with CAD100 | 82.7 ± 13.1 | 90.4 ± 11.7 | 86.3 ± 6.5 | 0.914 |
| Experienced observers | ||||
| Interpretation without CAD | 74.5 ± 2.1 | 85.2 ± 4.5 | 79.9 ± 1.5 | 0.871 |
| Interpretation with CAD | 76.0 ± 4.5 | 86.2 ± 5.1 | 81.1 ± 3.8 | 0.897 |
AUC, area under the curve; CAD, computer-aided detection.
The results for diagnostic performance in the experienced observer group are also shown in Table 1. The interpretation without CAD showed sensitivity of 74.5 ± 2.1%, specificity of 85.2 ± 4.5% and accuracy of 79.9 ± 1.5%. The interpretation with CAD showed sensitivity of 76.0 ± 4.5%, specificity of 86.2 ± 5.1% and accuracy of 81.1 ± 3.8%.
The receiver operating characteristic curves are shown in Figure 2, and the corresponding AUC values are shown in Table 1. In the inexperienced group, the AUC without and with CAD was 0.728 and 0.780, respectively, and the AUC with CAD100 was 0.914. In the experienced group, the AUC without and with CAD was 0.871 and 0.897, respectively.
Figure 2.
Receiver operating characteristic curves. The blue dotted line shows the curve for the interpretation without computer-aided detection (CAD) in the inexperienced group. The blue solid line shows the curve for the interpretation with CAD in the inexperienced group. The blue broken line shows the curve for the interpretation with CAD100 in the inexperienced group. The red dotted line shows the curve for the interpretation without CAD in the experienced group. The red solid line shows the curve for the interpretation with CAD in the inexperienced group. FPF, false-positive fraction; TPF, true-positive fraction. For colour image see online.
The results of comparisons of AUC values according to the different observer conditions and between the two observer groups are shown in Table 2. In the inexperienced group, the AUC with CAD was significantly larger than that without CAD (p = 0.000029; χ2 test), and the AUC with CAD100 was significantly larger than those with the other conditions (vs without CAD: p = 0.000000; vs with CAD: p = 0.000004; χ2 test). In the experienced group, there was no significant difference in the AUC values for interpretation without and with CAD.
Table 2.
Comparison of area under the curve
| Conditions of interpretation | χ2 test | |
|---|---|---|
| Inexperienced group | ||
| without CAD vs with CAD | p = 0.000029 | ** |
| without CAD vs with CAD100 | p = 0.000000 | ** |
| with CAD vs with CAD100 | p = 0.000004 | ** |
| Experienced group | ||
| without CAD vs with CAD | p = 0.105733 | |
| without CAD in inexperienced group | ||
| vs without CAD in experienced group | p = 0.000000 | ** |
| vs with CAD in experienced group | p = 0.000000 | ** |
| with CAD in inexperienced group | ||
| vs without CAD in experienced group | p = 0.000079 | ** |
| vs with CAD in experienced group | p = 0.000000 | ** |
| with CAD100 in inexperienced group | ||
| vs without CAD in experienced group | p = 0.635058 | |
| vs with CAD in experienced group | p = 0.584225 |
CAD, computer-aided detection.
** p < 0.001.
The AUC values without and with CAD in the inexperienced group were significantly smaller than the AUC values in the experienced group (p-values with χ2 test are shown in Table 2). There were no significant differences between the AUC with CAD100 in the inexperienced group and the AUC values without and with CAD in the experienced group (Table 2).
Discussion
Panoramic radiographs can detect periapical and periodontal diseases, as well as jawbone cysts and tumours.4 These images are also used for screening of osteoporosis and carotid artery calcification.5–8 However, diagnosis of the maxillary sinus is often difficult for inexperienced dentists or residents. Regarding maxillary sinusitis, easily recurring and refractory sinusitis (eosinophilic sinusitis) has been reported.17 Maxillary sinus carcinoma has been reported to be induced from Schneider papilloma.18,19 Therefore, early diagnosis and treatment of maxillary sinus lesions is important.20
CAD systems for chest radiographs have been developed to support the screening of lung lesions at group medical examinations.21 These systems were created based on differences in the left and right densities.12 Using this technique, CAD systems for maxillary sinus lesions on panoramic radiographs have been developed.14,15 Under this background, the present study targeted evaluation of unilateral maxillary sinusitis.
The diagnostic performance in the inexperienced group increased after reference to the CAD results, whereas that in the experienced group did not increase. When panoramic radiographs with CAD100 were focused, the diagnostic performance in the inexperienced group was elevated to an equivalent level of that in the experienced group. Therefore, the CAD system with high diagnostic performance is desired. Further improvements to the CAD system would enhance the support for diagnosis by inexperienced observers.
Application of CAD systems to telediagnosis has been attempted.22 The practical use of this application would help general dental practitioners. Misdiagnosis or oversight would be prevented by reference to the CAD results.
There are several limitations to the present study. Regarding the impact of the familiarity with interpretation on the diagnostic performance, it is predicted that effects would be reduced by taking the interval of more than 2 weeks between two interpretations. In the main experiment of a large number of subjects, it was difficult to confirm the effects. Therefore, prior to the main experiment, we performed the preliminary experiments of randomly extracted 30 cases. As a result, there was no difference in diagnostic performance between the first and second interpretations, when performed with an interval of longer than 2 weeks.
Furthermore, the CAD system of this study was based on the difference in densities between the left and right maxillary sinus. Therefore, the present study targeted evaluation of unilateral maxillary sinusitis. The development of CAD systems for detection of bilateral maxillary sinus disease would be desirable. This system also did not have the ability to distinguish mucous retention cyst from mucosal thickening or fluid collection. This is the reason why patients with mucous retention cyst were included in the study. In the future, development of the CAD system that can differentiate them using information such as the radio-opaque shape will be desired.
In conclusion, CAD systems for panoramic radiographs can support the diagnosis of maxillary sinusitis by inexperienced dentists or dental residents.
Acknowledgments
Acknowledgments
The authors thank the observers for participating in the imaging interpretation in this study.
References
- 1.Schubert MS. Allergic fungal sinusitis: pathophysiology, diagnosis and management. Med Mycol 2009; 47: S324–330. doi: http://dx.doi.org/10.1080/13693780802314809 [DOI] [PubMed] [Google Scholar]
- 2.Troeltzsch M, Pache C, Troeltzsch M, Kaeppler G, Ehrenfeld M, Otto S, et al. Etiology and clinical characteristics of symptomatic unilateral maxillary sinusitis: a review of 174 cases. J Craniomaxillofac Surg 2015; 43: 1522–9. doi: http://dx.doi.org/10.1016/j.jcms.2015.07.021 [DOI] [PubMed] [Google Scholar]
- 3.Obayashi N, Ariji Y, Goto M, Izumi M, Naitoh M, Kurita K, et al. Spread of odontogenic infection originating in the maxillary teeth: computerized tomographic assessment. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 2004; 98: 223–31. doi: http://dx.doi.org/10.1016/j.tripleo.2004.05.014 [DOI] [PubMed] [Google Scholar]
- 4.Suomalainen A, Pakbaznejad Esmaeili E, Robinson S. Dentomaxillofacial imaging with panoramic views and cone beam CT. Insights Imaging 2015; 6: 1–16. doi: http://dx.doi.org/10.1007/s13244-014-0379-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Taguchi A, Suei Y, Sanada M, Ohtsuka M, Nakamoto T, Sumida H, et al. Validation of dental panoramic radiography measures for identifying postmenopausal women with spinal osteoporosis. AJR Am J Roentgenol 2004; 183: 1755–60. doi: http://dx.doi.org/10.2214/ajr.183.6.01831755 [DOI] [PubMed] [Google Scholar]
- 6.Ishii K, Taguchi A, Nakamoto T, Ohtsuka M, Sutthiprapaporn P, Tsuda M, et al. Diagnostic efficacy of alveolar bone loss of the mandible for identifying postmenopausal women with femoral osteoporosis. Dentomaxillofac Radiol 2007; 36: 28–33. doi: http://dx.doi.org/10.1259/dmfr/28366679 [DOI] [PubMed] [Google Scholar]
- 7.Božič M, Ihan Hren N. A novel method of dental panoramic tomogram analysis: a perspective tool for a screening test for osteoporosis. J Craniomaxillofac Surg 2013; 41: 808–15. doi: http://dx.doi.org/10.1016/j.jcms.2013.01.033 [DOI] [PubMed] [Google Scholar]
- 8.Yamada S, Uchida K, Iwamoto Y, Sugino N, Yoshinari N, Kagami H, et al. Panoramic radiography measurements, osteoporosis diagnoses and fractures in Japanese men and women. Oral Dis 2015; 21: 335–41. doi: http://dx.doi.org/10.1111/odi.12282 [DOI] [PubMed] [Google Scholar]
- 9.Ohba T, Ogawa Y, Hiromatsu T, Shinohara Y. Experimental comparison of radiographic techniques in the detection of maxillary sinus disease. Dentomaxillofac Radiol 1990; 19: 13–17. doi: http://dx.doi.org/10.1259/dmfr.19.1.2387471 [DOI] [PubMed] [Google Scholar]
- 10.Ohba T, Ogawa Y, Shinohara Y, Hiromatsu T, Uchida A, Toyoda Y. Limitations of panoramic radiography in the detection of bone defects in the posterior wall of the maxillary sinus: an experimental study. Dentomaxillofac Radiol 1994; 23: 149–53. doi: http://dx.doi.org/10.1259/dmfr.23.3.7835515 [DOI] [PubMed] [Google Scholar]
- 11.Damante JH, Filho LI, Silva MA. Radiographic image of the hard palate and nasal fossa floor in panoramic radiography. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 1998; 85: 479–84. doi: http://dx.doi.org/10.1016/S1079-2104(98)90078-6 [DOI] [PubMed] [Google Scholar]
- 12.Li Q, Katsuragawa S, Ishida T, Yoshida H, Tsukuda S, MacMahon H, et al. Contralateral subtraction: a novel technique for detection of asymmetric abnormalities on digital chest radiographs. Med Phys 2000; 27: 47–55. doi: http://dx.doi.org/10.1118/1.598856 [DOI] [PubMed] [Google Scholar]
- 13.Maduskar P, Muyoyeta M, Ayles H, Hogeweg L, Peters-Bax L, van Ginneken B. Detection of tuberculosis using digital chest radiography: automated reading vs interpretation by clinical officers. Int J Tuberc Lung Dis 2013; 17: 1613–20. doi: http://dx.doi.org/10.5588/ijtld.13.0325 [DOI] [PubMed] [Google Scholar]
- 14.Hayashi T. A new screening pathway for identifying asymptomatic patients using dental panoramic radiographs. Computer-aided diagnosis. Proc SPIE Med Imaging 2012; 8315: pp. 83152K. [Google Scholar]
- 15.Hara T, Mori S, Kaneda T, Hayashi T, Katsumata A, Fujita H. Automated contralateral subtraction of dental panoramic radiographs for detecting abnormalities in paranasal sinus. Medical Imaging 2011: Computer-Aided Diagnosis. Proc SPIE; 7963: pp. 79632R. [Google Scholar]
- 16.Yoshiura K, Ban S, Hijiya T, Yuasa K, Miwa K, Ariji E, et al. Analysis of maxillary sinusitis using computed tomography. Dentomaxillofac Radiol 1993; 22: 86–92. doi: http://dx.doi.org/10.1259/dmfr.22.2.8375560 [DOI] [PubMed] [Google Scholar]
- 17.Fujieda S, Sakashita M, Tokunaga T, Okano M, Haruna Y, Yoshikawa M, et al. Practice guideline for eosinophilic rhinosinusitis [in Japanese.] Nihon Jibiinkoka Gakkai Kaiho 2015; 118: 728–35. doi: http://dx.doi.org/10.3950/jibiinkoka.118.728 [PubMed] [Google Scholar]
- 18.Yu HX, Liu G. Malignant transformation of sinonasal inverted papilloma: a retrospective analysis of 32 cases. Oncol Lett 2014; 8: 2637–41. doi: http://dx.doi.org/10.3892/ol.2014.2539 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Garcia AS, Bravo-Calderón DM, Ferreira MP, Oliveira DT. Squamous cell carcinoma arising from inverted schneiderian papilloma: a case report with oral involvement. Case Rep Otolaryngol 2014; 2014: 478092. doi: http://dx.doi.org/10.1155/2014/478092 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Kreppel M, Safi AF, Scheer M, Nickenig HJ, Zöller J, Preuss S, et al. The importance of early diagnosis in patients with maxillary sinus carcinoma. Eur Arch Otorhinolaryngol 2015 Sep 7. Epub ahead of print. doi: http://dx.doi.org/10.1007/s00405-015-3776-5 [DOI] [PubMed] [Google Scholar]
- 21.Song W, Xu Y, Xie YM, Fan L, Qian JZ, Jin ZY. Improved marking and characterizing of pulmonary nodules on digital radiographs using a computer-aided diagnosis system. Chin Med Sci J 2007; 22: 139–43. [PubMed] [Google Scholar]
- 22.Méndez AJ, Souto M, Tahoces PG, Vidal JJ. Computer aided diagnosis for breast masses detection on a telemammography system. Comput Med Imaging Graph 2003; 27: 497–502. doi: http://dx.doi.org/10.1016/S0895-6111(03)00035-1 [DOI] [PubMed] [Google Scholar]