Key Points
Question
How effective are deep learning (DL) models in predicting conductive hearing loss (CHL) from otoscopic images of ears with otitis media with effusion?
Findings
In this diagnostic/prognostic study including 2790 otoscopic images from 1239 patients, a DL model was developed using otoscopic images and validated in multiple centers. The DL model predicted CHL with an accuracy of 81%.
Meaning
This research to predict CHL using a DL model revealed great potential for the quick and accurate prediction of degree of conductive hearing loss, and offers opportunities to reduce patient waiting time and avoid unnecessary referral.
This retrospective study examines using deep learning techniques and a logistic regression model based on tympanic membrane features to assess conductive hearing loss from otoscopic images.
Abstract
Importance
Otitis media with effusion (OME) is one of the most common causes of acquired conductive hearing loss (CHL). Persistent hearing loss is associated with poor childhood speech and language development and other adverse consequence. However, to obtain accurate and reliable hearing thresholds largely requires a high degree of cooperation from the patients.
Objective
To predict CHL from otoscopic images using deep learning (DL) techniques and a logistic regression model based on tympanic membrane features.
Design, Setting, and Participants
A retrospective diagnostic/prognostic study was conducted using 2790 otoscopic images obtained from multiple centers between January 2015 and November 2020. Participants were aged between 4 and 89 years. Of 1239 participants, there were 209 ears from children and adolescents (aged 4-18 years [16.87%]), 804 ears from adults (aged 18-60 years [64.89%]), and 226 ears from older people (aged >60 years, [18.24%]). Overall, 679 ears (54.8%) were from men. The 2790 otoscopic images were randomly assigned into a training set (2232 [80%]), and validation set (558 [20%]). The DL model was developed to predict an average air-bone gap greater than 10 dB. A logistic regression model was also developed based on otoscopic features.
Main Outcomes and Measures
The performance of the DL model in predicting CHL was measured using the area under the receiver operating curve (AUC), accuracy, and F1 score (a measure of the quality of a classifier, which is the harmonic mean of precision and recall; a higher F1 score means better performance). In addition, these evaluation parameters were compared to results obtained from the logistic regression model and predictions made by three otologists.
Results
The performance of the DL model in predicting CHL showed the AUC of 0.74, accuracy of 81%, and F1 score of 0.89. This was better than the results from the logistic regression model (ie, AUC of 0.60, accuracy of 76%, and F1 score of 0.82), and much improved on the performance of the 3 otologists; accuracy of 16%, 30%, 39%, and F1 scores of 0.09, 0.18, and 0.25, respectively. Furthermore, the DL model took 2.5 seconds to predict from 205 otoscopic images, whereas the 3 otologists spent 633 seconds, 645 seconds, and 692 seconds, respectively.
Conclusions and Relevance
The model in this diagnostic/prognostic study provided greater accuracy in prediction of CHL in ears with OME than those obtained from the logistic regression model and otologists. This indicates great potential for the use of artificial intelligence tools to facilitate CHL evaluation when CHL is unable to be measured.
Introduction
Otitis media with effusion (OME) is one of the most common middle ear conditions presenting at otolaryngology and audiology clinics. Evidence suggests that approximately 90% of preschool children will experience OME at least once, and 30% to 40% are likely to have recurrent episodes.1,2 In the natural course of OME, more than 20% of children will have a conductive hearing loss (CHL) of more than 35 dB and 5% to 10% more than 50 dB.3,4,5 The CHL in patients with OME is largely caused by fluid accumulation in the middle ear cavity, which increases stiffness and mass of the tympanum.6 Without timely and appropriate intervention, persistent CHL in preschool children with OME is associated with poor speech and language development and other adverse consequences, for example behavior issues and educational difficulties.6,7,8
According to the clinical practice guidelines recommended by the American Academy of Otolaryngology in 2016,6 otoscopy, pure tone audiometry (PTA), and tympanometry are suggested as essential tools for OME diagnosis. Although PTA is considered the gold standard for assessing hearing sensitivity, it has several limitations owing to the subjective nature of its measurement, requiring a high degree of cooperation from the patients to achieve accurate and reliable hearing thresholds. Indeed, Browning et al9 and Kemaloglu et al10 have indicated that younger children are unlikely to be able to concentrate long enough to complete the test.
The otoscope is a fundamental tool used in otologic practice to examine tympanic membrane condition by observing its shape, color, and integrity.11,12 However, otoscopic examination and otoscopic images are currently unable to provide information on hearing levels. A number of studies have shown that deep learning (DL) is an effective tool to classify pathologic conditions from medical images, information that can then support diagnosis and treatment.13,14,15 In the field of otology, previous DL studies have mainly used otoscopic images as input data and focused on how to achieve a high diagnostic accuracy in classifying middle ear disorders.16,17,18,19,20,21,22
The aim of this study was to predict the degree of CHL from otoscopic images using DL techniques and a logistic regression model based on tympanic membrane features. The importance of this novel research is that it could enable a quick and accurate prediction of the degree of CHL, and thus provide solutions to reduce patient waiting time and avoid unnecessary referral.
Methods
Study Design and Data Set
In this multicenter study, otoscopic images of OME cases were obtained from 3 tertiary care medical centers in south China, including Sun Yat-sen Memorial Hospital of Sun Yat-sen University, The Third Affiliated Hospital of Sun Yat-sen University, and Zhongshan People’s Hospital. All data were collected between January 2015 and November 2020 and reviewed retrospectively. Cases of OME were diagnosed according to clinical otologic practice that included medical history, physical examination with otoscopes, and audiological tests (PTA and tympanometry). Inclusion criteria required that otoscopic images and audiological assessment results were measured at the same time and on individual OME ears. Ears with OME with a history of middle ear surgery (eg, grommet insertion) were excluded. The high incidence of nasopharyngeal tumors in the south of China is associated with 8% to 29% of patients there having OME after radiotherapy.23,24 However, owing to the low incidence in the other regions of China, to avoid confusion, radiation-induced OME ears were excluded in the present study. In addition, otoscopic images of OME ears labeled with an incorrect ear side were excluded. This study was approved by the institutional review board of Sun Yat-sen Memorial Hospital (reference number SYSEC-KY-KS-2021-191), and written informed consent was waived owing to the retrospective nature of the study.
Quality Control for Otoscopic Images
All otoscopic images were taken by otologists in routine otologic practice. They were captured in standard high-resolution using 2 different otoscopes, a 4-mm STORZ 0° endoscope (KARL STORZ, Germany) or a 2.7-mm TIAN SONG 0° endoscope (TIAN SONG, China). Images were stored in JPEG or PNG format in the range 500 × 500 to 700 × 700 pixels. Only otoscopic images where the pars flaccida and pars tensa were captured intact were included in this study. Furthermore, only otoscopic images taken using standard white light were deemed eligible for developing the DL model. Overexposed and underexposed images, and images where the tympanic membrane was largely blocked by earwax were excluded. The 1 to 3 best quality otoscopic images captured from different angles of each ear were selected for analysis. Overall, PTA hearing thresholds at frequencies of 500, 1000, 2000, and 4000 Hz were measured in left and right ears. The air-bone gap (ABG) was calculated at each frequency and an average of the 4 frequencies calculated. We chose averaged ABG across the 4 frequencies greater than 10 dB as the criteria of conductive hearing loss.25 The severity of OME and changes to the tympanic membrane is associated with different degrees of conductive hearing loss at different individual frequencies.26,27,28 Consequently, the DL model was used to predict CHL using different criteria: ABG greater than 10 dB at each frequency, averaged air conduction hearing threshold more than 500, 1000, 2000, 4000 Hz greater than 25 dB, and averaged ABG of the 4 frequencies greater than 20 dB. Changes in the tympanic membrane used to develop the logistic regression model included attic retraction pocket, atelectasis, type of fluid (full, air-fluid level, bubble), color of tympanic membrane, color of fluid, myringosclerosis, and opacification.16,25,27 An otologist (J.B.Z.) was assigned to collect otoscopic images, other clinical data, and record changes in the tympanic membrane. Another otologist (Y.X.C.) with more than 10 years of clinical experience reviewed the data independently. Any discrepancies were discussed with another experienced otologist (S.J.C.) until consensus was reached. The detailed criteria for the otologist to predict CHL on the basis of otoscopic images are described in the eMethods in the Supplement.
Convolutional Neural Network Model Development and Validation
Convolutional neural network (CNN) model development and validation included data preprocessing, strategies to minimize overfittings, model selection, construction of a prediction system, and main hyperparameters and are detailed in the eResults in the Supplement. To evaluate the performance of the DL model, the discriminative performance was assessed from the area under the receiver operating characteristic (AUC) curve, accuracy, F1 score (a measure of the quality of a classifier, which is the harmonic mean of precision and recall, higher F1 score means better performance) and comparison with otologists’ prediction and the outcomes obtained using the logistic regression model, all detailed in eFigure 1 in the Supplement.
Deep Learning Model Interpretability
To interpret how the DL model produced the diagnostic outputs it was crucial to visualize what the discriminative regions in the otoscopic images used were by the DL model. A gradient-weighted heatmap was obtained from the final layer of the neural network, which highlighted the discriminative regions in the original otoscopic image using blue to red colors.
Experimental Equipment and Statistical Analysis
The experiments were performed using Python (version 3.6 in Keras, version 2.2.4) to develop the diagnostic model based on TensorFlow with 4 Titan XP 256 GB GPU (version 1.12.0). All statistical analyses were carried out using SPSS statistical software (version 26, IBM). The categorical variables were compared using the χ2 test and Fisher exact test. The continuous variables were compared using the Kruskal-Wallis rank-sum test. The AUC, accuracy, and F1 score were calculated, and 95% CIs of AUC were estimated using bootstrapping with 1000 replicates. Two-sided P < .05 indicated statistical significance.
Results
Summary of Otoscopic Images
After application of inclusion and exclusion criteria, a total of 1239 ears with clinical data and 2790 otoscopic images were taken from the patients for use in model development and internal validation. Within the data set, 738 ears with 1678 otoscopic images were from Sun Yat-sen Memorial Hospital of Sun Yat-sen University; and 330 ears with 721 otoscopic images were from The Third Affiliated Hospital of Sun Yat-sen University; and 171 ears with 391 otoscopic images were from Zhongshan people’s hospital. The OME ears with otoscopic images were randomly split into the training set (1034 ears with 2232 otoscopic images) and validation set (205 ears with 558 otoscopic images).
Summary of Clinical Characteristics
Clinical diagnosis of CHL was taken on the basis of the averaged ABG across 4 frequencies being greater than 10 dB. Overall, 996 ears (80.4%) demonstrated CHL. The data set comprised 1239 participants aged between 4 and 89 years. This included 209 ears from children and adolescents (aged 4-18 years 16.9%), 804 ears from adults (aged 18-60 years, 64.9%), and 226 ears from older people (aged >60 years, 18.2%); 679 ears (54.8%) were from men. Ears with tympanogram A (5.0%) were further confirmed with tympanotomy, whereas 835 ears (67.40%) with tympanogram B, and 342 ears (27.6%) with tympanogram C did not need further testing to confirm.
Atelectasis (753 [60.7%]) and attic retraction pocket (922 [74.3%]) of any stage were most commonly seen. Pearl gray (736 [59.5%]) was the most common tympanic membrane color, and amber (1088 [87.8%]) the most common fluid color. Full (898 [72.5%]) was the most common type of fluid status. Statistical analysis revealed that demographic factors, type of tympanogram, sex, changes of tympanic membrane, and age were not significantly different between the training and validation sets. Detailed characteristics of the ears in the training and validation sets are summarized and compared in Table 1.
Table 1. Basic Characteristics of Participants.
| Characteristic | No. (%) | P value | |
|---|---|---|---|
| Training set (n = 1024) | Validation set (n = 215) | ||
| Age, median (IQR), y | 44 (27-58) | 44 (25-60) | .93 |
| Male sex | 562 (54.9) | 117 (54.4) | .90 |
| Tympanogram type | |||
| Tympanogram A | 52 (5.0) | 10 (4.7) | .35 |
| Tympanogram B | 698 (68.2) | 137 (13.4) | |
| Tympanogram C | 274 (26.8) | 68 (31.6) | |
| Pathologic features of tympanic membranes | |||
| Atelectasis | 633 (61.8) | 120 (55.9) | |
| Stage I | 562 (54.9) | 106 (49.3) | .44 |
| Stage II | 51 (5.0) | 11 (5.1) | |
| Stage III | 10 (1.0) | 1 (0.5) | |
| Stage IV | 10 (1.0) | 2 (0.9) | |
| Attic retraction pocket | 768 (75) | 154 (71.6) | |
| Stage I | 561 (54.8) | 111 (51.6) | .30 |
| Stage II | 189 (18.5) | 38 (17.7) | |
| Stage III | 9 (0.9) | 4 (1.9) | |
| Stage IV | 9 (0.9) | 1 (0.5) | |
| Color of tympanic membrane | |||
| Pearl gray | 593 (57.9) | 143 (66.5) | .07 |
| White | 41 (4) | 19 (8.8) | |
| Transparent | 179 (17.5) | 53 (24.7) | |
| Color of fluid | |||
| Amber | 904 (88.3) | 184 (85.6) | .54 |
| Transparent | 5 (0.5) | 2 (0.9) | |
| Yellow-white | 39 (3.8) | 12 (5.7) | |
| Dark | 76 (7.6) | 17 (7.9) | |
| Volume of fluid | |||
| Bubble | 89 (8.7) | 17 (7.9) | .87 |
| Air-fluid level | 192 (18.8) | 43 (20.0) | |
| Full | 743 (72.6) | 155 (72.1) | |
| Atrophy | 97 (9.5) | 20 (9.3) | .94 |
| Opacification | |||
| Marginal opacification | 117 (11.4) | 17 (7.9) | .01 |
| Full opacification | 53 (5.2) | 23 (10.7) | |
| Myringosclerosis | 56 (5.5) | 9 (4.2) | .44 |
Evaluation of DL Model Performance
To commence, receiver operating characteristic (ROC) curves were plotted and AUC of the DL model calculated to show diagnostic performance. The DL diagnostic model achieved an AUC of 0.74 (95% CI, 0.67-0.80) with ABG over 4 frequencies greater than 10 dB. The F1 score was 0.97 (95% CI, 0.95-0.99), and accuracy 81% (95% CI, 77%-85%). With ABG at 0.5 kHz, the AUC was 0.74 (95% CI, 0.65-0.83), 1 kHz 0.79 (95% CI, 0.68-0.85), 2 kHz 0.68 (95% CI, 0.63-0.75), and at 4 kHz 0.65 (95% CI, 0.55-0.78), respectively. When the DL model was applied to diagnose those with a CHL using averaged ABG across the 4 frequencies greater than 20 dB, the AUC was 0.69 (95% CI, 0.60-0.78), with F1 score of 0.83 (95% CI, 0.77-0.88), and accuracy of 73% (95% CI, 65%-80%) (Table 2). When human and artificial intelligence were compared, the performance of the 3 otologists in predicting CHL by reviewing the otoscopic images was lower compared with the DL model in terms of accuracy (81% vs 30%, 16%, and 39%). The DL model took just 2.5 seconds to complete its prediction of CHL, whereas the 3 otologists took 633 seconds, 645 seconds, and 692 seconds, respectively. The logistic regression model based on tympanic membrane changes provided a lower AUC of 0.62 (95% CI, 0.45-0.75), F1 score of 0.82 (95% CI, 0.79-0.85), and accuracy of 76% (95% CI, 70%-81%) (eFigure 2 in the Supplement; Table 3).
Table 2. Performance in Different Characteristics Groups.
| Group | AUC (95%CI) | F1 score (95% CI) | Accuracy, % (95% CI) |
|---|---|---|---|
| Male | 0.75 (0.62-0.86) | 0.91 (0.88-0.95) | 84 (79-90) |
| Female | 0.71 (0.60-0.80) | 0.86 (0.81-0.91) | 76 (68-83) |
| Age, y | |||
| <18 | 0.65 (0.47-0.81) | 0.83 (0.76-0.90) | 72 (62-82) |
| 18-60 | 0.76 (0.67-0.90) | 0.89 (0.85-0.93) | 80 (75-87) |
| >60 | 0.78 (0.63-0.90) | 0.92 (0.88-0.96) | 86 (78-93) |
| Tympanogram A | 0.78 (0.44-1.00) | 0.58 (0.21-0.80) | 49 (20-73) |
| Tympanogram B | 0.74 (0.64-0.84) | 0.93 (0.90-0.95) | 87 (82-91) |
| Tympanogram C | 0.72 (0.60-0.82) | 0.85 (0.78-0.90) | 74 (66-82) |
Abbreviation: AUC, area under the receiver operating curve.
Table 3. Performance of the DL Model, Logistic Regression Model, and Otologists.
| Group | AUC (95% CI) | F1 score (95% CI) | Accuracy, % (95% CI) |
|---|---|---|---|
| Otologist 1 (5 y) | NA | 0.18 (0.20-0.38) | 30 (18-33) |
| Otologist 2 (10 y) | NA | 0.09 (0.06-0.24) | 16 (11-23) |
| Otologist 3 (10 y) | NA | 0.25 (0.22-0.47) | 39 (22-48) |
| ABG of 500 Hz >10 dBa | 0.74 (0.65-0.83) | 0.87 (0.81-0.90) | 78 (70-84) |
| ABG of 1000 Hz >10 dBa | 0.79 (0.68-0.85) | 0.92 (0.86-0.94) | 85 (78-88) |
| ABG of 2000 Hz >10 dBa | 0.68 (0.63-0.75) | 0.53 (0.48-0.59) | 66 (60-73) |
| ABG of 4000 Hz >10 dBa | 0.65 (0.55-0.78) | 0.81 (0.75-0.87) | 72 (63-80) |
| Averaged ABG>10 dBa | 0.74 (0.67-0.81) | 0.89 (0.86-0.92) | 81 (77-85) |
| Averaged ABG>20 dBa | 0.69 (0.60-0.78) | 0.83 (0.77-0.88) | 73 (65-80) |
| Logistic regression model | 0.62 (0.45-0.75) | 0.82 (0.79-0.85) | 76 (70-81) |
Abbreviations: ABG, air-bone gap; AUC, area under the receiver operating curve; DL, deep learning; NA, not applicable.
Deep learning model.
Performance of the model was lower in the children and adolescent (aged 4-18 years) group than in the adult (aged 18-60 years) and older (aged >60 years) groups. Tympanogram B and C had a higher accuracy than tympanogram A (87%, 74%, 49%, respectively) (Table 2). The clinical characteristics of OME ears where CHL was predicted rightly or wrongly by the DL are illustrated in Table 4. A higher percentage of wrong cases was seen in stage I atelectasis (52.5% vs 48.6%), stage I attic retraction pocket (55.0% vs 50.9%), and bubble (20.0% vs 5.1%), with few errors in myringosclerosis (0% vs 5.1%).
Table 4. Characteristics of the DL Model Missed and Identified CHL.
| Characteristic | No. (%) | P value | |
|---|---|---|---|
| Missed (n = 40) | Identified (n = 175) | ||
| Age, median (IQR), y | 39 (18-65) | 46 (29-60) | .11 |
| Male sex | 16 (40) | 101 (57.8) | .04 |
| Tympanogram | |||
| Tympanogram A | 6 (15) | 4 (2.3) | .001 |
| Tympanogram B | 19 (47.5) | 118 (67.4) | |
| Tympanogram C | 15 (37.5) | 53 (30.3) | |
| Features of tympanic membranes | |||
| Atelectasis | 22 (55) | 98 (56) | |
| Stage I | 21 (52.5) | 85 (48.6) | .01 |
| Stage II | 1 (2.5) | 10 (5.7) | |
| Stage III | 0 | 1 (0.6) | |
| Stage IV | 0 | 2 (1.1) | |
| Attic retraction pocket | 24 (60) | 130 (74.3) | |
| Stage I | 22 (55) | 89 (50.9) | .07 |
| Stage II | 2 (5) | 36 (20.6) | |
| Stage III | 0 | 4 (2.3) | |
| Stage IV | 0 | 1 (0.6) | |
| Color of tympanic membranes | |||
| Pearl gray | 26 (65) | 117 (66.9) | .87 |
| White | 3 (7.5) | 16 (9.1) | |
| Transparent | 11 (27.5) | 42 (24.0) | |
| Color of fluid | |||
| Amber | 34 (85) | 150 (85.7) | .85 |
| Transparent | 0 | 2 (1.1) | |
| Yellow-white | 3 (7.5) | 9 (5.1) | |
| Dark | 3 (7.5) | 14 (8.0) | |
| Volume of fluid | |||
| Bubble | 8 (20) | 9 (5.1) | <.001 |
| Air-fluid level | 16 (40) | 27 (15.4) | |
| Full | 16 (40) | 139 (79.4) | |
| Atrophy | 1 (2.5) | 19 (10.9) | .10 |
| Opacification | |||
| Marginal opacification | 3 (7.5) | 14 (8.0) | .70 |
| Full opacification | 3 (7.5) | 21 (12.0) | |
| Myringosclerosis | 0 | 9 (5.1) | .14 |
Abbreviations: CHL, conductive hearing loss; DL, deep learning.
The DL Model Interpretability
The heatmap (Figure) highlights the changes in the tympanic membrane in the otoscopic images. These regions are what otologists based their estimates of CHL on, such as atelectasis, attic retraction pocket, and air-fluid level.
Figure. Heatmap Illustration Overlaid by Deep Learning on Original Otoscopic Images.
The red color represents a high discriminative region of the otoscopic images for identification of conductive hearing loss. A, No tympanic membrane changes; B, air-fluid level; C and D, atelectasis and attic retraction pocket.
Discussion
The importance of this novel research to predict hearing level using a DL model is that it provides solutions to reduce patient waiting time and avoid unnecessary referral. In the present study, we have demonstrated that DL was used to predict CHL in averaged ABG greater than 10 dB (accuracy of 81%). Notably, the DL model acquired the highest accuracy at 1 kHz (85%). The DL model provided values of AUC, F1 score, and accuracy greater than the logistic regression model and otologists reviewing otoscopic images. Moreover, the DL model only took 2.5 seconds to complete prediction of CHL in the validation set, whereas the 3 otologists took 633 seconds, 645 seconds, and 692 seconds, respectively. The study also demonstrated that the DL model used features changes in the tympanic membrane to predict CHL, such as atelectasis, attic retraction pocket, and air-fluid level.
Evidence has shown that the applications of artificially intelligent technology in health care may be an alternative solution, or provide additional support, for diagnostic decision making. In the field of ENT and clinical audiology, previous studies have shown DL models to be useful diagnostic tools to assist otologists in terms of distinguishing various otologic diseases on the basis of otoscopic images.17,18,19,20,21,22 For example, Cai et al22 proved the capability of artificial intelligence to diagnose OME automatically with an accuracy of 93%. In addition, a recent literature review29 indicated that DL models have achieved accuracy from 75.3% to 99% in predicting noise-induced hearing loss. However, it is noteworthy that the DL algorithm for predicting the degree of CHL using otoscopic images did not attain the excellent performance seen in automated diagnosis of ear diseases in previous studies (ie, accuracy over 90%).18,19,20,21,22 The possible explanation is that OME may cause structural alterations in the tympanic cavity,12 not just tympanic membrane changes. The otoscopic images only provide features of the tympanic membrane and fluid, without providing other features associated with CHL, such as status of the structures in the tympanic cavity and mobility of the tympanic membrane.25,30 Similar experiences occurred in studies in other fields. For example, Noorbakhsh et al31 used hematoxylin and eosin–scanned images to develop CNN architectures for tumor or normal and gene variation classification. Their CNN architectures acquired an AUC of 0.99 in 19 cancer type diagnosis, with only AUCs of 0.65 to 0.80 in TP53 variations. In addition, Coudary et al32 developed a DL model that classified lung histopathologic slides into adenocarcinoma (LUAD), squamous cell carcinoma, and normal lung tissue. This model showed an averaged AUC of 0.97. When DL was used to predict commonly mutated genes in LUAD, the AUCs were decreased, ranging from 0.64 to 0.86.
The heatmap in the Figure shows that tympanic membrane changes such as atelectasis and attic retraction pocket are the distinctive regions used by the DL model to predict CHL. Previous studies have correlated these features with CHL.25,27 Tympanic membrane changes and fluid could limit the mobility of the tympanic membrane and cause erosion of middle ear structures.6,25,27,30,33 However, otologists and the logistic regression model based on tympanic membrane changes show a worse performance in predicting CHL. The possible reason is that these tympanic membrane changes were classified for OME diagnostic assessment, rather than CHL assessment.34,35,36 Moreover, the manual classification of tympanic membrane changes did not contain sufficient detail to reflect the condition of tympanic membrane, such as air-fluid level, not supplying information on how much fluid was in the tympanic cavity. However, the DL model used the whole otoscopic image as input data to avoid missing important detailed information. As a result, the DL model offered a faster and more accurate prediction of CHL than the otologists, which may be useful when applied to the clinical environment.
Although the group younger than 18 years showed the lowest accuracy level (72%), it was still significantly higher than the prediction perceived by parents using a symptom questionnaire in the study by Swierniak et al,37 with an accuracy of 17.8%. Previous studies have suggested that the diagnostic accuracy for detecting OME using otoscopy is between 60% and 70%,38 whereas the sensitivity and specificity of tympanometry can reach to 70% to 90% for the detection of OME, but it is dependent on patient cooperation.39,40 Furthermore, it should be noted that there are several influencing factors that affect the observation of the tympanic membrane and thus the accuracy in diagnosis during otoscopic examination, such as in the conditions when the external ear canal is partly or completely blocked by cerumen, or when children are unable or unwilling to cooperate with the examination.41,42,43,44 Therefore, it is crucial to obtain high-quality otoscopic images as input data to ensure a reliable and accurate DL model for predicting CHL in ears with OME.
These results that explore the relationship between otoscopic images and CHL provide great potential for internal medicien physicians, general practitioners, pediatricians, and other clinicians in rural areas to use the current DL model to predict CHL by sending their otoscopic images to a centralized server for diagnosis of OME. Patients can then be triaged to otologists through new medical patterns.
Limitations and Future Studies
Although our algorithm achieved an overall accuracy greater than 80%, there are some limitations to this study. First, owing to the nature of retrospective data, lack of a standardized otoscopic image acquisition protocol may have limited their quality and consistency. Therefore, to improve the DL model performance, further prospective randomized studies with standardized otoscopic image acquisition and annotation protocols are recommended. Second, the mobility of the tympanic membrane, status of structures in the tympanic cavity and other clinical factors associated with CHL could have been included in the prediction model. Classifying on the basis of morphological features of the tympanic membrane and their changes could improve performance accuracy in evaluating hearing loss.
Conclusions
In this diagnostic/prognostic study, a clinically useful DL model has been developed using DL training on 2790 otoscopic images from patients located in multiple centers. These results first show that DL algorithms can predict CHL using otoscopic images to an accuracy of 81%. The DL model acquired a better performance in less time than a logistic regression model and otologists at predicting CHL. We intend to improve the performance and application of this DL model with a larger prospective study. A standardized otoscopic images acquisition and annotation protocol should be discussed and published, which will further enhance DL used on otoscopic images.
eMethods
eResults
eFigure 1. Illustration of Hearing Loss Prediction System
eFigure 2. Receiver-Operator Characteristic (ROC) Curves and Corresponding AUC
References
- 1.Tos M. Epidemiology and natural history of secretory otitis. Am J Otol. 1984;5(6):459-462. [PubMed] [Google Scholar]
- 2.Williamson IG, Dunleavey J, Bain J, Robinson D. The natural history of otitis media with effusion--a three-year study of the incidence and prevalence of abnormal tympanograms in four South West Hampshire infant and first schools. J Laryngol Otol. 1994;108(11):930-934. [DOI] [PubMed] [Google Scholar]
- 3.Fria TJ, Cantekin EI, Eichler JA. Hearing acuity of children with otitis media with effusion. Arch Otolaryngol. 1985;111(1):10-16. doi: 10.1001/archotol.1985.00800030044003 [DOI] [PubMed] [Google Scholar]
- 4.Roberts J, Hunter L, Gravel J, et al. Otitis media, hearing loss, and language learning: controversies and current research. J Dev Behav Pediatr. 2004;25(2):110-122. doi: 10.1097/00004703-200404000-00007 [DOI] [PubMed] [Google Scholar]
- 5.Vanneste P, Page C. Otitis media with effusion in children: pathophysiology, diagnosis, and treatment. A review. J Otol. 2019;14(2):33-39. doi: 10.1016/j.joto.2019.01.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Rosenfeld RM, Shin JJ, Schwartz SR, et al. Clinical practice guideline: otitis media with effusion executive summary (update). Otolaryngol Head Neck Surg. 2016;154(2):201-214. doi: 10.1177/0194599815624407 [DOI] [PubMed] [Google Scholar]
- 7.Ito M, Takahashi H, Iino Y, et al. Clinical practice guidelines for the diagnosis and management of otitis media with effusion (OME) in children in Japan, 2015. Auris Nasus Larynx. 2017;44(5):501-508. doi: 10.1016/j.anl.2017.03.018 [DOI] [PubMed] [Google Scholar]
- 8.Bess FH, Dodd-Murphy J, Parker RA. Children with minimal sensorineural hearing loss: prevalence, educational performance, and functional status. Ear Hear. 1998;19(5):339-354. doi: 10.1097/00003446-199810000-00001 [DOI] [PubMed] [Google Scholar]
- 9.Browning GG; MRC Multi-centre Otitis Media Study Group . Influence of age, type of audiometry and child’s concentration on hearing thresholds. Br J Audiol. 2000;34(4):231-240. doi: 10.3109/03005364000000133 [DOI] [PubMed] [Google Scholar]
- 10.Kemaloğlu YK, Gündüz B, Gökmen S, Yilmaz M. Pure tone audiometry in children. Int J Pediatr Otorhinolaryngol. 2005;69(2):209-214. doi: 10.1016/j.ijporl.2004.08.018 [DOI] [PubMed] [Google Scholar]
- 11.Lieberthal AS, Carroll AE, Chonmaitree T, et al. The diagnosis and management of acute otitis media. Pediatrics. 2013;131(3):e964-e999. doi: 10.1542/peds.2012-3488 [DOI] [PubMed] [Google Scholar]
- 12.Areias B, Parente MPL, Santos C, Gentil F, Natal Jorge RM. The human otitis media with effusion: a numerical-based study. Comput Methods Biomech Biomed Engin. 2017;20(9):958-966. doi: 10.1080/10255842.2017.1316382 [DOI] [PubMed] [Google Scholar]
- 13.Gargeya R, Leng T. Automated identification of diabetic retinopathy using deep learning. Ophthalmology. 2017;124(7):962-969. doi: 10.1016/j.ophtha.2017.02.008 [DOI] [PubMed] [Google Scholar]
- 14.Ai T, Yang Z, Hou H, et al. Correlation of chest CT and RT-PCR testing for coronavirus disease 2019 (COVID-19) in China: a report of 1014 cases. Radiology. 2020;296(2):E32-E40. doi: 10.1148/radiol.2020200642 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Bai HX, Wang R, Xiong Z, et al. Artificial intelligence augmentation of radiologist performance in distinguishing COVID-19 from pneumonia of other origin at chest CT. Radiology. 2020;296(3):E156-E165. doi: 10.1148/radiol.2020201491 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Myburgh HC, van Zijl WH, Swanepoel D, Hellström S, Laurent C. Otitis media diagnosis for developing countries using tympanic membrane image-analysis. EBioMedicine. 2016;5:156-160. doi: 10.1016/j.ebiom.2016.02.017 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Myburgh HC, Jose S, Swanepoel DW, Laurent C. Towards low cost automated smartphone- and cloud-based otitis media diagnosis. Biomed Signal Process Control. 2018;39:34-52. doi: 10.1016/j.bspc.2017.07.015 [DOI] [Google Scholar]
- 18.Cha D, Pae C, Seong SB, Choi JY, Park HJ. Automated diagnosis of ear disease using ensemble deep learning with a big otoendoscopy image database. EBioMedicine. 2019;45:606-614. doi: 10.1016/j.ebiom.2019.06.050 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Lee JY, Choi S-H, Chung JW. Automated classification of the tympanic membrane using a convolutional neural network. Appl Sci (Basel). 2019;9(9). doi: 10.3390/app9091827 [DOI] [Google Scholar]
- 20.Khan MA, Kwon S, Choo J, et al. Automatic detection of tympanic membrane and middle ear infection from oto-endoscopic images via convolutional neural networks. Neural Netw. 2020;126:384-394. doi: 10.1016/j.neunet.2020.03.023 [DOI] [PubMed] [Google Scholar]
- 21.Wu Z, Lin Z, Li L, et al. Deep learning for classification of pediatric otitis media. Laryngoscope. 2021;131(7):E2344-E2351. doi: 10.1002/lary.29302 [DOI] [PubMed] [Google Scholar]
- 22.Cai Y, Yu JG, Chen Y, et al. Investigating the use of a two-stage attention-aware convolutional neural network for the automated diagnosis of otitis media from tympanic membrane images: a prediction model development and validation study. BMJ Open. 2021;11(1):e041139. doi: 10.1136/bmjopen-2020-041139 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Chao WY, Wang CF, Chang SJ. Ventilation tube in adults with middle-ear effusion. J Otolaryngol. 1999;28(5):278-281. [PubMed] [Google Scholar]
- 24.Christensen JG, Wessel I, Gothelf AB, Homøe P. Otitis media with effusion after radiotherapy of the head and neck: a systematic review. Acta Oncol. 2018;57(8):1011-1016. doi: 10.1080/0284186X.2018.1468085 [DOI] [PubMed] [Google Scholar]
- 25.Ungkanont K, Charuluxananan S, Komoltri C. Association of otoscopic findings and hearing level in pediatric patients with otitis media with effusion. Int J Pediatr Otorhinolaryngol. 2010;74(9):1063-1066. doi: 10.1016/j.ijporl.2010.06.006 [DOI] [PubMed] [Google Scholar]
- 26.Cai T, McPherson B, Li C, Yang F. Pure tone hearing profiles in children with otitis media with effusion. Disabil Rehabil. 2018;40(10):1166-1175. doi: 10.1080/09638288.2017.1290698 [DOI] [PubMed] [Google Scholar]
- 27.Canali I, Rosito LPS, Longo VD, da Costa SS. Audiometric pattern in moderate and severe tympanic membrane retraction. Otol Neurotol. 2021;42(6):e716-e723. doi: 10.1097/MAO.0000000000003099 [DOI] [PubMed] [Google Scholar]
- 28.Cai T, McPherson B. Hearing loss in children with otitis media with effusion: a systematic review. Int J Audiol. 2017;56(2):65-76. doi: 10.1080/14992027.2016.1250960 [DOI] [PubMed] [Google Scholar]
- 29.Chen F, Cao Z, Grais EM, Zhao F. Contributions and limitations of using machine learning to predict noise-induced hearing loss. Int Arch Occup Environ Health. 2021;94(5):1097-1111. doi: 10.1007/s00420-020-01648-w [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Takahashi H, Honjo I, Hasebe S, Sudo M, Tanabe M. The diagnostic and prognostic value of eardrum mobility in otitis media with effusion. Eur Arch Otorhinolaryngol. 1999;256(4):189-191. doi: 10.1007/s004050050137 [DOI] [PubMed] [Google Scholar]
- 31.Noorbakhsh J, Farahmand S, Foroughi Pour A, et al. Deep learning-based cross-classifications reveal conserved spatial behaviors within tumor histological images. Nat Commun. 2020;11(1):6367. doi: 10.1038/s41467-020-20030-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Coudray N, Ocampo PS, Sakellaropoulos T, et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat Med. 2018;24(10):1559-1567. doi: 10.1038/s41591-018-0177-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Li Y, Hunter LL, Margolis RH, et al. Prospective study of tympanic membrane retraction, hearing loss, and multifrequency tympanometry. Otolaryngol Head Neck Surg. 1999;121(5):514-522. doi: 10.1016/S0194-5998(99)70049-5 [DOI] [PubMed] [Google Scholar]
- 34.Kaleida PH, Fireman P. Diagnostic assessment of otitis media. Clin Allergy Immunol. 2000;15:247-262. [PubMed] [Google Scholar]
- 35.Sadé J, Berco E. Atelectasis and secretory otitis media. Ann Otol Rhinol Laryngol. 1976;85(2)(suppl 25 Pt 2):66-72. doi: 10.1177/00034894760850S214 [DOI] [PubMed] [Google Scholar]
- 36.Tos M, Stangerup SE, Larsen P. Dynamics of eardrum changes following secretory otitis. a prospective study. Arch Otolaryngol Head Neck Surg. 1987;113(4):380-385. doi: 10.1001/archotol.1987.01860040042014 [DOI] [PubMed] [Google Scholar]
- 37.Swierniak W, Gos E, Skarzynski PH, Czajka N, Skarzynski H. The accuracy of parental suspicion of hearing loss in children. Int J Pediatr Otorhinolaryngol. 2021;141:110552. doi: 10.1016/j.ijporl.2020.110552 [DOI] [PubMed] [Google Scholar]
- 38.American Academy of Family Physicians; American Academy of Otolaryngology-Head and Neck Surgery; American Academy of Pediatrics Subcommittee on Otitis Media With Effusion . Otitis media with effusion. Pediatrics. 2004;113(5):1412-1429. doi: 10.1542/peds.113.5.1412 [DOI] [PubMed] [Google Scholar]
- 39.Watters GW, Jones JE, Freeland AP. The predictive value of tympanometry in the diagnosis of middle ear effusion. Clin Otolaryngol Allied Sci. 1997;22(4):343-345. doi: 10.1046/j.1365-2273.1997.00023.x [DOI] [PubMed] [Google Scholar]
- 40.Anwar K, Khan S, Rehman HU, Javaid M, Shahabi I. Otitis media with effusion: accuracy of tympanometry in detecting fluid in the middle ears of children at myringotomies. Pak J Med Sci. 2016;32(2):466-470. doi: 10.12669/pjms.322.9009 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Brodie KD, David AP, Kriss H, Chan DK. Outcomes of an early childhood hearing screening program in a low-income setting. JAMA Otolaryngol Head Neck Surg. 2022. doi: 10.1001/jamaoto.2021.4430 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.McManus B, Harbarger C, Grillis A, et al. Otoscopy and tympanometry outcomes from the National Health and Nutrition Examination Survey (NHANES). Am J Otolaryngol. 2022;43(2):103332. doi: 10.1016/j.amjoto.2021.103332 [DOI] [PubMed] [Google Scholar]
- 43.Esposito S, Bianchini S, Argentiero A, Gobbi R, Vicini C, Principi N. New approaches and technologies to improve accuracy of acute otitis media diagnosis. Diagnostics (Basel). 2021;11(12):2392. doi: 10.3390/diagnostics11122392 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Marchisio P, Pipolo C, Landi M, et al. ; Italian Earwax Study Group . Cerumen: A fundamental but neglected problem by pediatricians. Int J Pediatr Otorhinolaryngol. 2016;87:55-60. doi: 10.1016/j.ijporl.2016.05.014 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eMethods
eResults
eFigure 1. Illustration of Hearing Loss Prediction System
eFigure 2. Receiver-Operator Characteristic (ROC) Curves and Corresponding AUC
