Interpretation of tympanic membrane findings varies according to level of experience

Abstract

The authors administered a written survey consisting of colour images of 28 tympanic membranes (TM) to four groups of examiners: 86 medical students, 68 paediatric residents, 62 paediatricians and seven expert otoscopists. For each image, examiners were asked to indicate whether redness or bulging of the TM was present and to assign a diagnosis. The odds of diagnosing acute otitis media (AOM) was examined according to perceived redness or bulging of the TM. Medical students and paediatricians attributed approximately equal importance to isolated redness and isolated bulging of the TM. In contrast, experts placed significantly greater importance on isolated bulging of the TM compared with isolated redness of the TM. The authors suggest that training curricula emphasize the strong association between a bulging TM and a diagnosis of AOM, while de-emphasizing the association between TM redness and AOM.

Because acute otitis media (AOM) is one of the most frequent indications for prescribing antibiotics in children, it is important to have evidence-based criteria for its diagnosis (1). However, little empirical data regarding the relative importance of tympanic membrane characteristics in the diagnosis of AOM are available. Specifically, it is unclear whether isolated redness of the tympanic membrane (TM) is a key diagnostic finding. In both Canadian and American guidelines for the diagnosis and management of AOM, TM redness is neither necessary nor sufficient for the diagnosis of AOM (2,3). Yet, in our day-to-day encounters with medical students, residents and colleagues, it appears that redness often plays a large role in the diagnostic process. Previous studies have documented that, compared with expert otoscopists, medical students, residents and paediatricians have difficulty distinguishing AOM from otitis media with effusion (OME) (4–7). We hypothesized that these difficulties may be related, in part, to differences in the relative importance placed on redness and bulging of the TM in the diagnostic process. The objective of the present study was to test this hypothesis.

METHODS

A convenience sample of third- and fourth-year medical students, peadiatric residents and paediatricians affiliated with the Children’s Hospital of Pittsburgh (Pittsburgh, Pennsylvania, USA) was recruited. The University of Pittsburgh Institutional Review Board approved the present study.

Each participant completed a written survey consisting of 28 colour images of TMs, in which they were asked to indicate whether, in their judgment, the TM was bulging and/or red, and whether they would diagnose AOM.

The TM images used in the present study were obtained using an otoendospcope from children six to 24 months of age enrolled in a longitudinal study that examined the efficacy of influenza vaccination in preventing AOM during the 1999 and 2000 respiratory seasons. As previously reported, 135 images were randomly selected to be reviewed by a panel of seven expert otoscopists to assess the relative contribution of various signs to the diagnosis of AOM (8). Expert otoscopists were otolaryngologists or paediatricians with significant research and clinical experience diagnosing AOM. There was no overlap between experts in the authors’ previous study and the paediatricians participating in the current study. After excluding images, in which ≤4 experts agreed on the final diagnosis (n=2) and images of normal TMs (n=24), 28 of the remaining 109 images were randomly selected for inclusion in the written survey. Participants who answered ≤80% of survey questions were excluded.

The proportion of cases in which each group of examiners endorsed a particular TM finding was calculated first. Subsequently, the proportion of cases in which AOM was diagnosed for all four possible combinations of redness and bulging of the TM was computed. Next, limiting to the characteristics in isolation (ie, TM red but not bulging and TM bulging but not red, respectively), ORs (and corresponding 95% CIs) were calculated for each group of examiners, comparing the odds of diagnosing AOM when the TM was judged as being bulging with the odds of diagnosing AOM when the TM was judged as being red. The OR reflects the relative importance of bulging compared with redness in the diagnosis of AOM. SAS PROC GENMOD (SAS Institute, USA) was used to fit the logistic regression models using generalized estimating equations to account for the correlation that arises from repeated assessments of the same images. Finally, sensitivity and specificity for the diagnosis of AOM was calculated according to level of experience using the diagnosis of the majority of experts as the gold standard.

RESULTS

Ninety-nine medical students, 72 residents and 91 paediatricians were asked to complete the survey. A total of 216 participants (86 medical students, 68 residents, 62 paediatricians) agreed to participate; 16 participants with incomplete surveys were excluded.

Students, residents and paediatricians were more likely to describe images as bulging or red, respectively, compared with expert otoscopists. The proportions of TMs judged as being red (irrespective of bulging) were 55%, 55%, 52% and 39% by medical students, paediatric residents, paediatricians and experts, respectively (P<0.05 between experts and each of the other groups). The corresponding numbers for TMs judged as being bulging (irrespective of redness) were 61%, 62%, 60%, and 47% (P<0.05 between experts and each of the other groups), respectively.

The proportion of cases diagnosed as AOM according to the various combinations of bulging and redness is shown in Figure 1. Further detail is provided in the Table 1. When the TM was noted to be red but not bulging, the likelihood of diagnosing AOM was highest among medical students, lowest among expert otoscopists, and intermediate among residents and paediatricians.

Figure 1).

Proportion of cases diagnosed as acute otitis media according to perceived redness and bulging of the tympanic membrane

TABLE 1.

Proportion of cases diagnosed as acute otitis media (AOM) according to redness and bulging of the tympanic membrane (TM)^*

TM judged	Medical students		Paediatric residents		Paediatricians		Expert otoscopists
	AOM % (n/N)	No AOM % (n/N)	AOM % (n/N)	No AOM % (n/N)	AOM % (n/N)	No AOM % (n/N)	AOM % (n/N)	No AOM % (n/N)
Neither red nor bulging	8 (43/564)	92 (521/564)	8 (37/439)	92 (402/439)	6 (27/451)	94 (424/451)	0 (0/79)	100 (79/79)
As red, not bulging	42 (118/279)	58 (161/279)	28 (68/240)	72 (172/240)	36 (52/146)	64 (94/146)	25 (6/24)	75 (18/24)
As bulging, not red	48 (196/408)	52 (212/408)	64 (233/364)	36 (131/364)	68 (241/357)	32 (116/357)	88 (36/41)	12 (5/41)
As both red and bulging	94 (869/922)	6 (53/922)	94 (698/741)	6 (43/741)	95 (619/650)	5 (31/650)	98 (51/52)	2 (1/52)

^*Twenty-eight images were assessed independently by 78 medical students, 64 paediatric residents, 58 paediatricians and seven experts

Among medical students, the ratio of the odds of diagnosing AOM in those with perceived bulging of the TM (in isolation) to the odds of diagnosing AOM in those with perceived redness of the TM (in isolation) was 1.2 (95% CI 0.8 to 1.8; P=0.34). The respective corresponding ORs among paediatric residents, paediatricians and experts were 3.0 (95% CI 2.4 to 3.9; P<0.001), 1.5 (95% CI 1.0 to 2.3; P=0.09) and 19.1 (95% CI 3.2 to 115.2; P=0.02).

AOM was diagnosed by a majority of the experts in 14 of the 28 (50%) cases. Using the diagnosis endorsed by the majority of expert otoscopists as the gold standard, medical students diagnosed 66% of AOM correctly (sensitivity) and 54% of OME cases correctly (specificity). The corresponding values for residents were 75% and 59%; among pediatricians, these values were 84% and 67%.

DISCUSSION

Medical students and practicing paediatricians placed approximately equal importance on isolated redness and isolated bulging of the TM. The odds of diagnosing AOM in those with the respective findings were similar (ie, ORs nearing 1) in each of the two groups. In contrast, experts placed significantly greater importance on isolated bulging of the TM compared with isolated redness of the TM (ie, the odds of a diagnosis of AOM was 19 times higher in cases of TM bulging alone versus TM redness alone).

These differences in the interpretation of TM findings were further compounded by the fact that medical students, residents and paediatricians identified more TMs as bulging and/or red. Collectively, these findings explain, in part, the previously reported tendency of medical students, residents and paediatricians to diagnose children with OME as having AOM (4–7). The resultant unnecessary use of antimicrobials for children with OME likely contributes to the emergence of bacterial resistance (8).

The paediatric residents surveyed had values that were between medical students and expert otoscopists (ie, the odds of a diagnosis of AOM was three times higher in cases of TM bulging alone versus TM redness alone). This may be a reflection of the intensive curriculum on the diagnosis of AOM that is available to residents at the Children’s Hospital of Pittsburgh.

We did not directly ask participants to rate the importance of redness or bulging because we did not want to disclose the purpose of the study and risk altering their behaviour. In addition, we were skeptical about the validity and usefulness of self-report. Instead, we asked participants to evaluate whether TM images were red and/or bulging, and to commit to a diagnosis. Analysis of our data enabled us to empirically determine the assumptions underlying the diagnostic algorithms being used by the participants without having to directly ask about these assumptions.

We intentionally did not focus on diagnostic accuracy in this study. Previous studies have documented that clinicians have difficulty distinguishing between AOM and OME (4–7). Instead, we sought to understand the reasons for the reported low accuracy rates. Specifically, we were interested in understanding how interpretation of TM characteristics influences the final diagnosis. Accordingly, the focus of our analysis was to establish the relative importance of TM findings in the diagnostic process. This information is useful because it clarifies how expert and nonexperts differ in their interpretation of TM findings and because it can directly inform training programs.

Our study was limited by the use of images instead of actual paediatric patients. However, our findings are in accord with our previous prospective report describing the diagnosis of AOM by expert otoscopists in clinical practice, in which we found that the contribution of findings other than bulging to the diagnosis of AOM was relatively minor (10).

We suggest that training curricula and practice guidelines emphasize the strong association between a bulging TM and a diagnosis of AOM, while de-emphasizing the association between isolated redness of the TM and AOM.