Determination of the Minimal Important Difference for the Acute Otitis Media Severity of Symptom Scale

Abstract

Background

We previously developed and validated the Acute Otitis Media Severity of Symptom (AOM-SOS) scale for rating symptoms of acute otitis media (AOM) in young children. In this report, we sought to estimate the minimal important difference (MID) for change in AOM-SOS scores.

Materials and Methods

In a group of children 6 to 24 months of age with AOM enrolled in a recently reported placebo-controlled clinical trial of antibiotic efficacy, we compared changes in AOM-SOS scores with parental assessments of change over a 24-hour period. Mean absolute and mean relative change in scores in children reportedly exhibiting only a small degree of improvement were considered in arriving at an estimated MID. We then compared the proportions of children in the antibiotic and placebo groups, respectively, whose AOM-SOS scores changed more than the estimated MID at various time points following enrollment.

Results

Data were available for 277 children. Children whose parents reported only a small degree of improvement 24 hours after enrollment had a mean decrease in AOM-SOS score of 3.8, or 55%, from baseline. We found the relative decrease more telling than the absolute decrease. The proportions of children in the antibiotic and placebo groups, respectively, whose AOM-SOS scores had decreased <55% on Day 7 were 12.3% and 23.8% (P=0.02), and during Days 4 to 7 were 28% and 40% (P=0.046).

Conclusions

We estimated the MID for change in AOM-SOS scores in young children and described use of the MID as an added metric in interpreting results from a clinical trial of antibiotic efficacy.

BACKGROUND

Patient-reported outcome measures (PROs), or in the case of preverbal children, parent-reported outcome measures, constitute potentially useful tools for measuring improvement or worsening of symptoms, quality of life, and functional status from the patient’s or parent’s perspective. The US Food and Drug Administration has strongly recommended that PROs be used in trials of drug efficacy.¹ However, even if a statistically significant change in a PRO score is observed, the degree of change may not be meaningful clinically. Further, it may be difficult to relate a specific quantitative change in patient- or parent-reported outcome to a change in the patient’s functional or behavioral condition. To address these issues, some authors have suggested estimating the minimally important difference (MID) for a measure, i. e., the smallest absolute or relative change in the measure that patients--or their proxies-- consider important.² The proportion of patients demonstrating a change at least as large as the MID may then serve as a useful summary statistic,³ and in clinical trials in which PROs are reported, knowledge of the MIDs for those measures may be useful in interpreting the trials’ results. The MID has been estimated for measures of various pediatric conditions such as asthma,⁴ but to our knowledge, it has not been applied to scales that have been developed for rating the severity of symptoms in young children with acute otitis media (AOM).

Our objective here was to estimate the MID for the current version of such a measure that we have previously developed and validated, namely, Version 4 of the Acute Otitis Media Severity of Symptoms (AOM-SOS) scale.^5,6 This version of the scale consists of 5 discrete items: tugging of ears, crying, irritability, difficulty sleeping, and fever. Parents are asked to rate these symptoms, in comparison with the child’s usual state, as “none,” “a little,” or “a lot,” with corresponding scores of 0, 1, and 2. Scores may thus range from 0 to 10.

In estimating the MID for the AOM-SOS scale, we considered whether absolute or relative change in the AOM-SOS score over time would be more telling. We also explored two approaches for using the MID in comparing outcomes of differing treatment groups in clinical trials. In one approach, we compared the proportions of children with scores above the MID at a single time point; in the other approach, we compared the proportions over a period of time.

METHODS

Estimation of the MID

We estimated the MID using data we collected for this purpose while conducting a recently completed clinical trial. As previously reported,⁷ we followed a cohort of 291 symptomatic children 6 to 24 months of age with AOM diagnosed using stringent criteria. Briefly, we randomized children to receive either amoxicillin-clavulanate or placebo in a 1:1 ratio at the time of diagnosis (Day 1, baseline), and we asked parents to complete the AOM-SOS scale twice daily for 3 days and daily thereafter for 7 days. We also administered a single-item global rating of change scale (GRC) by telephone on Day 2. We chose Day 2 because, based on our clinical experience, responses on Day 2 were expected to exhibit the highest variability. We asked the parent to rate whether their child was “better”, “the same,” or “worse”. If the child was described as “better” we asked the parent to rate the child’s degree of improvement on a 7-point scale (1=almost the same; 2=a little better; 3=somewhat better; 4=moderately better; 5=a good deal better; 6=a great deal better; 7=a very great deal better). If the child was described as “worse”, we asked the parent to rate the degree of worsening on the same 7-point scale as above except that “worse” was substituted for “better,” and the items were assigned negative scores (−1 to −7). Children described as “the same” were assigned a score of zero.

We considered children with a GRC score of −1, 0 or +1 as having experienced “no change,” those with a score of 2 or 3 as having experienced “small degree of improvement,” those with a score of 4 or 5 as having experienced “moderate degree of improvement,” and those with a score of 5 to 7 points as having experienced a large degree of improvement. For each of these four GRC categories we calculated the mean change in AOM-SOS score from baseline to Day 2, using as the Day 2 score the mean of the Day 2 AM and Day 2 PM scores. We based our estimate of the MID for the AOM-SOS score on the mean change in that score in children who had a GRC score of either 2 or 3. We did not estimate the MID for worsening because scores worsened in only 5 children.

We calculated the degree of change in the AOM-SOS score using both absolute differences (i.e. Day 2 score minus Day 1 score) and relative differences (i.e. Day 2 score minus Day 1 score divided by the Day 1 score). To investigate whether absolute or relative change from baseline was the more meaningful, we examined change in AOM-SOS scores for children in the highest and lowest quartiles of AOM-SOS baseline scores. We also examined the correlations between the measure of change and the baseline score. Because subjects with higher baseline values tend to exhibit larger changes (in keeping with the phenomenon of regression to the mean), indices of change that are highly correlated with baseline values are relatively problematic. Accordingly, the degree of such correlation is considered an important criterion in deciding whether to use absolute or relative change.⁸

Application of the MID

We then considered use of the MID by applying the estimated MID to data from the aforementioned amoxicillin-clavulanate vs. placebo trial.³ In performing this analysis, we considered two different approaches.

In the first approach, we compared the proportion of children in each treatment group who had a change greater than the estimated MID at single points in time, using a two-sample t-test. For those points we chose Day 5 and Day 7 arbitrarily, both for the purpose of illustration and for comparison with the second method described below. Of note, although we had determined the MID using data from the small subset of children with minimal improvement on Day 2, we assumed that value of MID would be applicable also to time-points other than Day 2. It seemed likely that a change in AOM-SOS score considered by parents to reflect a small degree of improvement at Day 2 would also be so considered later on during the course of the AOM episode.

In the second approach, we used the MID to identify children who exhibited failure to improve meaningfully at any time between Days 4 and 7, inclusive, using Day 4 as a starting point because we did not expect children to improve substantially before Day 3. We defined “failure to improve meaningfully” as the occurrence of at least 2 AOM-SOS scores between Days 4 and 7 that decreased less than the MID (i.e., <55%) from baseline. This approach is in keeping with standard practice in clinical care and research, and precludes a child with only a single high AOM-SOS score--that is likely spurious--from being classified as having “failed to improve meaningfully”.

RESULTS

Of the 291 children enrolled in the clinical trial, 14 had Version 4 AOM-SOS scores of <3 at enrollment and were excluded from the present analysis. The mean age of the remaining 277 children was 12.6 ± 4.9 months; of these, 146 (53%) had bilateral AOM. Baseline AOM-SOS scores were 3 to 5 in 107 children (39%), 6 or 7 in 86 children (31 %), and 8 to 10 in 84 children (30%).

Table 1 illustrates the calculation we used in arriving at the estimated MID. Among children who were considered by their parents to have exhibited a small degree of improvement from Day 1 to Day 2 the mean absolute change in AOM-SOS score from Day 1 to Day 2 was 3.8. This corresponded to a mean 55% decrease in score relative to baseline. Absolute change in AOM-SOS score from baseline was highly correlated with the baseline value (rho=0.47, p<.001); in contrast, the corresponding correlation between relative change and baseline was low and not significant (rho =0.05, p = 0.41). The difference in correlation between absolute change and relative change was further apparent when we divided the sample into quartiles and compared lowest and highest quartiles. The mean absolute change in AOM-SOS score in children with GRC score of 2 or 3 varied from 2.5 in the lowest quartile to 4.8 in the highest quartile, whereas in such children the relative change in score remained relatively constant, at 56% in the lowest quartile and 55% in the highest quartile.

Table 1.

Mean absolute and relative change in Version 4 AOM-SOS score from Day 1 to Day 2 according to parental global rating of change (GRC) score.

Parental global rating of change from Day 1 to Day 2	Mean absolute change in Version 4 AOM-SOS score from Day 1 to Day 2 (mean relative change)*
	All Children N = 256†	Children with the highest quartile baseline AOM-SOS score (≥8) N = 82	Children with the lowest quartile baseline AOM-SOS score (≤5) N = 97
No appreciable change (GRC score −1, 0, or +1)	2.2 (35%) N=136	3.0 (34%) N=35	1.2 (29%) N=60
Small degree of improvement (GRC score 2 or 3)	3.8 (55%) N=74	4.8 (55%) N=32	2.5 (56%) N=21
Moderate degree of improvement (GRC score 4 or 5)	4.3 (63%) N=33	6.1 (68%) N=13	2.3 (57%) N=10
Large degree of improvement (GRC score 6 or 7)	4.9 (85%) N=13	7.5 (88%) N=2	3.8 (84%) N=6

AOM-SOS denotes Acute Otitis Media Severity of Symptoms.

^*Day 1 was the day of enrollment. Relative change in AOM-SOS score from Day 1 to Day 2 = [Day 2 score minus Day 1 score]/Day 1 score.

^†Not included in this table are 16 children with missing AOM-SOS and/or GRC scores and 5 children whose condition worsened from baseline to Day 2.

The shaded row shows the estimated minimal important difference (MID).

Table 2 demonstrates findings from our clinical trial with respect to differences between treatment groups as measured by the proportions failing to improve meaningfully as defined above. The proportions of such children in the antibiotic and placebo groups, respectively, were 12% and 24% on Day 7 (P=0.02), and 28% and 40% during Days 4 to 7 (P=0.046). Differences between treatment groups on Day 5 were less marked and nonsignificant.

Table 2.

Findings from our previously reported clinical trial⁶ using the estimated minimal important difference (MID) of the Version 4 AOM-SOS scale to define failure of symptoms to improve appreciably.

Outcome Measure	Amoxicillin-Clavulanate Group†	Placebo Group†	P value
	Proportion (%)
Failure of symptoms to improve appreciably on Day 5*	30/132 (23)	41/130 (32)	0.14
Failure of symptoms to improve appreciably on Day 7	16/130 (12)	30/126 (24)	0.02
Failure of symptoms to improve appreciably during Days 4 to 7	38/136 (28)	55/137 (40)	0.046

AOM-SOS denotes Acute Otitis Media Severity of Symptoms.

^*Day 1 was the day of enrollment.

^†4 and 6 children in the amoxicillin-clavulanate and placebo groups, respectively, were lost to follow-up.

DISCUSSION

We found that, for the Version 4 AOM-SOS scale, a relative reduction of 55% from the baseline score corresponded to a degree of symptomatic improvement that parents considered appreciable. Therefore, reduction of that degree may be considered the MID for the scale in this group of children. The strong correlation we found between absolute change and the baseline score, together with the lack of correlation between relative change and the baseline score, suggests that relative change is the more appropriate measure for the AOM-SOS scale and this data set.

Typically, the MID has been used to assess the response of patients at a fixed point in time. However, for children with AOM, there is a risk that invalid conclusions might result from assessment at a single time because symptoms may fluctuate naturally or may be attributable to associated upper or lower respiratory tract infection rather than to AOM specifically. For these reasons, in clinical trials involving children with AOM, assessment of the children over time would appear preferable. The data shown in Table 2 support this hypothesis: differences between the amoxicillin-clavulanate and placebo treatment groups were apparent on Day 7 but not on Day 5. In contrast, using the MID to define lack of improvement over a more extended period–i.e. during Days 4 to 7–seemed reasonable because that is a time period during which clinical experience suggests that differences would likely be observed. Further, as an adjunct to reporting the difference in mean AOM-SOS scores between the two treatment groups–we had reported a statistically significant but nonetheless modest difference in score of 0.5 over the initial 7 days of treatment⁷–reporting the between-group differences in the proportion of children who failed to improve appreciably provides a measure that may be easier to interpret. In the present instance, the differences in symptomatic response we found using the MID appear more consonant with the between-group differences we found in otoscopic findings than were the differences we found in mean AOM-SOS scores.

Importantly, the findings of the present analysis may not be generalizable to other populations of children with AOM whose behaviors and whose parental perceptions may differ. Future trials in other populations are therefore needed to confirm the findings of this report. Also, although consideration of the MID may help in interpreting the results of AOM clinical trials, we do not suggest that the MID should constitute the principal endpoint in analyzing symptomatic response in such trials, not only because of the loss of statistical power inherent in reduction of a continuous scale to a dichotomous outcome, but also because, as in the present analysis, outcomes would vary depending on the time point chosen for the between-group comparison. Finally, the MID estimated in this report should not be confused with the expected clinically minimum difference between treatment groups (also sometimes referred to as the MID); rather, the MID, as defined in this report, is useful in determining which children who do and do not respond to therapy. Thus, as demonstrated in this report, the usefulness of the MID lies mainly in its ability to broaden and make more interpretable the depiction of clinical trial outcomes.