The Early Treatment for Retinopathy of Prematurity Clinical Trial: presentation by subgroups versus analysis within subgroups

Short abstract

An ETROP study in infants with retinal ablation of high‐risk prethreshold ROP: subgroup results

The Early Treatment for Retinopathy of Prematurity (ETROP) Study showed that retinal ablation for high‐risk prethreshold retinopathy of prematurity (ROP) improves structural and functional outcomes at 9 months' corrected age, compared with conventional management.¹ As reported in this issue of the British Journal of Ophthalmology, (see p 1378) the benefit for structural outcome extends at least until 2 years of age.² To discern a significant benefit for earlier treatment, the study randomised 401 infants who had prethreshold disease, and who also had a risk for unfavourable structural outcome ⩾15%. Approximately 80% of the infants had high‐risk disease in both eyes (one treated at prethreshold and the other managed conventionally). The other 20% of infants had one eye eligible, and that eye was randomised to early treatment at prethreshold or to conventional management. The fellow eye was managed conventionally. More details about risk calculation, patient selection and study design are presented elsewhere.¹,³,⁴

Interest in subgroups in the study has occurred since the publication of the original report from the ETROP Study.¹,⁵ The purpose of this communication is to discuss treatment decisions that might be based on subgroup findings. We delineate two distinct ways of viewing the results of data in clinical trials when the data are (1) presented by subgroups and (2) analysed within subgroups.

Presentation by subgroups

Virtually all clinical trials report data by giving the overall results for the primary hypothesis followed by tables that present the results in separate subgroups. The ETROP Study reported findings for various subgroups, including the International Classification for Retinopathy of Prematurity (ICROP) categories (eg, zone I, stage 1 or 2; zone II stage 3, no plus disease), and the presence of bilateral versus asymmetrical disease. In the ETROP Study, presentation of the data by subgroup gives the reader a more complete view of the study results. It allows one to examine the data for consistency of the findings, and to examine the plausibility that the intervention is a cause of findings that have been observed in the study. Considerable attention has been focused on whether analysis of these subgroups can be used to refine study findings, and perhaps to minimise treatment of infants in whom the disease would have regressed without requiring treatment. Several authors have interpreted the evidence for efficacy (or lack of efficacy) within selected subgroups.⁶,⁷,⁸

Attention to the relative efficacy of early treatment in selected subgroups is potentially important, but caution in the interpretation of these comparisons is advised. In planning and later interpreting the results, it is important to note the original intent of the study design. The ETROP Study was planned to compare the results of an overall treatment effect between two groups of eyes, with sample size calculations set accordingly.⁹ This represents a test of the primary hypothesis of the study. The outcome of this overall comparison should guide the conclusion and shape the way that results are integrated into clinical practice.

Analysis within subgroups

Some studies (not the ETROP Study) are designed to include secondary hypotheses, as there is a desire to learn whether one or more subgroups of patients require special attention in the results of the study.¹⁰ When there are secondary hypotheses, these subgroup analyses have a more important purpose. The results represent additional findings that emanate from the study as it was designed. In essence, these are studies within the main study. Sample size considerations for secondary hypotheses are usually part of the design. The results of these planned comparisons help shape the design, analysis and conclusions. The implementation of the results in clinical practice could include different recommendations for selected subgroups of patients.

Distinguishing these two approaches

Difficulties arise when readers do not distinguish between studies with only one primary hypothesis, and studies that also have one or more secondary hypotheses. In the first instance, an interesting result is observed in the course of presenting data by subgroups. It may seem that some patients benefit whereas others do not. The ETROP Study was designed with only one primary hypothesis about the efficacy of treating high‐risk prethreshold eyes. It is tempting to search through the results and question the efficacy of earlier intervention for zone II, stage 2+ eyes or for eyes in infants who have asymmetrical disease. After reviewing the data, these questions can and should be considered, but they are not part of the original design. These post‐hoc comparisons of treatment effect should be reviewed with appropriate caution. Drawing conclusions about subgroups can lead to the unwarranted conclusion that the treatment does not work in some subgroups. In this instance, data are presented for a different purpose, and the subgroups often have very small sample sizes.¹ Corroborating evidence should be added to these (exploratory) observations before forming conclusions.

For the ETROP Study, the sample size calculations were based on data from 613 prethreshold eyes from the CRYO‐ROP Study. Data from the infants (eyes) were used to calculate the proportion at risk for having an unfavourable structural outcome. The sample size for the ETROP Study was 333 high‐risk prethreshold eyes in each of the two treatment groups.³ The study design did not specify any secondary hypotheses with corresponding sample size requirements. Therefore, the power to separate out subgroups and produce valid findings is low. The prudent approach is to behave as the overall study results indicate¹¹,¹²–that is, on the basis of the ETROP Study results, treatment of eyes with high‐risk prethreshold ROP should occur as soon as possible.

As an example, for the ETROP Study, we presented data of infants with symmetrical disease (80% of infants), where both eyes were high‐risk prethreshold, and infants with asymmetrical disease (20% of infants). These two sets of data were combined to give the overall results for the study. Infants with symmetrical disease represented 80% of the study population and contributed most of the statistical power of the study. The number of infants with symmetrical disease who had outcomes that differed in the two eyes (discordant pairs) was 33. With matched‐pairs data, the discordant pairs are known as the “effective sample size” and for the ETROP Study (n = 33). It would be inappropriate to carry out an unplanned subgroup analyses on the basis of 33 observations. Although the data are presented by ICROP strata and by symmetrical and asymmetrical disease, treatment recommendation based on these small subgroup sample sizes is ill‐advised.

To provide a clinical algorithm as opposed to a computer‐based algorithm, we developed a two‐stage procedure, with immediate treatment for type 1 eyes and observation of type 2 eyes, with treatment if these eyes become type 1. This clinical algorithm was not derived from a subgroup analysis of eyes in the randomised trial but was based on an analysis of findings for all prethreshold eyes (low‐risk and high‐risk).

As with any clinical trial, the results of our trial may be applied differently from doctor to doctor in clinical practice. The willingness of parents to accept treatment may also differ. Sound clinical judgement and discussion with parents is always needed to decide what is best in a given situation. The results of the ETROP Study indicate that intervention at the onset of prethreshold ROP disease in high‐risk eyes reduces the risk of blindness. In the ETROP Study, two examples illustrate the transition of the results into clinical practice. High‐risk prethreshold eyes with zone II, stage 2+ disease had similar results when treated early or managed conventionally; however, the risk of an unfavourable structural outcome was quite high (20.6%). Guided by the overall results, clinicians are advised to treat early. On the other hand, some infants with high‐risk prethreshold ROP showed a low rate of unfavourable outcome (infants with type 2 prethreshold ROP). In this situation, these eyes and their clinical characteristics can be closely observed for disease progression.

Presentation of data by subgroups enriches the value of any large study by strengthening the confidence in the overall conclusion and by helping to shape individual clinical judgements in applying the results of a trial in clinical practice. Subgroup presentations may also help generate hypotheses for future studies. However, conclusions about treatment comparisons within subgroups that are counter to conclusions based on the overall study population should be made with considerable caution unless they result from analyses that were planned as part of the original design of the study.