A Novel Approach to Clinical Change Endpoints: A Win Ratio Analysis of the INCREASE Trial

To the Editor:

The INCREASE (Inhaled Treprostinil in Pulmonary Hypertension Due to Interstitial Lung Disease) study was a 16-week, double-blind, randomized controlled trial that evaluated inhaled treprostinil in patients with pulmonary hypertension associated with interstitial lung disease (ILD). The study met the primary endpoint of change in 6-minute-walk distance (6MWD) at Week 16 as well as the secondary endpoint of time to clinical worsening, which was defined as a >15% decline in 6MWD, cardiopulmonary hospitalization, lung transplantation, or death (1). The inherent shortcoming of a composite endpoint is that it treats all components as equally important, as they are analyzed as the time to the first event. It is therefore possible that patients might meet less important components of the composite endpoint before progressing to more meaningful outcomes. A relatively new concept in clinical trial analyses is the “win ratio,” whereby events are analyzed according to clinical relevance rather than time to the first event (2, 3). In this post hoc analysis of the INCREASE study, we instituted a win ratio analysis with prioritization of clinical events.

Methods

Three clinical worsening events (death, cardiopulmonary hospitalization, and a ⩾15% decrease in 6MWD) and one clinical improvement event were reanalyzed using the win ratio methodology (4). Only events that occurred during the 16-week, randomized, placebo-controlled study were included. We used the win ratio analysis in two algorithms: algorithm 1 was a hierarchy accounting for the occurrence of and time to events, while algorithm 2 was a simple hierarchy of event occurrence. The win ratio is defined as the number of winners in the treatment group divided by the number of winners in the placebo group. Winners are determined as follows: for algorithm 1, the event hierarchy from most to least important was death, cardiopulmonary hospitalization, ⩾15% 6MWD decrease from baseline, and clinical improvement. The latter was defined as a ⩾15% increase in 6MWD together with a ⩾30% reduction in N-terminal pro–B-type natriuretic peptide without any clinical worsening event. For any pair of subjects from the inhaled treprostinil and placebo arms, the winner was whoever did not have the event first for the first three worsening events, while for the fourth event of clinical improvement, the winner was whoever did have the event. For the first event, if both died, the one who survived longer was the winner. If neither died, the winner was whomever did not have a cardiopulmonary hospitalization first. The other two events were evaluated sequentially in a similar fashion (Figure 1). If neither subject experienced the events of interest, it was deemed a tie, and no winner was recorded. Algorithm 2 compares whether the target event occurs without time as a factor. A similar sequential logic was applied. The original description of the win ratio recommends comparing matched pairs (i.e., only comparing patients matched by prognostic variables). However, matching can be subjective and difficult to achieve, especially with relatively small sample sizes. We, therefore, used the “unmatched” win ratio method, comparing all possible pairings between the treatment and placebo groups (4, 5).

Figure 1.

Flowchart of the two win ratio algorithms depicting the order of events analyzed. iTre = inhaled treprostinil; NT-proBNP = N-terminal pro–B-type natriuretic peptide.

A sensitivity analysis was performed in which the win ratio was calculated for events occurring after Week 4, given that the patients were up-titrating the dose of inhaled treprostinil during the first 4 weeks and were mostly on subtherapeutic doses. In the sensitivity analysis, Week 4 was regarded as the baseline, with 12 weeks of follow-up during which patients continued in their blinded arms.

Results

There were 326 patients randomized in the INCREASE study, all of whom were included in the win ratio analyses. The clinical worsening events occurring in each treatment arm are depicted in Figure 2.

Figure 2.

Number of events included in the win ratio analysis by assigned treatment group in the INCREASE trial. INCREASE = Inhaled Treprostinil in Pulmonary Hypertension Due to Interstitial Lung Disease; 6MWD = 6-minute-walk distance.

Algorithms 1 and 2 for the primary analyses both demonstrate that the inhaled treprostinil group had a higher win ratio over placebo for all events (Table 1). Mortality, cardiopulmonary hospitalizations, 6MWD decline, and clinical improvement events contributed 21%, 35%, 25%, and 19%, respectively, of all determined matches in algorithm 1 and 22%, 33%, 27%, and 18%, respectively, in algorithm 2. The overall win ratio estimates were 1.551 (confidence interval [CI], 1.068–2.252; P = 0.021) and 1.655 (CI, 1.131–2.422; P = 0.010) in favor of inhaled treprostinil over placebo for algorithms 1 and 2, respectively. The sensitivity analysis accounting for a 4-week dose up-titration confirmed the win ratio results from algorithms 1 and 2, with overall higher win ratio estimates of 1.854 (CI, 1.241–2.769; P = 0.003) and 1.994 (CI, 1.326–2.997; P = 0.001) for algorithms 1 and 2, respectively (Table 1).

Table 1.

Primary (Weeks 0–16) and sensitivity (Weeks 4–16) analyses

	Death	CP Hospitalization	6MWD Decline	Clinical Improvement	Total
Primary win ratio analysis
Algorithm 1
iTre wins	1,642 (11%)	2,796 (19%)	2,489 (17%)	2,182 (15%)	9,109 (61%)
Placebo wins	1,479 (10%)	2,484 (17%)	1,185 (8%)	726 (5%)	5,874 (39%)
Total	3,121 (21%)	5,280 (35%)	3,674 (25%)	2,908 (19%)	14,983 (100%)
Win ratio	1.551 (CI, 1.068–2.252, P = 0.021)
Algorithm 2
iTre wins	1,836 (12%)	2,756 (18%)	2,938 (19%)	2,121 (14%)	9,651 (62%)
Placebo wins	1,510 (10%)	2,370 (15%)	1,229 (8%)	721 (5%)	5,830 (38%)
Total	3,346 (22%)	5,126 (33%)	4,167 (27%)	2,842 (18%)	15,481 (100%)
Win ratio	1.655 (CI, 1.131–2.422, P = 0.010)
Sensitivity analysis
Algorithm 1
iTre wins	1,483 (11%)	2,282 (18%)	2,555 (20%)	2,140 (16%)	8,460 (65%)
Placebo wins	1,154 (9%)	1,445 (11%)	1,204 (9%)	761 (6%)	4,673 (35%)
Total	2,637 (20%)	3,727 (29%)	3,759 (29%)	2,901 (22%)	13,024 (100%)
Win ratio	1.854 (CI, 1.241–2.769, P = 0.003)
Algorithm 2
iTre wins	1,661 (12%)	2,282 (17%)	3,017 (22%)	2,079 (15%)	9,039 (67%)
Placebo wins	1,160 (9%)	1,427 (11%)	1,191 (9%)	756 (6%)	4,534 (33%)
Total	2,821 (21%)	3,709 (27%)	4,208 (27%)	2,835 (21%)	13,573 (100%)
Win ratio	1.994 (CI, 1.326–2.997, P = 0.001)

Definition of abbreviations: 6MWD = 6-minute-walk distance; CI = confidence interval; CP = cardiopulmonary; iTre = inhaled treprostinil; NT-proBNP = N-terminal pro–B-type natriuretic peptide.

The event hierarchy was ordered as follows: death, cardiopulmonary hospitalization, ⩾15% 6MWD decrease from baseline, and clinical improvement (⩾15% increase in 6MWD together with a ⩾30% reduction in NT-proBNP).

Discussion

In this novel post hoc analysis of patients enrolled in the INCREASE study, the inhaled treprostinil arm had a higher win ratio than the placebo arm for each of the four clinical events. Although some of the numeric differences in events, such as the one for mortality, were small, this was in the context of a relatively short-term study. This hierarchical approach has merit in that the most important clinical events are analyzed ahead of less important events and are not “lost” if patients met the composite clinical worsening composite through a less meaningful component. This approach also allows clinical improvement to contribute to the analysis. By capturing both worsening and improvement in a single analysis, this approach reflects the global impact of therapy. The win ratio approach of assessing meaningful clinical change, therefore, is a novel and more balanced way to assess complicated composite endpoints.

The study was inherently limited by the relatively short period of follow-up time of 16 weeks used in the INCREASE study. Whether these results will hold up over a longer period is uncertain, but it is also unknown if the difference between groups could become more evident over a longer period. In the original publication of the INCREASE study, treatment with inhaled treprostinil was associated with a significantly lower risk of clinical worsening using a Cox proportional hazards model (1). The concept of comparing the win ratio with the inverse of the Cox hazard ratio has been used previously in cardiovascular trials (6). Unfortunately, we were unable to perform this comparison, as our win ratio hierarchy uses different components, including one that captures clinical improvement.

Conclusions

We have demonstrated the clinical benefits of inhaled treprostinil in patients with pulmonary hypertension associated with ILD through a win ratio analytic approach. The sequential evaluation of events in order of clinical importance clearly has merit in the interpretation of composite endpoints over more traditional “first come, first served” approaches. Our analysis is further unique in that we incorporate clinical improvement into the sequence of events and thereby present a novel approach to future endpoint design not only in pulmonary hypertension associated with ILD but also in ILD and pulmonary arterial hypertension clinical trials.