Application of the Wei-Lachin Multivariate One-Sided Test to Multiple Event-time Outcomes

Abstract

Background/Aims

Cardiovascular outcome trials, among others, aim to assess the beneficial effects of a treatment on multiple event-time outcomes, such as the time to a myocardial infarction (MI) and the time to a stroke. The traditional approach is to conduct a simple analysis of a composite outcome defined as the time to the first component event using a logrank test or the Cox Proportional Hazards regression model. This ignores information from other component events after the first. The composite outcome analysis also treats all initial outcome events as equally important, e.g. non-fatal MI is as important as cardiovascular death.

Methods

Herein we describe the application of the Wei-Lachin multivariate one-sided (or one-directional) test to the analysis of multiple event-time outcomes. The test is based on the unweighted mean of the treatment group coefficients from individual Cox proportional hazards models fit to the outcomes, where the covariance of the set of coefficients is obtained from a partitioning of the information sandwich estimate. A weighted test is also described, weighting the outcomes by a scoring of their clinical importance. These and other methods are compared with application to the Prevention of Events with Angiotensin Converting Enzyme Inhibition (PEACE) cardiovascular outcome study.

Results

The Wei-Lachin test provides an inference with strong control of the type 1 error probability on the difference between groups for the set of outcomes considered. However, it does not provide an inference on the individual components specifically with control of the overall type 1 error probability. By direct computation of relative efficiency and by simulation we show that the power of the Wei-Lachin one-directional test can be greater than that of the traditional composite outcome analysis based on the time to the first observed component event.

Conclusion

The Wei-Lachin multivariate one-directional test may be more powerful than the traditional analysis of a composite outcome defined as the time to the first of the component outcomes experienced by each subject.

Introduction

In a cardiovascular outcomes trial, subjects may experience a Major Adverse Cardiovascular Event (MACE) usually defined as the time to either non-fatal MI, or non-fatal stroke, or cardiovascular death, whichever occurs first. Often a MACE+ outcome is employed that includes other related outcomes such as a revascularization procedure, or hospitalization for unstable angina, or hospitalization for congestive heart failure. The traditional analysis, commonly termed a composite analysis, consists of standard survival analysis of the time to the first event, so that each subject contributes a single event or right censored time. However, this fails to capture the total burden of disease, such as when a revascularization procedure is followed by cardiovascular death, the latter not entering into the analysis. This may also sacrifice power.

Other methods for the analysis of composite outcomes include the weighted composite of Bakal et al. [1], the Win Ratio [2] and the ordered risk profile [3]. All three methods attempt to compare the treatment groups with respect to the incidence of the component events and their relative importance or severity in a manner that is clinically meaningful both to physicians and patients. Others have proposed analyses of the individual components with an adjustment for multiple tests, or using a gatekeeping strategy based on an a priori specified order of testing, or apportioning the type 1 error probability over primary and secondary outcomes, see [4].

Herein we describe a simple test, originally proposed by Wei and Lachin [5], that can provide a more powerful approach to the analysis of multiple event times. The method can also employ weights to reflect the perceived relative severity of each outcome. The method is applied to a past-completed cardiovascular outcomes study.

The Wei-Lachin One Directional Multivariate Test

Background

Wei and Lachin [5] describe a multivariate analysis of a vector of K two-group Aalen-Gill or weighted logrank statistics for K event times, where each subject can experience one or more of the K events. In addition to the usual K df MANOVA-like omnibus test, they suggested that a simple sum of the rank statistics could provide a 1 df test of the joint null hypothesis for the K outcomes against a restricted alternative hypothesis of “stochastic ordering” wherein the distribution function for each outcome in one group dominates that of the other. This alternative implies that the values of the K outcomes for one group tend to be less than those of the other, though not necessarily to the same degree. This alternative hypothesis is also called a one-directional or multivariate one-sided hypothesis.

The Wei-Lachin one-directional test can then be applied to any vector of statistics comparing two groups for multiple outcomes. Lachin [6] described the application to the analysis of repeated measures using summary statistics such as the mean difference or the Mann-Whitney difference parameter estimate. Lachin [7] describes the application to the analysis of multiple outcomes, possibly measured on different scales, such as to a joint analysis of means, proportions and lifetimes, and more generally to any set of model-based analyses.

Application to Multiple Event Times Under Proportional Hazards

The Wei-Lachin 1 df one-directional test is described in terms of the sum or simple mean of the differences between groups in a set of summary measures. Herein, since the analysis is conducted under proportional hazards assumptions, the test is described in terms of the estimated log hazard ratio, or the group coefficient β̂_j for the j-th outcome, where a negative value represents a reduction in risk with the experimental therapy (E) versus control (C).

The coefficient estimates are obtained by fitting a separate proportional hazards regression model, perhaps covariate-adjusted, for each outcome, e.g. time to the first non-fatal MI, time to first non-fatal stroke, and time to cardiovascular death, where any one patient could have an event in any or all of the component analyses.

For simplicity consider the case where each subject can experience either or both of two events A and B, with log hazard ratios β_a and β_b, respectively, for experimental versus control E:C. These hazard ratios are conveniently estimated from separate Cox proportional hazards (PH) models to yield a vector of coefficient estimates β̂ = (β̂_a β̂_b)^T. Pipper, et al. [8] then show that the model-based analysis can provide an estimate of the covariances of the parameter estimates by partitioning the information matrices from the two models. See also [7]. Thus, the estimated covariance matrix of the vector of coefficient estimates can be obtained from the two models

$$\hat{\mathbf{\Sigma }}=\hat{V}(\hat{\beta })=\left[\begin{array}{cc}\hfill {\hat{\sigma }}_a^2\hfill & \hfill {\hat{\sigma }}_{ab}\hfill \\ \hfill {\hat{\sigma }}_{ab}\hfill & \hfill {\hat{\sigma }}_b^2\hfill \end{array}\right].$$ (1)

The covariance is provided by supplemental programming or by the the mmm function in the R package multcomp. Asymptotically, under the marginal PH model assumption, β̂ is distributed as bivariate normal with expectation β and covariance matrix Σ, each of which is estimated consistently from the model based estimates.

Then the null and alternative hypotheses of interest are

$$H_0:{\beta }_a=0\text{and}{\beta }_b=0$$ (2)

$$H_{1S}:{\beta }_a\le 0\text{and}{\beta }_b\le 0\text{and}\mathit{\text{sum}}({\beta }_a,{\beta }_b)<0.$$

Thus, H_1S designates that the experimental therapy is at least as effective as control for both outcomes and is superior to control for either or both outcomes. This is called the multivariate one-directional alternative hypothesis.

The null hypothesis H₀ can be tested against the restricted alternative H_1S using the simple Wei-Lachin test of the form

$$Z_S=\frac{\mathbf{J}\prime \hat{\beta }}{\sqrt{\mathbf{J}\prime \hat{\mathrm{\Sigma }}\mathbf{J}}}=\frac{{\hat{\beta }}_a+{\hat{\beta }}_b}{{\hat{\sigma }}_S}$$ (3)

$${\hat{\sigma }}_S^2=\hat{V}({\hat{\beta }}_a+{\hat{\beta }}_b)=[{\hat{\sigma }}_a^2+{\hat{\sigma }}_a^2+2{\hat{\sigma }}_{ab}]$$

where J = (1 1)′. Asymptotically Z_S ~ N(0, 1) under H₀ from Slutsky’s theorem. Since a negative value for β̂ (or Z_S) favors the treatment, the test rejects H₀ in favor of H_1S when Z_S ≤ Z_α at level α one-sided. The test Z_S can also be expressed as the ratio of the unweighted mean coefficient $\hat{\overline{\beta }}$ relative to its SE (see below).

The alternative hypothesis H_1S states that there is a preponderance of benefit of E relative to C, that leads to the one-sided rejection region for the test. However, it is possible to conduct a two-sided one-directional test that either E is superior to C or C is superior to E. In that case, the Wei-Lachin 1 df test statistic Z_S is referred to the two-sided critical value rather than the one-sided value to determine statistical significance. Herein we describe the one-sided test.

Also, note that the test can also be generalized to employ a weighted combination of the estimates of the form

$$Z_S=\frac{\mathbf{W}\prime \hat{\beta }}{\sqrt{\mathbf{W}\prime \hat{\mathrm{\Sigma }}\mathbf{W}}}=\frac{w_a{\hat{\beta }}_a+w_b{\hat{\beta }}_b}{{[w_a^2{\hat{\sigma }}_a^2+w_b^2{\hat{\sigma }}_b^2+2w_a{w}_b{\hat{\sigma }}_{ab}]}^{1/2}}=\frac{{\hat{\overline{\beta }}}_w}{\sqrt{\hat{V}\left({\hat{\overline{\beta }}}_w\right)}},$$ (4)

where W′J = 1. The weights (pre-specified) can reflect the relative importance or severity of the different outcomes, or be determined by statistical optimality criteria. Thus, the test can be described in terms of the weighted average of the group coefficient estimates, ${\hat{\overline{\beta }}}_w$, relative to its standard error. This also provides a summary measure of the overall difference between groups for which one or two-sided confidence limits can be provided. For w_j = 1/K (for K measures) then ${\hat{\overline{\beta }}}_w=\hat{\overline{\beta }}$ is simply the unweighted mean of the coefficients that provides the same test as in (3).

Application under non-Proportional Hazards

The above test applies under the assumption that the model is correctly specified for each outcome, i.e. the proportional hazards assumption applies. When the model specifications may not apply, Lin and Wei [9] describe a robust information sandwich estimate of the covariance matrix of the coefficient estimates.

Wei, Lin and Weissfeld [10] then show that partitioning of the robust covariance matrices from separate models can be used to provide the joint covariance matrix of the parameter estimates from the separate models. Their approach also fits separate marginal models to provide estimates of β̂_a and β̂_b, and then uses the Lin-Wei [9] robust estimate of the covariance matrix to obtain the covariance estimate σ̂_ab. The Wei-Lachin 1 df one-directional test can then be computed from this joint covariance matrix.

Lin [11] also describes a joint model that can be estimated under the “working” model assumption of independence of the two outcomes, again with a consistent estimate of the covariance matrix obtained from the robust information matrix. These methods provide consistent estimates for the coefficients and the covariance matrix when the PH model assumptions do not strictly apply.

Another possible consideration is that cardiovascular death is a competing risk for other non-fatal outcomes. Nevertheless, a Cox PH model analysis in which non-fatal event times are censored by death can still be justified since Prentice et al. [12] have shown that such a PH model provides an evaluation of covariate effects on the underlying cause-specific hazard for the event. Alternately, it would be possible to employ the Fine and Gray [13] generalization of the PH model for the analysis of competing risks. Herein the simple PH model is employed.

The method could also be generalized to include all events of all types in all subjects using the multiplicative intensity model generalization of the Cox PH model for recurrent events. Some have recommended, however, that recurrent events of a specific type could reduce power because a subject who has experienced an initial event of a specific type (e.g. non-fatal MI) may then be at high risk of additional events of that type and that risk may not be affected by treatment.

Relative Efficiency Versus A Composite Outcome Analysis

Theoretically, for every alternative hypothesis defined by a specific pair of values for β_a and β_b there is an optimal linear weighted statistic of the form in (4) with weights that maximize the power to detect a difference between groups. However, the actual values of the parameters, and thus of the optimal weight vector, are unknown. Nevertheless, under mild conditions, Frick [14,15] shows that the Wei-Lachin test is a robust or maximin test that maintains good power over a range of parameter values. See [7] for technical details.

Lachin [7] then describes the power of a Wei-Lachin test of two event-time outcomes under a bivariate exponential model with a shared frailty, that would apply approximately to an analysis using a test based on multiple Cox PH models with correlated event times. The supplemental Appendix also derives the hazard rate for a time-to-first-event composite outcome that can be used with the equations of Lachin and Foulkes [16] to assess sample size or power for the composite anlaysis.

For a specified set of hazard rates, Lachin [7] shows that a total N = 394, evenly divided between groups, provides a one-sided one-directional Wei-Lachin test with 90% power to detect hazard ratios of HR_a = 0.8 and HR_b = 2/3. Under this same model, the composite outcome would have a hazard ratio HR_m = 0.7 and the analysis would require a total N = 436 to provide 90% power with a one-sided test at the 0.05 level. Since the relative efficiency of two tests is proportional to the ratio of the sample sizes needed to provide a given level of power, for this example, the Wei-Lachin test of the two outcomes has a relative efficiency of 1.11 relative to the composite outcome test, or an 11% greater efficiency.

A simulation study of the relative power of the Wei-Lachin test versus the composite analysis is presented in the Supplemental Appendix B. The results show that the Wei-Lachin test is largely more powerful than the standard approach in cases of practical interest where the treatment effect is in the same direction for both outcomes, i.e. the log hazard rates log(HR_a) and log(HR_b) have the same sign.

Example – The PEACE Study

The Prevention of Events with Angiotensin Converting Enzyme Inhibition (PEACE) study [17] assessed whether treatment with ACE inhibition with trandolapril (ACEi, n=4158) versus placebo (n=4132), when added to standard therapy, would reduce the risk of cardiovascular outcomes in patients with stable coronary artery disease. The primary outcome was the composite of death from cardiovascular causes, non-fatal myocardial infarction, or coronary revascularization. Over an average of 4.8 years of follow-up, the primary outcome was observed in 21.9% in the ACEi group versus 22.5% in the placebo group, with a hazard ratio of 0.96; 95% confidence interval 0.88 to 1.06; p = 0.43 two-sided (0.22 one-sided).

Table 1 presents the numbers of subjects (cases) with each type of event. In addition to the primary outcome we also analyze two other composite outcomes: traditional MACE (major adverse cardiovascular event) consisting of cardiovascular death, non-fatal MI or non-fatal stroke, and MACE+CHF consisting of either a MACE outcome or hospitalization for congestive heart failure. For all analyses, one-sided p-values are shown, although in practice two-sided values might be employed.

Table 1.

Numbers of subjects (cases) with each type of cardiovascular event and for the primary outcome composite, traditional MACE and a composite of MACE plus CHF.

	# Cases
Outcome	ACEi (n=4158)	Placebo (n=4132)	A:P HR	95% CI	One-sided p
CV death	146	152	0.95	0.76, 1.19	0.34
Non-fatal MI	222	220	1.0	0.83, 1.21	0.5
Revascularization	740	746	0.98	0.88, 1.08	0.34
Non-fatal stroke	55	75	0.72	0.51, 1.03	0.035
CHF	105	134	0.77	0.6, 1.0	0.025
Primary Composite	909	929	0.96	0.88, 1.06	0.22
MACE	392	417	0.93	0.81, 1.07	0.16
MACE+CHF	449	492	0.90	0.79, 1.02	0.06

Among the components of the primary outcome, there is a small benefit with ACEi versus placebo for CV death and revascularization, but a slight benefit with placebo for non-fatal MI. As a result there is a small non-significant benefit with ACEi for the composite primary outcome. However, there is a nominally significant (one-sided) benefit with ACEi for non-fatal stroke, so the traditional MACE outcome (CV death, non-fatal MI or stroke) shows greater benefit for ACEi than did the primary outcome. There was also a nominally significant benefit of ACEi with congestive heart failure and the composite of MACE+CHF showed an even greater suggestion of benefit with a HR = 0.9 and one-sided p = 0.06.

To conduct Wei-Lachin analyses, separate Cox PH models were fit for the component events within each composite outcome and the R function mmm function used to provide the estimated covariance matrix of the treatment group coefficients. The coefficients and covariance matrix are presented in Appendix Table 1 of the supplementary material. The unweighted mean of the coefficients for the components of each composite outcome and its SE were obtained. From these, the Wei-Lachin one-directional tests were then computed for the set of components of each composite outcome, with the results shown in Table 2.

Table 2.

The unweighted mean coefficient of the components of each composite outcome with the one-sided upper confidence bound (UB) and corresponding one-sided p-value from the Wei-Lachin one-directional test, as well as the two-sided confidence interval (LL, UL) and p-value.

			One-sided		Two-sided
Outcome (Components)	$\hat{\overline{\beta }}$	HR	UB	p-value	LL	UL	p-value
Primary (1, 2, 3)	−0.0245	0.976	1.075	0.339	0.870	1.094	0.677
MACE (1, 2, 4)	−0.1244	0.883	1.007	0.060	0.756	1.032	0.119
MACE+CHF (1, 2, 4, 5)	−0.1576	0.854	0.964	0.016	0.740	0.986	0.032

For the primary outcome, there is essentially no difference between groups for any of the three component outcomes (Table 1) and the Wei-Lachin test provides a larger one-sided p-value than did the composite outcome analysis (Table 2). For the MACE and MACE+CHF analyses, both of which showed a non-significant but beneficial trend with treatment in the composite analysis, the Wei-Lachin p-values were smaller, and in the case of MACE+CHF, would have yielded a statistically significant result either one- or two-sided (0.016 or 0.033).

Alternative Methods

The model-based Wei-Lachin test is not the only alternative analysis to the standard composite approach. In the following additional methods are described with application to the MACE+CHF outcome.

Robust Marginal Models

Wei, Lin and Weissfeld [10] were among the first to show that the covariance matrix of the coefficients from multiple models could be obtained by partitioning the Lin-Wei [9] robust information sandwich. Their method is provided by the SAS procedure PHREG or the coxphreg function in the R survival package. See also Therneau and Grambsch [18]. However, neither program computes the joint covariance matrix of the parameter estimates from the multiple models that would be required to compute the Wei-Lachin test. Rather, additional computations are required as described in the older SAS program wlw.sas. For the above example, the results using the Wei-Lin-Weissfeld approach using either SAS or R are virtually identical to the Wei-Lachin test using the model-based covariance matrix estimate described above.

Multivariate Tests

Other tests might be considered that are directed towards a different specification of the alternative hypothesis. The most general test is the T²-like MANOVA test on K-df that is directed towards the omnibus alternative hypothesis H_1O: (β_a, β_b) ≠ (0, 0) of a group difference in either direction (benefit or harm) for either or both measures. For the case of a test of two means, Lachin [7] shows that the Wei-Lachin sum test is more powerful than the omnibus test, or separate tests with a multiplicity adjustment, when the restricted alternative hypothesis H_1S in (2) applies. For the PEACE study MACE+CHF outcome(s) described above, the MANOVA test provides a chi-square value of 7.39 on 4-df with p = 0.117. From Table 1, clearly none of the separate tests for the individual outcomes would approach statistical significance with a correction for multiplicity.

Another approach is based on the restricted alternative H_1A: β_a = β_b = β ≠ 0 which assumes that the expected value of the treatment group coefficient (or HR) is the same for each outcome. In that case an efficient test of H₀ versus H_1A is provided by a weighted average of the form in (4) with weights that are inversely proportional to the variances, of the form

$$\mathbf{W}={(\mathbf{J}\prime {\hat{\mathrm{\Sigma }}}^{-1}\mathbf{J})}^{-1}\mathbf{J}\prime {\hat{\mathrm{\Sigma }}}^{-1}.$$ (5)

This computation is provided by the SAS PHREG using the “average” option. Note that this alternative hypothesis is more restrictive than the one-directional alternative to which the Wei-Lachin test is directed. For the PEACE study MACE+CHF outcome, this yields a common coefficient estimate ${\hat{\overline{\beta }}}_A=-0.1059$, or a hazard ratio of 0.90, with SE = 0.0602 and a corresponding Z-test value of −1.76 with a one-sided p = 0.04.

Alternately, since the Wei-Lin-Weissfeld approach is fit using a PH model that is stratified with a distinct background hazard function for each outcome, the set of models can be fit using a single group coefficient for all outcomes. See Therneau and Grambsch [18]. When applied to the PEACE MACE+CHF outcome, this approach provides an estimate of the assumed common coefficient β̂ = −0.1070, or HR = 0.90, with SE = 0.068 and one-sided p = 0.06. The results are similar to the results obtained from the weighted average of the separate coefficients for each outcome using the weights in (5).

Weighted Wei-Lachin Analyses

The weighted Wei-Lachin test in (5) could also be conducted using a weighted average of the coefficients for the different outcomes with weights that reflect the perceived importance or severity of each outcome. For example, Bakal et al. [1] employed weights 1.0 for CV death, 0.5 for non-fatal stroke, 0.3 for hospitalization for CHF, and 0.2 for non-fatal MI. The corresponding weight vector, rescaled to sum to 1, has elements W = (0.5 0.25 0.15 0.1)′ for each component event. The weighted test then yields a weighted average ${\hat{\overline{\beta }}}_w=-0.036$ corresponding to a HR = 0.965 with 95% two-sided limits of 0.927, 1.003 and one-sided p = 0.037.

Discussion

The objective of an analysis of multiple event-time outcomes, as in the PEACE cardiovascular outcomes study, is to assess whether the experimental treatment provides a preponderance of benefit for a set of outcomes rather than any one specific outcome. Heretofore, the most commonly used approach has been to construct a composite outcome defined as the time to the first of the component events. Herein, we describe a simple Wei-Lachin test that is based on separate analyses of each component outcome event, not just the first of all component events as in the composite outcome analysis. The test is expressed as the simple sum of the group coefficient estimates over all outcomes, or their average. Through application to the PEACE study, computations of power and simulations, we demonstrate that the simple Wei-Lachin test can provide greater power than the composite analysis.

It would also be possible to further extend the Wei-Lachin test to the analysis of all recurrent component events over time using the multiplicative intensity model extension of the Cox PH model, or the robust counterpart proportional rate model [19]. Unfortunately, the recurrent event times were not included in the publicly available PEACE study data set, and it was not possible to illustrate analyses of all recurrent events.

On the other hand, methods such as the Wei-Lachin analysis that employ multiple event times may not be desirable in situations wherein the occurrence of an event of one type often prompts a change in therapy administered to a patient. Under an intent-to-treat or pragmatic trial design, anything that happens to the patient after randomization can be viewed as part of the ‘real life’ experience with an initial treatment assignment. However, a change in treatment prompted by an outcome event obviously limits the ability to ascribe outcomes to the pharmacologic or other mechanism of the agent. None of these methods are designed to account for such a change in treatment. In this case the time-to-first event composite analysis will in fact be less affected by the change in treatment than the Wei-Lachin or other methods, and on that basis might be preferred.

As in the traditional composite analysis, it is then desirable to evaluate the treatment group effect on each component outcome separately in a set of secondary analyses, involving multiple estimates and tests. Usually these component-wise inferences are considered descriptive and no adjustment for multiple tests is applied.

In conclusion, the Wei-Lachin simple sum test is directed to testing the joint null hypothesis of no difference for multiple outcomes against the restricted one-directional alternative hypothesis that the treatment has a preponderance of positive effects on the multiple outcomes considered, while not being harmful for any. The test has good power when such one-directional differences are observed and may be more powerful than the traditional composite analysis of the time to the firs-observed outcome event.