Inferring Latent Heterogeneity Using Many Feature Variables Supervised by Survival Outcome

Abstract

In cancer studies, it is important to understand disease heterogeneity among patients so that precision medicine can particularly target high-risk patients at the right time. Many feature variables such as demographic variables and biomarkers, combined with a patient’s survival outcome, can be used to infer such latent heterogeneity. In this work, we propose a mixture model to model each patient’s latent survival pattern, where the mixing probabilities for latent groups are modelled through a multinomial distribution. The Bayesian information criterion (BIC) is used for selecting the number of latent groups. Furthermore, we incorporate variable selection with the adaptive lasso into inference so that only a few feature variables will be selected to characterize the latent heterogeneity. We show that our adaptive lasso estimator has oracle properties when the number of parameters diverges with the sample size. The finite sample performance is evaluated by the simulation study, and the proposed method is illustrated by two datasets.

1 |. INTRODUCTION

A typical clinical trial is designed to test a drug/vaccine on a large and diverse group of patients and hopefully the One-Size-Fits-All approach is successful. The benefit for this approach is the quick availability of an effect drug/vaccine to the broadly targeted population with the unmet medical need. However, with much less low-hanging fruits available, it becomes more challenging to develop a blockbuster drug/vaccine that works for all studies population. Especially in more advanced and hard-to-treat diseases such as oncology, patients often present a heterogeneous survival experience, and their disease outcomes may range from early death to spontaneous regression of the tumor followed by cure. This traditional one-size-fit-all approach may not be cost and time effective due to the highly heterogeneity of the study population. As technology advancing, more personal clinical, genetic, genomic, and environmental information and other baseline characteristic variables are available before the clinical study. Consequently, sponsors are looking into ways to conduct study in a more homogeneous subgroup with much higher probability of success to develop new medicines effectively. Thus, a challenging statistical problem with strong scientific/clinical interest in drug discovery and development is the identification of patient subgroups with different survival experience. Recently, Liao and Liu¹ demonstrated that many Kaplan Meier survival curves commonly seen in oncology trials can be reconstructed using a mixture of two or three parametric survival profiles. In other words, a disease population can be approximately decomposed into two or three latent groups with unique corresponding survival behavior in each latent subgroup.

Statistical methods to identify such latent groups have received increasing attention in the past several decades, where the majority of the broad literature focuses on latent class modeling. Among them, one popular subclass of latent class models is a mixture model that considers finite mixture components to represent the unobserved heterogeneity in the data. For survival outcomes, Farewell² discussed the use of mixture models by assuming a fraction of long-term survivors. Larson and Dinse³ used a parametric mixture model to analyze competing risks data where the mixing parameters correspond to the marginal probabilities of various failure types. Altstein and Li⁴ studied a semiparametric accelerated failure time mixture model on a latent subgroup with time-to-event data in randomized clinical trials, and Shen and He⁵ proposed a structured Logistic-Normal mixture model to identify subgroups. These methods had no discussion of variable selection. Wu et al.⁶ extended Shen and He’s work⁵ and introduced a backward elimination algorithm to select important variables. However, such an algorithm is computational intensive, and there is no theoretical justification for this procedure. More recently, Bussy et al.⁷ proposed a Quasi-Newton Expectation Maximization (QNEM) algorithm to detect patients subgroups. They only considered discrete survival time. In their algorithm, the negative log-likelihood is penalized by the elastic-net in every iteration, which is computationally intensive and there is no theoretical justification. Bennis et al.⁸ proposed a neural network architecture to estimate a finite mixture of two-parameter Weibull distributions with right-censored data. This approach consists multiple network layers, including fully connected layers, which has complicated structure and poor interpretability. Moreover, this approach did not have variable selection or theoretical justifications.

In practice, datasets tend to be rich in information, and baseline characteristics are often considered to be predictive of the latent subgroup membership. A large number of covariates, such as demographic characteristics and biomarkers, may be involved in the procedures of identifying latent subgroups. Variable selection becomes crucial to reducing dimensions without losing much information and it helps in understanding important features. Hence, it is natural to introduce variable selection in latent subgroup identification and think that only a few covariates are truly predictive of latent subgroup membership. Various approaches in variable selection from data without time-to-event outcomes and heterogeneity have been greatly discussed in the literature, including test-based approaches (see Chatfield⁹, Harrell Jr et al.¹⁰, Steyerberg et al.¹¹) and penalty-based methods (see Tibshirani¹², Meinshausen and Buhlmann¹³, Fan and Li¹⁴, Zou¹⁵). There has also been extensive work discussing variable selection for survival outcomes (see Tibshirani¹⁶, Fan and Li¹⁷) or for mixture Gaussian models (see Law et al.^18,19; Raftery and Dean²⁰, Khalili and Chen²¹). However, no literature takes the time-to-event data into consideration for variable selection when potential heterogeneity may be in the population.

In this paper, to predict the latent subgroup membership for future individuals and identify the important variables that are predictive of latent subgroup membership for individuals with specific survival profiles, we propose a method for variable selection in latent subgroup identification with time-to-event data. More specifically, we model the survival distribution through a mixture of Weibull distributions, where each mixture represents a latent subgroup. The latent group membership is then modelled via a multinomial distribution that may vary with feature variables. To select important feature variables for characterizing the latent groups, the EM algorithm is first applied to obtain the initial maximum likelihood estimate, and then the adaptive lasso penalty is introduced for variable selection. We show that our proposed estimator enjoys the oracle property when the number of covariates diverges with the sample size. The rest of the paper is organized as follows. Section 2 details the proposed method for variable selection in latent subgroup identification for individuals with specific survival profiles. Section 3 provides the theoretical properties of the proposed method. Section 4 shows the finite sample performance of our proposed method via a simulation study. Two real data examples in Section 5 demonstrate the applications of the proposed method.

2 |. METHODOLOGY

2.1 |. Model

We assume that the whole population consists of K different subgroups. Each group of patients will follow a specific survival profile. More specifically, we assume that the kth group has a survival distribution S(t, η_k), which has a parametric form with unknown parameters η_k, for k = 1, …, K. In this paper, we assume that the survival outcome for each latent subgroup follows a Weibull distribution, which is a commonly used distribution in survival analysis due to its flexibility and reliability¹. The functional form of the Weibull distribution for the kth latent subgroup is givens by $S\left(t,{\eta }_k\right)=\text{exp}\left\{-{\left(\frac{t}{{\lambda }_k}\right)}^{{\kappa }_k}\right\}$, where η_k = (κ_k, λ_k)^T, κ_k is the shape parameter and λ_k is the scale parameter.

We let T denote the time to event and X denote all the baseline covariates, which the number of baseline covariates could be large. To classify each patient into one of the survival groups using the baseline covariates X (X contains constant 1), we introduce a latent group membership B and assume

$$P\left(T>t\mid B=k,X\right)=S\left(t,{\eta }_k\right)$$ (1)

and

$$P\left(B=k\mid X\right)=\frac{\text{exp}\left\{{\beta }_k^{\mathbf{\text{T}}}X\right\}}{{\sum }_{k=1}^K\text{exp}\left\{{\beta }_k^{\mathbf{\text{T}}}X\right\}}={\pi }_k(X,\beta )$$ (2)

for k = 1, …, K, where β₁ = 0 and β₂, …, β_k are unknown parameters. Therefore, the latent group membership determines which group the patient should belong to and this membership depends on the baseline covariates through a multinomial distribution. Clearly, the proposed model implies that the marginal survival distribution for T takes a mixture form:

$$P\left(T>t\mid X\right)=\sum _{k=1}^{K}S\left(t,{\eta }_k\right){\pi }_k(X,\beta ),$$

where $\beta ={\left({\beta }_2^T,\dots ,{\beta }_K^T\right)}^T$. To conduct a future trial, for any new patient with baseline covariates X = x, we then classify this patient into group k with maximal value ${\beta }_k^{T}x$, i.e., the most likely group membership.

2.2 |. Initial Estimate

Suppose that we have right-censored observations from n i.i.d patients, denoted by

$$\left\{Y_i=T_i\wedge C_i,{\Delta }_i=I\left(T_i\le C_i\right),X_i,i=1,\dots ,n\right\},$$

where C_i is the censoring time. Assuming that the censoring time is independent of T_i given X_i, and is independent of the latent group membership B_i, we obtain the observed data log-likelihood function as

$$l_{n,obs}\left(\theta \right)=\sum _{i=1}^n\left[{\Delta }_i\text{log}\left\{\sum _{k=1}^{K}f\left(Y_i,{\eta }_k\right){\pi }_k\left(X_i;\beta \right)\right\}+\left(1-{\Delta }_i\right)\text{log}\left\{\sum _{k=1}^{K}S\left(Y_i,{\eta }_k\right){\pi }_k\left(X_i;\beta \right)\right\}\right],$$ (3)

where θ = (η^T, β^T)^T, ${\eta }^T={\left({\eta }_1^T,\dots ,{\eta }_K^T\right)}^T$, and f(t, η_k) = −S′(t, η_k).

To estimate β, we introduce B₁, …, B_n as the latent group membership for each subject and use the EM algorithm to compute the maximum likelihood estimators, treating the B′s as missing data. In the E-step, at the kth iteration, we compute the expected log-likelihood based on the current estimates of all parameters, conditional on the observed data, which is equivalent to calculating the posterior probability of B_i = k given the observed data for k = 1, …, K and i = 1, …, n. More specifically, this posterior probability is

$$q_{ik}=\frac{f\left(Y_i,{\eta }_k\right){\pi }_k\left(X_i;\beta \right)}{{\sum }_{k=1}^{K}f\left(Y_i,{\eta }_k\right){\pi }_k\left(X_i;\beta \right)}$$

if Δ_i = 1, and it is

$$q_{ik}=\frac{S\left(Y_i,{\eta }_k\right){\pi }_k\left(X_i;\beta \right)}{{\sum }_{k=1}^{K}S\left(Y_i,{\eta }_k\right){\pi }_k\left(X_i;\beta \right)}$$

if Δ_i = 0. In the M-step, we compute the estimates that maximize the expected log-likelihood obtained in the E-step,

$$l_n(\eta ,\beta )=\sum _{i=1}^n\sum _{k=1}^K{q}_{ik}\left[{\Delta }_i\text{log}\left(f\left(Y_i,{\eta }_k\right)\right)+\left(1-{\Delta }_i\right)\text{log}\left(S\left(Y_i,{\eta }_k\right)\right)+\text{log}\left({\pi }_k\left(X_i;\beta \right)\right)\right].$$ (4)

To estimate the survival distribution parameter η, we implement the Newton-Raphson algorithm to update the estimate based on the expected log-likelihood function (4). The expected log-likelihood function (4) is essentially a weighted multinomial regression. To obtain the maximum likelihood estimate $\tilde{\beta }$, for each iteration, we apply a one-step Newton-Raphson in the M-step to update the estimate. After convergence, we obtain the maximum likelihood estimates $\tilde{\eta }$ and $\tilde{\beta }$. It is easy to see that the expected log-likelihood function (4) in the M-step increases at each iteration, which implies that the algorithm is guaranteed to converge and will stay unchanged once converged.

To determine the best number of latent subgroups in the data, we consider several choices of the number of latent subgroups. For each potential number of latent subgroups, we apply a similar procedure as stated in this section to obtain the initial estimates. The value of the log-likelihood based on the initial estimates is computed afterwards. The BIC, as suggested by Nylund et al.²², is then calculated to determine the best number of latent subgroups for the data. Nylund et al.²² evaluated the performance of several information criteria for correctly identifying the number of groups. The performance of BIC for determining the best number of latent subgroups is also evaluated in Section 4.

2.3 |. Variable Selection for Latent Groups

The objective function (4) in the M-step is essentially a weighted multinomial regression, with weights being the posterior probability of B_i = k given the observed data for k = 1, …, K and i = 1, …, n. We use this objective function to accommodate penalties for variable selection. Because of the strict concavity of the objective function (4), we can derive nice theoretical properties for the estimator after variable selection.

Among many penalty functions, we apply the convex adaptive lasso penalty to the objective function (4). The weight for each coefficient in the adaptive lasso penalty is related to the importance of the corresponding covariate and helps to adaptively penalize each coefficient by tuning each coefficient with a different parameter. Zou¹⁵ shows that the adaptive lasso enjoys the oracle properties by inflating the weights for zero-coefficient covariates and enables the weights of nonzero-coefficient covariates to converge to a finite constant. The data-dependent adapting weights can be the reciprocal of any consistent estimator of β (Zou¹⁵). Here we consider the maximum likelihood estimator $\tilde{\beta }$. The penalized objective function becomes

$$-l_n\left(\tilde{\eta },\beta \right)+\lambda \sum _{k=1}^K\sum _{j=1}^d\frac{\left|{\beta }_{kj}\right|}{{\left|{\tilde{\beta }}_{kj}\right|}^{\gamma }},$$ (5)

where γ is a prespecified positive constant and the commonly used value is γ = 1, and β = (β₁₁, β₁₂, …, β_1d, β₂₁, …, β_Kd)^T. Here, we do not introduce a penalty on η, and $\tilde{\eta }$ is the maximum likelihood estimator. Hence, minimizing (5) is equivalent to applying the adaptive lasso penalty to a weighted multinomial regression.

To obtain the adaptive lasso estimates $\widehat{\beta }$, we minimize the penalized objective function (5) via a two-step strategy. The first step is to calculate the maximum likelihood estimates $\left(\tilde{\eta },\tilde{\beta }\right)$ that optimize (4) by an iterative Newton-Raphson update. Denote θ = (η, β). Define the gradient vector $\nabla l_n\left(\theta \right)=\frac{\partial l_n\left(\theta \right)}{\partial \theta }$ and the Hessian matrix ${\nabla }^2{l}_n\left(\theta \right)=\frac{{\partial }^2{l}_n\left(\theta \right)}{\partial \theta \partial {\theta }^T}$. The Newton-Raphson update is

$${\theta }^{\left(t+1\right)}={\theta }^{\left(t\right)}-{{\left({{\nabla }^2{l}_n\left(\theta \right)|}_{\theta ={\theta }^{\left(t\right)}}\right)}^{-1}\nabla l_n\left(\theta \right)|}_{\theta ={\theta }^{\left(t\right)}}.$$ (6)

The second step is to obtain the adaptive lasso estimates $\widehat{\beta }$ by minimizing (5) via a coordinate descent algorithm, where the coefficients are iterated over to minimize (5).

Hence, to minimize the penalized objective function (5) for any fixed γ, we use the following procedure.

• Step 1.

  Use the EM algorithm and the Newton-Raphson update (6) to compute the maximum likelihood estimates $\tilde{\eta }$ and $\tilde{\beta }$.

• Step 2.

  Calculate the weights in the adaptive lasso penalty, ${\tilde{w}}_i$ for i = 1, …, n, by using $\tilde{\beta }$.

• Step 3.

  Compute the weights, q_ik for i = 1, 2, …, n, and k = 1, 2, …, K, in the weighted multinomial regression $l_n\left(\tilde{\eta },\beta \right)$ by using $\tilde{\eta }$ and $\tilde{\beta }$.

• Step 4.

  Apply the coordinate descent algorithm to minimize the penalized objective function (5) until the convergence criterion is met.

In Step 3, the weights in the weighted multinomial regression are obtained by plugging in the estimates $\tilde{\eta }$ and $\tilde{\beta }$, since $\tilde{\eta }$ and $\tilde{\beta }$ are consistent maximum likelihood estimates, and the weights are fairly close to the true weights, which is shown in the Supporting Information. The minimization in Step 4 is based on the coordinate descent algorithm, which can be implemented via a statistical package such as glmnet in R.

To select the data-dependent tuning parameter λ in the proposed algorithm, we use V -fold cross validation. We consider λ from a set of grid points and partition the data into V subsets with equal size. For each point λ, we compute the coefficients using V − 1 subsets and obtain the deviance residual on the V th subset by using these coefficients V times. Averaging over V deviance residuals, we have an average deviance residual associated with one point of λ. We then select among the average deviance residuals and have the best choice for the tuning parameter λ that yields the smallest average deviance residual. After variable selection, we reapply the EM algorithm and maximize (3) by including the selected important covariates. We then classify patients to their most likely latent subgroup based on the post-selection maximum likelihood estimates.

3 |. THEORETICAL PROPERTIES

In this section, we describe the asymptotic properties of our estimators when the number of parameters grows with the sample size. With a slight abuse of notation, we write ${\beta }_n={\left({\beta }_{n\mathbf{1}}^T,\dots ,{\beta }_{nK}^T\right)}^T={\left({\beta }_{n\mathbf{1}},\dots ,{\beta }_{n{p}_n}\right)}^T$, where p_n is the number of variables, and ${\theta }_n={\left({\eta }^T,{\beta }_n^T\right)}^T$. We consider the penalized objective function based on n samples,

$$Q_n\left({\theta }_n\right)=l_n\left({\theta }_n\right)-n{\lambda }_n\sum _{j=\mathbf{1}}^{p_n}\left|{\beta }_{nj}\right|/{\left|{\tilde{\beta }}_{nj}\right|}^{\gamma }.$$

Denote the true values of θ_n by θ_n0. We write θ_n0 as ${\left({\eta }^T,{\beta }_{n\mathbf{10}}^T,{\beta }_{n\mathbf{20}}^T\right)}^T$, where

$${\beta }_{n\mathbf{10}}={\left({\beta }_{n\mathbf{10}},{\beta }_{n\mathbf{20}},\dots ,{\beta }_{nq\mathbf{0}}\right)}^T$$

consists of all q nonzero components and

$${\beta }_{n\mathbf{20}}={\left({\beta }_{n\left(q+1\right)0},{\beta }_{n(q+2)0},\dots ,{\beta }_{n{p}_{n}0}\right)}^T$$

consists of the remaining zero components. Correspondingly, we have the adaptive lasso estimator ${\widehat{\theta }}_n={\left({\widehat{\eta }}^T,{\widehat{\beta }}_{n\mathbf{1}}^T,{\widehat{\beta }}_{n\mathbf{2}}^T\right)}^T$.

We require the following regularity conditions.

• (C1)The function S(t, η_k) for k = 1, 2, …, K is non-increasing and continuously differentiable.
• (C2)
  Let g(X_i, Y_i, Δ_i, θ_n) Denote the probability density for observation {X_i, Y_i, Δ_i}, for i = 1, 2, …, n. The observations {X_i, Y_i, Δ_i, i = 1, 2, …, n} are independent and identically distributed. Let λ_min(A) and λ_max(A) denote the minimum and maximum eigenvalues of a positive definite matrix A, respectively. Assume that, for all i, the Fisher information matrix

  $$I_n\left({\theta }_n\right)=E\left[\left(\frac{\mathbf{\partial }\text{log}g\left(X_i,Y_i,{\Delta }_i,{\theta }_n\right)}{\mathbf{\partial }{\theta }_n}\right){\left(\frac{\mathbf{\partial }\text{log}g\left(X_i,Y_i,{\Delta }_i,{\theta }_n\right)}{\mathbf{\partial }{\theta }_n}\right)}^T\right]$$

  satisfies

  $$C_1\le {\lambda }_{min}\left\{I_n\left({\theta }_n\right)\right\}\le {\lambda }_{max}\left\{I_n\left({\theta }_n\right)\right\}\le C_2$$

  and, for j, l = 1, 2, …, p_n,

  $$E{\left[{\left(\frac{\partial \text{log}g\left(X_i,Y_i,{\Delta }_i,{\theta }_n\right)}{\partial \eta }\right)}^T\left(\frac{\partial \text{log}g\left(X_i,Y_i,{\Delta }_1,{\theta }_n\right)}{\partial \eta }\right)\right]}^2\le C_3$$

  and

  $$E{\left[\frac{\partial \text{log}g\left(X_i,Y_i,{\Delta }_i,{\theta }_n\right)}{\partial {\beta }_{nj}}\frac{\partial \text{log}g\left(X_i,Y_i,{\Delta }_i,{\theta }_n\right)}{\partial {\beta }_{nl}}\right]}^2\le C_4,$$

  where C₁, C₂, C₃ and C₄ are positive constants.
• (C3)
  θ_n0 is contained in a large enough open set. For all θ_n within this open set, the third derivatives of g(X_i, Y_i, Δ_i, θ_n) with respect to β_n satisfy

  $$\left|\frac{{\partial }^3\text{log}g\left(X_i,Y_i,{\Delta }_i,{\theta }_n\right)}{\partial {\beta }_{nj}\partial {\beta }_{nl}\partial {\beta }_{nm}}\right|\le M_{\mathit{\text{njlm}}}\left(X_i,Y_i,{\Delta }_i\right)$$

  and

  $$E\left[M_{\mathit{\text{njlm}}}^2\left(X_i,Y_i,{\Delta }_i\right)\right]\le C_5,\text{ where }{C}_5\text{ is a positive constant}$$

  for j, l, m = 1, 2, …, p_n.
• (C4) Assume that

  $$\underset{1\le j\le q}{\text{min}}\frac{\left|{\beta }_{nj0}\right|}{{\lambda }_n}\to \infty ,\text{ as }n\to \infty .$$

Condition (C1) requires S(t, η_k), k = 1, 2, …, k to be a valid survival distribution. Conditions (C2) and (C3) are similar to conditions (F) and (G) in Fan et al.²³, which assume that the likelihood function has reasonably good behavior. Condition (C4) is used to establish the oracle property of the adaptive lasso estimator and already implicitly assumed in a finite dimensional setting. This condition is exactly condition (H) in Fan et al.²³, which allows nonzero coefficients to vanish and can be distinguished at a rate by the penalized likelihood.

Under conditions (C1) – (C4), we have the following asymptotic results for our estimators.

Theorem 1.

Denote the maximum likelihood estimates of l_n,obs(θ_n) by ${\tilde{\theta }}_n$, where

$$l_{n,obs}\left({\theta }_n\right)=\sum _{i=1}^n\left[{\Delta }_i\text{log}\left\{\sum _{k=1}^{K}f\left(Y_i,{\eta }_k\right){\pi }_k\left(X_i;{\beta }_n\right)\right\}+\left(1-{\Delta }_i\right)\text{log}\left\{\sum _{k=1}^{K}S\left(Y_i,{\eta }_k\right){\pi }_k\left(X_i;{\beta }_n\right)\right\}\right]$$

If $p_n^4/n\to 0$ as n → ∞, then $\Vert {\tilde{\theta }}_n-{\theta }_{n0}\Vert =O_p\left(\sqrt{p_n}{n}^{-1/2}\right)$

Theorem 2.

If $\sqrt{n{p}_n}{\lambda }_n=O\left(1\right)$ and $p_n^4/n\to 0$ as n → ∞, then there is a unique maximizer ${\widehat{\theta }}_n$ of Q_n(θ_n) such that ${\widehat{\theta }}_n-{\theta }_{n0}=O_p\left\{\sqrt{p_n}\left(n^{-1/2}\right)\right\}$.

Finally, we provide the asymptotic distribution of the adaptive lasso estimator. We let

$$b_n={\left\{0,\dots ,0,{\lambda }_n\text{sign}\left({\beta }_{n\mathbf{10}}\right)/{\left|{\tilde{\beta }}_{n\mathbf{1}}\right|}^{\gamma },\dots ,{\lambda }_n\text{sign}\left({\beta }_{nq\mathbf{0}}\right)/{\left|{\tilde{\beta }}_{nq}\right|}^{\gamma }\right\}}^T,$$

${\theta }_{n1}={\left({\eta }^T,{\beta }_{n1}^T,0^T\right)}^T$ and ${\theta }_{n10}={\left({\eta }_0^T,{\beta }_{n10}^T,0^T\right)}^T$. Let s be the number of parameters for the survival distributions of the K latent subgroups. The first s zeros contained in b_n are due to the fact that we do not penalize the parameters of the survival distributions.

Theorem 3.

If nλ_n → 0, $\sqrt{n/p_n}{\lambda }_n\to \infty$ and $p_n^5/n\to 0$ as n → ∞, then under Theorem 1, the adaptive lasso estimator ${\widehat{\theta }}_n$ has the following properties:

1. ${\widehat{\beta }}_{n2}$ with probability tending to 1;

2. $$\sqrt{n}{A}_n{I}_n^{-1/2}\left({\theta }_{n10}\right)\left\{I_n\left({\theta }_{n10}\right)\right\}\left[{\widehat{\theta }}_{n1}-{\theta }_{n10}+{\left\{I_n\left({\theta }_{n10}\right)\right\}}^{-1}{b}_n\right]{\to }_{D}N(0,G)$$

   where A_n is a r × (s + q) matrix such that $A_n{A}_n^T\to G$, and G is a r × r non-negative symmetric matrix.

One key to the proofs is to obtain a uniform approximation rate for the weights in the expression of Q_n(θ_n). For this, we use the result established in Theorem 1. The proofs of Theorems 2 and 3 then follow the standard arguments in variable selection for parametric models, including the existence of the local maximum in a neighborhood of the true parameters and verification of the fact that the oracle estimator attains this local maximum, but with careful verification of certain approximation rates in terms of p_n. The details of the proof are given in the Supporting Information. The theoretical properties for the post-selection estimator, that is, the maximum likelihood estimator of selected important variables after refitting the model without the adaptive lasso penalty, could be easily obtained. Under Theorem 3, the probability that adaptive lasso estimator of unimportant variables does not equal to zero tends to 0. Therefore, the post-selection estimator has the same asymptotic distribution as the adaptive lasso estimator of important variables, which is stated in Theorem 3.

4 |. SIMULATION STUDIES

We conduct the simulation study that assumes two latent subgroups exist. We consider 10, 30 and 50 covariates in the regression model and only a few of covariates have nonzero effects. The covariates X = (X₁, X₂, …, X_p), where p = 10, 30, 50, are generated from standard normal distribution with moderate correlations. Time to event data for each latent subgroup follow a different Weibull distribution with scale parameter λ and shape parameter κ. The censoring time is generated from an exponential distribution, where the mean is calibrated by a prespecified censoring rate of 10%.

The true values of the scale parameters (i.e., λ₁ and λ₂) of the Weibull distributions for two latent subgroups are set to be 1 and 4.5 respectively, and the true values of the shape parameters (i.e., κ₁ and κ₂) are 1 and 3 for two latent subgroups respectively. Around 40% of the individuals belong to latent subgroup 1 with a 2-year survival probability of 13.5%. Sixty percent of the individuals are in latent subgroup 2 and have a 2-year survival probability of 91.5%. The subgroup-specific survival curves are illustrated in Figure 1. The true β associated with latent subgroup membership is calibrated such that the true proportions of the two subgroups are 40% and 60%, respectively. Three scenarios in the simulation study are described below. The sensitivity analysis that evaluates the proposed model when the link function of the multinomial distribution for the latent subgroup membership is nonlinear is included in the Supporting Information.

FIGURE 1.

The true survival curves in the simulation study.

• Scenario 1.
  10 covariates are independently generated from a standard normal distribution, and first three of them are important covariates. Subgroup 1 is regarded as the reference group and β₁ is set to 0.

  $$\beta =\left(\begin{array}{l}{\beta }_{\mathbf{1}}\hfill \\ {\beta }_{\mathbf{2}}\hfill \end{array}\right)=\left(\begin{array}{ccccccc}0,& 0,& 0,& 0,& 0,& \dots ,& 0\\ 0.4,& 0.2,& -0.6,& -0.3,& 0,& \dots ,& 0\end{array}\right)$$
• Scenario 2.

  30 covariates are generated from standard normal distribution, and first eight of them have nonzero effects.

  Correlations between X₁ and X₂, X₃ and X₄, X₇ and X₁₀ are set to be 0.2, 0.3 and 0.2, respectively.

  $$\beta =\left(\begin{array}{l}{\beta }_{\mathbf{1}}\hfill \\ {\beta }_{\mathbf{2}}\hfill \end{array}\right)=\left(\begin{array}{cccccccccccc}0,& 0,& 0,& 0,& 0,& 0,& 0,& 0,& 0,& 0,& \dots ,& 0\\ 0.4,& 0.2,& -0.6,& -0.3,& 0.5,& -0.5,& 0.7,& -0.7,& 0.5,& 0,& \dots ,& 0\end{array}\right)$$
• Scenario 3.

  50 covariates are generated from standard normal distribution, and first eight of them have nonzero effects. Correlations between X₁ and X₂, X₃ and X₄, X₇ and X₁₀ are set to be 0.2, 0.3 and 0.2, respectively. True values of regression coefficients for important variables are set to be the same as in scenario 2.

To implement the EM algorithm to obtain the maximum likelihood estimators of the β’s and the survival distribution parameters κ_k, λ_k for k = 1, 2, …, K, the stopping criteria for EM is l_obs(θ^(k+1))−l_obs(θ^(k)) < 10⁻⁴, where θ = (κ₁, λ₁, …, κ_k, λ_k, β^T)^T. For γ in the adaptive lasso penalty, we use γ = 1 for all simulation studies. For each simulated dataset, we first identify the number of latent subgroups by applying our method for estimation and calculating BIC. Once the number of latent subgroups is determined, we use the EM algorithm to obtain the maximum likelihood estimates and then implement the adaptive lasso procedure to perform variable selection. We consider the grid 2⁻¹⁶, 2⁻¹⁵, …, 2¹⁵, 2¹⁶ for the tuning parameter λ, and report the results that yield the smallest value of average deviance residual. After variable selection, the EM algorithm is reapplied to the models with only selected covariates. We repeat the simulation 1000 times and consider sample sizes of n = 300, 1000 and 3000.

We first calculate BIC for models assuming no latent subgroups, two, and three latent subgroups, in datasets that truly consisted of two latent subgroups. BIC suggests that approximately 100% of the datasets with the sample size of 1000 consist of two latent groups. When the sample size increases to 3000, all the datasets are correctly detected consisting of two latent subgroups by the BIC criterion.

Table 1 summarizes the prediction accuracy, along with standard errors, for models without and after variable selection for all three scenarios, and also reports the average number of correct and incorrect zero coefficients and corresponding standard errors. The prediction accuracy is calculated by applying the decision rule obtained from the training set to a validation set with sample size 10, 000. Compared to the optimal accuracy rate, our method performs well for all three scenarios, especially for models with only selected important covariates. The prediction accuracy is approaching to the optimal accuracy rate as the sample size increases. When the number of covariates increases, our method also works well in terms of prediction accuracy and variable selection results. The optimal accuracy rate is 1–Bayes error rate, where the Bayes error rate is calculated via the formula $1-E\left(\underset{k}{\text{max }}P\left(B=k\mid X\right)\right)$, is the lowest possible test error rate. For scenario 1, when the sample size is 300, important variables are correctly selected in approximately 80% of the datasets, and unimportant variables are selected in approximately 15% of the datasets. As the sample size increases to 1000, important variables are identified in over 99% of the datasets. Meanwhile, the ability to shrink zero coefficients to zero is also improved: the rate of incorrectly selecting unimportant variables is below 5%. For scenario 2 when the number of covariates increases to 30, important variables can be correctly identified in around 80% of the datasets when the sample size is 300, while the unimportant variables are selected in around 32% of the datasets. The ability to identify important variables and shrink zero coefficients to zero is improved when the sample size grows to 1000: important covariates can be picked out in approximately 95% of the datasets, and our method selects unimportant variables in only 5% of the datasets. For scenario 3 with 50 covariates, when the sample size is 300, around 80% of the datasets can correctly distinguish important variables and the rate of incorrectly selecting unimportant variables is 35%. As the sample size increases to 1000, the rates of identifying important variables and selecting unimportant variables are 80% and 11%, respectively. When the sample size further grows to 3000, over 99% of the datasets can correctly recognize important variables and the rate of incorrectly selecting unimportant variables decreases to around 1%. Table 2 reports the accuracy of nonzero coefficient post-selection estimates, their standard errors and coverage probabilities for nominal 95% confidence intervals for scenario 1. Due to the limited space, we report results for scenario 2 and 3 in the Supporting Information. To obtain the standard errors for the maximum likelihood estimates, we use the Louis formula²⁴ because the latent group membership is treated as missing data in our method. For these three scenarios, we observe similar results: the post-selection estimates are slightly biased on small samples and the bias can be reduced by increasing the sample size; the 95% confidence intervals for the post-selection estimators based on the estimated coefficients and standard errors have accurate coverage for the true parameters.

TABLE 1.

Results from the simulation study with 2 latent groups

	Accuracy (SE)		Comparison
N	without variable selection	after variable selection	Group 2 vs. 1
			Corr. (SE)	Incorr. (SE)
Scenario 1: 10 independent covariates, 3 of them are important.
	The optimal accuracy rate is 0.648.
300	0.617 (0.019)	0.619 (0.030)	5.70 (1.754)	0.63 (0.876)
1000	0.636 (0.009)	0.640 (0.010)	6.53 (0.946)	0.08 (0.278)
3000	0.643 (0.006)	0.645 (0.015)	6.90 (0.326)	0.00 (0.045)
Scenario 2: 30 covariates with moderate correlations, 8 of them are important.
	The optimal accuracy rate is 0.732.
300	0.675 (0.020)	0.680 (0.029)	13.47 (6.574)	0.86 (1.146)
1000	0.716 (0.007)	0.724 (0.008)	19.71 (3.245)	0.24 (0.459)
3000	0.728 (0.005)	0.731 (0.005)	21.41 (1.256)	0.03 (0.159)
Scenario 3: 50 covariates with moderate correlations, 8 of them are important.
	The optimal accuracy rate is 0.732.
300	0.653 (0.019)	0.666 (0.030)	27.39 (10.290)	0.84 (1.178)
1000	0.705 (0.008)	0.720 (0.010)	36.45 (6.969)	0.22 (0.430)
3000	0.724 (0.005)	0.731 (0.005)	40.86 (2.783)	0.03 (0.179)

Note. Each column corresponds to prediction accuracy and standard errors, average number of correct (Corr.) and incorrect (Incorr.) zero coefficients and standard errors from 1000 simulated datasets.

TABLE 2.

Maximum likelihood Estimates after variable selection, their standard errors, and coverage probabilities for nominal 95% confidence intervals from the simulation study with 2 latent groups

N	Parameter	Bias	SE	SEE	CP
300	k1	0.035	0.113	0.109	0.954
	λ1	0.015	0.274	0.254	0.852
	k2	0.096	0.422	0.354	0.915
	λ2	−0.013	0.210	0.190	0.902
	β0	−0.003	0.350	0.318	0.901
	β1	−0.033	0.186	0.110	0.605
	β2	−0.002	0.277	0.172	0.866
	β3	0.005	0.202	0.145	0.807
1000	k1	0.013	0.062	0.061	0.946
	λ1	−0.000	0.166	0.160	0.890
	k2	0.007	0.203	0.190	0.931
	λ2	−0.015	0.109	0.109	0.940
	β0	0.010	0.195	0.187	0.916
	β1	−0.008	0.101	0.080	0.883
	β2	−0.017	0.101	0.097	0.949
	β3	−0.006	0.088	0.089	0.962
3000	k1	0.004	0.036	0.036	0.942
	λ1	0.002	0.099	0.099	0.925
	k2	0.006	0.110	0.111	0.952
	λ2	−0.003	0.065	0.064	0.949
	β0	0.002	0.111	0.112	0.939
	β1	0.000	0.049	0.050	0.954
	β2	−0.007	0.056	0.055	0.948
	β3	−0.005	0.051	0.051	0.947

Note: SE, standard error; SEE, mean of standard error estimator; CP, coverage probability for nominal 95% confidence interval.

5 |. REAL DATA APPLICATION

Our proposed method is applied to two datasets (see Supporting Information for the other real data example). We apply the proposed methodology to data from a breast cancer clinical trial to study the potential heterogeneity of patients in terms of their survival outcomes and investigate important variables that are associated with such heterogeneity. The data was collected from a large clinical trial, IBCSG Trial VI²⁵, in premenopausal women with node-positive breast cancer to study both the duration of adjuvant chemotherapy and the reintroduction of delayed chemotherapy. Patients were randomized in a two by two factorial design to receive the following: (A) cyclophosphamide, methotrexate, and fluorouracil (CMF) for six consecutive cycles (CMF*6); (B) CMF*6 plus three single cycles of reintroduction CMF; (C) CMF*3; and (D) CMF*3 plus three single cycles of reintroduction CMF. The patients’ quality of life (QOL) was also measured at baseline and was hypothesized to contain prognostic information and reflect breast cancer progression. Four aspects of QOL, including physical well-being, mood, appetite and perceived coping, were assessed by a self-assessment QOL questionnaire. In addition to treatment effects and patients’ QOL, disease-free survival (median follow-up of 7.47 years, rescaled to [0, 1]), event status, age at baseline, estrogen receptor (ER) status (1=positive, 0=negative) and the number of positive nodes of the tumor (i.e., node group, 1=number of positive nodes > 4, 0=else) are also considered in the data. After excluding missing values, data are available for 962 patients. The median follow-up for disease free survival (DFS) is 7.47 years and the event rate is around 45%. We rescale the DFS to [0, 1] and standardize continuous variables such as age and the four measures of QOL for computation.

Our first step is to investigate whether latent subgroups exist in the data. BIC suggests that two latent subgroups are contained in the data with a value of 806.1. Values of BIC of models assuming no latent subgroup and three latent subgroups in the data are 962.0 and 886.5 respectively. We assume that the survival outcomes for patients in different latent subgroups follow different Weibull distributions. Next, survival distributions and coefficients for covariates are estimated by the mixture model via the EM algorithm and Newton-Raphson. The estimated shape parameters of the Weibull distribution for two latent subgroups are 3.05 and 1.90, respectively. The corresponding scale parameter estimates are 0.24 and 1.43. The regression coefficient estimates for the covariates are summarized in Table 3. Based on initial estimates from this model, we then implement our variable selection procedure to identify important covariates that are associated with latent subgroup membership assignment. After variable selection, we refit the model using the selected important covariates. The estimated Weibull distributions for two latent subgroups are pretty close to those without variable selection: the survival distribution for latent group 1 has shape parameter estimate of 3.04 and scale parameter estimate of 0.24; the survival distribution for latent group 2 has shape parameter estimate of 1.91 and scale parameter estimate of 1.43. The results of the adaptive lasso estimator and maximum likelihood estimator after variable selection can be found in Table 3. The latent subgroup membership is associated with age and the number of positive nodes. The latent subgroup membership for each individual is predicted afterwards, based on maximum likelihood estimates without variable selection and after variable selection. The model without variable selection yields 74 out of 962 individuals belong to latent group 1, which makes up about 7.7% of the total individuals in the data. After variable selection, 63 individuals (≈ 6.5%) belong to latent group 1. Kaplan-Meier curves for the two latent subgroups are utilized to illustrate the survival profiles, which is shown in the left panel of Figure 2.

TABLE 3.

Parameter estimates for the IBCSG trial data

	Overall models		Treatment-specific models
			Treatment B		Treatment C
Covariates	MLE w/o var. sel. (p-value)	MLE after var. sel. (p-value)	MLE w/o var. sel. (p-value)	MLE after var. sel. (p-value)	MLE w/o var. sel. (p-value)	MLE after var. sel. (p-value)
(intercept)	1.21	1.21	1.09	1.05	0.90	0.87
	(<0.0001)	(<0.0001)	(0.0017)	(0.0019)	(0.0176)	(0.0009)
age	0.25	0.25	0.16	0.16	0.57	0.55
	(0.0034)	(0.0030)	(0.3316)	(0.3309)	(0.0015)	(0.0017)
node	−1.06	−1.05	−1.74	−1.77	−1.26	−1.16
	(<0.0001)	(<0.0001)	(<0.0001)	(0.0033)	(0.0007)	(0.0011)
ER status	0.13	0	0.32	0.32	0.03	0
	(0.4869)	(−)	(0.3811)	(0.3740)	(0.9277)	(−)
physical	0.08	0	−0.17	−0.08	0.22	0
	(0.4456)	(−)	(0.4591)	(0.7095)	(0.2959)	(−)
mood	−0.24	0	−0.02	−0.42	−0.36	0
	(0.0375)	(−)	(0.0129)	(0.0451)	(0.1271)	(−)
appetite	0.04	0	0.26	0	0.02	0
	(0.6430)	(−)	(0.2178)	(−)	(0.8964)	(−)
cope	0.21	0	0.27	0	0.32	0
	(0.0302)	(−)	(0.2008)	(−)	(0.0953)	(−)
trtB	−0.03	0	-	-	-	-
	(0.8884)	(−)	(−)	(−)	(−)	(−)
trtC	−0.17	0	-	-	-	-
	(0.4925)	(−)	(−)	(−)	(−)	(−)
trtD	−0.08	0	-	-	-	-
	(0.7354)	(−)	(−)	(−)	(−)	(−)

Note. “node”: the number of positive nodes in the tumor; “physical”: physical well-being; “cope”: perceived coping.

FIGURE 2.

Kaplan-Meier Curves for latent subgroups and for subgroups determined by age and the number of positive nodes

Note: “Time” is disease free survival and rescaled to [0, 1].

The results of variable selection indicate that treatment does not have a significant effect on the latent subgroup membership assignment. To further explore the heterogeneity of patients under different therapeutic procedures, we apply our method to the datasets of patients under each treatment. According to BIC, two latent subgroups are detected among patients with treatment B and patients with treatment C, and no latent subgroup is identified among patients with treatment A and treatment D. More specifically, for treatment A, the BICs for models assuming no latent subgroup, two and three latent subgroups are calculated as 257.5, 311.7 and 382.4 respectively. For treatment B, the BICs for these three models are 258.2, 221.0 and 347.2 respectively. For treatment C, BICs for these three models become 225.0, 203.9 and 260.0 respectively. For treatment D, the corresponding BIC values for these three models are 250.0, 278.4 and 305.1. Table 3 reports the estimated regression coefficients for patients under treatment B and patients under treatment C. Assuming that the survival outcomes for patients follow a Weibull distribution, among patients under treatment B, the survival distribution estimates yield a shape parameter of 3.06 and a scale parameter of 0.24 for latent group 1, and a shape parameter of 1.88 and a scale parameter of 1.44 for latent group 2, based on the model without variable selection. For patients under treatment C, the survival distribution estimates yield a shape parameter of 2.57 and a scale parameter of 0.28 for latent group 1, and a shape parameter of 2.21 and a scale parameter of 1.74 for latent group 2, based on the model without variable selection. After variable selection, two estimated Weibull distributions yield ${\widehat{\kappa }}_1=2.88$, ${\widehat{\lambda }}_1=0.27$, ${\widehat{\kappa }}_2=1.87$ and ${\widehat{\lambda }}_2=1.84$ for patients under treatment B, and ${\widehat{\kappa }}_1=2.57$, ${\widehat{\lambda }}_1=0.28$, ${\widehat{\kappa }}_2=2.10$ and ${\widehat{\lambda }}_2=1.75$ for patients under treatment C. The middle and right panels of Figure 2 demonstrate survival profiles for two latent groups from patients under treatment B and treatment C. After we obtain the predicted latent group membership, A logrank test is performed to evaluate the difference between survival profiles for two latent groups from patients under treatment B and treatment C. P-values from logrank test are smaller than 0.0001, which implies that, among patients under treatment B and treatment C, two latent groups are significantly different in terms of their survival profiles.

Based on the results of variable selection, we find that for patients under treatment B, the latent subgroup membership assignment is associated with age, the number of positive nodes, ER status, physical well-being and mood. When looking at patients under treatment C, only age and the number of positive nodes are predictive of the latent subgroup membership assignment, which agrees with the findings in the overall model. With these findings, we conclude that some latent subgroups of patients under treatment B and treatment C respond to the treatment differently due to some important covariates such as age and the number of positive nodes. Therefore, it is of interest to further study how the treatment works differently for some subgroups that are determined by important covariates. We create four subgroups of patients based on dichotomized age and the number of positive nodes. More explicitly, we dichotomize age by a threshold of 40 years, which is learned from previous findings about the IBCSG trial²⁶. A Cox proportional hazards model with treatment as the only covariate is then applied to each subgroups of patients to evaluate the treatment effect. In the subgroups of patients aged less than 40 years and with more than 4 positive nodes, treatment C has a significant effect on the survival outcomes (p-value=0.031). Compared with treatment A, the hazard ratio of treatment C is 2.48 with 95% confidence interval (1.085, 5.666), which implies that the hazard for patients treated with CMF*3 is higher than for patients treated with CMF*6. Kaplan-Meier curves for patients under each treatment in different subgroups are demonstrated in Figure 3.

FIGURE 3.

Kaplan-Meier Curves for patients under each treatment in different subgroups

Note: Top-left panel: subgroup of patients with age less than 40 years old and the number of positive nodes less than 4. Top-right panel: subgroup of patients with age more than 40 years old and the number of positive nodes less than 4. Bottom-left panel: subgroup of patients with age less than 40 years old and the number of positive nodes more than 4. Bottom-right panel: subgroup of patients with age more than 40 years old and the number of positive nodes more than 4. Treatment A is the reference for hazard ratio and corresponding confidence interval estimates. P-value corresponds to the logrank test of treatment effect for each subgroup. “Time” is disease free survival and rescaled to [0, 1].

6 |. CONCLUSION

In this article, we propose a novel algorithm to detect the latent subgroups for individuals with different survival profiles and identify important covariates that are associated with the latent subgroup membership assignment. We have shown that our proposed estimator is consistent and enjoys the oracle properties when the number of covariates diverges with the sample size. Our proposed method can simultaneously estimate the unknown survival distributions and the coefficients that are predictive of latent subgroup membership assignment. The data with a large number of covariates can be handled well through a penalized objective function. This proposed methodology would potentially work as an exploratory step in clinical trial settings before implementing a subgroup analysis to study the treatment effect. The selected important covariates may help to explicitly determine the subgroup and discover how patients in different subgroups respond differently to treatments. Specific treatments could be developed for a target group of patients subsequently. Furthermore, using this proposed algorithm, we could directly classify patients into high-risk and low-risk groups based on their survival profiles. Since the identified classes have distinct survival distributions, each class is clinically meaningful, corresponding to patients with either long or short survival trajectories. Thus, the obtained classes can be useful to differentiate subgroup of patients at least in the following direction. First, the obtained latent classes can be used for patient recruitment in conducting future clinical trials. For example, we can recruit more patients from the high-risk group to empower trials. Another potential application is as illustrated in the real data application, our method can be used to identify subgroups of patients who may more benefit from one treatment as compared to the rest and to explore the baseline characteristics of these subgroups of patients, or their intersections, based on selected important covariates.

In our proposed algorithm, the survival distributions for latent subgroups are assumed to follow Weibull distributions with unknown parameters. It is easy to extend out methodology to other parametric distributions, such as the exponential distribution and the lognormal distribution. Our parametric framework could also be weakened by assuming that the baseline hazard function is semi-parametric and we estimate the baseline cumulative hazard function using the Breslow’s estimator.

The distribution of the latent subgroup membership, given baseline covariates, is assumed to be a multinomial distribution. This assumption could be relaxed by considering a tree-based partition on the data, which could be used to identify latent subgroups and select important covariates in a nonparametric framework. The tree-based method for latent subgroup identification may be helpful for handling the data with many covariates. Lastly, the way we select the best number of latent subgroups is to essentially apply the proposed method to the data by assuming a different number of latent subgroups in the data. A nonparametric approach for determining the best number of latent subgroups could be established as well. We could implement the tree-based partition procedure based on several choices of the number of latent subgroups assumed in the data, compute BIC values for each choice and select the number of latent subgroups associated with the smallest value of BIC. Semi-parametric approaches and their theoretical justification are currently being investigated.

We assume that the survival distributions from different latent groups are distinct, which implies that the mixture model can be identifiable. The selection on number of latent subgroups using BIC can also help to distinguish non-identifiable cases, where BIC will suggest that no latent subgroups exists in the data. It has been well understood that the model selection method with a fixed number of covariates using BIC criterion²⁷ can identify the true model consistently^28,29. For the situation with a diverging number of covariates, the asymptotic behavior of a slightly modified version of BIC criterion has been greatly discussed as well and the consistency in linear regression model selection with a diverging number of covariates for penalized estimators has been studied³⁰. When the number of covariates diverges in a mixture model setting, although the BIC criterion works well in our empirical studies, we are not aware of the theoretical results of BIC criterion in this situation. We will pursue this interesting topic in our future work.

Moreover, our proposed approach does not cover the ultra-high dimensional case in which the dimensionality p_n is much larger than the sample size n. In this case, some problems need to be solved. For example, the estimate of the number of latent groups based on BIC may not be consistent³¹, and the variable selection procedure will be challenged³². Further work on applying our proposed method to ultra-high dimensional data needs to be done. This work focuses on identifying latent groups who have a distinct survival experience. The same idea can be extended to study latent groups who may respond to treatments differently. The latter will be particularly characterized by different treatment effects which can be constant or time-varying. Variable selection will also be important to determine a small list of feature variables for medical decisions. We will pursue these extensions in future work.