Scalable and Robust Latent Trajectory Class Analysis Using Artificial Likelihood

SUMMARY:

Latent trajectory class analysis is a powerful technique to elucidate the structure underlying population heterogeneity. The standard approach relies on fully parametric modeling and is computationally impractical when the data include a large collection of non-Gaussian longitudinal features. We introduce a new approach, the first based on artificial likelihood concepts, that avoids undue modeling assumptions and is computationally tractable. We show that this new method provides reliable estimates of the underlying population structure and is from 20 to 200 times faster than conventional methods when the longitudinal features are non-Gaussian. We apply the approach to explore subgroups among research participants in the early stages of neurodegeneration.

1. Introduction

To investigate the structure of a heterogeneous population exhibiting a complex phenotype that changes over time, it is desirable to take into account the longitudinal trajectories of many features. For example, separating people in the early stages of neurodegeneration into subgroups based upon the trajectories of a collection of relevant clinical features and biomarkers would be helpful for conducting preventive trials of Alzheimer’s and related neurodegenerative diseases. Latent trajectory class analysis is a powerful statistical approach to explore the structure, based upon the intuitive premise that both the observed longitudinal trajectories within features and the statistical associations between features are manifestations of latent classes in the study population (Proust et al., 2006). This approach is rooted within a rigorous statistical framework of finite mixture models and, thus, provides inferences with many appealing properties: the most efficient large-sample estimator of model parameters can be obtained by maximum likelihood; the uncertainty of results can be assessed with analytical standard errors and confidence intervals; and subjects can be assigned more accurately to subgroups based on the probabilities of latent class membership (McLachlan and Peel, 2000).

In recent years, several models for use in latent trajectory class analysis have been introduced (e.g., Proust-Lima et al., 2014;Muthen, 2001). While these models are attractive in many respects, they require the investigator to posit the full probabilistic mechanism that generated the longitudinal trajectories. Such modeling assumptions are subject to misspecification and can result in misleading conclusions (Bauer and Curran, 2003). Moreover, maximum likelihood estimation of the model parameters is computationally prohibitive when we have a large collection of longitudinal features that do not follow multivariate Gaussian distributions and require numerical integration, such as class-specific nonlinear mixed effects models. To illustrate, consider a hypothetical latent trajectory class analysis with 5 latent classes and 30 non-Gaussian variables measured longitudinally. When the search for the maximum likelihood estimator is conducted over 100 random starting values in the parameter space, with an average of 10 iterations of the expectation-maximization algorithm applied per random starting value, this medium-sized analysis would require solving 150,000 separate nonlinear optimization problems each involving numerical integration. Rosen et al.’s (2000) mixture-of-marginals model avoids numerical integration, but provides probabilities of latent class membership using only the marginal information at specific time points, a critical limitation in studies where we seek to classify subjects into subgroups based upon the longitudinal courses of their features. An extended method of moments approach is available, as in Reboussin et al. (1999) and Reboussin et al. (2002), where one compares each observation to its unconditional mean, marginalizing over the latent classes. Although this method of moments approach is robust, its lack of a likelihood function makes it difficult to assess the number of classes or predict the participants’ latent class membership probabilities. Moreover, the method of moments, and its extensions, are well known to yield less precise estimators than the maximum likelihood approach (Casella and Berger, 2002).

To address this need for a scalable and robust method of latent trajectory class analysis, we propose a new approach, the first based upon artificial likelihood concepts, that avoids undue model assumptions, yields probabilistic predictions of class membership based on observed trajectories, and offers dramatically simpler computations.

2. Finite Mixture Models

Let $Y={\left(Y_1^T,\dots ,Y_n^T\right)}^T$ denote a random sample of size n, where each Y_i is a random vector. Assume that there are C latent classes of the population, where C is possibly unknown. Denote each subject’s unobserved latent class membership indicator vector as Z_i = (z_i1, . . ., z_iC). Since an individual belongs to exactly one latent class, for each subject exactly one of these indicators is one and the rest are zeroes. Briefly assume that the random vector Y_i has a fully-specified probability density function, and let f_c(y_i; θ)(c = 1, . . ., C), denote the component density for class c, where θ is a vector of unknown parameters. To assess the effect of a vector of baseline covariates x_i on the relative frequency of belonging to latent class c, possibly the trivial case where x_i = 1, we adopt a latent polytomous logistic regression model (Bandeen-Roche et al., 1997)

$$Pr\left(z_{ic}=1\mid x_i\right)=p_c\left(\alpha ,x_i\right)=\frac{\exp \left(x_i^T{\alpha }_c\right)}{{\sum }_{d=1}^C\exp \left(x_i^T{\alpha }_d\right)}(c=1,\dots ,C),$$ (1)

where α₁ = 0 for identifiability and $\alpha ={\left({\alpha }_1^T,\dots ,{\alpha }_C^T\right)}^T$ . It follows that the probability density function for Y_i under a C-component mixture model is (McLachlan and Peel, 2000)

$$f\left(y_i;\psi \right)={\sum }_{c=1}^C{p}_c\left(\alpha ,x_i\right)f_c\left(y_i;\theta \right),$$

Where $\psi ={\left({\alpha }^T,{\theta }^T\right)}^T$ denotes the vector of all unknown parameters.

As outlined in McLachlan and Krishnan (2008), the expectation-maximization algorithm is the most widely used approach for maximum likelihood estimation of finite mixture models. The log likelihood for a C-component finite mixture model is

$$l(\alpha ,\theta )=\sum _{i=1}^n\log \left\{{\sum }_{c=1}^C{p}_c\left(\alpha ,x_i\right)f_c\left(y_i;\theta \right)\right\}.$$

To identify the estimators of (α, θ) that maximize the likelihood function, one jointly solves the score equations S (α) = ∂l(α, θ)/∂α = 0 and S (θ) = ∂l(α, θ)/∂θ = 0, where

$$S(\alpha )=\sum _{i=1}^n\sum _{c=1}^C{\tau }_{ic}\frac{\partial \log p_c\left(\alpha ,x_i\right)}{\partial \alpha },S(\theta )=\sum _{i=1}^n\sum _{c=1}^C{\tau }_{ic}\frac{\partial \log f_c\left(y_i;\theta \right)}{\partial \theta }.$$ (2)

Here, τ_ic denotes the posterior probability that subject i belongs to class c, and is obtained using Bayes Rule as

$${\tau }_{ic}={\tau }_{ic}\left(\alpha ,\theta ;y_i,x_i\right)=\mathrm{Pr}\left(z_{ic}=1\mid y_i,x_i;\alpha ,\theta \right)=\frac{p_c\left(\alpha ,x_i\right)e^{w_c\left(y_i;\theta \right)}}{{\sum }_{d=1}^C{p}_d\left(\alpha ,x_i\right)e^{w_d\left(y_i;\theta \right)}}.$$ (3)

The weights in (3) are the subject-specific log likelihood ratios w_c (y_i; θ) = log{f_c (y_i; θ)/f₁ (y_i; θ)}, measuring the strength of the evidence in support of the hypothesis that person i belongs to latent class c versus the alternative hypothesis that person i belongs to latent class one. The fully parametric expectation-maximization algorithm consists of alternating between solving the score equations, S(α) = 0 and S(θ) = 0, and updating the subject-specific posterior probabilities of class membership, τ_ic, until a pre-specified convergence criterion is met.

3. The Proposed Latent Trajectory Class Model

Denote the observed longitudinal data by {(y_ijk,t_ijk),i = 1, . . ., n,j = 1, . . ., J,k = 1, . . ., m_ij}, where y_ijk denotes the kth measurement of the jth feature variable for the ith subject and t_ijk denotes the time of the observation. The component means, variances, and temporal correlations of the features can be modeled using class-specific generalized estimating equations with an autoregressive correlation structure (Liang and Zeger, 1986), where for each latent class c = 1, . . ., C

$$E\left(y_{ijk}\mid z_{ic}=1\right)={\mu }_{ijkc}(\beta )=h_j\left({\beta }_{0jc}+t_{ijk}{\beta }_{1jc}\right),\mathrm{var}\left(y_{ijk}\mid z_{ic}=1\right)={\sigma }_{ijkc}^2(\varphi ,\beta )={\varphi }_{jc}{V}_j\left({\mu }_{ijkc}\right),$$ (4)

and corr$\left(y_{ijk},y_{ijl}\mid z_{ic}=1\right)={\rho }_{ijklc}(\gamma )={\gamma }_{jc}^{\left|t_{ijk}-t_{ijl}\right|},k<l$ . All parameters are allowed to be distinct over features and latent classes, h_j (·) is a known inverse link function, and V_j (·) is a known function specifying the mean-variance relationship. We have assumed that time t_ijk enters the regression model of E (y_ijk | z_ic = 1) linearly, but we could easily extend the model to include nonlinear time effects by entering a higher-order polynomial of t_ijk. Moreover, if desired, the model is flexible enough to accommodate additional covariates. As is standard in latent class analysis, we make the local independence assumption that the J features are independent within a given latent class (McLachlan and Peel, 2000). Denote the collection of unknown parameters that summarize the longitudinal trajectories of the feature variables by θ = {(β_0jc, β_1jc, φ_jc, γ_jc),j = 1, . . ., J, c = 1, . . ., C}. Estimation of θ is key to conceptualizing the underlying structure of the population.

We deliberately avoid fully specifying the component densities f_c(y_i;θ), since for non-Gaussian features these would typically require burdensome numerical integration and tenuous assumptions about the third- and higher-order joint moments of the longitudinal observations.

4. Artificial Likelihood-Based Expectation-Solution Algorithm

Briefly assume that the number of latent classes, C, is known. To mimic the expectation-maximization algorithm without requiring the full specification of the component densities f_c (y_i; θ), we make two important changes. First, we can safely replace the component score functions, ∂ log f_c (y_i; θ) /∂θ, in (2) with much simpler component estimating functions g_c (θ; y_i) satisfying E {g_c (θ; y_i) | z_ic = 1} = 0, without compromising the unbiasedness of the resulting estimating function. That is, we define the quasi-score for θ by

$$U(\theta )=\sum _{i=1}^n\sum _{c=1}^C{\tau }_{ic}{g}_c\left(\theta ;y_i\right),$$ (5)

which is unbiased in the sense that

$$E\{U(\theta )\}=\sum _{i=1}^{n}E\left[\sum _{c=1}^{C}E\left\{z_{ic}{g}_c\left(\theta ;y_i\right)\mid y_i,x_i;\alpha ,\theta \right\}\right]=\sum _{i=1}^{n}E\left[\sum _{c=1}^{C}E\left\{z_{ic}{g}_c\left(\theta ;y_i\right)\mid Z_i;\alpha ,\theta \right\}\right]=0.$$

Under mild regularity conditions, the standard theory of estimating functions indicates that our robust estimator of θ, satisfying U (θ) = 0, is consistent and asymptotically normal as n → ∞ (McCullagh and Nelder, 1989, Chapter 9). Natural choices of the components g_c (θ;y_i) are orthogonal generalized estimating equations (Hardin and Hilbe, 2003), where

$$g_c\left(\theta ;y_i\right)=\left(\begin{array}{l}\frac{\partial {\mu }_{ic}{(\beta )}^T}{\partial \beta }{V}_{ic}^{-1}(\theta )\left\{y_i-{\mu }_{ic}(\beta )\right\}\hfill \\ \frac{\partial {\sigma }_{ic}^2{(\varphi ,\beta )}^T}{\partial \varphi }\left\{s_{ic}^2-{\sigma }_{ic}^2(\varphi ,\beta )\right\}\hfill \\ \frac{\partial {\eta }_{ic}{(\gamma ,\varphi ,\beta )}^T}{\partial \gamma }\left\{r_{ic}-{\eta }_{ic}(\gamma ,\varphi ,\beta )\right\}\hfill \end{array}\right)(c=1,\dots ,C).$$ (6)

To form the column vectors in (6), we have stacked all the relevant quantities, including the squared residuals s_ijkc = {y_ijk − μ_ijkc (β)}², the cross-products of the residuals r_ijklc = {y_ijk − μ_ijkc (β)}{y_ijl − μ_ijlc (β)}, and the covariances

$${\eta }_{ijklc}=E\left(r_{ijklc}\right)={\rho }_{ijklc}(\gamma ){\varphi }_{jc}{V}_j{\left\{{\mu }_{ijkc}(\beta )\right\}}^{1/2}{V}_j{\left\{{\mu }_{ijlc}(\beta )\right\}}^{1/2},k<l$$

The component variance-covariance matrices V_ic = var(y_i | z_ic = 1) appearing in (6) are block diagonal and thus easily invertible.

Rosen et al. (2000) similarly considered embedding generalized estimating equation components into a finite mixture model. Their model, however, relied critically on a series of predictions of latent class membership using only the marginal information at each specific time point, and did not provide probabilities of latent class membership based on the longitudinal trajectories, as given by τ_ic, which are often of key scientific interest.

Thus, our key insight concerns the posterior probabilities of class membership, τ_ic, which are specified by (3) and play an essential role both in recovering the unknown latent class memberships and in achieving a quasi-score (5) that is unbiased. We can achieve simple and robust predictions of these probabilities

$${\tilde{\tau }}_{ic}={\tilde{\tau }}_{ic}\left(\alpha ,\theta ;y_i,x_i\right)=\frac{p_c\left(\alpha ,x_i\right)e^{{\tilde{w}}_c\left(y_i;\theta \right)}}{{\sum }_{d=1}^C{p}_d\left(\alpha ,x_i\right)e^{{\tilde{w}}_d\left(y_i;\theta \right)}},(c=1,\dots ,C),$$ (7)

where the ${\tilde{w}}_c\left(y_i;\theta \right)$ are approximations of the log likelihood ratios w_c based on the generalized estimating equation model (4). A simple approximation of w_c (y_i; θ) is the linear projection of the log likelihood ratio (Li, 1993), which in our latent class context takes the form

$${\tilde{w}}_c\left(y_i;\theta \right)=\frac{1}{2}{\left({\mu }_{ic}-{\mu }_{i1}\right)}^T\left\{V_{ic}^{-1}\left(y_i-{\mu }_{ic}\right)+V_{i1}^{-1}\left(y_i-{\mu }_{i1}\right)\right\},(c=1,\dots ,C).$$ (8)

Given the current estimates of α and θ, the expectation step of our Expectation-Solution algorithm consists of updating the predicted class membership probabilities ${\tilde{\tau }}_{ic}$ using (7). In the solution step, we treat the current values of ${\tilde{\tau }}_{ic}$ as fixed and find the value of (α, θ) that jointly solves

$$\tilde{S}(\alpha )=\sum _{i=1}^n\sum _{c=1}^C{\tilde{\tau }}_{ic}\frac{\partial \log p_c\left(\alpha ,x_i\right)}{\partial \alpha }=0,\tilde{U}(\theta )=\sum _{i=1}^n\sum _{c=1}^C{\tilde{\tau }}_{ic}{g}_c\left(\theta ;y_i\right)=0.$$

Standard errors of the fitted parameters are available using the standard theory of estimating functions (McCullagh and Nelder, 1989, Chapter 9). See Web Appendix A for details about the asymptotic variance matrix of the estimator of the parameters.

As is typically the case with finite mixture models, the proposed latent trajectory model is not identifiable, since the latent classes lack a natural ordering. In practice, this lack of identifiability is not generally of concern under the expectation-solution approach to inference (McLachlan and Peel, 2000). If necessary, an order can be imposed on the latent classes based on the average of their relative frequencies, ${\sum }_i{p}_1\left(\alpha ,x_i\right)/n⩾\dots ⩾{\sum }_i{p}_C\left(\alpha ,x_i\right)/n.$ .

The proposed Expectation-Solution algorithm is implemented as macros in the statistical environment SAS and the SLTCA package in R software. Discussion of initialization of the algorithm, stopping criteria, and convergence properties is available in Web Appendix B.

5. Selecting the Number of Latent Classes

An important issue is determining the appropriate number of latent classes, C, using objective criteria. Even when fully parametric models are used, the assumptions underlying standard model selection criteria, including the Akaike information criterion and the Bayesian information criterion, do not hold in latent class analysis owing to the non-standard parameter space. Naive use of these criteria typically leads to over-estimation of the number of latent classes, and it is recommended that these criteria be augmented with a penalty for the entropy of the fuzzy classification matrix with elements $\left\{{\tilde{\tau }}_{ic}\right\}$ (McLachlan and Peel, 2000; Celeux and Soromenho, 1996). Since we wish to avoid fully parametric methods, we propose an analogue of this augmented criterion based on artificial likelihood concepts.

The starting point for our approach is the extended quasi-likelihood information criterion, which is intended for use with first-order generalized estimating equations (Wang and Hin, 2010). In our context, for a specific longitudinal feature and latent class, it is given by ${\text{EQIC}}_{jc}=-2Q_{jc}^{+}+p\log n$, where p is the number of estimated parameters and $Q_{jc}^{+}$ refers to the extended quasi-likelihood under a working independence model, i.e.,

$$Q_{jc}^{+}=\sum _{i=1}^n\sum _{k=1}^{m_{ij}}{Q}_{jc;ik}^{+}=\sum _{i=1}^n\sum _{k=1}^{m_{ij}}\left[Q_{jc}\left(y_{ijk},\widehat{\theta }\right)-\frac{1}{2}\log \left\{{\widehat{\varphi }}_{jc}{V}_j\left({\widehat{\mu }}_{ijkc}\right)\right\}\right].$$

We adapt the extended quasi-likelihood information criterion to our finite mixture model setting as follows. First, we incorporate the essential role of τ̃_ic to predict the probability that subject i belongs to class c. Second, we sum over all features and latent classes. Third, we add a penalty for the entropy of the fuzzy classification matrix. These changes result in our new criterion, which we call the classification entropy extended quasi-likelihood information criterion, given by

$$\text{CE}-\text{EQIC}=\sum _{c=1}^C\sum _{j=1}^J-2Q_{jc}^{\ast }+p\log n-2\sum _{i=1}^n\sum _{c=1}^C{\tilde{\tau }}_{ic}\log {\tilde{\tau }}_{ic},$$ (9)

where $Q_{jc}^{**}={\Sigma }_{i=1}^n{\Sigma }_{k=1}^{m_j}{\tilde{\tau }}_{ic}{Q}_{jc;ik}^{+}$. We select the number of classes C so as to minimize the CE-EQIC. In situations where our latent class approach yields multiple roots for a given specification of C, the CE-EQIC can also be used to select the best root.

Several variations of the above model selection criterion are also possible, including:

1. a cross-validation approach, which may be used to allow the scale parameter ø_jc and the quasi-likelihood Q_jc to be estimated independently of one another (Dietterich, 1998);

2. the Bayesian information criterion type penalty for model complexity, namely plogn, may be replaced by a penalty for generalized estimating equation models considered by Pan (2001), $2{\Sigma }_c{\Sigma }_{j}tr{R}_{jc}{M}_{jc}^{-1}$, where R_jc and M_jc are the class- and feature-specific robust and model-based variance matrix estimates of the parameters; and

3. either an empirical likelihood criterion (Hong et al., 2003) or a quadratic inference function criterion (Wang and Qu, 2009) may be used in place of $Q_{jc}^{+}$ .

As in the fully-parametric setting, the above measures are useful objective guides, but in practice it is also often recommended to incorporate scientific judgment when determining the number of latent classes.

6. Results

6.1. Scalability

Unlike traditional methods of latent trajectory class analysis, our artificial likelihood approach simplifies computations by avoiding numerical integration of non-Gaussian features measured longitudinally. Moreover, our approach requires matrix inversions of only the component variance-covariance matrices V_ic = var(y_i | z_ic = 1) appearing in (6) and (8), which are block diagonal and thus easily invertible.

We compared the computational efficiency of the conventional full-likelihood approach with our artificial-likelihood method when there were n = 10000 subjects and J = 100 longitudinal features. We generated longitudinal binary observations under a logistic regression mixed-effects model with subject-, feature- and class-specific random slopes and intercepts, four equally-spaced time points, and C = 2 latent classes. This model was intended to be favorable to a full-likelihood based nonlinear mixed-effects modeling approach. To ensure comparability of results, both the full-likelihood software and the artificial-likelihood software were written as macros in the statistical environment SAS, both were run on the same computing platform, both were initialized at the same starting point in the parameter space, and both used the same convergence criteria in the iterative algorithm to estimate the parameters. The full-likelihood code relied upon built-in routines in SAS for numerical integration and nonlinear maximization via a dual quasi-Newton algorithm. We optimized the full-likelihood code, but not the artificial-likelihood code, to exploit the fact that each of the 100 features had 16 unique patterns of longitudinal observations and hence it was possible to perform a computationally more efficient grouped-data analysis. Without this optimization, or if the data were ungrouped, the full-likelihood method would have been much slower. Even with this optimization, fitting a correctly specified random slopes and intercepts model using the full-likelihood method took 4185.1 CPU minutes, 200 times slower than the artificial-likelihood method, which took 20.5 CPU minutes. Fitting a random intercepts model, in which the slopes were regarded as non-random for computational convenience, the full-likelihood method (425.2 CPU minutes) remained 20 times slower than the artificial-likelihood approach.

6.2. Selecting the number of latent classes

We assessed the performance of our model selection criterion by generating data under a two-class model and comparing the fits of two- and three-class models. We generated data under a trivial version of the latent polytomous logistic regression model (1), where x_i = (1, x_i2)^T and x_i2 was a randomly assigned binary covariate with a corresponding regression coefficient of zero. The relative frequencies of the two classes were 67% and 33%. For n = 500 subjects, we generated J = 6 longitudinal features at six evenly spaced timepoints, including two variables that were dependent Gaussian, two variables that were dependent Poisson using the data generation approach of Dalthorp and Madsen (2007), and two that were dependent binary using the approach of Qaqish (2003). The specified parameter values are shown in Table 1. In order to evaluate the reliability of the model selection criterion under different levels of heterogeneity between the two classes, we considered four scenarios with different specifications for the slope of Feature 6 in Class 2. In Scenario 1, the slope was 5, leading to the largest separation between the two classes. The slope in Scenario 2 to 4 was 3.75, 2.5 and 1.25, respectively, leading to increasingly more overlapped latent class patterns.

Table 1.

Class-specific intercepts and slopes of six longitudinal features generated under an autoregressive correlation structure with a correlation coefficient of 0.3. Distribution refers to time- and class-specific conditional distribution. Gaussian features had standard deviation σ_ijkc = 5.

			Class 1		Class 2
Scenarios	Feature	Distribution	Intercept	Slope	Intercept	Slope
1,2,3,4	Feature 1	Poisson	0–7	1–0	0–7	0–0
1,2,3,4	Feature 2	Poisson	3–0	0–0	3–0	−1–0
1,2,3,4	Feature 3	Binary	−0–5	1–0	−0–5	0–0
1,2,3,4	Feature 4	Binary	0–5	0–0	0–5	−1–0
1,2,3,4	Feature 5	Normal	20–0	0–0	20–0	0–0
1	Feature 6	Normal	5–0	0–0	5–0	5–0
2	Feature 6	Normal	5–0	0–0	5–0	3–75
3	Feature 6	Normal	5–0	0–0	5–0	2–5
4	Feature 6	Normal	5–0	0–0	5–0	1–25

We compared our augmented criterion CE-EQIC to a BIC-type version that failed to take into account the entropy of the fuzzy classification matrix, $EQI{C}^{**}={\Sigma }_{c=1}^C{\Sigma }_{j=1}^J-2Q_{jc}^{**}+$ p log n, as well as a version that used the Akaike-type penalty for model complexity $EQI{C}^{\text{n}}={\Sigma }_{c=1}^C{\Sigma }_{j=1}^J-2Q_{jc}^{**}+2p$. Table 2 shows that, as expected, CE-EQIC performed better than EQIC^∗ and EQIC^† in each scenario, selecting the correct number of classes in over 80% of samples. A cross-validation approach that allowed the scale parameter φ_jc and the quasi-likelihood Q_jc to be estimated independently of one another did not improve the model selection performance (data not shown). More comprehensive simulation studies will be reported elsewhere.

Table 2.

Number of latent classes selected in 500 simulated latent trajectory class analyses.

Scenario	Number of Classes	EQIC†	EQIC*	CE-EQIC
1	2	403	4ll	433
	3	97	89	67
2	2	383	390	405
	3	117	110	95
3	2	373	381	417
	3	127	119	83
4	2	382	388	426
	3	118	112	74

6.3. Estimation of parameters

Under each Scenario 1–4, we regarded as non-convergent the result for any simulated sample when the estimated covariate effect, that is, the element of α̂₂ that was not an intercept, exceeded three times the mean absolute deviation from the median value of the covariate effect among 500 simulations. In the progression of scenarios where the two classes became more overlapped, the convergence rate of the algorithm decreased from 97% to 82%. The performance of the converged solutions, however, was maintained at similarly high level across the four scenarios. Results for the most challenging scenario (Scenario 4) considered, where there was the least separation among classes, indicated that our method yielded estimates with small bias and confidence intervals with valid coverage probabilities (Table 3). Supplementary Tables D.1-D.3 in Web Appendix D present similar results for the less challenging scenarios considered.

Table 3.

Results of estimated parameters in 500 simulated latent trajectory class analyses under Scenario 4. The autoregressive correlation parameters are not shown for brevity. M.Bias, median bias; SE, empirical standard error; ASE, average of the analytical standard errors; C.Prob, coverage rate of the 95% confidence interval.

Convergence rate = 0.820 (90 non-convergence out of 500 simulations)

	Class 2 intercept					Class 2 covariate effect
	Bias	M . Bias	SE	ASE	C.Prob	Bias	M.Bias	SE	ASE	C.Prob
${\widehat{\alpha }}_2$	−0.044	−0.032	0.147	0.135	0.915	0.027	0.014	0.199	0.191	0.927
	Class 1					Class 2
	Bias	M.Bias	SE	ASE	C.Prob	Bias	M.Bias	SE	ASE	C.Prob
	−0.004	−0.004	0.037	0.036	0.959	0.000	0.000	0.035	0.033	0.932
${\widehat{\beta }}_0$	−0.001	−0.001	0.015	0.015	0.959	−0.000	−0.001	0.011	0.012	0.966
	0.000	0.009	0.134	0.138	0.961	0.004	0.005	0.097	0.096	0.944
	−0.005	−0.005	0.135	0.136	0.954	0.016	0.016	0.096	0.098	0.956
	−0.010	−0.007	0.312	0.331	0.959	−0.003	−0.009	0.233	0.232	0.956
	0.007	0.002	0.332	0.332	0.949	−0.008	−0.005	0.242	0.233	0.946
	Class 1					Class 2
	Bias	M.Bias	SE	ASE	C.Prob	Bias	M.Bias	SE	ASE	C.Prob
${\widehat{\beta }}_1$	−0.002	−0.002	0.020	0.018	0.961	0.000	0.001	0.025	0.028	0.968
	−0.004	−0.002	0.018	0.013	0.968	0.002	0.003	0.014	0.019	0.980
	−0.005	−0.005	0.112	0.112	0.951	−0.004	−0.002	0.060	0.061	0.959
	0.002	0.007	0.091	0.090	0.954	−0.013	−0.007	0.072	0.073	0.946
	−0.002	−0.005	0.205	0.210	0.959	0.002	0.004	0.155	0.148	0.932
	0.013	0.012	0.208	0.213	0.963	0.008	0.017	0.143	0.148	0.956

6.4. Estimation performance compared to a full-likelihood approach

We compared the proposed approach and a publicly available full-likelihood method, lcmm (Proust-Lima et al., 2017), to demonstrate the efficiency and robustness of our method. To facilitate comparisons between lcmm and our approach, we considered a simple scenario using only the two longitudinal normal features from Scenario 1 Table 1 in The results indicated that our approach yielded inferences that were comparable, in terms of bias and mean squared error, to the full likelihood approach (Table 4) By contrast, under a more complicated scenario with six longitudinal normal features, where the modeling framework underlying lcmm did not strictly apply, as discussed in Web Appendix C, our approach outperformed lcmm in terms of lower bias and smaller mean squared error (Supplementary Tables D.4 and D.5 in Web Appendix D).

Table 4.

Bias (mean squared error) of estimated parameters for the proposed method and lcmm in 500 simulated latent trajectory class analyses, under the scenario using only the two longitudinal normal features from Scenario 1 in Table 1.

Convergence rate for the proposed method is 1

	Class 2 intercept		Class 2 covariate effect
α̂2	Proposed method	lcmm	Proposed method	lcmm
	−0.006 (0.031)	−0.001 (0.019)	0.006 (0.038)	0.006 (0.038)
	Class 1		Class 2
	Proposed method	lcmm	Proposed method	lcmm
β̂0	0.002 (0.021)	−0.074 (0.009)	−0.001 (0.010)	−0.006 (0.009)
	−0.003 (0.020)	−0.001 (0.020)	−0.072 (0.005)	0.006 (0.011)
	Class 1		Class 2
	Proposed method	lcmm	Proposed method	lcmm
β̂1	−0.000 (0.008)	0.000 (< 0.001)	0.001 (0.005)	−0.088 (0.010)
	0.013 (0.034)	−0.002 (0.008)	0.001 (0.006)	0.003 (0.004)

6.5. Example

We explored the heterogeneity of mild cognitive impairment using trajectories of clinical features collected approximately annually on 6,034 participants in the Uniform Data Set as of the June 2015 data freeze by the U.S. National Alzheimer’s Coordinating Center. Generalized estimating equations with canonical links (6) were specified for each feature, including a binary measure of depression, two count variables for the number of impairments in instrumental activities of daily living and the number of neuropsychiatric symptoms, and ten continuous variables for normalized cognitive test scores. For normed test scores, a score of −1.5, say, indicated cognitive performance 1.5 standard deviations worse than a cognitively normal person of the same age, education level and race. Since the mean follow-up period was only 2.6 years, we considered only the linear effects of time in the trajectory model (6). We entered two baseline covariates thought to be associated with cognitive decline, namely decades of smoking and elevated Hachinski score (an indicator of probable cerebrovascular disease), in the latent polytomous logistic regression model (1). Following the recommendation by Weuve et al. (2012), we adjusted the trajectory analysis for selective attrition by including stabilized inverse probability of attrition weights at each visit based on our two baseline covariates as well as age, sex, race, and overall cognitive performance at baseline. A detailed description of this dataset is given in Hanfelt et al. (2018).

We fitted models with up to five classes. Given a fixed number of latent classes, we explored whether there were multiple solutions by initializing our Expectation-Solution algorithm from 100 random starting points. As shown in Supplementary Table D.6, our objective model selection criteria favored a four-class solution. Specifically, our four-class model separated participants into the following four groups: a benign amnestic group (7% of the participants) with subtle impairment in memory at baseline and improvement in all features over time; a benign non-amnestic group (13% of the participants) with subtle impairment in executive function at baseline, as well as elevated rate of depression, and improvement in all features over time; a mildly progressive amnestic multi-domain group (43% of the participants) with functional impairment and neuropsychiatric features; and a rapidly progressive amnestic multi-domain group (38% of the participants) with functional impairment and neuropsychiatric features (Figure 1). The trajectories of all 13 longitudinal features used for the analysis are shown in Figure 1. Elevated Hachinski score at baseline, but not decades of smoking, was associated with the latent classes, and led to an especially high risk of belonging to the rapidly progressive amnestic multi-domain class (Table 5). In future research, we will conduct a more comprehensive analysis of mild cognitive impairment that, in addition to clinical features, incorporates the trajectories of a large collection of biomarkers, many of which are non-Gaussian. Moreover, with longer follow-up we will consider nonlinear effects of time in model (6).

Figure 1.

Latent trajectory classes of mild cognitive impairment. IADLs: instrumental activities of daily living. MD: multidomain. The cognitive test scores were standardized based on age, education and race. Higher scores on Trails A and B indicated worse performance.

Table 5.

Estimated odds ratios (95 % confidence intervals) for the effects of baseline covariates.

	Benign, Amnestic	Benign, Non-Amnestic	Mildly Progressive, Amnestic Multi-Domain	Rapidly Progressive, Amnestic Multi-Domain
Elevated Hachinski	1	5.57 (1.69, 18.39)	5.74 (1.87, 17.59)	12.70 (3.21, 50.31)
Smoking (decades)	1	1.03 (0.96, 1.12)	1.12 (0.99, 1.26)	1.02 (0.91, 1.15)

7. Discussion

As seen in (8), the projected log likelihood ratio approximation ${\tilde{w}}_c\left(y_i;\theta \right)$ implemented in this paper depends on the choice of reference class and requires correct modeling of the component variance-covariance matrices V_ic in the GEE model. A potential alternative that avoids these limitations is to use empirical likelihood (Qin and Lawless, 1994) components given by

$$Q_c(\theta )=\underset{q_{1c},\dots ,q_{nc}}{\max }\left\{\prod _{i=1}^n{q}_{ic}:\sum _{i=1}^n{q}_{ic}{g}_c\left(y_i;\theta \right)=0,\sum _{i=1}^n{q}_{ic}=1,q_{ic}⩾0\right\},(c=1,\dots ,C).$$

Empirical likelihood ordinarily is used to compare hypotheses about model parameters θ, but under the context of latent class analysis, for any given value of θ we can assess the evidence in favor of the hypothesis that participant i belongs to latent class c rather than latent class 1 by

$${\check{w}}_c\left(y_i;\theta \right)=\log \left\{\frac{q_{ic}(\theta )/q_{ic}\left({\theta }_0\right)}{q_{i1}(\theta )/q_{i1}\left({\theta }_0\right)}\right\}=\log \left\{\frac{q_{ic}(\theta )}{q_{i1}(\theta )}\right\},(c=1,\dots ,C),$$

where θ₀ denotes the null hypothesis that all classes have the same longitudinal trajectories. This non-standard use of empirical likelihood might require special software to generate the individual empirical probabilities q_ic(θ). If the collection of longitudinal features is large, then the proposed approximation ${\tilde{w}}_c\left(y_i;\theta \right)$ is preferred to the empirical likelihood approximation ${\tilde{w}}_c\left(y_i;\theta \right)$, owing to the former’s computational simplicity and ability to distinguish among multiple roots (Small and Wang, 2003; Hanfelt and Liang, 1995; Li, 1993). When the number of longitudinal features is small, however, empirical likelihood might be considered. We will study the empirical likelihood approach in future research.

Supplemental methods might also enhance scalability. For example, in latent class applications where the estimation step is slower than the solution step, iterating between partial estimation steps that are limited to small blocks of observations and full solution steps can reduce the time to convergence by up to 50% (McLachlan and Ng, 2003). For datasets that are too massive to analyze locally, a one-pass method could be used. At each increment, we could randomly sample a chunk of observations small enough to work with locally, apply our latent class analysis, and compress the data accordingly (Bradley et al., 2000).

We have adopted the popular local independence assumption. This assumption can be relaxed for a subset of longitudinal features that are conditionally dependent given a latent class by replacing the simpler generalized estimating equation in (6) with a class-specific second-order generalized estimating equation that models the association between multiple features (Liang et al., 1992; Prentice, 1988). A future direction of research is to develop detailed assessments of model fit, including the detection of outliers and departures from model assumptions, along the lines of the pseudoclass residual diagnostics of Wang et al. (2005).

We have assumed implicitly that any missing follow-up information is missing completely at random within each latent class. For applications in which the missing completely at random assumption is untenable, such as in our mild cognitive impairment example, the proposed approach can be extended by incorporating inverse probability weights for the longitudinal features, as considered by Weuve et al. (2012).

Although we did not encounter any difficulties with multiple solutions in our brief simulation studies, in general the solution to our expectation-solution algorithm might not be unique. A recommended practice is to initialize the algorithm at multiple random starting values and to choose the solution that minimizes an objective model selection criterion (McLachlan and Peel, 2000), such as our proposed criterion ce-eqic. In future research, we will further investigate the performance of criteria to select the correct solution.