BSMHM2: Bayesian Semiparametric Mixed Hidden Markov Models for Delineating the Pathology of Alzheimer’s Disease

Abstract

Alzheimer’s disease (AD) is a firmly incurable and progressive disease. The pathology of AD usually evolves from cognitive normal (CN), to mild cognitive impairment (MCI), to AD. The aim of this paper is to develop a Bayesian semiparametric mixed hidden Markov modeling (BSMHM2) framework to characterize disease pathology, identify hidden states corresponding to the diagnosed stages of cognitive decline, and examine the dynamic changes of potential risk factors associated with the CN-MCI-AD transition. The BSMHM2 framework consists of two major components. The first one is a state-dependent semiparametric regression for delineating the complex associations between clinical outcomes of interest and a set of prognostic biomarkers across neurodegenerative states. The second one is a parametric transition model, while accounting for potential covariate effects on the cross-state transition. The inter-individual and inter-process differences are taken into account via correlated random effects in both components. Based on the Alzheimer’s Disease Neuroimaging Initiative dataset, we are able to identify four states of AD pathology, corresponding to common diagnosed cognitive decline stages, including CN, early MCI, late MCI, and AD and examine the effects of hippocampus, age, gender, and APOE-ε4 on degeneration of cognitive function across the four cognitive states.

1 Introduction

Alzheimer’s disease (AD) is a chronic neurodegenerative disease that usually starts slowly and worsens over time. The most common early symptom of AD is short-term memory loss, also referred to mild cognitive impairment (MCI). Patients at MCI state have high likelihood to transit to dementia or AD within a few years (Albert et al., 2011). Despite an increasing attention to its growing public threat, the cause of AD remains poorly understood. Thus, it is great interest to discovering or validating prognostic biomarkers that may identify subjects at great risk for future cognitive decline and investigating the functional effects of various biomarkers on the conversion from NC to AD.

The Alzheimer’s Disease Neuroimaging Initiative (ADNI) study began in 2004 and it collected imaging, generic, clinical, and cognitive data from subjects under cognitive normal (CN) controls and subjects with MCI or AD in order to delineate the complex associations among various characteristics of the clinical spectrum of AD. The ADNI-1 recruited approximately 800 subjects according to its initial aim and has been extended by three follow-up studies, namely, ADNI-GO, ADNI-2, and ADNI-3. ANDI-1 subjects had an option to refuse follow-up monitoring in subsequent studies. More information on ADNI can be obtained in the official website (www.adni-info.org). Functional assessment questionnaire (FAQ), an assessment of abilities to function independently in daily life, is widely used to monitor the decline of cognitive ability over time. The FAQ scores of each subject were obtained at baseline and then every 6 months across 9 years multiple study phases. For this longitudinal study, several central questions are naturally raised as follows:

• (i) How many hidden pathophysiological states exist in the progression of AD?

• (ii) Which factors should contribute to the neuro-degenerative pathology from one state (e.g., MCI) to another (e.g., AD)?

• (iii) Whether the identified risk factors are equally good predictors of cognitive decline at each state?

Given these questions, there is a particular need for the development of statistical models that delineate cognitive decline in terms of the pathophysiological states of AD.

Hidden Markov models (HMM) are well suited to the characterization of longitudinal data in terms of a set of hidden states (Cappé et al., 2005; Maruotti, 2011; Bartolucci et al., 2013). HMMs consist of two components: a transition model to describe the dynamic transition of hidden states and a conditional regression model to examine state-specific covariate effects on responses. Owing to their ability to simultaneously reveal the longitudinal association structure and dynamic heterogeneity of the observed process, HMMs and their variants have attracted significant attention from the medical, behavioral, social, environmental, and psychological sciences (Vermunt et al., 1999; Scott et al., 2005; Schmittmann et al., 2005; Bartolucci and Farcomeni, 2009; Bartolucci et al., 2013; Chow et al., 2013). In particular, HMMs have previously been applied to investigate diseases progression to identify latent pathophysiological states. For instance, Albert et al. (1994) used HMMs to analyze multiple sclerosis disease across relapse and remission states (see also Altman and Petkau, 2005; Altman, 2007). Ip et al. (2013) identified ten disable states on the basis of a 10-year follow-up study of late-life disability in elder adults, and examined the patterns and risk factors for transition among disable states. Song et al. (2016) revealed the dynamic change of treatment effectiveness in preventing cocaine use across three cocaine addiction states.

Despite the rapid development and wide applications of HMMs, existing literature has mainly focused on parametric HMMs, in which the forms of covariate effects on responses and on transition probabilities are pre-specified. One problem of parametric models is that they may be too restrictive to reflect correctly the reality because the complex relationships among variables are seldom known a priori, and a pre-specified parametric form tends to overlook the subtle pattern of a function. A more comprehensive analysis can be performed by incorporating nonparametric functions into HMMs so that the functional effects of interest can be discovered. To the best of our knowledge, however, such nonparametric modeling has not been introduced into the HMM framework.

In this study, we propose a Bayesian mixed semiparametric hidden Markov modeling (BMSHM2) framework to analyze the ADNI-I dataset. Similar to conventional HMMs, the proposed model consists of two major components. The first component is a state-dependent semiparametric regression to investigate the linear and nonlinear effects of covariates, such as hippocampus, age, gender, and APOE-ε4, on the clinical outcome of cognitive decline (e.g., FAQ score). The second component is a mixed continuation-ratio logit transition model to examine various covariate effects on the probabilities of transitioning among neurodenerative states. We introduce a random effect in both models in order to account for inter-individual differences and allow the random effects to be dependent by assigning a joint distribution for them. Such joint random effects enable the model to accommodate the situation where some omitted factors influence both the observed process and the hidden transition process (Wulfsohn and Tsiatis, 1997; Chi and Ibrahim, 2006). We develop a full Bayesian approach, along with Bayesian P-splines procedure and Markov chain Monte Carlo (MCMC) methods, for statistical inference. As far as we know, no previous study has ever been conducted either on the proposed BMSHM2 or on Bayesian HMMs. Also, this paper is the first to investigate the neurodegenerative pathology of AD.

Section 2 defines BMSHM2 and discusses the related identifiability issues. Section 3 introduces the Bayesian inference procedure. Section 4 illustrates the use of BMSHM2 in the analysis of the ADNI dataset. Section 5 demonstrates the empirical performance of the proposed methodology through a simulation study. Section 6 discusses the findings obtained from the analysis of the ADNI dataset. Technical details are provided in the Appendix.

2 Model description

2.1 Questions of Interest for ADNI-1

Data used in this article were obtained from the ADNI-1 database launched in 2003. A total of n = 633 patients at baseline, 6 months, 12 months, and 24 months (t = 1, …, 4) were considered in the analysis. We use the score of FAQ, denoted by y_it, to characterize the cognitive function of subject i at occasion t. Moreover, we observe a r × 1 vector of discrete covariates, denoted by b_it = (b_it,1, …, b_it,r)^T, and a q × 1 vector of continuous covariates, denoted by x_it = (x_it,1, …, x_it,q)^T. The covariates of interest include gender (1 = male; 0 = female), apolipoprotein E-ε4 (APOE-ε4), hippocampus, and age at baseline, in which APOE-ε4 is a known genetic risk factor for AD and is coded as 0, 1, and 2, denoting the number of APOE-ε4 alleles, and hippocampal volume is divided by whole brain volume to account for the confounding effect of brain size. Thus, APOE-ε4 (=1, b_it,1), APOE-ε4 (=2, b_it,2), and gender (b_it,3) are discrete, whereas hippocampus (x_it,1) and age at baseline (x_it,2) are continuous.

Several kinds of dependencies/heterogeneities are worthy of investigation. The first one is the dynamic heterogeneity across different groups. Figure 1 plots the individual trajectories of FAQ scores for 20 randomly selected samples, who were initially diagnosed as CN, MCI, and AD, respectively, at baseline. The cognitive decline patterns are apparently distinct over the groups, suggesting at least three (and probably more) distinct neurodegenerative states existent underlying the observations of FAQ score. The second one is the dependency of FAQ score on potential covariates, such as hippocampus, age at baseline, APOE-ε4, and gender. The third one is the serial dependency of the longitudinal observations, owing to relative persistence of neurodegenerative states. The last one is the heterogeneity caused by the existence of some omitted clinical or genetic indicators that influence both cognitive decline and its underlying transition. The BMSHM2 described below perfectly accommodates all these features.

Figure 1.

ADNI-1 data analysis results: individual trajectories of FAQ scores for 20 randomly selective samples whose baseline states are CN, MCI, and AD, respectively.

2.2 Model Setup

The BMSHM2 consists of two major components, including a conditional seminparametric regression model and a continuation-ratio logit transition model, as detailed below.

2.2.1 Conditional semiparametric regression model

Let y_it with subject i = 1, …, n at t = 1, …, T be the observation process. The hidden state process, Z_it, which takes values in {1, …, S}, is assumed to follow a first-order Markov chain. Given the hidden state Z_it = s, the conditional semiparametric regression model is defined as follows:

$$[y_{\mathit{\text{it}}}|Z_{\mathit{\text{it}}}=s]={\mu }_s+{\mathit{\gamma }}_s^T{\mathbf{b}}_{\mathit{\text{it}}}+\sum _{j=1}^q{f}_{\mathit{\text{sj}}}(x_{\mathit{\text{it}},j})+w_{i1}+{\delta }_{\mathit{\text{it}}},$$ (1)

where μ_s is a state-specific intercept, γ_s = (γ₁, …, γ_r) is a state-specific vector of fixed effect of discrete covariates, f_sj(·)s are state-specific unknown smoothing functions, b_it = (b_it,1, …, b_it,r)^T and x_it = (x_it,1, …, x_it,q)^T are r × 1 vector of discrete covariates and q × 1 vector of continuous covariates respectively, w_i1 is a subject-specific random effect, δ_it is a random residual independent of y_it, and [δ_it|Z_it = s] ~ N[0, ψ_s].

The conditional model defined by (1) extends the parametric regression to allow the additive nonparametric functions of covariates, so that the functional effects of interest can be discovered. Such nonparametric modeling provides great flexibility in fitting nonlinear effects whose forms need not be specified a priori. When used as an exploratory tool, the proposed model is able to help users to visually examine and interpret the functional effects of potential predictors on the response of interest. Moreover, the subject-specific random effect w_i1 permits additional dependencies elicited from other sources and thus avoids a large number of hidden states caused by possible residual correlation among responses.

2.2.2 Continuation-ratio logit transition model

Let p_itus denote the transition probability from state Z_i,t−1 = u at occasion t − 1 to state Z_it = s at occasion t for individual i. Based on the assumption of the first-order Markov chain, we have

$$p_{\mathit{\text{itus}}}=P(Z_{\mathit{\text{it}}}=s|Z_{i1},Z_{i2},\dots ,Z_{i,t-1}=u)=P(Z_{\mathit{\text{it}}}=s|Z_{i,t-1}=u).$$ (2)

The initial distribution of Z_i1 is assumed to be a multinomial with probabilities (τ₁, …, τ_S)^T such that τ_s ≥ 0 and ${\sum }_{s=1}^S{\tau }_s=1$. The distribution of ${\{Z_{\mathit{\text{it}}}\}}_{t=1}^T$ is then fully determined by the transition probabilities and the distribution of the initial state.

In the study of disease progression, the hidden states can often be naturally ranked (e.g., CN, MCI, and AD can be ranked from the best to the worst cognitive condition). Thus, we assume that the hidden states {1, …, S} are ordered and ϑ_itus = P(Z_it = s|Z_it ≥ s, Z_i,t−1 = u). Then, the transition across the ordered states can be described by continuation logits as follows: For t = 2, …, T, s = 1, …, S − 1, and u = 1, …, S,

$$\text{log }\left(\frac{P(Z_{\mathit{\text{it}}}=s|Z_{i,t-1}=u)}{P(Z_{\mathit{\text{it}}}>s|Z_{i,t-1}=u)}\right)=\text{log }\left(\frac{p_{\mathit{\text{itus}}}}{p_{\mathit{\text{itu}},s+1}+\dots +p_{\mathit{\text{ituS}}}}\right)=\text{logit}({\vartheta }_{\mathit{\text{itus}}}).$$ (3)

The parameterization in (3) is intended to facilitate the interpretation of transition to a state rather than to a better one. To examine the effects of potential predictors on the transition probabilities, we consider a continuation-ratio logit transition model as follows:

$$\text{logit}({\vartheta }_{\mathit{\text{itus}}})={\zeta }_{\mathit{\text{us}}}+{\mathit{\alpha }}^T{\mathbf{d}}_{\mathit{\text{it}}}+w_{i2},$$ (4)

where ζ_us is a state-specific intercept, ${\mathbf{d}}_{\mathit{\text{it}}}={({\mathbf{x}}_{\mathit{\text{it}}}^T,{\mathbf{b}}_{\mathit{\text{it}}}^T)}^T$ is the vector of covariates defined in (1), α is a (q + r) × 1 vector of regression coefficients that can be interpreted as conditional log odds ratios in a logistic regression, w_i2 is a subject-specific random effect that is distinct from but correlated with w_i1, and w_i = (w_i1, w_i2)^T is assumed to follow a multivariate normal distribution N(0, Φ). Similar to the proportional assumption in a cumulative logit model, α in (4) is assumed to be independent of u and s in order to maintain the order of the hidden states and avoid a tedious transition model, in which every transition elicits a set of parameters for all possible states of origination and destination. This outcome, in turn, greatly reduces the complexity and enhances the interpretability of the transition model.

Notably, random effects w_i1 and w_i2 play different roles in the conditional and transition models. While w_i1 in conditional model (1) relaxes the assumption that observations {y_it; i = 1, …, n, t = 1, …, T} are conditionally independent given the hidden state Z_it = s, w_i2 in transitional model (4) releases the assumption that hidden process Z_it is Markovian. Unlike the existing literature that usually treats w_i1 and w_i2 separately, we accommodate their possible correlation by assigning a joint distribution for w_i = (w_i1, w_i2)^T. Consequently, the possible correlation between the heterogeneities existent within the two stochastic processes can be appropriately addressed and examined through the covariance matrix Φ.

2.3 Model identification

The proposed model is not identifiable because of the following two model indeterminacies. The first is caused by the additive nonparametric functions involved in (1), in which each unknown function is not identifiable up to a constant. To address this problem, we need to impose constraints on the unknown functions to enforce their integrations in the ranges of predictors to zero (Panagiotelis and Smith, 2008; Song and Lu, 2010) as follows:

$${\int }_{{𝒳}_j}{f}_{\mathit{\text{sj}}}(x)\mathit{\text{dx}}=0,\text{ for }s=1,\dots ,S,j=1,\dots ,q,$$ (5)

where 𝒳_j is the support of x_j. The other model determinacy is the label switching problem elicited by the invariance of the likelihood function to a random permutation of the state labels, which results in a multi-modal posterior under a symmetric prior specification. We follow Frühwirth-Schnatter (2001) to conduct a permutation sampler to address this issue.

3 Bayesian Inference

3.1 Nonparametric modeling

The first critical issue in the Bayesian analysis of the proposed model is to estimate the nonparametric functions involved in (1). We consider the use of Bayesian P-splines (Berry et al., 2002; Lang and Brezger, 2004; Fahrmeir and Raach, 2007). The basic idea is to estimate the unknown smoothing functions through a sum of B-splines basis functions (De Boor, 2001) given a large number of knots in the domains of predictors. Specifically, f_sj(x_it,j), the functional effect of the jth covariate at state s for subject i at time t, can be approximated as follows:

$$f_{\mathit{\text{sj}}}(x_{\mathit{\text{it}},j})=\sum _{l=1}^L{\beta }_{\mathit{\text{sj}},l}{B}_l(x_{\mathit{\text{it}},j})={\mathit{\beta }}_{\mathit{\text{sj}}}^T\mathbf{B}(x_{\mathit{\text{it}},j}),$$ (6)

where L is the number of splines determined by the number of knots, β_sj = (β_sj,1, …, β_sj,L)^T is the vector of the unknown parameters, B(·)s’ are cubic B-splines basis functions, and B(x_it,j) = (B₁(x_it,j), …, B_L(x_it,j))^T. Usually, L taking a value from 10 to 30 provides sufficient flexibility in fitting most smooth functions.

One problem of applying (6) to approximate an unknown smooth function is the over-fitting caused by the use of a large number of knots. Eilers and Marx (1996) suggested the penalization of the coefficients of adjacent B-splines basis functions to prevent the overfitting. Such penalization can be implemented in the Bayesian framework by applying random walk priors to β_sj (Lang and Brezger, 2004; Fahrmeir and Raach, 2007; Song and Lu, 2010).

3.2 Prior distributions

We assign a truncated Gaussian priors for β_sj as follows:

$$p({\mathit{\beta }}_{\mathit{\text{sj}}}|{\nu }_{\mathit{\text{sj}}})={\left(\frac{1}{2\pi {\nu }_{\mathit{\text{sj}}}}\right)}^{L_{\mathit{\text{sj}}}/2}\text{ exp }\{-\frac{1}{2{\nu }_{\mathit{\text{sj}}}}{\mathit{\beta }}_{\mathit{\text{sj}}}^T{\mathbf{K}}_{\mathit{\text{sj}}}{\mathit{\beta }}_{\mathit{\text{sj}}}\}\mathrm{I}({\mathbf{1}}_{n_s}^T{\mathbf{B}}_{\mathit{\text{sj}}}{\mathit{\beta }}_{\mathit{\text{sj}}}=0),$$ (7)

where ν_sj is a smoothing parameter for controlling the amount of penalty, K_sj is a penalty matrix derived according to the random walk penalties proposed, L_sj is the rank of K_sj, 1_ns is an n_s × 1 vector with all elements equal to 1, n_s is the sample size at state s, B_sj is the sub-matrix of B_j = [B_l(x_it,j)]_nT×L without the rows where Z_it ≠ s, and the truncation term incorporates the identifiability constraint (5) into the splines approximation (6).

For the smoothing parameters ν_sj, we assign a highly dispersed but proper inverse gamma prior as follows:

$$p({\nu }_{\mathit{\text{sj}}}^{-1})\stackrel{D}{=}\text{Gamma}[{\nu }_1,{\nu }_2],$$ (8)

where ν₁ and ν₂ are hyperparameters whose values are pre-specified. A common choice for these hyperparameters is ν₁ = 1 and ν₂ is small (Fahrmeir and Raach, 2007; Song and Lu, 2010). We set ν₁ = 1 and ν₂ = 0.005 in the present study.

For the parameters involved in conditional model (1), conjugate-type priors are assigned as follows: for s = 1, …, S,

$$p({\mu }_s)\stackrel{D}{=}N[{\mu }_{s0},{\sigma }_{\mu s0}^2],p({\mathit{\gamma }}_s)\stackrel{D}{=}N[{\mathit{\gamma }}_{s0},{\mathbf{\sum }}_{s0}],$$ (9)

$$p({\mathbf{\Phi }}^{-1})\stackrel{D}{=}\text{Wishart}[{\mathbf{R}}_0,{\rho }_0],p({\psi }_s^{-1})\stackrel{D}{=}\text{Gamma}[{\stackrel{\sim }{\alpha }}_{s0},{\stackrel{\sim }{\beta }}_{s0}],$$

where μ_s0, ${\sigma }_{\mu s0}^2$, γ_s0, Σ_s0, α̃_s0, β̃_s0, R₀, and ρ₀ are hyperparameters with preassigned values.

Finally, for the parameters involved in transition model (4), we consider the following Gaussian priors:

$$p({\zeta }_{\mathit{\text{us}}})\stackrel{D}{=}N[{\zeta }_{\mathit{\text{us}}0},{\sigma }_{\zeta 0}^2],p(\mathit{\alpha })\stackrel{D}{=}N[{\mathit{\alpha }}_0,{\mathbf{H}}_{\alpha 0}],p({\tau }_s)\stackrel{D}{=}N[{\tau }_{s0},{\sigma }_{\tau 0}^2],$$ (10)

where ζ_us0, ${\sigma }_{\zeta 0}^2$, α₀, H_α0, τ_s0, and ${\sigma }_{\tau 0}^2$ are hyperparameters with preassigned values.

3.3 Posterior computation

Let y_i = (y_i1, …, y_iT)^T, Y = (y₁, …, y_N), D = (d₁₁, …, d_NT), Z_i = (Z_i1, …, Z_iT)^T, Z = (Z₁, …, Z_N), W = (w₁, …, w_N), and θ be the vector that includes all the unknown parameters in the proposed model. The complete-data log-likelihood function that is used to derive the posterior distributions and compute the model selection criterion is given by

$$\text{log }p(\mathbf{Y},\mathbf{D},\mathbf{W},\mathbf{Z}|\mathit{\theta })=\sum _{i=1}^n[\text{log }p({\mathbf{y}}_i|{\mathbf{d}}_i,{\mathbf{w}}_i,{\mathbf{Z}}_i,\mathit{\theta })+\text{log }p({\mathbf{Z}}_i|{\mathbf{d}}_i,{\mathbf{w}}_i,\mathit{\theta })+\text{log }p({\mathbf{w}}_i|\mathit{\theta })]=\sum _{i=1}^n\sum _{t=1}^T\text{log }p(y_{\mathit{\text{it}}}|{\mathbf{d}}_{\mathit{\text{it}}},w_{i1},Z_{\mathit{\text{it}}}=s,\mathit{\theta })+\sum _{i=1}^n\sum _{t=2}^T\text{log }p(Z_{\mathit{\text{it}}}=s|Z_{i,t-1}=u,{\mathbf{d}}_{\mathit{\text{it}}},w_{i2},\mathit{\theta })+\sum _{i=1}^n\text{log }p(Z_{i1}=s|\mathit{\theta })+\sum _{i=1}^n\text{log }p({\mathbf{w}}_i|\mathit{\theta })=-\frac{1}{2}\sum _{i=1}^n\sum _{t=1}^T[\text{log}(2\pi {\psi }_s)+{(y_{\mathit{\text{it}}}-{\mu }_s-{\mathit{\gamma }}_s^T{\mathbf{b}}_{\mathit{\text{it}}}-\sum _{j=1}^q{\mathit{\beta }}_{\mathit{\text{sj}}}^T\mathbf{B}(x_{\mathit{\text{it}},j})-w_{i1})}^2/{\psi }_s]+\sum _{i=1}^n\sum _{t=2}^T\text{log}(p_{\mathit{\text{itus}}})+\sum _{i=1}^n\text{log}(p_{i10s})-\frac{1}{2}\sum _{i=1}^n[\text{log}(4{\pi }^2|\mathbf{\Phi }|)+{\mathbf{w}}_i^T{\mathbf{\Phi }}^{-1}{\mathbf{w}}_i],$$ (11)

where

$$p_{\mathit{\text{itu}}1}=\frac{\text{exp}\{a_{\mathit{\text{itu}}1}\}}{1+\text{exp}\{a_{\mathit{\text{itu}}1}\}},p_{i1\mathit{\text{uS}}}=\prod _{j=1}^{S-1}\frac{1}{1+\text{exp}\{a_{\mathit{\text{ituj}}}\}},$$

$$p_{\mathit{\text{itus}}}=\frac{\text{exp}\{a_{\mathit{\text{itus}}}\}}{1+\text{exp}\{a_{\mathit{\text{itus}}}\}}\prod _{j=1}^{s-1}\frac{1}{1+\text{exp}\{a_{\mathit{\text{ituj}}}\}},s=2,\dots ,S-1,$$ (12)

$$p_{i10s}={\tau }_s,s=1,\dots ,S,$$

with a_itus = ζ_us + α^Td_it + w_i2.

The Bayesian estimate of θ is obtained by drawing samples from p(θ|Y), which is intractable because of the existence of latent states and random effects. We instead work on p(θ, Z, W|Y) and use a Gibbs sampler to implement the posterior simulation. Owing to the nonlinearity of the continuation-logit transition model and the existence of the nonparametric functions in the conditional regression, some full conditional distributions, especially those related to the transition model, have complex forms. MCMC methods, such as the forward filtering and backward sampling algorithm (Cappé et al., 2005) and the Metropolis-Hastings (MH) algorithm (Metropolis et al., 1953; Hastings, 1970), are employed to sample from them. The details are provided in the Appendix.

With the use of posterior samples, the hidden states can be estimated as follows:

$${\hat{Z}}_{\mathit{\text{it}}}=\text{arg }\underset{s\in \{1,\dots ,S\}}{\text{max}}P(Z_{\mathit{\text{it}}}=s|{\mathbf{y}}_i,\mathit{\theta })\approx \text{arg }\underset{s\in \{1,\dots ,S\}}{\text{max}}\frac{1}{M}\sum _{m=1}^M\mathrm{I}(Z_{\mathit{\text{it}}}^{(m)}=s),$$ (13)

where $Z_{\mathit{\text{it}}}^{(m)}$ denotes the latent allocation of y_it at the mth iteration, and $\frac{1}{M}{\sum }_{m=1}^M\mathrm{I}(Z_{\mathit{\text{it}}}^{(m)}=s)$ is the posterior mean of the latent allocations of y_it drawn from the MCMC iterations.

3.4 Determination of the number of hidden states

In the applications of BMSHM2 to the ADNI dataset, the states of the Markov chain can often naturally be interpreted as proxies for the neurodegenerative states, although a one-to-one correspondence between nominal HMM states and the clinical cognitive stages diagnosed by doctors is unnecessary. In this regard, a relevant question is how to determine the number of hidden states in the analysis of ADNI data. We propose the use of a modified deviance information criteria (DIC) to determine the number of hidden states and choose a plausible model for the ADNI data analysis.

The modified DIC, which was developed by Celeux et al. (2006) for model comparison in the presence of incomplete data, is defined as follows:

$$\text{DIC}=-4E_{\mathit{\theta },\mathbf{W},\mathbf{Z}}\{\text{log }p(\mathbf{Y},\mathbf{W},\mathbf{Z}|\mathit{\theta })|\mathbf{Y}\}+2E_{\mathbf{W},\mathbf{Z}}\{\text{log }p(\mathbf{Y},\mathbf{W},\mathbf{Z}|E_{\mathit{\theta }}[\mathit{\theta }|\mathbf{Y},\mathbf{W},\mathbf{Z}])|\mathbf{Y}\}.$$ (14)

where log p(Y, W, Z|θ) is the complete-data log-likelihood function shown in (11). The expectations involved in (14) can be approximated using the posterior samples collected through MCMC methods (Celeux et al., 2006). In model selection, the model with the smallest value of DIC is selected.

4 Alzheimer’s Disease Neuroimaging Initiative Data Analysis

4.1 Data description

The ADNI was launched in 2003 by the National Institute on Aging (NIA), the National Institute of Biomedical Imaging and Bioengineering (NIBIB), the Food and Drug Administration (FDA), private pharmaceutical companies and nonprofit organizations, as a $60 million, 5-year public private partnership. The initial goal of ADNI was to recruit 800 adults, aged 55 to 90, to participate in the research – approximately 200 cognitively normal older individuals to be followed for 3 years, 400 people with MCI to be followed for 3 years, and 200 people with early AD to be followed for 2 years.

We focused on 633 subjects who were all followed up at baseline, 6 months, 12 months, and 24 months. For each subject, we included his/her clinical, genetic, and imaging variables at the four time points. The clinical characteristics include gender (0 = male; 1 = female), age at baseline, and FAQ score. The FAQ score is an assessment of abilities to function independently in daily life and is widely used to monitor the decline of cognitive ability over time. The genetic variables include APOE gene because mutations in APOE raise the risk of progression from amnestic MCI to AD (Petersen et al., 2005). The APOE SNPs, rs429358 and rs7412 were separately genotyped in ADNI-1. These two SNPs together define a 3-allele haplotype, namely, the ε2, ε3, and ε4 variants. Among these variants, APOE-ε4 has been identified as a risk factor for early onset of AD (e.g., Okuizumi et al., 1994). Thus, we considered the presence of APOE-ε4 as a covariate in this analysis. APOE-ε4 is coded as 0, 1, and 2, denoting the number of APOE-ε4 alleles. Furthermore, the logarithm of the ratio of hippocampal volume over whole brain volume was included as a covariate because published reports (Kesslak, Nalcioglu and Cotman, 1991; Jack et al., 1992; Dickerson and Wolk, 2013) revealed that the atrophy of the hippocampal formation was a significant diagnostic marker of clinical dementia. Table 1 summarizes the basic characteristics of the aforementioned variables for the samples under consideration. Males account for about 56.2% in the samples. Mean values of patients’ age (in year), adjusted hippocampal volume, and corresponding FAQ score are 73.0, −5.0, and 4.1, respectively. 34.6% patients carry one APOE-ε4 allele while only 9.8% carry two APOE-ε4 alleles.

Table 1.

Characteristics of the study samples in the ADNI-1 dataset

Mean Age (in years)	73.0(6.9)
Gender (Male percentage)	56.2%
Mean log(hippocampus/whole brain volume)	−5.0(0.2)
Mean FAQ score	4.1(6.5)
One APOE-ε4 allele carriers	34.6%
Two APOE-ε4 alleles carriers	9.8%

The numbers in parentheses are standard deviations.

4.2 Data analysis

The aims of this ADNI data analysis are (I) to identify the hidden states of the neurodegenerative pathology on the basis of 633 patients enrolled in the ADNI-1, (II) to reveal a set of potential covariates that influence the between-states transition, and (III) to investigate the linear and/or functional covariate effects on cognitive decline across the hidden states of the AD progression.

We fitted BMSHM2 with the FAQ score as the response y_it, the clinical and genetic variables, gender and APOE-ε4, as covariates in b_it, and hippocampus and age at baseline as covariates in x_it. Three continuous variables, FAQ score, hippocampus, and age, were standardized prior to analysis. We first determined the number of hidden states. We considered five competing models M_k, k = 1, …, 5, where M_k represents a BMSHM2 with k states. A total of 24 equidistant knots were used to construct cubic P-splines, and the second-order random walk penalties were used for the Bayesian P-splines to estimate the unknown smooth functions. Given the lack of prior information, we assign the hyperparameters in (9) and (10) to reflect vague prior information as follows: μ_s0 = ζ_us0 = τ_s0 = 0, ${\sigma }_{\mu s0}={\sigma }_{\zeta 0}^2={\sigma }_{\tau 0}^2=1$, α̃_s0 = 9, β̃_s0 = 4, ρ₀ = 7, R₀ = 4I₂, α₀ and γ_s0 is a vector with all elements being zero, H_α = I₅ and Σ_s0 = I₃ where I_r is a r-dimensional identity matrix. We used the random permutation sampler to search for a suitable identifiability constraint to solve the label switching problem. The MCMC algorithm converged within 2,000 iterations for all competing models. We collected a total of 10,000 observations after discarding 2,000 burn-in iterations to calculate DIC. The DIC values corresponding to M₁ to M₅ were 20,175, 1,823, 1,001, 950, and 1615, respectively. Thus, the four-state model M₄ was selected.

To examine the necessity of the random effects in the proposed model, we considered another competing model M_N: a four-state BMSHM2 without random effects. The DIC value for M_N is 1,122, which suggests an evident advantage of the proposed mixed effect model in the presence of high dependency/heterogeneity in longitudinal observations. Thus, M₄ was selected for the subsequent analysis. The estimation results obtained under M₄ are reported in Table 2 (parametric part) and Figure 2 (nonparametric part).

Table 2.

ADNI-1 data analysis results: parameter estimation results.

Parameters in conditional regression model
State 1			State 2			State 3			State 4
Par.	Est	SE	Par.	Est	SE	Par.	Est	SE	Par.	Est	SE
μ1	−0.556	0.013	μ2	0.029	0.114	μ3	1.108	0.134	μ4	2.494	0.127
ψ1	0.013	0.001	ψ2	0.135	0.019	ψ3	0.187	0.025	ψ4	0.397	0.057
γ11	0.016	0.018	γ21	0.163	0.098	γ31	0.102	0.131	γ41	0.443	0.140
γ12	0.079	0.033	γ22	0.260	0.132	γ32	0.307	0.157	γ42	0.407	0.193
γ13	0.011	0.015	γ23	0.038	0.100	γ33	−0.252	0.142	γ43	−0.557	0.137
Parameters in probability transition model
Par.	Est	SE	Par.	Est	SE	Par.	Est	SE	Par.	Est	SE
τ1	0.938	0.110	τ2	−0.303	0.373	τ3	1.132	0.330	α1	0.500	0.080
α2	0.035	0.067	α3	−0.342	0.141	α4	−0.728	0.230	α5	0.085	0.162
ζ11	2.477	0.158	ζ21	−1.714	0.342	ζ31	−3.244	0.508	ζ41	−3.252	0.512
ζ12	2.540	0.493	ζ22	1.198	0.411	ζ32	−1.661	0.479	ζ42	−3.114	0.525
ζ13	1.877	0.779	ζ23	2.618	0.445	ζ33	1.298	0.307	ζ43	−1.845	0.432
Covariance matrix of random effects
Par.	Est	SE	Par.	Est	SE	Par.	Est	SE
φ11	0.024	0.002	φ22	0.223	0.051	φ12	−0.007	0.006

Figure 2.

ADNI-1 data analysis results: estimates of the unknown smooth functions. The solid curves represent the pointwise mean curves, and the dashed curves represent the 2.5%-and 97.5%- pointwise quantiles. line y = 0 has been shown on each picture by red dot-dash to illustrate the range of significant effect for each variable.

We have the following observations. First, μ₁, μ₂, μ₃, and μ₄ are ranked in an ascending order, indicating that patients in state 1 got the lowest score of FAQ, whereas those in state 4 got the highest. That is, patients’ ability to function independently in daily life steadily deteriorated from state 1 to state 4. According to the existing literature (Kantarci et al., 2013), state 1 to state 4 can be explained as cognitive normal (CN), early mild cognitive impairment (EMCI), late mild cognitive impairment (LMCI), and AD, respectively.

Second, the functional effect of hippocampus on FAQ exhibits a descending trend as hippocampus grows regardless of states. Specifically, in CN state, people with bigger hippocampus volume tend to have slightly better memory. This finding is in line with the common sense that hippocampus plays an important role in the consolidation of information from short-term memory to long-term memory. In EMCI and LMCI states, the descending trend of the functional effect of hippocampus on FAQ becomes much more pronounced. This result implies that atrophy in hippocampus increasingly impaires patients’ cognitive ability. The published reports (e.g., Dickerson and Wolk, 2013; Kesslak, Nalcioglu and Cotman, 1991; Jack et al., 1992) also indicate that the volume loss of the hippocampus is greatly associated with clinical dementia. In AD state, the effect of hippocampal volume on FAQ is not significant because patients’ cognitive ability and memory have already been damaged by serious hippocampus atrophy.

Third, the effect of age on FAQ is nonsignificant in CN and EMCI states, implying that for those who have cognitive normal function or undergo only EMCI, age may not be a decisive factor for the reduction of cognitive ability. On the contrary, age exhibits nonlinear effects on FAQ in LMCI and AD states. The age effect is nonsignificant for relatively young patients but becomes significantly positive for elderly patients (say, over 85 years old). The positive effect increases with age and gets an even sharper rise in AD state. Such age effect was also discovered by previous studies (e.g. Gao et al., 1998; Lindsay et al., 2002).

Fourth, as for the fixed effects of discrete variables, a significantly negative effect of gender on FAQ appears in state 4, which means that male AD patients are in a better condition than females in terms of independent abilities. The two APOE-ε4 alleles (b_it,1 and b_it,2) have significantly positive effects on FAQ in state 4 and b_it,1 also has a slightly positive effect on FAQ in other states. This finding agrees with the newly published clinical research output that the presence of ε4 alleles in the APOE gene is the only genetic variant broadly accepted as increasing risk for late-onset AD dementia (Albert et al., 2011).

Fifth, in the transition model, hippocampus positively affects the probability of transitioning from a state to a better one, indicating that controlling loss of hippocampal volume would be beneficial to prevent the deterioration of cognitive ability. Similar to previous studies (e.g., Lee et al., 2015), APOE-ε4 alleles have negative effects on the probability of transitioning from a state to a better one, reconfirming that APOE-ε4 alleles are important risk factors for the development of AD.

Sixth, the variances of the two random effects are significant, reconfirming the necessity of the random effects proposed. However, the corvariance between the two random effects is nonsignificant, showing that some omitted clinical or genetic indicators influenced outcomes of the observation process or probabilities of the transition process but did not affect the two processes simultaneously.

Moreover, we estimated the hidden states of all patients at four time points based on Equation (13). Around 98% posterior transition patterns are from a state to a severer one, which is in line with the common knowledge of irreversibility of AD. Table 3 reports patients’ estimated hidden states and their diagnosed status by doctors. For CN, LMCI, and AD states, a majority of the estimated states are consistent with those diagnosed by doctors. For EMCI state, however, 835 (67%) EMCI patients diagnosed by doctors were classified into CN state by our procedure. Such vague demarcation between CN and EMCI was also found and discussed in the literature (e.g., Petersen, 2004).

Table 3.

ADNI-1 data analysis results: comparison of estimated hidden states and diagnosis status

	Estimates	CN	EMCI	LMCI	AD	Total
Diagnosis
CN		840	21	1	0	862
EMCI		835	232	148	28	1243
LMCI		9	21	39	21	90
AD		23	49	106	159	337
Total		1707	323	294	208	2532

5 Simulation Study

We conduct Monte Carlo simulations to assess the empirical performance of the proposed method in estimation of the nonparametric functions and model parameters.

5.1 Model setup

We consider a BMSHM2 with four hidden states (S = 4), a continuous response y_it, three discrete covariates (r = 3), and two continuous covariates (q = 2) to mimic the scenario of the ADNI study. For i = 1, …, 700 and t = 1, …, 9, b_it,1, b_it,2, and b_it,3 are all generated from the Bernoulli distribution with the probability of success 0.5, and x_it,1 and x_it,2 are generated from U (−1, 1) and N (0, 1), respectively. The conditional model is defined as

$$[y_{\mathit{\text{it}}}|Z_{\mathit{\text{it}}}=s]={\mu }_s+{\gamma }_{s1}{b}_{\mathit{\text{it}},1}+{\gamma }_{s2}{b}_{\mathit{\text{it}},2}+{\gamma }_{s3}{b}_{\mathit{\text{it}},3}+f_{s1}(x_{\mathit{\text{it}},1})+f_{s2}(x_{\mathit{\text{it}},2})+w_{i1}+{\delta }_{\mathit{\text{it}}},$$ (15)

where f₁₁(x_it,1) = −1.305+exp(x_it,1), f₁₂(x_it,2) = 0.55+sin(1.5x_it,2)+x_it,2, f₂₁(x_it,1) = 0.06−log((1 + x_it,1)/(1 − x_it,1)), $f_{22}(x_{\mathit{\text{it}},2})=0.125+x_{\mathit{\text{it}},2}^3$, f₃₁(x_it,1) = −0.05 − 0.8x_it,1, f₃₂(x_it,2) = −0.275 + cos(2x_it,2) + 0.5x_it,2, $f_{41}(x_{\mathit{\text{it}},1})=-0.13-x_{\mathit{\text{it}},1}^3$ and f₄₂(x_it,2) = −0.85 + 1.5x_it,2.

The transition model is defined as

$$\text{logit}({\vartheta }_{\mathit{\text{itus}}})={\zeta }_{\mathit{\text{us}}}+{\alpha }_1{x}_{\mathit{\text{it}},1}+{\alpha }_2{x}_{\mathit{\text{it}},2}+{\alpha }_3{b}_{\mathit{\text{it}},1}+{\alpha }_4{b}_{\mathit{\text{it}},2}+{\alpha }_5{b}_{\mathit{\text{it}},3}+w_{i2}.$$ (16)

The true population values of the unknown parameters are set as μ = (μ₁, μ₂, μ₃, μ₄) = (−5, −1, 3, 7), τ = (τ₁, τ₂, τ₃, τ₄) = (0.27, 0.27, 0.23, 0.23), ζ₁₁ = ζ₂₁ = ζ₃₁ = ζ₄₁ = −1, ζ₁₂ = ζ₂₂ = ζ₃₂ = ζ₄₂ = −1/2, ζ₁₃ = ζ₂₃ = ζ₃₃ = ζ₄₃ = 1/2, γ₁ = (γ₁₁, γ₁₂, γ₁₃) = (−1, 0.5, 0.5), γ₂ = (γ₂₁, γ₂₂, γ₂₃) = (1, 1, 0.5), γ₃ = (γ₃₁, γ₃₂, γ₃₃) = (−0.5, −0.5, −0.5), γ₄ = (γ₄₁, γ₄₂, γ₄₃) = (0.5, −1, −1), α = (α₁, α₂, α₃, α₄, α₅)^T = (1, −1, −0.5, 0.5, 1), ψ = (ψ₁, ψ₂, ψ₃, ψ₄) = (1, 0.64, 0.36, 0.25), and Φ is a correlation matrix with off diagonal elements −0.5. Based on the above setup, we simulate 100 datasets for analysis.

5.2 Simulation results

We used a total of 24 equidistant knots to construct the cubic B-splines of the covariates. Again, the second-order random walk penalties were used for the Bayesian P-splines to estimate the unknown smooth functions. The prior inputs in (9) and (10) were assigned as follows: μ_s0 = ζ_us0 = τ_s0 = 0, ${\sigma }_{\mu s0}={\sigma }_{\zeta 0}^2={\sigma }_{\tau 0}^2=1$, α̃_s0 = 9, β̃_s0 = 4, ρ₀ = 7, R₀ = 4I₂, α₀ and γ_s0 are vectors with all elements being zero, H_α = I₅ and Σ_s0 = I₃, where I_r is a r × r identity matrix. We conducted a few test runs to decide the number of burn-in iterations required for convergence and found that the MCMC algorithm converged within 2,000 iterations. Therefore, we obtain Bayesian results using 5,000 observations after discarding 2,000 burn-in iterations. The performance of the Bayesian estimates is assessed through the bias (BIAS) and the root mean square errors (RMSE) between the Bayesian estimates and the true population values of the parameters.

Table 4 summarizes the result of parameter estimation based on the 100 datasets. The BIAS and RMSE for most of the parameters are close to zero, indicating a satisfactory performance of Bayesian estimation regarding the parametric part. Figure 3 depicts the averages of the pointwise posterior means of the nonparametric functions, along with their 2.5%- and 97.5%- pointwise quantiles. The posterior means of the nonparametric functions are close to their true curves and all the ranges of the two pointwise quantiles are relatively small, indicating that the estimated functions can correctly recover the true functional relationships between the response and covariates. In this simulation, the average of the correct classification rates calculated through Equation (13) based on the 100 datasets is 91%. Considering the complexity of proposed model, this result is satisfactory.

Table 4.

Bayesian estimates of the parameters in the simulation study.

Parameters in conditional regression model
State 1			State 2			State 3			State 4
Par.	Bias	RMSE	Par.	Bias	RMSE	Par.	Bias	RMSE	Par.	Bias	RMSE
μ1	0.010	0.098	μ2	0.024	0.115	μ3	−0.046	0.195	μ4	−0.006	0.106
ψ1	−0.006	0.042	ψ2	0.011	0.047	ψ3	0.003	0.044	ψ4	0.016	0.028
γ11	−0.002	0.105	γ21	−0.045	0.188	γ31	0.025	0.147	γ41	−0.020	0.112
γ12	0.002	0.091	γ22	−0.022	0.126	γ32	0.030	0.168	γ42	0.018	0.089
γ13	−0.013	0.090	γ23	−0.013	0.098	γ33	0.026	0.126	γ43	0.002	0.093
Parameters in probability transition model
Par.	Bias	RMSE	Par.	Bias	RMSE	Par.	Bias	RMSE	Par.	Bias	RMSE
τ1	0.008	0.137	τ2	−0.030	0.187	τ3	−0.048	0.236	α1	0.007	0.068
α2	0.010	0.064	α3	−0.024	0.103	α4	−0.011	0.122	α5	−0.029	0.112
ζ11	0.040	0.125	ζ21	0.042	0.132	ζ31	0.034	0.120	ζ41	0.035	0.131
ζ12	0.050	0.168	ζ22	0.031	0.156	ζ32	0.037	0.162	ζ42	0.034	0.167
ζ13	−0.034	0.197	ζ23	−0.032	0.195	ζ33	−0.027	0.186	ζ43	−0.032	0.197
Covariance matrix of random effects
Par.	Bias	RMSE	Par.	Bias	RMSE	Par.	Bias	RMSE
φ11	0.004	0.066	φ22	0.017	0.161	φ12	0.001	0.067

Figure 3.

Estimates of the unknown smooth functions in the simulation study: The solid curves represent the true curves, and the dashed curves represent the estimated posterior means and the 2.5%- and 97.5%- pointwise quantiles based on 100 replications, respectively.

To reveal the sensitivity of the Bayesian estimates to the input of priors, we disturbed the prior inputs as follows: μ_s0 = ζ_us0 = τ_s0 = 2, ${\sigma }_{\mu s0}={\sigma }_{\zeta 0}^2={\sigma }_{\tau 0}^2=2$, α̃_s0 = 3, β̃_s0 = 2, ρ₀ = 4, R₀ = 2I₂, α₀ and γ_s0 are vectors with all elements being two, H_α = 2I₅ and Σ_s0 = 2I₃. The obtained results are similar and not reported.

6 Discussion

The BMSHM2 was developed and successfully applied to the ADNI data analysis. Although HMMs and their variants have already been extensively used for longitudinal data analysis, a majority of applications restrict analysis in a parametric framework. Nonetheless, examples of using HMMs to classify and characterise the neurodegenerative states of AD pathology are not prevalent, especially in a semiparametric context. In this study, we extended parametric HMMs to accommodate the functional effects of hippocampus and age on cognitive decline across four neurodegenerative states, namely, CN, EMCI, LMCI, and AD. The functional effect of hippocampus on cognitive function exhibited a descending trend as hippocampus grows regardless of states. This descending trend became more pronounced for EMCI and LMCI states than for CN and AD states, implying that atrophy in hippocampal volume had increasingly impaired patients’ cognitive ability, especially during the progression from EMCI to LMCI. On the contrary, age affected cognitive function mainly in LMCI and AD states. Elderly LMCI or AD patients suffered from more increasing neurodegeneration than relatively young patients.

Our model incorporates correlated random effects to account for individual and/or contextual differences in the progression of cognitive decline and in between-state transition. Large inter-individual variability is a prominent feature of the ADNI dataset and many other longitudinal datasets. As we demonstrated in the ADNI study, accounting for such differences can dramatically improve model fit, as evidenced by an apparent improvement in DIC value between models with and without random effects. In addition, the correlation between the random effects enhances the model capability of accommodating the situation where some omitted covariates influence both the state-dependent observation process and the hidden-state transition process. Another appealing feature of this study is that it implements a full Bayesian analysis along with efficient MCMC methods. The sampling-based Bayesian approach is not only applicable to the current parameter-rich BMSHM2 but also possesses potential to address highly complex problems with which huge challenges are confronted by ML-based procedures.

The present study can be extended in several directions: First, we considered the nonparametric modeling only in the conditional model. Generalizing the parametric transition model to a semiparametric or nonparametric one can further enhance model flexibility and analytic power. However, the statistical analysis of such comprehensive models can be challenging because the computational burden and sample size often limit the complexity of candidate models. Thus, the feasibility of this extension requires further investigation. Second, in the application to the ADNI dataset, a highly comprehensive characterization of cognitive function is to group the FAQ, Alzheimer’s Disease Assessment Scale, and Mini-Mental State Examination into an integrated latent construct through multivariate techniques such as factor analysis (e.g. Song et al., 2016). Finally, this study did not consider missing data. Given that missingness is very common in longitudinal settings, accommodation of missing responses and/or missing covariates in the context of BMSHM2s is both of scientific interest and of practical value. These advances certainly require substantial efforts for further investigation.

Appendix

Full Conditional Distributions

(I) Full conditional distributions of Z_it

We follow Baum et al. (1970) to adopt a recursive method to sample Z_it from the full conditional distribution efficiently as follows:

Let y_i = {y_i1, …, y_iT } and D_i = {d_i1, …, d_iT}, then we have

$$p(Z_{\mathit{\text{it}}}|·)\propto p({\mathbf{y}}_i,{\mathbf{D}}_i,{\mathbf{w}}_i,Z_{\mathit{\text{it}}}|\mathit{\theta })$$ (A1)

$$=p(y_{i1},\dots ,y_{\mathit{\text{it}}},{\mathbf{d}}_{i1},\dots ,{\mathbf{d}}_{\mathit{\text{it}}},{\mathbf{w}}_i,Z_{\mathit{\text{it}}}|\mathit{\theta })\times p(y_{\mathit{\text{it}}+1},\dots ,y_{\mathit{\text{iT}}},{\mathbf{d}}_{\mathit{\text{it}}+1},\dots ,{\mathbf{d}}_{\mathit{\text{iT}}}|{\mathbf{w}}_i,Z_{\mathit{\text{it}}},\mathit{\theta })$$ (A2)

$$\doteq q_{\mathit{\text{it}}}({\mathbf{y}}_i,{\mathbf{D}}_i,{\mathbf{w}}_i,Z_{\mathit{\text{it}}}|\mathit{\theta })\times {\overline{q}}_{\mathit{\text{it}}}({\mathbf{y}}_i,{\mathbf{D}}_i|{\mathbf{w}}_i,Z_{\mathit{\text{it}}},\mathit{\theta }).$$ (A3)

We first initialize q_i1(y_i, D_i, w_i, Z_it|θ) = p(y_i1, d_i1, w_i, Z_it|θ) = p(y_i1, d_i1|Z_i1, θ)p(Z_i1|θ) and calculate q_it(y_i, D_i, w_i, Z_it|θ) for t = 2, …, T, in a recursion manner as follows:

$$q_{\mathit{\text{it}}}({\mathbf{y}}_i,{\mathbf{D}}_i,{\mathbf{w}}_i,Z_{i1}|\mathit{\theta })=q_{\mathit{\text{it}}}(y_{i1},\dots ,y_{\mathit{\text{it}}},{\mathbf{d}}_{i1},\dots ,{\mathbf{d}}_{\mathit{\text{iT}}},{\mathbf{w}}_i,Z_{i1}|\mathit{\theta })=\sum _{u=1}^{S}p(y_{i1},\dots ,y_{\mathit{\text{it}}},{\mathbf{d}}_{i1},\dots ,{\mathbf{d}}_{\mathit{\text{iT}}},{\mathbf{w}}_i,Z_{\mathit{\text{it}}},Z_{i,t-1}=u|\mathit{\theta })=\sum _{u=1}^{S}p(y_{i1},\dots ,y_{\mathit{\text{it}}},{\mathbf{d}}_{i1},\dots ,{\mathbf{d}}_{\mathit{\text{iT}}},{\mathbf{w}}_i,Z_{i,t-1}=u|\mathit{\theta })\times p(Z_{\mathit{\text{it}}}|Z_{i,t-1}=u,w_{i2},\mathit{\theta })\times p(y_{\mathit{\text{it}}},{\mathbf{d}}_{\mathit{\text{it}}}|Z_{\mathit{\text{it}}},w_{i1},\mathit{\theta })]=\sum _{u=1}^S[q_{\mathit{\text{it}}}({\mathbf{y}}_{i,t-1},{\mathbf{D}}_i,{\mathbf{w}}_i,Z_{i,t-1}=u|\mathit{\theta })\times p(Z_{\mathit{\text{it}}}|Z_{i,t-1}=u,w_{i2},\mathit{\theta })\times p(y_{\mathit{\text{it}}},{\mathbf{d}}_{\mathit{\text{it}}}|Z_{\mathit{\text{it}}},w_{i1},\mathit{\theta })],$$ (A4)

where p(Z_it|Z_i,t−1 = u, w_i2, θ), p(y_it, d_it|Z_it, w_i1, θ) and p(w_i|θ) can be calculated through Equation (11).

Similarly, we initialize q̄_iT (y_i, D_i|w_i, Z_iT, θ) = 1 and calculate q̄_it(y_i, D_i|w_i, Z_it, θ) for t = T − 1, …, 1 as follows:

$$\begin{gathered} {\overline{q}}_{\mathit{\text{it}}}({\mathbf{y}}_i,{\mathbf{D}}_i|{\mathbf{w}}_i,Z_{\mathit{\text{it}}},\mathit{\theta })=p(y_{\mathit{\text{it}}+1},\dots ,y_{\mathit{\text{iT}}},{\mathbf{x}}_{\mathit{\text{it}}+1},\dots ,{\mathbf{d}}_{\mathit{\text{iT}}}|{\mathbf{w}}_i,Z_{\mathit{\text{it}}},\mathit{\theta }) \\ =\sum _{u=1}^{S}p(y_{\mathit{\text{it}}+1},\dots ,y_{\mathit{\text{iT}}},{\mathbf{d}}_{\mathit{\text{it}}+1},\dots ,{\mathbf{d}}_{\mathit{\text{iT}}},Z_{\mathit{\text{it}}+1}=u|{\mathbf{w}}_i,Z_{\mathit{\text{it}}},\mathit{\theta }) \end{gathered}$$ (A5)

$$=\sum _{u=1}^S[p(y_{\mathit{\text{it}}+1},\dots ,y_{\mathit{\text{iT}}},{\mathbf{d}}_{\mathit{\text{it}}+1},\dots ,{\mathbf{d}}_{\mathit{\text{iT}}}|Z_{\mathit{\text{it}}+1}=u,{\mathbf{w}}_i,\mathit{\theta })\times p(Z_{\mathit{\text{it}}+1}=u|Z_{\mathit{\text{it}}},w_{i2},\mathit{\theta })\times p(y_{\mathit{\text{it}}+1},{\mathbf{d}}_{\mathit{\text{it}}+1}|Z_{\mathit{\text{it}}+1}=u,w_{i1},\mathit{\theta })]$$ (A6)

$$=\sum _{u=1}^S[{\overline{q}}_{\mathit{\text{it}}}({\mathbf{y}}_i,{\mathbf{D}}_i|{\mathbf{w}}_i,Z_{\mathit{\text{it}}+1}=u,\mathit{\theta })\times p(Z_{\mathit{\text{it}}+1}=u|Z_{\mathit{\text{it}}},w_{i2},\mathit{\theta })\times p(y_{\mathit{\text{it}}+1},{\mathbf{d}}_{\mathit{\text{it}}+1}|Z_{\mathit{\text{it}}+1}=u,w_{i1},\mathit{\theta })].$$

Thus, Z_it can be directly generated from (A1) when all q_it(·)s and q̄_it(·)s defined in (A4) and (A5) are well calculated.

(II) Full conditional distributions of w_i

$$p({\mathbf{w}}_i|·)\propto p({\mathbf{y}}_i,{\mathbf{D}}_i|{\mathbf{w}}_i,Z_{i1},\dots ,Z_{\mathit{\text{iT}}},\mathit{\theta })\times p(Z_{i1},\dots ,Z_{\mathit{\text{iT}}}|{\mathbf{w}}_i,\mathit{\theta })\times p({\mathbf{w}}_i|\mathit{\theta })\propto \text{exp }\{-\frac{1}{2}\sum _{t=1}^T{(y_{\mathit{\text{it}}}-{\mu }_s-{\mathit{\gamma }}_2^T{\mathbf{b}}_{\mathit{\text{it}}}-\sum _{j=1}^q{\mathit{\beta }}_{\mathit{\text{sj}}}^T{\mathbf{B}}_{\mathit{\text{it}},j}-w_{i1})}^2\mathrm{I}(Z_{\mathit{\text{it}}}=s)/{\psi }_s+\sum _{t=2}^T\text{log}(p_{\mathit{\text{itus}}})\mathrm{I}(Z_{i,t-1}=u,Z_{\mathit{\text{it}}}=s)-\frac{1}{2}{\mathbf{w}}_i^T{\mathbf{\Phi }}^{-1}{\mathbf{w}}_i\},$$ (A7)

where p_itu0 and p_itus can be calculated via Equation (12).

(III) Full conditional distributions of μ_s, γ_s, ψ_s, and Φ

$$\begin{array}{cc}[{\mu }_s|·]~N[{\mu }_s^{\ast },{\sigma }_{\mu s}^{\ast }],\hfill & [{\mathit{\gamma }}_s|·]~N[{\mathit{\gamma }}_s^{\ast },{\mathbf{\sum }}_s^{\ast }],\hfill \\ [{\psi }_s^{-1}|·]~\text{Gamma}[{\stackrel{\sim }{\alpha }}_s^{\ast },{\stackrel{\sim }{\beta }}_s^{\ast }],\hfill & [{\mathbf{\Phi }}^{-1}|·]~\text{Wishart}[{\mathbf{R}}^{\ast },N+{\rho }_0],\hfill \end{array}$$ (A8)

where R* = (R₀ + WW^T)⁻¹, $\mathbf{W}={({\mathbf{w}}_1^T,\dots ,{\mathbf{w}}_N^T)}^T$ and

$${\sigma }_{\mu s}^{\ast }={(n_s{\psi }_s^{-1}+{\sigma }_{\mu s0}^{-1})}^{-1},{\mathbf{\sum }}_s^{\ast }={(\sum _{i=1}^N\sum _{t=1}^T{\mathbf{b}}_{\mathit{\text{it}}}{\mathbf{b}}_{\mathit{\text{it}}}^T{\psi }_s^{-1}+{\mathbf{\sum }}_{s0}^{-1})}^{-1},$$

$${\mu }_s^{\ast }={\sigma }_{\mu s}^{\ast }[{\psi }_s^{-1}\sum _{i=1}^n\sum _{t=1}^T\mathrm{I}(Z_{\mathit{\text{it}}}=s)(y_{\mathit{\text{it}}}-{\mathit{\gamma }}_s^T{\mathbf{b}}_{\mathit{\text{it}}}-\sum _{j=1}^q{\mathit{\beta }}_{\mathit{\text{sj}}}^T{\mathbf{B}}_{\mathit{\text{it}},j}-w_{i1})+{\sigma }_{\mu s0}^{-1}{\mu }_{s0}],$$

$${\mathit{\gamma }}_s^{\ast }={\mathbf{\sum }}_s^{\ast }[{\psi }_s^{-1}\sum _{i=1}^n\sum _{t=1}^T\mathrm{I}(Z_{\mathit{\text{it}}}=s){\mathbf{b}}_{\mathit{\text{it}}}(y_{\mathit{\text{it}}}-{\mu }_s-\sum _{j=1}^q{\mathit{\beta }}_{\mathit{\text{sj}}}^T{\mathbf{B}}_{\mathit{\text{it}},j}-w_{i1})+{\mathbf{\sum }}_{s0}^{-1}{\mathit{\gamma }}_{s0}],$$

$${\stackrel{\sim }{\alpha }}_s^{\ast }=n_s/2+{\stackrel{\sim }{\alpha }}_{s0},{\stackrel{\sim }{\beta }}_s^{\ast }={\stackrel{\sim }{\beta }}_{s0}+\frac{1}{2}[\sum _{i=1}^n\sum _{t=1}^T\mathrm{I}(Z_{\mathit{\text{it}}}=s){(y_{\mathit{\text{it}}}-{\mu }_s-{\mathit{\gamma }}_s^T{\mathbf{b}}_{\mathit{\text{it}}}-\sum _{j=1}^q{\mathit{\beta }}_{\mathit{\text{sj}}}^T{\mathbf{B}}_{\mathit{\text{it}},j}-w_{i1})}^2].$$

(IV) Full conditional distributions of β_sj and θ_sj

$$[{\mathit{\beta }}_{\mathit{\text{sj}}}|·]~N[{\mathit{\beta }}_{\mathit{\text{sj}}}^{\ast },{\mathbf{H}}_{\mathit{\text{sj}}}]\mathrm{I}({\mathbf{1}}_{n_s}^T{\mathbf{B}}_{\mathit{\text{sj}}}{\mathit{\beta }}_{\mathit{\text{sj}}}=0),$$ (A9)

where ${\mathbf{H}}_{\mathit{\text{sj}}}={({\psi }_s^{-1}{\mathbf{B}}_{\mathit{\text{sj}}}^T{\mathbf{B}}_{\mathit{\text{sj}}}+{\nu }_{\mathit{\text{sj}}}^{-1}{\mathbf{K}}_{\mathit{\text{sj}}})}^{-1},{\mathit{\beta }}_{\mathit{\text{sj}}}^{\ast }={\psi }_s^{-1}{\mathbf{H}}_{\mathit{\text{sj}}}{\mathbf{B}}_{\mathit{\text{sj}}}^T{\mathbf{y}}_s^{\ast }$, and ${\mathbf{y}}_s^{\ast }=\{y_{\mathit{\text{it}},s}^{\ast }\}$ is an n_s × 1 vector with

$$y_{\mathit{\text{it}},s}^{\ast }=y_{\mathit{\text{it}}}-{\mu }_s-{\mathit{\gamma }}_s^T{\mathbf{b}}_{\mathit{\text{it}}}-\sum _{l\ne j,l=1}^q{\mathit{\beta }}_{\mathit{\text{sl}}}^T{\mathbf{B}}_{\mathit{\text{it}},l}-w_{i1},\text{ for }{Z}_{\mathit{\text{it}}}=s.$$

According to Panagiotelis and Smith (2008), sampling an observation β_sj from truncated normal (A9) is equivalent to sampling an observation ${\mathit{\beta }}_{\mathit{\text{sj}}}^{(\mathit{\text{temp}})}$ from $N[{\mathit{\beta }}_{\mathit{\text{sj}}}^{\ast },{\mathbf{H}}_{\mathit{\text{sj}}}]$ and then transforming ${\mathit{\beta }}_{\mathit{\text{sj}}}^{(\mathit{\text{temp}})}$ to β_sj by

$${\mathit{\beta }}_{\mathit{\text{sj}}}={\mathit{\beta }}_{\mathit{\text{sj}}}^{(\mathit{\text{temp}})}-{\mathbf{H}}_{\mathit{\text{sj}}}{\mathbf{Q}}_{\mathit{\text{sj}}}^T{({\mathbf{Q}}_{\mathit{\text{sj}}}{\mathbf{H}}_{\mathit{\text{sj}}}{\mathbf{Q}}_{\mathit{\text{sj}}}^T)}^{-1}{\mathbf{Q}}_{\mathit{\text{sj}}}{\mathit{\beta }}_{\mathit{\text{sj}}}^{(\mathit{\text{temp}})},$$ (A10)

where ${\mathbf{Q}}_{\mathit{\text{sj}}}={\mathbf{1}}_{n_s}^T{\mathbf{B}}_{\mathit{\text{sj}}}$. Moreover,

$$[{\nu }_{\mathit{\text{sj}}}^{-1}|·]~\text{Gamma }[{\nu }_1+\frac{L}{2},{\nu }_2+\frac{1}{2}{\mathit{\beta }}_{\mathit{\text{sj}}}^T{\mathbf{K}}_{\mathit{\text{sj}}}{\mathit{\beta }}_{\mathit{\text{sj}}}].$$ (A11)

(V) Full conditional distributions of τ_s, ζ_us, and α

$$p({\tau }_s|·)\propto \text{exp }\{\sum _{u=s}^S\sum _{i=1}^n\text{log}(p_{i10u})\times \mathrm{I}(Z_{i1}=u)-\frac{{({\tau }_s-{\tau }_{s0})}^2}{2{\sigma }_{\tau 0}^2}\},$$

$$p({\zeta }_{\mathit{\text{us}}}|·)\propto \text{exp }\{\sum _{\upsilon =s}^S\sum _{i=1}^n\sum _{t=2}^T\text{log}(p_{\mathit{\text{itu}}\upsilon })\times \mathrm{I}(Z_{\mathit{\text{it}}}=\upsilon ,Z_{i,t-1}=u)-\frac{{({\zeta }_{\mathit{\text{us}}}-{\zeta }_{\mathit{\text{us}}0})}^2}{2{\sigma }_{\zeta 0}^2}\},$$ (A12)

$$p(\mathit{\alpha }|·)\propto \text{exp }\{\sum _{i=1}^n\sum _{t=2}^T\text{log}(p_{\mathit{\text{itus}}})\times \mathrm{I}(Z_{\mathit{\text{it}}}=s,Z_{i,t-1}=u)-\frac{1}{2}{(\mathit{\alpha }-{\mathit{\alpha }}_0)}^T{\mathbf{H}}_{\alpha 0}^{-1}(\mathit{\alpha }-{\mathit{\alpha }}_0)\},$$

where p_itu0 and p_itus can be calculated via Equation (12).