Bayesian adaptive group lasso with semiparametric hidden Markov models

Abstract

This paper presents a Bayesian adaptive group least absolute shrinkage and selection operator method to conduct simultaneous model selection and estimation under semiparametric hidden Markov models. We specify the conditional regression model and the transition probability model in the hidden Markov model into additive nonparametric functions of covariates. A basis expansion is adopted to approximate the nonparametric functions. We introduce multivariate conditional Laplace priors to impose adaptive penalties on regression coefficients and different groups of basis expansions under the Bayesian framework. An efficient Markov chain Monte Carlo algorithm is then proposed to identify the nonexistent, constant, linear, and nonlinear forms of covariate effects in both conditional and transition models. The empirical performance of the proposed methodology is evaluated via simulation studies. We apply the proposed model to analyze a real data set that was collected from the Alzheimer’s Disease Neuroimaging Initiative study. The analysis identifies important risk factors on cognitive decline and the transition from cognitive normal to Alzheimer’s disease.

1 ∣. INTRODUCTION

Hidden Markov models (HMMs) have been widely used in the medical, behavioral, social, environmental, and psychological sciences where longitudinal data are frequently collected.^1-6 Basically, HMMs are designed to have two parts: a transition model to investigate the effects of covariates on the dynamic transition process of hidden states and a conditional regression model to examine state-specific covariate effects on the response of interest. In these two parts, the effect of a covariate on the response or on the transition process can be nonexistent, constant, linear, or nonlinear. Identifying the specific forms of such covariate effects is useful not only in achieving a parsimonious model but also in obtaining enhanced parameter estimation and attractive interpretations.

Conventional studies on HMMs have focused on a parametric framework, wherein the forms of covariate effects on responses and/or on transition probabilities are prespecified. However, one fundamental issue overlooked by these parametric HMMs is that the complex relationships among variables are seldom known a priori, and the parametric form is thus too restrictive to correctly reflect the reality. Several nonparametric approaches have been investigated recently to relax the parametric assumption of HMMs. Yau et al⁷ developed a Bayesian nonparametric HMM, where the sampling distribution of the observations at each state was assumed unknown and modeled via a mixture of Dirichlet processes. Although their method did not rely on the distributional assumption of the observed process, it cannot reveal the functional effects of potential explanatory variables on the outcome of interest. Song et al⁸ considered Bayesian P-splines for describing the nonparametric relation among latent variables in HMMs, but they did not consider the model selection problem.

Model selection is an important issue beyond estimation in the application of HMMs. Classical model selection methods are mainly developed on the basis of a pairwise comparison through common model selection criteria, such as the Akaike information criterion and the Bayesian information criterion. However, such pairwise-based procedure usually becomes increasingly computationally demanding when the search dimension is high. An appealing alternative is to adopt least absolute shrinkage and selection operator (lasso)–type variable selection techniques. Choi et al⁹ applied lasso to correlated HMMs to detect the important parameters in transition models. Städler and Mukherjee¹⁰ introduced L₁ penalization to obtain a sparse HMM with state-specific graphical models. However, the preceding studies consider only parametric HMMs. Recently, some variants of lasso, such as group lasso, adaptive lasso, and adaptive group lasso, have been developed to manage group variables and address the issue of lasso and group lasso possibly suffering from appreciable bias. Owing to the computational efficiency and stability of the Bayesian approach, the Bayesian analogs of lasso and its variants have been proposed.^11,12 However, the available Bayesian lasso-type methods are all developed in the context of cross-sectional models without between-state transitions, thereby making them inapplicable to the proposed semiparametric HMMs.

In this paper, we propose a Bayesian adaptive group lasso (BaGlasso) procedure to conduct simultaneous model selection and estimation for semiparametric HMMs. With the use of basis expansion and appropriate penalties, the non-parametric relationships that subsume nonexistent, constant, linear, and nonlinear relationships between covariates and the response can be automatically identified. The proposed procedure has the following appealing features: first, the group effects and additional correlation within the basis expansion are well addressed by the group lasso, thus ensuring estimation accuracy. Second, adaptive penalties imposed on different groups of coefficients enable us to achieve an efficient variable selection. Finally, the proposed procedure avoids tedious pairwise comparisons among competing models with different combinations of covariates in the conditional and transition models. This entirely data-driven feature not only relaxes the dependence on experts’ knowledge in empirical studies but also reduces the computational burden. To the best of our knowledge, this study is the first to introduce Bayesian lasso-type procedure into semiparametric HMMs.

The proposed method is motivated by a real study conducted by the Alzheimer’s Disease Neuroimaging Initiative (ADNI). A set of biomarkers, namely, gender, age, educational levels, marital status, hippocampal volume, and apolipoprotein E (APOE)-ϵ4, is collected across several time points in this data set. The purpose of this study is to detect the potential risk factors of Alzheimer’s disease (AD) from two perspectives. First, considering that the pathology of AD usually evolves from cognitive normal (CN) to mild cognitive impairment (MCI) to dementia, characterizing the disease pathology, identifying hidden states that correspond to the diagnosed stages of cognitive decline, and examining the potential risk factors of the neurodegenerative transition are of scientific interest and practical value. Given that the effects of biomarkers on the pathology from one state to another may vary across nonexistent, constant, linear, and nonlinear ones, allowing their forms to be unspecified and introducing penalties to penalize unimportant effects can reveal the patterns of the effects to the greatest extent. Previous studies¹³ pointed out that the relationships between some biomarkers and cognitive decline are variant across different states. Therefore, identifying the significant state-specific risk factors of cognitive decline and investigating the subtle forms of their effects are of great interest. However, existing relevant research either restricts the examination of the above relationships under a parametric framework or emphasizes only estimation. The proposed methodology enables us to perfectly accommodate all the aforementioned features and provide new insights into the prevention of AD.

The rest of this paper is organized as follows. Section 2 introduces the semiparametric HMM and discusses the associated identifiability issue. Section 3 illustrates the statistical inference of the proposed model. Specifically, BaGlasso for simultaneous variable selection and parameter estimation as well as the deviance information criterion (DIC) for the determination of the number of hidden states are presented. Section 4 investigates the empirical performance of the proposed method via simulation studies. Section 5 presents an application of the proposed method to the aforementioned ADNI study. Several important biomarkers are detected to have significant functional effects on patients’ cognitive decline across neurodegenerative states and/or on transition probabilities. The extension of the model is discussed in Section 6.

2 ∣. MODEL DESCRIPTION

2.1 ∣. Semiparametric HMMs

Let y_it with subject i = 1, … , n at t = 1, … , T be the observation process. Z_i = (Z_i1, … , Z_iT)′, the hidden-state sequence, is commonly assumed to follow a first-order Markov chain taking values in a finite set {1, … , S). Given the hidden state Z_it, the conditional semiparametric regression model is formulated as follows:

$$[y_{it}\mid Z_{it}=s]={\mu }_s+{\alpha }_s^{\prime }{\mathbf{c}}_{it}+\sum _{j=1}^q{f}_{sj}(x_{itj})+{\delta }_{it},$$ (1)

where c_it = (c_it1, & , c_itp)′ and x_it = (x_it1, & , x_itq)′ are a p × 1 vector of discrete covariates and a q × 1 vector of continuous covariates, respectively; intercept μ_s, fixed effects ${\alpha }_s^{\prime }=({\alpha }_{s1},\dots ,{\alpha }_{sp})$, and unknown smoothing function f_sj(·)s are all defined as state-specific to address the heterogeneity underlying the observations; δ_it is a random residual independent of y_it; and [δ_it∣Z_it = s] ~ N[0, ψ_s].

In addition to the observable process, the hidden process, Z_i, is formulated as follows: let p_itus denote the transition probability from state Z_i,t–1 = u at occasion t – 1 to state Z_it = s at occasion t for individual i. Then, we have

$$p_{itus}=P(Z_{it}=s\mid Z_{i1},Z_{i2},\dots ,Z_{i,t-1}=u)=P(Z_{it}=s\mid Z_{i,t-1}=u).$$ (2)

Notably, model (2) is guaranteed by the assumed property of Markov chain. A common setting for the initial distribution of Z_i1 is the multinomial distribution with probability (π₁, … , π_S)′, such that π_s ≥ 0 and ${\mathbf{Z}}_i=(Z_{i1},\dots ,Z_{iT}{)}^{\prime }$. Thus, the hidden-state sequence Z_i = (Z_i1, … , Z_iT)′ is fully specified by the initial and transition probabilities.

Considering that the hidden states usually have natural ranking information in empirical studies, we assume the hidden states {1, … , S) to be ordered and consider a continuation-ratio logit model¹⁴ as follows: for t = 2, … , T, s = 1, … , S – 1, and u = 1, … , S, we have

$$\log \left(\frac{P(Z_{it}=s\mid Z_{i,t-1}=u)}{P(Z_{it}>s\mid Z_{i,t-1}=u)}\right)=\log \left(\frac{p_{itus}}{p_{itu,s+1}+\cdots +p_{ituS}}\right)={\zeta }_{us}+{\tilde{\alpha }}^{\prime }{\mathbf{c}}_{it}+\sum _{j=1}^q{g}_j(x_{itj}),$$ (3)

where the left-hand side is the log odds of transition to state s rather than to a state that is higher than s given Z_i,t–1 = u, ζ_us is a transition-specific intercept, c_it = (c_it1, … , c_itp)′ and x_it = (x_it1, … ,x_itq)′ are the covariate vectors defined in (1), $\tilde{\alpha }=({\tilde{\alpha }}_1,\dots ,{\tilde{\alpha }}_p{)}^{\prime }$ is a p × 1 vector of fixed effect, and g_j(·)s are unknown smoothing functions. Let ϑ_itus = P(Z_it = s∣Z_it ≥ s, Z_i,t–1 = u). Then, the continuation-ratio logits in (3) can be rewritten as

$$\begin{array}{cc}\hfill \log & \left(\frac{P(Z_{it}=s\mid Z_{i,t-1}=u)}{P(Z_{it}>s\mid Z_{i,t-1}=u)}\right)\hfill \\ \hfill & =\log \left(\frac{P(Z_{it}=s,Z_{i,t-1}=u)}{P(Z_{it}\ge s,Z_{i,t-1}=u)-P(Z_{it}=s,Z_{i,t-1}=u)}\right)\hfill \\ \hfill & =\log \left(\frac{P(Z_{it}=s,Z_{i,t-1}=u)∕P(Z_{it}\ge s,Z_{i,t-1}=u)}{1-P(Z_{it}=s,Z_{i,t-1}=u)∕P(Z_{it}\ge s,Z_{i,t-1}=u)}\right)\hfill \\ \hfill & =\log \left(\frac{P(Z_{it}=s\mid Z_{it}\ge s,Z_{i,t-1}=u)}{1-P(Z_{it}=s\mid Z_{it}\ge s,Z_{i,t-1}=u)}\right)\hfill \\ \hfill & =\log \left(\frac{{\vartheta }_{itus}}{1-{\vartheta }_{itus}}\right).\hfill \end{array}$$

Thus, the continuation-ratio logit (3) can be rewritten as a conventional logistic regression model as follows:

$$\log \text{it}({\vartheta }_{itus})={\zeta }_{us}+{\tilde{\alpha }}^{\prime }{\mathbf{c}}_{it}+\sum _{j=1}^q{g}_j(x_{itj}),$$ (4)

where logit(ϑ_itus) is the log odds of Z_it = s given Z_it ≥ s and Z_i,t–1 = u. In model (3) or (4), $\tilde{\alpha }$ and g_j(−)s are assumed to be independent of u and s. This proportional odds assumption is compulsory in modeling an ordinal variable because it ensures the that P(Z_it < 1) < P(Z_it < 2) < ⋯ < P(Z_it < S) for ordered states 1 < 2 < ⋯ < S.^14,15 Moreover, the proportional odds assumption avoids a tedious inference, in which every possible transition of origination and destination elicits a set of parameters, and it, in turn, greatly reduces the complexity and enhances the interpretability of the transition model.

2.2 ∣. Nonparametric modeling

We use linear basis expansion to estimate the nonparametric functions f_sj(·) and g_j(·) in (1) and (3). Given that g_j(·) can be regarded as a special case (without a state-specific setting) of f_sj(·) we describe only the modeling of f_sj(·) in this section. Specifically, f_sj(x_itj) can be approximated as follows:

$$f_{sj}(x_{itj})=\sum _{m=1}^{M_j}{\beta }_{sjm}{h}_m(x_{itj})={\beta }_{sj}^{\prime }{\mathbf{h}}_{itj},$$ (5)

where h_m(·)s are basis functions, such as piecewise polynomials or natural cubic splines,¹⁶ h_itj = (h₁(x_itj), …, h_Mj (x_itj))′, and M_j is the number of basis functions that are used to estimate the jth unknown smoothing function. For notational simplicity, M_j is set to be invariant to states. An extension to relax this assumption is straightforward.

An important issue regarding the model selection of (1) and (3) is whether a functional effect, eg, f_sj(·), truly exists or not. In this study, we utilize a norm ∥·∥ to quantify the magnitude of nonparametric function f_sj. Let x_sj and H_sj denote the submatrix of x_j = (x_11j, … ,x_nTj)′ and H_j, respectively, with the rows corresponding to Z_it ≠ s deleted, where H_j is formed by

$${\mathbf{H}}_j=\left(\begin{array}{c}\hfill {\mathbf{h}}^{\prime }(x_{11j})\hfill \\ \hfill ⋮\hfill \\ \hfill {\mathbf{h}}^{\prime }(x_{nTj})\hfill \end{array}\right)={\left(\begin{array}{ccc}\hfill h_1(x_{11j})\hfill & \hfill \cdots \hfill & \hfill h_{M_j}(x_{11j})\hfill \\ \hfill ⋮\hfill & \hfill \ddots \hfill & \hfill ⋮\hfill \\ \hfill h_1(x_{nTj})\hfill & \hfill \cdots \hfill & \hfill h_{M_j}(x_{nTj})\hfill \end{array}\right)}_{nT\times M_j}.$$ (6)

The norm of f_sj, ∥f_sj∥, is defined as $\sqrt{E(f_{sj}^2({\mathbf{x}}_{sj}))}$. Then, f_sj = 0 is equivalent to ∥f_sj∥ = 0. On the basis of (5), ∥f_sj can be approximated by $\Vert {\beta }_{sj}{\Vert }_{{\mathrm{G}}_{sj}}=({\beta }_{sj}^{\prime }{\mathbf{G}}_{sj}{\beta }_{sj}{)}^{1∕2}$ with positive definite matrix ${\mathbf{G}}_{sj}={\mathbf{H}}_{sj}^{\prime }{\mathbf{H}}_{sj}∕n_s$, where n_s is the number of subjects staying in state s. Denote $\Vert {\widehat{f}}_{sj}\Vert$ as the estimator of ∥f_sj∥. In the model selection procedure, if $\Vert {\widehat{f}}_{sj}\Vert =0$, then f_sj = 0. The nonparametric function g_j(x_itj) can be similarly approximated by

$$g_j(x_{itj})=\sum _{m=1}^{M_j}{\tilde{\beta }}_{jm}{h}_m(x_{itj})={\tilde{\beta }}_j^{\prime }{\mathbf{h}}_{itj},$$ (7)

where ${\tilde{\mathit{\beta }}}_{jm}$, h_m(·), h_itj, M_j, and ${\tilde{\mathit{\beta }}}_j$ are defined in the same manner as those in (5). Likewise, ∥g_j∥ can be approximated by $\Vert g_j{\Vert }_{{\tilde{\mathrm{G}}}_j}=({\tilde{\beta }}_j^{\prime }{\tilde{\mathbf{G}}}_j{\tilde{\beta }}_j{)}^{1∕2}$, where ${\tilde{\mathbf{G}}}_j={\mathbf{H}}_j^{\prime }{\mathbf{H}}_j∕(n\times (T-1))$.

Let y_i = (y_i1 , ⋯ , y_iT)′, $\mathbf{Y}=({\mathbf{y}}_1^{\prime },\dots ,{\mathbf{y}}_n^{\prime }{)}^{\prime }$, ${\mathbf{d}}_{it}=({\mathbf{c}}_{it}^{\prime },{\mathbf{x}}_{it}^{\prime }{)}^{\prime }$, ${\mathbf{D}}_i=({\mathbf{d}}_{i1}^{\prime },\dots ,{\mathbf{d}}_{iT}^{\prime }{)}^{\prime }$, $\mathbf{D}=({\mathbf{D}}_1^{\prime },\dots ,{\mathbf{D}}_n^{\prime }{)}^{\prime }$, Z_i = (Z_il, … ,Z_iT)′, $\mathbf{Z}=({\mathbf{Z}}_1^{\prime },\dots ,{\mathbf{Z}}_n^{\prime }{)}^{\prime }$, and θ be the vector that includes all the unknown parameters. With the linear basis expansion, the complete-data log-likelihood function is given by

$$\begin{array}{cc}\hfill & \log p(\mathbf{Y},\mathbf{D},\mathbf{Z}\mid \theta )=\sum _{i=1}^n\left[\log p({\mathbf{y}}_i\mid {\mathbf{D}}_i,{\mathbf{Z}}_i,\theta )+\log p({\mathbf{Z}}_i\mid {\mathbf{D}}_i,\theta )\right]\hfill \\ \hfill =& \sum _{i=1}^n\sum _{t=1}^T\log p(y_{it}\mid {\mathbf{d}}_{it},Z_{it}=s,\theta )+\sum _{i=1}^n\sum _{t=2}^T\log p(Z_{it}=s\mid Z_{i,t-1}=u,{\mathbf{d}}_{it},\theta )+\sum _{i=1}^n\log p(Z_{i1}=s\mid \theta )\hfill \\ \hfill =& -\frac{1}{2}\sum _{i=1}^n\sum _{t=1}^T\left[\log (2\pi {\Psi }_s)+(y_{it}-{\eta }_{it}{)}^2∕{\Psi }_s\right]+\sum _{i=1}^n\sum _{t=2}^T\log (p_{itus})+\sum _{i=1}^n\log (p_{i10s}),\hfill \end{array}$$ (8)

where

$$\begin{array}{cc}\hfill {\eta }_{it}& ={\mu }_s+{\alpha }_s^{\prime }{\mathbf{c}}_{it}+\sum _{j=1}^q{\beta }_{sj}^{\prime }{\mathbf{h}}_{itj},p_{i10s}={\pi }_s,s=1,\dots ,S,\hfill \\ \hfill p_{itu1}& =\frac{\text{exp}\{a_{itu1}\}}{1+\text{exp}\{a_{itu1}\}},p_{i1uS}=\prod _{j=1}^{S-1}\frac{1}{1+\text{exp}\{a_{ituj}\}},\hfill \\ \hfill p_{itus}& =\frac{\text{exp}\{a_{itus}\}}{1+\text{exp}\{a_{itus}\}}\prod _{j=1}^{s-1}\frac{1}{1+\text{exp}\{a_{ituj}\}},s=2,\dots ,S-1,\hfill \end{array}$$ (9)

with $a_{itus}={\zeta }_{us}+{\tilde{\alpha }}^{\prime }{\mathbf{c}}_{it}+{\sum }_{j=1}^q{\tilde{\beta }}_j^{\prime }{\mathbf{h}}_{itj}$.

2.3 ∣. Related issues

The proposed model is not identifiable because of the following two model indeterminacies. First, the basis functions involved in basis expansion may contain constant parts. When applying such constant basis functions in every f_sj(·) and/or g_j(·), each unknown function is not identifiable up to a constant. To address this issue, we need to impose the following constraints on the unknown functions to enforce their integrations in the ranges of predictors to zero^17,18:

$${\int }_{{\chi }_j}{f}_{sj}(x)dx=0,\text{for}s=1,\dots ,S,j=1,\dots ,q,$$ (10)

where ${\chi }_j$ is the domain of x_j. Second, the label switching problem, which is caused by the invariance of the likelihood function to a random permutation of the state labels, arises and leads to a multimodal posterior under a symmetric prior specification. We address this issue by imposing constraint μ₁ < ⋯ < μ_S on posterior samples.

3 ∣. BAYESIAN ANALYSIS

3.1 ∣. Adaptive group lasso penalties

We explain the key idea of the adaptive group lasso penalties in the context of a simple linear regression model: y = μ1_n + Xβ + δ, where y is the response vector, μ is an intercept, 1_n is an n-dimensional vector of all elements being 1, X is a standardized design matrix, δ is the vector of residuals, δ ~ N(0, ψI_n), and I_n is an n-dimensional identity matrix. Tibshirani¹⁹ first introduced the lasso procedure for simultaneous model selection and parameter estimation of the above linear regression. The lasso estimator of β can be expressed as

$${\text{argmin}}_{\beta }\left\{(\mathbf{y}-\mu 1_n-\mathbf{X}\beta {)}^{\prime }(\mathbf{y}-\mu 1_n-\mathbf{X}\beta )+\gamma \sum _{h=1}^p\mid {\beta }_h\mid \right\},$$ (11)

where γ ≥ 0 can be regarded as an L₁-penalty that automatically shrinks unimportant covariate effects to 0. Given that the covariates in X are standardized to the same scale, the magnitudes of the coefficients in β can represent the significance of predictors. If some elements of β are close to 0, then the corresponding covariates are unimportant and can be removed from the model.

However, when simply applying lasso to the proposed semiparametric HMMs, at least two problems exist. First, lasso is originally designed for the selection of individual variables. Yuan and Lin²⁰ showed that lasso tends to select more factors than necessary in the presence of group variables. Moreover, the pairwise correlations among group variables jeopardize the model selection accuracy of the lasso estimator.²¹ In this study, high correlations exist among the basis functions h_m(x_it)s in the conditional and transition models because they can be viewed as different transformations of x_it. Consequently, the linear basis expansion involves group variables and should not be treated separately. Second, lasso applies the same tuning parameter γ to different regression coefficients, thereby introducing the same amount of shrinkage to different covariate effects. This inflexible setting may add considerable bias to the resulting estimates.^22,23

To address the aforementioned issues, Yuan and Lin²⁰ proposed group lasso to perform model selection among group variables. Wang and Leng²⁴ further developed adaptive group lasso to assign different tuning parameters to different groups of regression coefficients. Let $\alpha =({\alpha }_1^{\prime },\dots ,{\alpha }_S^{\prime }{)}^{\prime }$, ${\beta }_s^{\prime }=({\beta }_{s1}^{\prime },\dots ,{\beta }_{sq}^{\prime }{)}^{\prime }$, $\beta =({\beta }_1^{\prime },\dots ,{\beta }_S^{\prime }{)}^{\prime }$, $\tilde{\beta }=({\tilde{\beta }}_1^{\prime },\dots ,{\tilde{\beta }}_q^{\prime }{)}^{\prime }$, and ${\theta }^{\ast }=({\alpha }^{\prime },{\tilde{\alpha }}^{\prime },{\beta }^{\prime },{\tilde{\beta }}^{\prime }{)}^{\prime }$. On the basis of the proposed model defined in (1)–(7), the adaptive group lasso estimator can be formulated as

$${\text{arg min}}_{{\theta }^{\ast }}\left\{\sum _{i=1}^n\sum _{t=1}^T(y_{it}-{\eta }_{it}{)}^{\prime }(y_{it}-{\eta }_{it})-\sum _{i=1}^n\sum _{t=2}^T\log (p_{itus})-P({\theta }^{\ast })\right\},$$ (12)

where η_η is the mean ofy_it, p_itus is the transition probability defined in (2) and (9), and

$$P({\theta }^{\ast })=\sum _{s=1}^S\sum _{h=1}^p{\gamma }_{\alpha sh}\mid {\alpha }_{sh}\mid +\sum _{h=1}^p{\tilde{\gamma }}_{\alpha h}\mid {\tilde{\alpha }}_h\mid +\sum _{s=1}^S\sum _{j=1}^q{\gamma }_{\beta sj}\Vert {\beta }_{sj}{\Vert }_{{\mathrm{G}}_{sj}}+\sum _{j=1}^q{\tilde{\gamma }}_{\beta j}\Vert {\tilde{\beta }}_j{\Vert }_{{\tilde{\mathrm{G}}}_j},$$ (13)

in which α_sh, ${\tilde{\alpha }}_h$, β_sj, and ${\tilde{\beta }}_j$ are coefficients of fixed effects and basis functions in the conditional and transition models; γ_ash, ${\tilde{\gamma }}_{\alpha h}$, γ_βsj and ${\tilde{\gamma }}_{\beta j}$ are the corresponding tuning parameters; and the norms ∥β_sj∥_Gsl and $\Vert {\tilde{\beta }}_j{\Vert }_{{\tilde{\mathrm{G}}}_l}$ are defined in Section 2.2. Notably, the coefficients of discrete covariates, namely, α_sh and ${\tilde{\alpha }}_h$, are simply assigned adaptive penalties, whereas the coefficients of unknown smooth functions β_sj and ${\tilde{\beta }}_j$, which have groupwise features, are assigned adaptive group lasso penalties. The initial probabilities p_i10s are excluded from (13) because they are independent of ${\tilde{\alpha }}_h$ and ${\tilde{\beta }}_j$.

Yuan and Lin²⁰ argued that the penalty function in (13) is intermediate between the L₁-penalty used in lasso and the L₂-penalty used in ridge regression. Therefore, the adaptive group lasso not only has the same advantages of lasso in model selection but also alleviates the problem caused by the existence of high pairwise correlation among basis functions. Furthermore, with the use of different tuning parameters γ_βsj and ${\tilde{\gamma }}_{\beta j}$, the adaptive group lasso automatically imposes large penalties on groups of unimportant coefficients to efficiently shrink them to 0. Moreover, the penalty terms ∥β_sj∥_Gsj and $\Vert {\tilde{\beta }}_j{\Vert }_{{\tilde{\mathrm{G}}}_j}$ can be regarded as the scaled version of the groupwise prediction penalty suggested by Buhlmann and Van De Geer.²⁵ With the great power of adaptive group lasso, the estimation of all unknown parameters and the structure detection for important functional covariate effects on the observed response and on the hidden-state process can be simultaneously and efficiently obtained.

3.2 ∣. BaGlasso and prior specification

Under the Bayesian framework, the adaptive group lasso procedure can be implemented by introducing a multivariate conditional Laplace prior to the regression coefficients in ${\theta }^{\ast }=({\alpha }^{\prime },{\tilde{\alpha }}^{\prime },{\beta }^{\prime },{\tilde{\beta }}^{\prime }{)}^{\prime }$ as follows:

$$p({\theta }^{\ast }\mid \Psi ,{\sigma }^2)\propto \text{exp}\left(-\sum _{h=1}^p\left(\frac{{\gamma }_{\alpha sh}}{\sqrt{{\Psi }_s}}\mid {\alpha }_{sh}\mid +\frac{{\tilde{\gamma }}_{\alpha h}}{\sqrt{{\sigma }^2}}\mid {\tilde{\alpha }}_h\mid \right)-\sum _{j=1}^q\left(\frac{{\gamma }_{\beta sj}}{\sqrt{{\Psi }_s}}\Vert {\beta }_{sj}{\Vert }_{{\mathrm{G}}_{sj}}+\frac{{\tilde{\gamma }}_{\beta j}}{\sqrt{{\sigma }^2}}\Vert \tilde{\beta }{\Vert }_{{\tilde{\mathrm{G}}}_j}\right)\right),$$ (14)

where ψ = (ψ₁, … , ψ_S)′. This conditional Laplace prior can be represented as a scale mixture of normals with an exponential mixing density, leading to a hierarchical representation of the full model as follows: for i = 1, … , n, t = 1, … , T, s = 1, … ,S, h = 1, … , p, and j = 1, … , q, we have

$$\begin{array}{cc}\hfill & y_{it}\mid Z_{it}=s,{\mu }_s,{\mathbf{c}}_{it},{\alpha }_s,{\beta }_s,{\Psi }_s\sim N({\eta }_{it},{\Psi }_s),\hfill \\ \hfill & {\alpha }_s\mid {\Psi }_s,{\tau }_{\alpha s1}^2,\dots ,{\tau }_{\alpha sp}^2\stackrel{\text{ind}}{\sim }{N}_p(0,{\Psi }_s,{\Sigma }_{\alpha s}),{\Sigma }_{\alpha s}=\text{diag}({\tau }_{\alpha s1},\dots ,{\tau }_{\alpha sp})\hfill \\ \hfill & \tilde{\alpha }\mid {\sigma }^2,{\tilde{\tau }}_{\alpha 1}^2,\dots ,{\tilde{\tau }}_{\alpha p}^2\sim N_p(0,{\sigma }^2{\tilde{\Sigma }}_{\alpha }),{\tilde{\Sigma }}_{\alpha }=\text{diag}({\tilde{\tau }}_{\alpha 1},\dots ,{\tilde{\tau }}_{\alpha p})\hfill \\ \hfill & {\beta }_{sj}\mid {\Psi }_s,{\tau }_{\beta sj}^2\stackrel{\text{ind}}{\sim }{N}_{M_j}\left(0,{\Psi }_s{\tau }_{\beta sj}^2{\mathbf{G}}_{sj}^{-1}\right),{\tilde{\beta }}_j\mid {\sigma }^2,{\tilde{\tau }}_{\beta j}^2\stackrel{\text{ind}}{\sim }{N}_{M_j}\left(0,{\sigma }^2{\tilde{\tau }}_{\beta j}^2{\tilde{\mathbf{G}}}_j^{-1}\right),\hfill \\ \hfill & {\tau }_{\alpha sh}^2\stackrel{\text{ind}}{\sim }\text{Gamma}\left(1,\frac{{\gamma }_{\alpha sh}^2}{2}\right),{\tilde{\tau }}_{\alpha h}^2\stackrel{\text{ind}}{\sim }\text{Gamma}\left(1,\frac{{\tilde{\gamma }}_{\alpha h}^2}{2}\right)\hfill \\ \hfill & {\tau }_{\beta sj}^2\stackrel{\text{ind}}{\sim }\text{Gamma}\left(\frac{M_j+1}{2},\frac{{\gamma }_{\beta sj}^2}{2}\right),{\tilde{\tau }}_{\beta j}^2\stackrel{\text{ind}}{\sim }\text{Gamma}\left(\frac{M_j+1}{2},\frac{{\tilde{\gamma }}_{\beta j}^2}{2}\right)\hfill \\ \hfill & {\Psi }_s^{-1}\stackrel{\text{ind}}{\sim }\text{Gamma}({\alpha }_{\Psi s0},{\beta }_{\Psi s0}),{\sigma }^{-2}\sim \text{Gamma}({\alpha }_{\sigma 0},{\beta }_{\sigma 0},\hfill \end{array}$$ (15)

where $\stackrel{\text{ind}}{\sim }$ represents “independently distributed according to” and η_it is defined in (9). For the tuning parameters γ_αsh, ${\tilde{\gamma }}_{\alpha h}$, γ_βsj, and ${\tilde{\gamma }}_{\beta j}$, we assign gamma priors as follows:

$$\begin{array}{cc}\hfill p({\gamma }_{\alpha sh}^2)& \stackrel{\text{ind}}{\sim }\text{Gamma}({\alpha }_{\alpha sh0},{\beta }_{\alpha sh0}),p({\tilde{\gamma }}_{\alpha h}^2)\stackrel{\text{ind}}{\sim }\text{Gamma}({\tilde{\alpha }}_{\alpha h0},{\tilde{\beta }}_{\alpha h0}),\hfill \\ \hfill p({\gamma }_{\beta sj}^2)& \stackrel{\text{ind}}{\sim }\text{Gamma}({\alpha }_{\beta sj0},{\beta }_{\beta sj0}),p({\tilde{\gamma }}_{\beta j}^2)\stackrel{\text{ind}}{\sim }\text{Gamma}({\tilde{\alpha }}_{\beta j0},{\tilde{\beta }}_{\beta j0}),\hfill \end{array}$$ (16)

where α_αsh0, ${\tilde{\alpha }}_{\alpha h0}$, α_βsj0, ${\tilde{\alpha }}_{\beta j0}$, β_αsh0, ${\tilde{\beta }}_{\alpha h0}$, β_βsj0, and ${\tilde{\beta }}_{\beta j0}$ are hyperparameters with prespecified values. We follow a common practice in the literature^11,12 to set ${\alpha }_{\alpha sh0}={\tilde{\alpha }}_{\alpha h0}={\alpha }_{\beta sj0}={\tilde{\alpha }}_{\beta j0}=1$, ${\beta }_{\alpha sh0}={\tilde{\beta }}_{\alpha h0}=0.1$, and ${\beta }_{\beta sj0}={\tilde{\beta }}_{\beta j0}=0.01$ to obtain relatively dispersed gamma priors. The key idea of BaGlasso is to properly update the tuning parameters by using the data, thereby automatically imposing large penalties on unimportant coefficients. This target can be naturally achieved by introducing dispersed priors with small hyperparameters β_αsh0, ${\tilde{\beta }}_{\alpha h0}$, β_βsj0, and ${\tilde{\beta }}_{\beta j0}$. We explain this regularization procedure further through the posterior distribution of the tuning parameters as follows:

$$\begin{array}{cc}\hfill & p\left(\phantom{\mid }{\tau }_{\alpha sh}^{-2}\mid \cdot \right)\sim \text{In}-\text{Gaussian}\left(\sqrt{\frac{{\gamma }_{\alpha sh}^2{\Psi }_s}{\mid {\alpha }_{sh}{\mid }^2}},{\gamma }_{\alpha sh}^2\right),p\left(\phantom{\mid }{\tilde{\tau }}_{\alpha h}^{-2}\mid \cdot \right)\sim \text{In}-\text{Gaussian}\left(\sqrt{\frac{{\tilde{\gamma }}_{\alpha sh}^2{\sigma }^2}{\mid {\tilde{\alpha }}_{sh}{\mid }^2}},{\tilde{\gamma }}_{\alpha sh}^2\right),\hfill \\ \hfill & p\left(\phantom{\mid }{\tau }_{\beta sj}^{-2}\mid \cdot \right)\sim \text{In}-\text{Gaussian}\left(\sqrt{\frac{{\gamma }_{\beta sj}^2{\Psi }_s}{\Vert {\beta }_{sj}{\Vert }_{{\mathrm{G}}_{sj}}}},{\gamma }_{\beta sj}^2\right),p\left(\phantom{\mid }{\tilde{\tau }}_{\beta j}^{-2}\mid \cdot \right)\sim \text{In}-\text{Gaussian}\left(\sqrt{\frac{{\tilde{\gamma }}_{\beta sj}^2{\sigma }^2}{\Vert {\tilde{\beta }}_j{\Vert }_{{\tilde{\mathrm{G}}}_j}}},{\tilde{\gamma }}_{\beta j}^2\right),\hfill \\ \hfill & p\left(\phantom{\mid }{\gamma }_{\alpha sh}^2\mid \cdot \right)\sim \text{Gamma}\left({\alpha }_{\alpha sh0}+1,{\beta }_{\alpha sh0}+\frac{{\tau }_{\alpha sh}^2}{2}\right),p\left(\phantom{\mid }{\tilde{\gamma }}_{\alpha h}^2\mid \cdot \right)\sim \text{Gamma}\left({\tilde{\alpha }}_{\alpha h0}+1,{\tilde{\beta }}_{\alpha h0}+\frac{{\tilde{\tau }}_{\alpha h}^2}{2}\right),\hfill \\ \hfill & p\left(\phantom{\mid }{\gamma }_{\beta sj}^2\mid \cdot \right)\sim \text{Gamma}\left({\alpha }_{\beta sj0}+\frac{M_j+1}{2},{\beta }_{\beta sj0}+\frac{{\tau }_{\beta sj}^2}{2}\right),\hfill \\ \hfill & p\left(\phantom{\mid }{\gamma }_{\beta j}^2\mid \cdot \right)\sim \text{Gamma}\left({\tilde{\alpha }}_{\beta j0}+\frac{M_j+1}{2},{\tilde{\beta }}_{\beta j0}+\frac{{\tau }_{\beta j}^2}{2}\right),\hfill \end{array}$$ (17)

where “In-Gaussian(·)” denotes the inverse Gaussian distribution. We omit the tedious subscripts and use generic terms τ and γ to simplify notations below. On the basis of (17), if the coefficients are significant, then τ² tends to be large. As a result, the corresponding tuning parameter γ is dominated by τ², leading γ to be mostly data driven. If the coefficients are insignificant, then τ² tends to be small. Consequently, the corresponding tuning parameter γ is dominated by the dispersed prior information, leading to a large value of γ. Thus, the degree of dispersion of the gamma priors in (16) determines the amount of penalties imposed on unimportant predictors. This rationale explains why we assign higher dispersed priors to ${\gamma }_{\beta sj}^2$ and ${\tilde{\gamma }}_{\beta j}^2$ than to ${\gamma }_{\alpha sh}^2$ and ${\tilde{\alpha }}_{\alpha h}^2$ because the coefficients of the nonlinear parts of basis functions are more difficult to shrink to 0 than those of the linear parts.

To conduct a full Bayesian analysis, we specify appropriate prior distributions for other unknown parameters, such as μ_s, π_s, and ζ_us. For u = 1, … ,S and s = 1, … , S, the following Gaussian priors are considered:

$$p({\mu }_s)\stackrel{\text{ind}}{\sim }N\left({\mu }_{s0},{\sigma }_{\mu s0}^2\right),p({\pi }_s)\stackrel{\text{ind}}{\sim }N\left({\pi }_{s0},{\sigma }_{\pi s0}^2\right),p({\zeta }_{us})\stackrel{\text{ind}}{\sim }N\left({\zeta }_{us0},{\sigma }_{\zeta us0}^2\right),$$ (18)

where μ_s0, ${\sigma }_{\mu s0}^2$, π_s0, ${\sigma }_{\pi s0}^2$, ζ_us0, and ${\sigma }_{\zeta us0}^2$ are hyperparameters with preassigned values.

3.3 ∣. Posterior inference

The Bayesian estimate of θ can be obtained through the mean or mode of the posterior samples drawn from p(θ∣Y). However, directly sampling from p(θ∣Y) is intractable because of the existence of latent states. To address this issue, we adopt the data augmentation technique to work on p(θ, Z∣Y) and utilize the Gibbs sampler to simulate each of the unknowns from its full conditional distribution iteratively. Owing to the nonlinearity of the continuation-logit transition model, the full conditional distributions related to the transition model have complex forms. Thus, Markov chain Monte Carlo (MCMC) methods, such as the forward filtering and backward sampling algorithm²⁶ and the Metropolis-Hastings algorithm,^27,28 are used to sample from them. The details of the full conditional distributions are provided in the Appendix.

For nonparametric functions involved in (1) and (3), as suggested by Li et al,²⁹ a functional effect of a covariate is detected as significant and included in the regression if at least one of its coefficients of the basis expansion has a two-sided 95% credible interval estimate that does not cover zero. The latent state Z_it, which usually has actual meaning in empirical studies, is also of great interest for scientists. By using posterior samples, we can estimate the hidden state as follows:

$${\widehat{Z}}_{it}=\text{arg}\underset{s\in \{1,\dots ,S\}}{\max }P(Z_{it}=s\mid {\mathrm{y}}_i,\theta )\approx \text{arg}\underset{s\in \{1,\dots ,S\}}{\max }\frac{1}{L}\sum _{l=1}^L\mathrm{I}\left(Z_{it}^{(l)}=s\right),$$ (19)

where $Z_{it}^{(l)}$ denotes the latent allocation of y_it at the lth iteration, and $\frac{1}{L}{\sum }_{l=1}^M\mathrm{I}(Z_{it}^{(l)}=s)$ is the posterior mean of the latent allocations of y_it drawn from the MCMC iterations.

3.4 ∣. Determination of the number of hidden states

In the analysis of HMMs, the number of hidden states, S, is usually determined a priori. We use a modified DIC, which was developed by Celeux et al,³⁰ for model comparison in the presence of incomplete data, to determine the number of hidden states of the proposed model. The modified DIC is defined as follows:

$$\text{DIC}=\overline{D(\theta )}+p_D,$$ (20)

where $\overline{D(\theta })=E_{\theta ,\mathrm{Z}}[-2\log p(\mathbf{Y},\mathbf{Z}\mid \theta )\mid \mathbf{Y}]$ is the posterior mean deviance to reflect the goodness of fit of the model, p_D is the effective number of parameters to penalize an overcomplex model, and p_D = E_θ,Z[−2logp(Y, Z∣θ)∣Y] + 2E_Z[logp(Y, Z)∣E_θ[θ∣Y, Z])∣Y]. The expectations involved in (20) can be approximated by averaging the posterior samples collected through the MCMC algorithm.^30,31 The model with the smallest value of DIC is selected.

4 ∣. SIMULATION STUDY

This section contains two simulations: Simulation 1 assesses the empirical performance of the proposed BaGlasso for simultaneous estimation and variable selection in the context of semiparametric HMMs, and Simulation 2 examines the performance of the DIC in determining the number of hidden states in semiparametric HMMs.

4.1 ∣. Simulation 1

We consider 100 simulated data sets, each consisting of n = 700 subjects and T = 9 time points. For each data set, observations are generated from a two-state semiparametric HMM with a continuous response y_it, two discrete covariates c_it = (c_it1, c_it2)^r (p = 2), and three continuous covariates x_it = (x_it1, x_it2, x_it3)′ (q = 3). For i = 1, … , 700 and t = 1, … , 9, c_it1 and c_it2 are independently generated from the Bernoulli distribution with a probability of success of 0.5, and x_it1, x_it2, and x_it3 are generated from U(−1, 1), N(0, 1), and $N(\sqrt{t},1)$, respectively, and they are standardized to the same scale beforehand. Here, x_it1 and x_it2 are set as time-invariant covariates, whereas x_it3 is set as a time-variant one. The conditional regression model is defined as follows:

$$[y_{it}\mid Z_{it}=s]={\mu }_s+{\alpha }_{s1}{c}_{it1}+{\alpha }_{s2}{c}_{it2}+f_{s1}(x_{it1})+f_{s2}(x_{it2})+f_{s3}(x_{it3})+{\delta }_{it},$$ (21)

where f₁₁(x_it1) = 0, f₁₂(x_it2) = sin(1.5x_it2) + x_it2 – 0.6, f₁₃(x_it1) = −0.8x_it3, f₂₁(x_it1) = 2.08 – exp(x_it1), f₂₂(x_it2) = 0, and f₂₃(x_it3) = −0.105 + cos(2x_it3) + 0.5x_it3.

The transition model is defined as

$$\log \text{it}({\vartheta }_{itus})={\zeta }_{us}+{\tilde{\alpha }}_1{c}_{it1}+{\tilde{\alpha }}_2{c}_{it2}+g_1(x_{it1})+g_2(x_{it2})+g_3(x_{it3}),$$ (22)

where g₁(x_it1) = −log(2 + x_it1)/(2 – x_it1), g₂(x_it2) = 1.5x_it2, and g₃(x_it3) = 0. The true population values of the unknown parameters are set as μ = (μ₁, μ₂)′ = (−1, 1)′, π = (π₁, π₂)′ = (0.5, 0.5)′, ζ₁₁ = ζ₂₁ = 0.5, α₁ = (α₁₁, α₁₂)′ = (0, 0.5)′, α₂ = (α₂₁, α₂₂)′ = (−0.5, 0)−, $\tilde{\alpha }=({\tilde{\alpha }}_1,{\tilde{\alpha }}_2{)}^{\prime }=(-1,0{)}^{\prime }$, and ψ = (ψ₁, ψ₂)′ = (0.36, 0.16)′.

In this study, we use a simple version of natural cubic splines derived from a truncated power series basis function¹⁶ to approximate the nonparametric functions: h_j1(x_itj) = 1, h_j2(x_itj) = x_itj, and h_j,m+2 = u_jm(x_itj) − u_j,Mj–1(x_itj) for m = 1, … , M_j – 2, where $u_{j,m}(x_{itj})=[(x_{itj}-{\kappa }_j{M}_j{)}_{+}^3-(x_{itj}-{\kappa }_{jm}{)}_{+}^3]∕({\kappa }_j{M}_j-{\kappa }_{jm})$, and κ_jm, m = 1, … , M_j, are the knots taken in the range of x_itj. The prior inputs in (15), (16), and (18) are assigned as follows: μ_s0 = ζ_us0 = π_s0 = 0, ${\sigma }_{\mu s0}^2={\sigma }_{\zeta us0}^2={\sigma }_{\pi 0}^2=1$, α_ψs0 = α_σ0 = 9, β_ψs0 = β_σ0 = 4, ${\alpha }_{\alpha sh0}={\tilde{\alpha }}_{\alpha h0}={\alpha }_{\beta sj0}={\tilde{\alpha }}_{\beta j0}=1$, ${\beta }_{\alpha sh0}={\tilde{\beta }}_{\alpha h0}=0.1$, and ${\beta }_{\beta sj0}={\tilde{\beta }}_{\beta j0}=0.01$. For each x_itj, M_j = 10 knots are used. We impose the constraint μ₁ < μ₂ in each MCMC iteration to avoid label switching and check the convergence of the algorithm using the estimated potential scale reduction (EPSR) proposed by Gelman et al.³² The MCMC algorithm converges within 5000 iterations. Thus, we collect posterior samples with a size of 20 000 with the first 10 000 as burn-in iterations. The performance of Bayesian estimates is assessed through the bias (BIAS) and the root-mean-square error (RMSE) between the Bayesian estimates and the true population values of the parameters.

Table 1 summarizes the estimation results on the basis of the 100 data sets. The BIAS and RMSE for most of the parameters are close to zero, indicating a satisfactory performance of Bayesian estimation regarding the parametric part. Figure 1 depicts the averages of the pointwise posterior means of the nonparametric functions, along with their 2.5% and 97.5% pointwise quantiles. Three nonexistent functions are successfully shrunk to almost zero by the proposed BaGlasso procedure. The posterior means of other nonzero nonparametric functions are close to their true curves, and all the ranges of the 2.5% and 97.5% pointwise quantiles are small, indicating that the estimated nonparametric curves can correctly recover the complex functional relationships between the response and covariates. Moreover, the average of the correct classification rates calculated from (19) is approximately 95%, implying the good performance of the proposed method in identifying the hidden states of the observations.

TABLE 1.

Bayesian estimates of the parameters in the simulation study

Parameters in the Conditional Regression Model
State 1				State 2
Par	True	Est	RMSE	Par	True	Est	RMSE
μ1	−1.0	−0.969	0.041	μ2	1.0	1.006	0.033
α11	0.0	−0.000	0.025	α21	−0.5	−0.499	0.015
α12	0.5	0.501	0.023	α22	0.0	0.001	0.015
ψ1	0.36	0.392	0.034	ψ2	0.16	0.191	0.032

Parameters in the Probability Transition Model
Par	True	Est	RMSE	Par	True	Est	RMSE
${\tilde{\alpha }}_1$	−1.0	−0.985	0.080	${\tilde{\alpha }}_2$	0.0	−0.000	0.055
π1	0.5	0.528	0.036	π2	0.5	0.472	0.036
ζ11	0.5	0.501	0.152	ζ21	0.5	0.504	0.152

Abbreviation: RMSE, root-mean-square error.

FIGURE 1.

Estimates of the unknown smooth functions in the simulation study. The solid curves represent the true curves, and the dashed curves represent the estimated posterior means and the 2.5% and 97.5% pointwise quantiles on the basis of 100 replications

To reveal the sensitivity of Bayesian estimates to the input of prior distributions, we disturb the prior input as follows: μ_s0 = ζ_us0 = π_s0 = 2, ${\sigma }_{\mu s0}^2={\sigma }_{\zeta us0}^2={\sigma }_{\pi s0}^2=2$, α_ψs0 = 3, β_ψs0 = 2, ${\alpha }_{\alpha sh0}={\tilde{\alpha }}_{\alpha h0}={\alpha }_{\beta sj0}={\tilde{\alpha }}_{\beta j0}=1$, ${\beta }_{\alpha sh0}={\tilde{\beta }}_{\alpha h0}=0.5$, and ${\beta }_{\beta sj0}={\tilde{\beta }}_{\beta j0}=0.01$. The Bayesian results obtained under the disturbed prior are similar and not reported.

Notably, this simulation study contains five covariates in the conditional and transition models, which result in a large number (2^2×5) of competing models with various combinations of covariates in both models. Traditional Bayesian model selection statistics, such as the Bayes factor and the DIC, are extremely time consuming in performing variable selection because they compare these competing models in a pairwise basis. By contrast, the proposed BaGlasso procedure automatically selects important predictors and avoids the tedious pairwise comparison, thereby greatly reducing the computational time. In this simulation study, the computing time for simultaneous variable selection and parameter estimation in each replication is 48 minutes using a PC Intel Core i7-6700 3.40-GHz CPU and 16 G of RAM.

4.2 ∣. Simulation 2

To examine the performance of the DIC in determining the number of hidden states of a semiparametric HMM, we consider five competing models M₁, M₂, M₃, M₄, and M₅, where M_s is a model defined by (1)–(3) with S = s, s = 1, … , 5. Here, M₄ is the true model, whereas M₁, M₂, M₃, and M₅ are models with incorrect numbers of hidden states. To mimic the scenario of the ADNI data set in the subsequent real example, we generate 100 data sets from (1)–(3) with S = 4, n = 633, T = 4,p = 4, and q = 3. For i = 1, … , 633 and t = 1, … , 4, c_it1 to c_it4 are independently generated from the Bernoulli distribution with a probability of success of 0.5, and x_it1, x_it2, and x_it3 are generated from U(−1, 1), N(0, 1), and $N(\sqrt{t},1)$, respectively, and they are standardized prior to analysis. The true functions are set as f₁₁(x_it1) = 0, f₁₂(x_it2) = sin(1.5x_it2)+ x_it2 – 0.6, f₁₃(x_it1) = –0.8x_it3, f₂₁(x_it1) = 2.08 – exp(x_it1), f₂₂(x_it2) = 0, f₂₃(x_it3) = −0.105 + cos(2x_it3) + 0.5x_it3, f₃₁(x_it1) = 0.5x_it1, f₃₂(x_it2) = 0, f₃₃(x_it1) = −x_it3, f₄₁(x_it1) = 2x_it1, f₄₂(x_it2) = 1.5x_it2, f₄₃(x_it3) = 0, g₁(x_it1) = −log(2 + x_it1)/(2 − x_it1), g₂(x_it2) = 1.5x_it2, and g₃(x_it3) = 0. The true population values of the unknown parameters are set as μ = (μ₁, μ₂, μ₃, μ₄)′ = (−4, −2, 2, 4)′, π = (π₁, π₂, π₃, π₄)′ = (0.25, 0.25, 0.25, 0.25)′, ζ₁₁ = ζ₂₁ = ζ₃₁ = ζ₄₁ = −1, ζ₁₂ = ζ₂₂ = ζ₃₂ = ζ₄₂ = 0, ζ₁₃ = ζ₂₃ = ζ₃₃ = ζ₄₃ = 1, α1 = (α₁₁, α₁₂, α₁₃, α₁₄)′ = (1, 0, 0.5, 1)′, α₂ = (α₂₁, α₂₂, α₂₃, α₂₄)′ = (0.5, −0.5, 0, −1)′, α₃ = (α₃₁, α₃₂, α₃₃, α₃₄)′ = (0.5, −1, 1, 0)′, α₄ = (α₄₁, α₄₂, α₄₃, α₄₄)′ = (0.5, 1, −0.5, 0)′, $\tilde{\alpha }=({\tilde{\alpha }}_1,{\tilde{\alpha }}_2,{\tilde{\alpha }}_3,{\tilde{\alpha }}_4{)}^{\prime }=(-1,0.5,0,1{)}^{\prime }$, and ψ = (ψ₁, ψ₂, ψ₃, ψ₄)′ = (0.16, 0.16, 0.16, 0.16)′. The prior distributions and other settings are specified in the same manner as in Simulation 1. On the basis of the 100 simulated data sets, the means and standard deviations of the DIC values for M₁ to M₅ are reported in Table 2, which suggests that the true model M₄ is consistently selected in each of the 100 replications.

TABLE 2.

Summary of deviance information criterion (DIC) values in the simulation study

Competing Model	DIC (mean)	DIC (std)	No. of Selections
M1	12 018	79	0
M2	10 912	92	0
M3	10 124	461	0
M4	8988	128	100
M5	10 052	158	0

Note: No. of selections represents the number of times that the DIC value of M_s (s = 1, … ,5) is the smanest among all competing models in 100 replications.

The computer code for conducting the preceding analyses is written in R and is freely available at http://www.sta.cuhk.edu.hk/xysong/codes/BaGLassoHMMs.

5 ∣. ADNI STUDY

To demonstrate the empirical utility of our proposed method, we conduct real data analysis on the basis of the ADNI study. The data set collected imaging, genetic, clinical, and cognitive data from participants under CN controls and participants with mild cognitive impairment or AD. ADNI-1 was first conducted in 2004, and several extensions, namely, ADNI-GO, ADNI-2, and ADNI-3, followed afterward. In this study, we focused on 633 participants collected from ADNI-1 and included their clinical and genetic variables at four time points, namely, baseline, 6 months, 12 months, and 24 months. Functional Assessment Questionnaire (FAQ), a widely used assessment of abilities to function independently in daily life, was used as a response variable (y_it) to reflect cognitive decline over time. Patients with higher FAQ scores have lower cognitive abilities. Three continuous covariates, namely, the logarithm of the ratio of hippocampal volume over whole brain (x_it1), age at baseline (x_it2), and years of education (x_it3), were considered. Moreover, we included a genetic variable, APOE-ϵ4 (c_it1 and c_it2), which was coded as 0, 1, and 2, denoting the number of APOE-ϵ4 alleles. Other discrete demographic characteristics, such as gender (c_it3, 0 = female; 1 = male) and marital status (c_it4, 0 = has been married; 1 = has not been married), were also included. The three continuous variables, namely, FAQ score, hippocampus, and age, were standardized prior to analysis. The main objective of this study is to examine the complex effects of potential risk factors on the transition of neurodegenerative states and on the cognitive decline of participants across different states.

We first determined the number of hidden states. We considered five competing models M_k, k = 1, … , 5, where M_k represents a semiparametric HMM defined in (1)–(3) with k states. We used natural cubic splines for h_itj and M_j = 10 in approximating the unknown smoothing functions. The hyperparameters were assigned in the same manner as those in the simulation study, and the identifiability constraint μ₁ < ⋯ < μ₅ was taken to avoid label switching. We generated several MCMC chains with different initial values to monitor the convergence of the MCMC algorithm. The EPSR plot depicted in Figure 2 indicated that the MCMC algorithm converged within 10 000 iterations. Therefore, we collected 10 000 observations after discarding 10 000 burn-in iterations to calculate the DIC values of the competing models.

FIGURE 2.

Plot of estimated potential scale reduction (EPSR) values for the parameters in the ADNI-1 (Alzheimer’s Disease Neuroimaging Initiative) data analysis. The horizontal dotted line is for EPSR = 1.2. MCMC, Markov chain Monte Carlo

The values of $\overline{D(\theta )}$, p_D, and DIC corresponding to M₁ to M₄ are reported in Table 3. When fitting the data to M₅, the MCMC algorithm broke down after several iterations. After carefully checking the results, we found that one of the states included only fewer than six subjects after several iterations. This phenomenon implies the nonexistence of such a state and the inapplicability of the five-state model in this study. On the basis of the results in Table 3, the four-state model M₄ with the smallest DIC was selected. Then, we used the proposed BaGlasso procedure to conduct a simultaneous estimation and variable selection under M₄. Results are presented in Table 4 (parametric part) and Figure 3 (nonparametric part), in which only significant functional effects are reported.

TABLE 3.

Summary of deviance information criterion (DIC) values in the analysis of the Alzheimer’s Disease Neuroimaging Initiative data set

Competing Model	$\overline{D(\theta )}$	PD	DIC
M1	6294	35	6329
M2	1434	69	1503
M3	1016	97	1113
M4	972	126	1098

TABLE 4.

Estimation results in the ADNI-1 (Alzheimer’s Disease Neuroimaging Initiative) data analysis: parametric part

Parameters in the Conditional Regression Model
State 1			State 2			State 3			State 4
Par	Est	SE	Par	Est	SE	Par	Est	SE	Par	Est	SE
μ1	−0.608	0.005	μ2	−0.200	0.032	μ3	0.948	0.075	μ4	2.466	0.127
α11	0.000	0.005	α21	0.059	0.040	α31	0.113	0.082	α41	0.256	0.151
α12	0.015	0.013	α22	0.012	0.040	α32	0.068	0.086	α42	0.120	0.143
α13	0.003	0.005	α23	0.019	0.031	α33	−0.303	0.107	α43	−0.427	0.157
α14	0.003	0.005	α24	0.008	0.030	α34	−0.047	0.073	α44	−0.115	0.143
ψ1	0.009	0.000	ψ2	0.073	0.008	ψ3	0.173	0.020	ψ4	0.437	0.052

Parameters in the Transition Model
Par	Est	SE	Par	Est	SE	Par	Est	SE	Par	Est	SE
${\tilde{\alpha }}_1$	−0.386	0.174	${\tilde{\alpha }}_2$	−0.821	0.253	${\tilde{\alpha }}_3$	0.012	0.078	${\tilde{\alpha }}_4$	−0.150	0.132
π1	0.592	0.022	π2	0.198	0.022	π3	0.149	0.018	π4	0.060	0.014
ζ11	2.513	0.165	ζ21	−1.459	0.246	ζ31	−3.278	0.451	ζ41	−3.343	0.500
ζ12	2.395	0.418	ζ22	1.498	0.253	ζ32	−1.674	0.331	ζ42	−3.320	0.498
ζ13	1.405	0.740	ζ23	2.840	0.447	ζ33	1.657	0.279	ζ43	−2.017	0.426

FIGURE 3.

ADNI-1 (Alzheimer’s Disease Neuroimaging Initiative) data analysis results: the estimates of significant unknown smooth functions at the corresponding states. The solid curves represent the pointwise mean curves, and the dashed curves represent the 2.5% and 97.5% pointwise quantiles. Line y = 0 is denoted in each picture by a red dot-dash line to illustrate the range of significant effects for each risk factor. FAQ, Functional Assessment Questionnaire [Colour figure can be viewed at wileyonlinelibrary.com]

We obtain the following observations: first, intercepts μ₁, μ₂, μ₃, and μ₄ were ranked in ascending order. Patients in state 1 had the lowest mean score of FAQ, whereas those in state 4 received the highest mean score. That is, patients’ cognitive ability reflected by independent functioning in daily life steadily deteriorated from state 1 to state 4. According to the existing literature,³³ state 1 to state 4 can be explained as CN, early mild cognitive impairment (EMCI), late mild cognitive impairment (LMCI), and AD, respectively.

Second, BaGlasso selected six significant functional effects across the states. The effect of hippocampus on the FAQ score exhibits a descending trend in all the states. Specifically, in the CN state, participants with a greater hippocampal volume tend to have slightly better memory. This result is consistent with the common sense that the hippocampus helps consolidate outside information from short-term memory to long-term memory. In EMCI and LMCI states, the magnitude of the functional effect of the hippocampus on FAQ becomes increasingly large, confirming that atrophy in hippocampal volume continuously impairs patients’ cognitive ability during the progression from EMCI to LMCI. Published medical reports^34-36 also revealed the similar result that the loss of hippocampal volume greatly affects dementia. In the AD state, preventing the loss of hippocampal volume is still beneficial to postpone cognitive decline, but this effect is significant only in a small range of hippocampal volume. The effect of age on FAQ is nonsignificant in the first three states, implying that age influences cognitive function mainly in the AD state. Relatively younger AD patients (around 75 years old) have better functional independence in daily life compared with elder ones. This age effect was also revealed by previous research.^37,38 The effect of educational level on FAQ is likewise significant only in the AD state. Such effect becomes large when educational level is high, indicating that patients with higher educational levels tend to experience more pronounced cognitive decline compared with patients with lower educational levels. This finding is in line with the existing literature.^39,40

Third, for the parametric part, gender has a negative effect on FAQ in the LMCI and AD states, implying that women suffer more serious cognitive decline than men in the late progression period of AD. This result agrees with existing medical reports.^41-43

Fourth, in the transition model, the functional effect of the hippocampus exhibits an ascending trend with the growth of hippocampal volume. In the progression of AD, patients with larger hippocampal volumes are more likely to remain in the current state rather than transit to a worse one compared with those with smaller hippocampal volumes. By contrast, patients with APOE-ϵ4 alleles are more likely to transit to a worse state rather than remain in the current one. Thus, APOE-ϵ4 alleles are important risk factors for the development of AD. This result is consistent with the existing finding.⁴⁴ However, the estimates of other covariates, such as age, educational level, gender, and marital status, were shrunk to nearly zero by BaGlasso, implying that conditional on hippocampus and APOE-ϵ4, the direct effects of age, educational level, gender, and marital status on the transition probability are weak.

For comparison, we reanalyzed the ADNI data set using a parametric HMM as follows:

$$\begin{array}{cc}\hfill [y_{it}\mid Z_{it}=s]& ={\mu }_s+{\alpha }_{s1}{c}_{it1}+{\alpha }_{s2}{c}_{it2}+{\alpha }_{s3}{c}_{it3}+{\alpha }_{s4}{c}_{it4}+{\beta }_{s1}{x}_{it1}+{\beta }_{s2}{x}_{it2}+{\beta }_{s3}{x}_{it3}+{\delta }_{it},\hfill \\ \hfill \log \text{it}({\vartheta }_{itus})& ={\zeta }_{us}+{\tilde{\alpha }}_1{c}_{it1}+{\tilde{\alpha }}_2{c}_{it2}+{\tilde{\alpha }}_3{c}_{it3}+{\tilde{\alpha }}_4{c}_{it4}+{\tilde{\beta }}_1{x}_{it1}+{\tilde{\beta }}_2{x}_{it2}+{\tilde{\beta }}_3{x}_{it3}.\hfill \end{array}$$

The Bayesian adaptive lasso procedure was used to perform estimation. Table 5 presents the results of parameters β_sj and ${\tilde{\beta }}_j$. The results of μ_s, ζ_us, α_sh, and ${\tilde{\alpha }}_h$ are similar to those in Table 4 and not reported. Several differences exist between the results obtained using the parametric and semiparametric HMMs. First, the parametric model shows a negative constant effect of the hippocampus on FAQ in the CN, EMCI, and LMCI states, whereas the semiparametric model reveals that these negative effects have a descending trend. Second, the parametric model indicates that the effects of the hippocampus, age, and educational level on FAQ are all insignificant in the AD state, whereas the semiparametric model reveals that these effects are actually significant in certain covariate ranges. Finally, the parametric model shows that the effect of age on FAQ is negative in the NC and EMCI states but positive in the LMCI state. This diverse effect is hard to interpret and probably caused by overlooking the subtle structure of the age effect in the parametric model.

TABLE 5.

Estimation results of the parametric hidden Markov model in the ADNI-1 (Alzheimer’s Disease Neuroimaging Initiative) data analysis

Parameters in the Conditional Regression Model
State 1			State 2			State 3			State 4
Par	Est	SE	Par	Est	SE	Par	Est	SE	Par	Est	SE
β11	−0.022	0.004	β21	−0.122	0.023	β31	−0.155	0.039	β41	−0.127	0.065
β12	−0.006	0.003	β22	−0.008	0.017	β32	0.070	0.034	β42	0.088	0.055
β13	−0.004	0.003	β23	−0.014	0.018	β33	0.030	0.029	β43	0.025	0.051

Parameters in the Transition Model
Par	Est	SE	Par	Est	SE	Par	Est	SE	Par	Est	SE
${\tilde{\beta }}_1$	0.351	0.042	${\tilde{\beta }}_2$	−0.033	0.034	${\tilde{\beta }}_3$	0.004	0.023

6 ∣. CONCLUSION

In this paper, we have introduced a BaGlasso procedure to conduct simultaneous variable selection and parameter estimation in the context of semiparametric HMMs. We developed a full Bayesian approach, along with efficient MCMC methods and the basis expansion technique, to implement the procedure and estimate nonparametric functions. The methodology was demonstrated by a simulation study and an application to the analysis of the ADNI data set. In the proposed model, covariates are allowed to affect both responses and transition probabilities. This feature enables the model to cope with general situations where certain covariates simultaneously influence the two stochastic processes in various ways. An alternative method of including covariates in HMMs is to use an exclusion restriction to split the overall set of covariates into two groups: one contains covariates affecting only the responses, and the other contains covariates affecting the hidden-state transition. However, determining such an exclusion restriction may be subjective and difficult to justify in practice, which, in turn, elicits model selection issues.

This study can be extended in several directions. First, in approximating nonparametric functions, we considered only a simple version of natural cubic splines. Highly sophisticated smoothing techniques, such as splines and local polynomial kernel methods, may be used to enhance the performance of estimation and variable selection. Second, we simply used a single indicator, FAQ, to reflect cognitive ability in the ADNI data analysis. A comprehensive way to characterize cognitive function is to account for other relevant tests, such as the Alzheimer’s Disease Assessment Scale and the Mini-Mental State Examination. Grouping such highly correlated but different perspectives into an integrated latent variable through factor analysis can improve the analytic power and interpretability of the model. Finally, our model framework includes only binary and continuous variables. Given that ordered and unordered categorical data are frequently encountered in medical, social, and psychological sciences, generalizing the existing framework to accommodate a wide variety of data types is of great interest.

APPENDIX A

FULL CONDITIONAL DISTRIBUTIONS

A.1 ∣. Full conditional distributions of Z_it

Let y_i = (y_i1, … , y_iT)′, ${\mathbf{d}}_{it}=({\mathbf{c}}_{it}^{\prime },{\mathbf{x}}_{it}^{\prime }{)}^{\prime }$, and ${\mathbf{D}}_i=({\mathbf{d}}_{i1}^{\prime },\dots ,{\mathbf{d}}_{iT}^{\prime }{)}^{\prime }$. Then, we have

$$\begin{gathered} p(Z_{it}\mid \cdot )\propto p({\mathbf{y}}_i,{\mathbf{D}}_i,Z_{it}\mid \theta ) \\ =p(y_{i1},\dots ,y_{it},{\mathbf{d}}_{i1},\dots ,{\mathbf{d}}_{it},Z_{it}\mid \theta )\times p(y_{i,t+1},\dots ,y_{iT},{\mathbf{d}}_{i,t+1},\dots ,{\mathbf{d}}_{iT}\mid Z_{it},\theta ) \\ =q_{it}({\mathbf{y}}_i,{\mathbf{D}}_i,Z_{it}\mid \theta )\times {\stackrel{‒}{q}}_{it}({\mathbf{y}}_i,{\mathbf{D}}_i\mid Z_{it},\theta ). \end{gathered}$$

We first initialize q_i1(y_i, D_i, Z_it∣θ) = p(y_i1, d_i1, Z_it∣θ) = p(y_i1∣d_i1, Z_i1, θ)p(Z_i1∣θ) and calculate q_it(y_i, D_i, Z_it∣θ) for t = 2, … , T, in a recursion manner as follows:

$$\begin{array}{cc}\hfill & q_{it}({\mathbf{y}}_i,{\mathbf{D}}_i,Z_{it}\mid \theta )=q_{it}(y_{i1},\dots ,y_{it},{\mathbf{d}}_{i1},\dots ,{\mathbf{d}}_{iT},Z_{it}\mid \theta )\hfill \\ \hfill =& \sum _{u=1}^{S}p(y_{i1},\dots ,y_{it},{\mathbf{d}}_{i1},\dots ,{\mathbf{d}}_{iT},Z_{it},Z_{i,t-1}=u\mid \theta )\hfill \\ \hfill =& \phantom{[}\sum _{u=1}^{S}p(y_{i1},\dots ,y_{it},{\mathbf{d}}_{i1},\dots ,{\mathbf{d}}_{iT}{Z}_{i,t-1}=u\mid \theta )\times p(Z_{it}\mid Z_{i,t-1}=u,{\mathbf{d}}_{it},\theta )\times p(y_{it}\mid {\mathbf{d}}_{it},Z_{it},\theta )]\hfill \\ \hfill =& \sum _{u=1}^S\left[q_{i,t-1}({\mathbf{y}}_i,{\mathbf{D}}_i,Z_{i,t-1}=u\mid \theta )\times p(Z_{i,t-1}=u,{\mathbf{d}}_{it},\theta )\times p(y_{it}\mid {\mathbf{d}}_{it},Z_{it}\theta )\right],\hfill \end{array}$$ (A1)

where p(Z_it∣Z_i,t–1 = u, d_it, θ) and p(y_it, d_it∣Z_it, w_i1, θ) can be calculated on the basis of (8).

Similarly, we initialize ${\stackrel{‒}{q}}_{iT}({\mathbf{y}}_i,{\mathbf{D}}_i\mid Z_{iT},\theta )=1$ and calculate ${\stackrel{‒}{q}}_{it}({\mathbf{y}}_i,{\mathbf{D}}_i\mid Z_{it},\theta )$ for t = T – 1, – , 1 as follows:

$$\begin{array}{cc}\hfill {\stackrel{‒}{q}}_{it}& ({\mathbf{y}}_i,{\mathbf{D}}_i\mid Z_{it},\theta )=p(y_{i,t+1},\dots ,y_{iT},{\mathbf{d}}_{i,t+1},\dots ,{\mathbf{d}}_{iT}\mid Z_{it},\theta )\hfill \\ \hfill =& \sum _{u=1}^{S}p(y_{i,t+1},\dots ,y_{iT},{\mathbf{d}}_{i,t+1},\dots ,{\mathbf{d}}_{iT},Z_{i,t+1}=u\mid Z_{it},\theta )\hfill \\ \hfill =& \sum _{u=1}^S[p(y_{i,t+1},\dots ,y_{iT},{\mathbf{d}}_{i,t+1},\dots ,{\mathbf{d}}_{iT}\mid Z_{i,t+1}=u,\theta )\times p(Z_{i,t+1}=u\mid Z_{it},{\mathbf{d}}_{i,t+1},\theta )\phantom{]}\hfill \\ \hfill & \phantom{[}\times p(y_{i,t+1}\mid {\mathbf{d}}_{i,t+1},Z_{i,t+1}=u,\theta )]\hfill \\ \hfill =& \sum _{u=1}^S\left[{\stackrel{‒}{q}}_{i,t+1}({\mathbf{y}}_i,{\mathbf{D}}_i\mid Z_{i,t+1}=u,\theta )\times p(Z_{i,t+1}=u\mid Z_{it},{\mathbf{d}}_{i,t+1},\theta )\times p(y_{i,t+1}\mid {\mathbf{d}}_{i,t+1},Z_{i,t-1}=u,\theta )\right]\hfill \end{array}$$ (A2)

Thus, Z_it can be directly generated from (A1) when all q_it(·) and ${\stackrel{‒}{q}}_{it}(\cdot )\mathrm{S}$ defined in (A1) and (A2) are well calculated.

A.2 ∣. Full conditional distributions of μ_s, α_s, and ψ_s

$$[{\mu }_s\mid \cdot ]\sim N\left[{\mu }_s^{\ast },{\sigma }_{\mu s}^{\ast }\right],[{\alpha }_s\mid \cdot ]\sim N\left[{\alpha }_s^{\ast },{\Sigma }_{\alpha s}^{\ast }\right],\left[{\Psi }_s^{-1}\mid \cdot \right]\sim \text{Gamma}\left[{\alpha }_{\Psi s}^{\ast },{\beta }_{\Psi s}^{\ast }\right]$$ (A3)

In the above equation, ${\alpha }_{\Psi s}^{\ast }=(n_s+p+{\sum }_{j=1}^q{M}_j)∕2+{\tilde{\alpha }}_{s0}$, ${\sigma }_{\mu s}^{\ast }=(n_s{\Psi }_s^{-1}+{\sigma }_{\mu s0}^{-1}{)}^{-1}$, and

$$\begin{array}{cc}\hfill & {\beta }_{\Psi s}^{\ast }={\tilde{\beta }}_{s0}+\frac{1}{2}\left[\sum _{i=1}^n\sum _{t=1}^T\mathrm{I}(Z_{it}=s){\left(y_{it}-{\mu }_s-{\alpha }_s^{\prime }{\mathbf{c}}_{it}-\sum _{j=1}^q{\beta }_{sj}^{\prime }{\mathbf{h}}_{itj}\right)}^2+\sum _{j=1}^q\frac{\Vert {\beta }_{sj}{\Vert }_{{\mathrm{G}}_{sj}}^2}{{\tau }_{\beta sj}^2}+\sum _{h=1}^p+\frac{\mid {\alpha }_{sh}{\mid }^2}{{\tau }_{\alpha sh}^2}\right],\hfill \\ \hfill & {\Sigma }_{\alpha s}^{\ast }={\left(\sum _{i=1}^N\sum _{t=1}^T{\mathbf{c}}_{it}{\mathbf{c}}_{it}^{\prime }{\Psi }_s^{-1}\mathrm{I}(Z_{it}=s)+{\mathbf{D}}_{\alpha s}^{-1}\right)}^{-1},{\mathbf{D}}_{\alpha s}=\text{diag}({\tau }_{\alpha s1}^2,\dots ,{\tau }_{\alpha sp}^2),\hfill \\ \hfill & {\mu }_s^{\ast }={\sigma }_{\mu s}^{\ast }\left[{\Psi }_s^{-1}\sum _{i=1}^n\sum _{t=1}^T\mathrm{I}(Z_{it}=s)\left(y_{it}-{\alpha }_s^{\prime }{\mathbf{c}}_{it}-\sum _{j=1}^q{\beta }_{sj}^{\prime }{\mathbf{h}}_{itj}\right)+{\sigma }_{\mu s0}^{-1}{\mu }_{s0}\right],\hfill \\ \hfill & {\alpha }_s={\Sigma }_s^{\ast }\left[{\Psi }_s^{-1}\sum _{i=1}^n\sum _{t=1}^T\mathrm{I}(Z_{it}=s){\mathbf{c}}_{it}\left(y_{it}-{\mu }_s-\sum _{j=1}^q{\beta }_{sj}^{\prime }{\mathbf{h}}_{itj}\right)+{\Sigma }_{\alpha s0}^{-1}{\alpha }_{s0}\right].\hfill \end{array}$$

A.3 ∣. Full conditional distributions of β_sj

$$[{\beta }_{sj}\mid \cdot ]\sim N\left[{\beta }_{sj}^{\ast },{\Sigma }_{sj}^{\ast }\right]\mathrm{I}(1_{n_s}^{\prime }{\mathbf{H}}_{sj}{\beta }_{sj}=0)$$ (A4)

In the above equation, ${\Sigma }_{sj}^{\ast }={\Psi }_s({\mathbf{H}}_{sj}^{\prime }{\mathbf{H}}_{sj}+{\tau }_{\beta sj}^{-1}{\mathbf{G}}_{sj}{)}^{-1}$, ${\beta }_{sj}^{\ast }={\Psi }_s^{-1}{\Sigma }_{sj}^{\ast }{\mathbf{H}}_{sj}^{\prime }{\mathbf{y}}_s^{\ast }$, and ${\mathbf{y}}_s^{\ast }=\{y_{its}^{\ast }\}$ is an n_s × 1 vector with

$$y_{its}^{\ast }=y_{it}-{\mu }_s-{\alpha }_s^{\prime }{\mathbf{c}}_{it}-\sum _{l\ne j,l=1}^q{\beta }_{sl}^{\prime }{\mathbf{h}}_{itl},\text{for}{Z}_{it}=s.$$

A.4 ∣. Full conditional distributions of π_s, ζ_us, and $\tilde{\alpha }$

$$\begin{array}{cc}\hfill & p({\pi }_s\mid \cdot )\propto \text{exp}\left\{\sum _{u=s}^S\sum _{i=1}^n\log (p_{i10u})\times \mathrm{I}(Z_{i1}=u)-\frac{({\pi }_s-{\pi }_{s0}{)}^2}{2{\sigma }_{\pi 0}^2}\right\}\hfill \\ \hfill & p({\zeta }_{us}\mid \cdot )\propto \text{exp}\left\{\sum _{\nu =s}^S\sum _{i=1}^n\sum _{t=2}^T\log (p_{itu\nu })\times \mathrm{I}(Z_{it}=\nu ,Z_{i,t-1}=u)-\frac{({\zeta }_{us}-{\zeta }_{us0}{)}^2}{2{\sigma }_{\zeta us0}^2}\right\}\hfill \\ \hfill & p(\tilde{\alpha }\mid \cdot )\propto \text{exp}\left\{\sum _{i=1}^n\sum _{t=2}^T\log (p_{itus})\times \mathrm{I}(Z_{it}=s,Z_{i,t-1}=u)-\frac{1}{2}(\tilde{\alpha }-{\tilde{\alpha }}_0{)}^{\prime }{\tilde{\mathbf{D}}}_{\alpha }^{-1}(\tilde{\alpha }-{\tilde{\alpha }}_0)\right\}\hfill \end{array}$$ (A5)

In the above equation, ${\tilde{\mathbf{D}}}_{\alpha }={\sigma }^2\text{diag}({\tilde{\tau }}_{\alpha 1}^2,\dots ,{\tilde{\tau }}_{\alpha p}^2)$, and p_itu0 and p_itus can be calculated on the basis of (9).

A.5 ∣. Full conditional distributions of ${\tilde{\beta }}_j$

$$p({\tilde{\beta }}_j\mid \cdot )\propto \text{exp}\left\{\sum _{i=1}^n\sum _{t=2}^T\log (p_{itus})\times \mathrm{I}(Z_{it}=s,Z_{i,t-1}=u)-\frac{1}{2}{\left({\tilde{\beta }}_j-{\tilde{\beta }}_{j0}\right)}^{\prime }{\tilde{\mathbf{D}}}_{\beta j}^{-1}\left({\tilde{\beta }}_j-{\tilde{\beta }}_{j0}\right)\right\}$$ (A6)

The above equation is with the constraint $1_{n(T-1)}^{\prime }{\mathbf{H}}_j{\tilde{\beta }}_j=0$, where ${\tilde{\mathbf{D}}}_{\beta j}={\sigma }^2{\tilde{\tau }}_{\beta j}^2{\tilde{\mathbf{G}}}_j^{-1}$, and p_itus can be calculated on the basis of (9).

Notably, the full conditional distributions in (A5) and (A6) are not familiar probability distributions. Therefore, the Metropolis-Hastings algorithm is used to sample from them. Besides, the full conditional distributions in (A4) and (A6) involve constraints, and the procedure for sampling from them can be found in the work of Song and Lu.¹⁸