Bayesian factor selection in a hybrid approach to confirmatory factor analysis

Abstract

To investigate latent structures of measured variables, various factor structures are used for confirmatory factor analysis, including higher-order models and more flexible bifactor models. In practice, measured variables may also have relatively small or moderate non-zero loadings on multiple group factors, which form cross loadings. The selection of correct and ‘identifiable’ latent structures is important to evaluate an impact of constructs of interest in the confirmatory factor analysis model. Herein, we first discuss the identifiability condition that allows several cross loadings of the models with underlying bifactor structures. Then, we implement Bayesian variable selection allowing cross loadings on bifactor structures using the spike and slab prior. Our approaches evaluate the inclusion probability for all group factor loadings and utilize known underlying structural information, making our approaches not entirely exploratory.

Through a Monte Carlo study, we demonstrate that our methods can provide more accurately identified results than other available methods. For the application, the SF-12 version 2 scale, a self-report health-related quality of life survey is used. The model selected by our proposed methods is more parsimonious and has a better fit index compared to other models including the ridge prior selection and strict bifactor model.

1. Introduction

Confirmatory factor analysis (CFA) is a type of structural equation modeling that deals specifically with measurement models, that is, the relationships between observed measures/indicators (e.g. test items, test scores, behavioral observation ratings) and latent constructs [8]. CFA differs from exploratory factor analysis (EFA) in that it starts with a robust theoretical foundation, whereby many factor loadings are fixed to zero. This approach results in a reduction of parameters to estimate in CFA, leading to smaller standard errors and the potential for more precise estimations compared to EFA [31]. However, prior knowledge may not be available, and specifying oversimplified factor loading structures may lead to biased estimates of the parameters in factor variance [3]. For instance, forcing non-zero cross loadings on minor group factors to zero substantially inflates the estimates of the factor correlations in CFA models [3]. Thus, identification of correct latent structures is important to evaluate the impact of constructs of interest and to interpret the CFA model. To overcome excessively restrictive CFAs, hybrid strategies to combine confirmatory analysis and exploratory factor analysis may be resorted such as exploratory structural equation modeling (ESEM) and Bayesian structural equation modeling [54].

In this paper, we adopt a Bayesian variable selection called the spike and slab shrinkage method in the context of a hybrid approach to CFA and compare our approach with other existing procedures such as different prior choices and post-processing methods using factor rotations [41]. The spike and slab prior is suitable when the researcher has strong prior knowledge that some variables have exactly zero coefficients, which makes suitable for CFA modeling. Commonly, the spike and slab method is based on the calculation of a certain probability as a function of existing data. In application to factor analyses, a major difficulty of using this technique is that latent factors are not observed unlike other more traditional regressions. Our method is a kind of hybrid as some pre-identified bifactor structures are incorporated into factor loading identification. We note that Lu, Chow and Loken [31] explore the spike and slab technique on candidates of cross loadings based on a distribution for each item in the factor analysis. Their method shows an improvement in factor variable selection over other Bayesian variable selections and traditional variable selections such as forward and backward selections. In this paper, we derive a different formation of inclusion probabilities in which an inclusion of each factor loading is evaluated based on a marginal conditional distribution. Additionally, our proposed approach is vetted to handle cases that the majority of group factor loadings are not fixed rather than handling a few cross loadings in the model. Our approach may simplify the calculation of inclusion probabilities, but we also show an improvement of variable selection performance compared to existing methods.

Various factor structures are used for CFA, including independent clusters model of CFA (ICM-CFA), higher-order models, and hierarchical factor models. As a more flexible alternative to these models, a bifactor measurement model [22] can be used to explain hierarchical latent structures of multidimensional variables. The bifactor measurement model uses a hybrid model that includes a general factor on which all items (indicators) load and one or more uncorrelated group factors on which subsets of items load. In a typical higher-order model, each group level factor loads onto the general factor (second-order factor). Thus, the higher-order model restricts factor structures, where the shared variance between the group-level factors is caused by a general factor [52]. For this reason, the second-order factor model is nested in the bifactor model, and the models are equivalent when a condition called proportionality is met [4,34,38]. When a bifactor model is used for CFA, the cross loadings are typically fixed at zero values, which may prevent nonidentification (i.e. nonunique model) due to approximate linear dependencies between the general factor and the group factors [55]. A simplified model may prevent nonconvergence, but CFA models may include large cross loadings [32]. To this end, it is important to discuss the model identifiability or uniqueness of the model when we alter basic bifactor models and include multiple cross loadings.

The outline of this paper is as follows. In Section 2, we review the existing literature on variable selection methods, including Bayesian variable selection with different priors. In Section 3, we introduce the Bayesian factor model and present a set of results regarding the identifiability of models with cross loadings. These findings provide us with ample flexibility to explore various cross loadings and justify the final model in terms of its identifiability. In addition, we propose a novel approach using the spike and slab shrinkage method to assess the inclusion probability of each factor loading based on the marginal conditional distribution. In Section 4, we conduct a Monte Carlo study to evaluate the performance of the proposed methods using various scenarios, including the number of items, factors, and the number of cross loadings. We investigate the percentage of correctly identified models and compare our results with those obtained using other existing methods. In Section 5, we apply the developed methods to analyze the Short Form-12 version 2 (SF-12v2), a self-report quality of life questionnaire. Finally, Section 6 presents the concluding remarks summarizing the key findings of our research.

2. Existing methods of latent variable selection

2.1. Classical variable selection

Variable selection has been a major topic in the statistical literature on regression. Classical variable selection procedures such as best subset selection, sequential searches including forward selection, and backward elimination and stepwise methods can be used for the factor analysis [17]. Stepwise methods [13] gained popularity in epidemiology and other fields for their computational simplicity [45]. However, the stepwise algorithm is generally ineffective at excluding anomalous variables from the factor model and ‘the poor selection accuracy of the stepwise approach suggests that it should be avoided’ [21].

Model comparison criteria such as the Akaike information criterion (AIC) [1] and the Bayesian information criterion (BIC) [46] can be used for the factor analysis. The AIC only provides information about the quality of a model relative to other models and does not provide information on the absolute quality of a model. With a small sample size relative to a large number of parameters/variables, the AIC often provides models with too many variables. The BIC often chooses models that are more parsimonious than the AIC, as the BIC penalizes bigger models due to the larger penalty term inherent in its formula [40]. These methods assess models estimated by the maximum likelihood procedure and are unable to provide suitable regularization parameters to regulate overfitting [19].

Commonly, factor structures are evaluated using fit indexes. Evaluating factor structures using fit indexes is an iterative process that involves exploring different model combinations to select the final candidate models. The model chi-square index directly uses the ratio of the maximum likelihood functions, which is assumed to have a chi-square distribution. In practice, the null hypothesis is often rejected due to the chi-square’s heightened sensitivity under large samples. To resolve this problem, approximate fit indexes that are not based on accepting or rejecting the null hypothesis have been developed. Approximate fit indexes can be further classified into absolute and incremental or relative fit indexes. An example of an incremental fit indexes is the CFI (comparative fit index) [6]. An example of an absolute fit index is the root-mean-square error of approximation (RMSEA). The RMSEA [48] measures the discrepancy due to the approximation per degree of freedom. The RMSEA tends to reward complex models with high degrees of freedom and large samples. It also tends to penalize simpler structural equation models when estimated with fewer variables analyzed at smaller sample sizes [29,37,53]. In contrast, the CFI is an index of ‘good fit’ ranging from 0 to 1, which quantifies the proportional improvement in structural equation model fit over a ‘null’ model [6,7,29]. It is less influenced by sample sizes and penalizes non-parsimonious models. However, the ‘null’ comparison model for the CFI may not be supported by the data, while the model may produce an unrealistic extreme value for comparison that gives rise to excessively generous assessments of model fit [43]. Researchers using structural equation models often provide multiple fit index values, such as the chi-square test statistic, the CFI, the RMSEA, and others. However, these fit indexes may leave readers to assess the strength of their claims based on subjective preference. Papers often report fit indexes, and authors characterize their findings based on personal opinions for meeting recommended cut-point criteria [43]. Absent any additional definitive criteria, adequate cut-point criteria of fit indexes (e.g. [24] can become matters more of semantic subjectivity than absolute validity [5,43]).

2.2. Bayesian factor analysis with different priors

Regularization methods, such as Bayesian model selection, offer automated and data-driven model selection, which reduces subjectivity and is relatively free of personal preference. These methods can assist proper model selection in CFA by carefully regulating the loss functions/log-likelihood functions to achieve sparse coefficient estimates. By defining the penalty functions as prior distributions, it is feasible to combine established regularization techniques with Bayesian models. Consequently, these Bayesian regularization methods can determine the distributions of unknown parameter values and exclude irrelevant parameters from the model structure. The Bayesian statistical approach has gained widespread recognition due to its ability to quantify uncertainty transparently and its generality across different kinds of simple and complex analyses. Moreover, the availability of computationally sophisticated Markov chain Monte Carlo (MCMC) sampling algorithms in the statistics and machine-learning communities has made Bayesian analysis highly feasible.

The ridge [20] and Lasso [49] penalties are two well-known regularization techniques that employ $L_2$-norm and $L_1$-norm, respectively. For Bayesian model selection, the prior choice similar to the ridge regression is the Gaussian distribution, $\left(\mathrm{\text{β}}|{\sigma }^2\right)\propto \text{exp}(-\frac{1}{2{\sigma }^2}\sum _{j=1}^p{\mathrm{\text{β}}}_j^2)$ where $\mathrm{\text{β}}$ is the coefficient of a model and ${\sigma }^2$is a variance parameter with prior distribution $p\left({\sigma }^2\right).$The prior choice of the Laplace distributions, $p\left(\mathrm{\text{β}}|{\sigma }^2\right)\propto \exp \left(-\frac{1}{2{\sigma }^2}\sum _{j=1}^p|{\mathrm{\text{β}}}_j|\right)$, regulates variable selection similar to the Lasso regression. Powerful Bayesian computational techniques can be employed for wide range of models, like parameter constraints and penalty functions, which often lead to optimization difficulties in the frequentist framework and may be handled with relative ease in the Bayesian framework. For example, sampling from the posterior distribution provides more information for estimation and inference that is difficult to carry out for the Lasso regression [9].

For the latent variable selections in the factor analysis framework, Muthén and Asparouhov [39] use Bayesian variable selection in which the ridge prior is used to shrink the cross loading elements in factor loadings toward zero. In their approach, they replace parameter specifications of exact zeros with approximate zeros based on the ridge prior. The proposed Bayesian approach is particularly beneficial in applications in which parameters are added to a conventional model leading to a non-identified model when using maximum likelihood methods. Additionally, Jacobucci, Brandmaier and Kievit [26] show that regularized structural equation modeling with Lasso prior outperforms non-Bayesian traditional structural equation modeling in situations with a large number of predictors and a small sample size.

Mixture priors such as spike and slab components have been used extensively for variable selection (e.g. [16,25,35]). The spike component, which concentrates its mass at values close to zero, allows shrinkage of small effects to zero, whereas the slab component has its mass spread over a wide range of plausible values for the regression coefficients [33]. For latent variable selections, Lu, Chow and Loken [31] explore variable selection problems in Bayesian factor analysis, where the spike and slab prior (SSP) is used to model the prior probability of nonzero factor loadings. In their approach, the ridge prior and SSP are used together, where the SSP is used to select a few cross loadings that are strongly associated with each factor. We note that Lu, Chow and Loken [31] focus on a few suspicious factor loadings to use the SSP, and have not evaluated their performance in the setting of the bifactor model. Let $\text{R}=\left[r_{\mathrm{jk}}\right]$ indicate a $p\times q$ matrix ( $p$: the number of items, $q$: the number of factors), where $r_{\mathrm{jk}}$’s are latent binary variables that can take on the value of either 1 or 0 to indicate selecting or not selecting factor loadings. In Lu, Chow and Loken [31] approach, promising factor loadings to be estimated are predicted by high posterior inclusion probabilities that are obtained based on multinomial distribution of $(r_{j1},\dots ,r_{\mathrm{jq}})$.

A recent study by Papastamoulis and Ntzoufras [41] introduces a post-processing approach to identify correct latent factor loading structures addressing the issue of invariance in Bayesian factor analytic models caused by orthogonal transformations. The proposed method involves a two-stage process that uses a varimax rotation to achieve a simple structure of factor loadings, followed by a correction for sign and permutation invariance. While the method can be used for identifying the factor loading structures, it primarily identifies orthogonal factor structures without a general factor. Therefore, the method may not be suitable for the bifactor model identification.

Most of these existing techniques can target to identify cross loadings. In our paper, we investigate a new Bayesian approach in the SSP framework to identify general confirmatory model structures as shown in the next section. We also compare our approach to previously mentioned Bayesian approaches in terms of the correctness of the model identification as shown in Section 4.

3. Method

In this section, we begin by discussing the general setting of Bayesian factor models. Then, we discuss one crucial issue in model searching, i.e. models’ identifiability. We will provide some results related to this issue for models with cross loadings as a variation of the bifactor model. These results offer us flexibility in model searching and help justify the final model’s identifiability. Then, we explore a spike and slab approach to inclusion probability, which evaluates each factor loading’s inclusion using a marginal conditional distribution. This method simplifies calculations and improves variable selection performance as well as incorporating some items’ bifactor structures.

3.1. Bayesian factor model

For Bayesian models considering latent variables, Lee [30] provides a thorough explanation for obtaining posterior distributions of latent variable quantities in Bayesian models, and we rely on Lee’s work as our primary source for notation and methodology. Let $X$ indicate $p$-variate random vector, $Y=X-E\left(X\right)$ and $V\left(Y\right)=\mathrm{\Sigma }$. Let $N$ indicate the sample size, $p$ indicate the number of manifest variables (or items) in the data, and $q$ indicate the number of latent variables or factors. A factor model is described as

$${\text{Y}}_{\text{i}}=\mathrm{\Lambda }{\text{f}}_{\text{i}}+{ϵ}_{\text{i}},\text{i}=1,2,\dots ,\text{N},$$ (1)

where $Y_i$ is a $p\times 1$ observed random vector, $\mathrm{\Lambda }$ is a $p\times q$ factor loading matrix, $f_i$ is a $q\times 1$ vector factor scores and ${ϵ}_i$ is a $p\times 1$ random vector of error measurements which is independent of $f_i$. The error vector ${ϵ}_i$ is normally distributed as $N\left(0,\mathrm{\Psi }\right)$, where $\mathrm{\Psi }$ is a diagonal matrix, and $f_i$ is distributed as $N\left(0,\mathrm{\Phi }\right)$ with some positive definite covariance matrix $\mathrm{\Phi }$. The quantities $f_i$ and ${ϵ}_i$ are independent. In the frequentist approach, $\mathrm{\Phi }$ is often considered as the identity matrix with variances of 1. We note that some constraints are necessary to identify the model in Equation (1), as discussed in Section 3.2. In standard Bayesian factor analysis, conjugate priors are used for the model parameters. Let ${\psi }_j$ and ${\mathrm{\Lambda }}_j$ be the $j$-th diagonal elements of $\mathrm{\Psi }$ and the vector consisted of the elements of $j$-th row of $\mathrm{\Lambda }$, respectively. We assume

$$f_i\sim N\left(0,\mathrm{\Phi }\right),\mathrm{\Phi }\sim I{W}_q\left({\text{R}}_0,{\rho }_0\right),{\mathrm{\Lambda }}_j|{\psi }_j\sim N\left({\mathrm{\Lambda }}_{0j},{\psi }_j{H}_{0j}\right),{\psi }_j^{-1}\sim \mathrm{Gamma}\left({\alpha }_{0j},{\mathrm{\text{β}}}_{0j}\right),$$

where $\mathrm{Gamma}\left(\alpha ,\mathrm{\text{β}}\right)$ represents the gamma distribution with shape parameter $\alpha$ and inverse scale parameter $\mathrm{\text{β}}$, and $I{W}_q$ denotes an $q$-dimensional inverse Wishart distribution. Scalars such as ${\alpha }_{0j},{\mathrm{\text{β}}}_{0j}$ ${\mathrm{\Lambda }}_{0j}$, ${\rho }_0$ and the positive definite matrices $H_{0j}$ and ${\text{R}}_0$ are hyperparameters whose values are frequently considered non-informative or presumed to be derived from prior knowledge obtained through previous studies. For noninformative priors, we use the following commonly used hyperparameters [30,31,44],

$${\alpha }_{0j}=2,{\mathrm{\text{β}}}_{0j}=1,H_{0j}=I_q,R_0^{-1}=I_q\left(p-q-1\right),{\rho }_0=p,{\mathrm{\Lambda }}_{0j}=0.$$

The conditional posterior distribution of $f_i$ given $(y_i,\mathrm{\Lambda },\mathrm{\Psi },\mathrm{\Phi })$ is equal to

$$({\text{f}}_{\text{i}}|{\text{y}}_{\text{i}},\mathrm{\Lambda },\mathrm{\Psi },\mathrm{\Phi })\sim \text{N}\left[{\left({\mathrm{\Phi }}^{-1}+{\mathrm{\Lambda }}^{\text{T}}{\mathrm{\Psi }}^{-1}\mathrm{\Lambda }\right)}^{-1}{\mathrm{\Lambda }}^{\text{T}}{\mathrm{\Psi }}^{-1}{\text{y}}_{\text{i}},{\left({\mathrm{\Phi }}^{-1}+{\mathrm{\Lambda }}^{\text{T}}{\mathrm{\Psi }}^{-1}\mathrm{\Lambda }\right)}^{-1}\right].$$ (2)

Let $\mathit{Y}=\left(y_1,\dots ,y_N\right)$ be a $p\times N$ data matrix and ${\text{Y }}_j^T$ be the $j$-th row of $\mathit{Y}\left(j=1,\dots ,p\right),A_j=(H_{0j}^{-1}+F{F}^T{)}^{-1},$ $a_j=A_j\left(H_{0j}^{-1}{\mathrm{\Lambda }}_{0j}+F{Y}_j\right)$, ${\mathrm{\text{β}}}_j={\mathrm{\text{β}}}_{0j}+1/2({\text{Y }}_j^T{Y}_j-a_j^T{A}_j^{-1}{a}_j+{\mathrm{\Lambda }}_{0\text{j}}^{\text{T}}{H}_{0j}^{-1}{\mathrm{\Lambda }}_{0j})$, where $F=\left(f_1,f_2,\dots ,f_N\right)$. The conditional posterior distributions of ( ${\mathrm{\Lambda }}_j,{\psi }_j^{-1}$) and $\mathrm{\Phi }$ given $\mathit{Y}$ and $F$ are expressed as

$$\begin{array}{rl}\left({\psi }_j^{-1}|\mathit{Y},F\right)& \sim \mathrm{Gamma}\left(\frac{N}{2}+{\alpha }_{0j},{\mathrm{\text{β}}}_j\right),\\ ({\mathrm{\Lambda }}_j|\mathit{Y},F,{\psi }_j^{-1})& \sim N\left({\alpha }_j,{\psi }_j{A}_j\right),\\ (\mathrm{\Phi }|\mathit{Y},\text{F)}& \sim \text{I}{\text{W}}_{\text{q}}\left[\left(\text{F}{\text{F}}^{\text{T}}+{\text{R}}_0^{-1}\right),\text{N}+{\rho }_0\right].\end{array}$$ (3)

In each iteration of the Gibbs sampler, we consecutively draw values from the conditional posterior distributions of the parameters. We repeat that process until sufficient values have been drawn so that the samples converged to the posterior distribution.

3.2. Identifiability of Bayesian confirmatory factor model

In the CFA model building, model identifiability needs to be satisfied for consistent parameter estimation and valid model description [14]. If the model is not identifiable, different sets of parameters may produce the same model-implied covariance matrix [14]. In a frequentist analysis, parameters are often put at zero (i.e. hard constraint) to produce an analysis that better reflects the researcher’s theories and prior beliefs. Freeing these parameters would result in a non-identified model in practical applications. In Bayesian approach, Muthén and Asparouhov [39] replace the hard constraint on cross loadings with small-variance informative priors (i.e. soft constraint) to represent a belief that the cross loadings may have large prior belief close to 0 [31]. To this end, it is important to know how many cross loadings can be incorporated when we search for the model so that the final model is identifiable. In this section, we discuss identifiable cross loading structures, altering basic bifactor models. Our results described below provide ample flexibility in the model identification procedure as well as final model structures can be justified by the results.

For the general factor model, a few conditions are required to ensure identification without label-switching and sign-reversal problems. Anderson and Rubin [2] suggest that model identification can be achieved by imposing at least $q^2$ restrictions on $(\mathrm{\Lambda },\mathrm{\Phi })$. Howe [23] and most recently Peeters [42] provide some detailed conditions to make sure rotational invariances of $\mathrm{\Lambda }$. Some common methods to achieve identifiability is to set some elements in $\mathrm{\Lambda }$ to fixed known values, e.g. one main loading of each latent factor to be fixed at 1 as well as $q\left(q-1\right)$ additional cross loadings to be fixed at 0 at appropriate places [2]. Alternatively, for a CFA model with at least three indicators (items), we can either set the variance of each factor to 1 (variance standardization method) or set the first loading of each factor to 1 (the marker method [47]). From a Bayesian point of view, this is equivalent to assigning the fixed values to the corresponding parameters with probability one [30].

In our CFA model, the models under consideration have bifactor structures allowing several cross loadings. Fang, Xu, Guo, Ying and Zhang [14] state that, under the standard bifactor model with the same sign factor loadings, the model parameters are identifiable if and only if one of the following conditions is met

1. If we have three or more group factors, each group factor needs at least three indicators.

2. When we have two group factors, each group factor should have a minimum of three indicators, and the set of group factors should be partitioned into two disjoint subsets of items, where each submatrix of factor loadings (consisted of general and group factor loadings) has full column ranks.

The same sign factor loadings are accomplished commonly in constructing a questionnaire by aligning the responses of question items in the same direction, e.g. positive responses have a larger value and indicating stronger relationship with constructs, while positive but smaller values usually indicate weaker relationship. In the spirit of the above identifiability conditions, we consider some identifiability conditions of bifactor model modified with a few cross loadings. In the following discussions, we largely adopt the notations from [14].

First, consider the standard bifactor model. Let $B=\left\{1,\dots ,p\right\}$ indicate the set with all item indexes. In this case, the cardinal number of $B,$ $|B|=p$. Let ${\mathrm{\Lambda }}^{\ast }=\left[{\mathit{\lambda }}_0^{\ast },{\mathit{\lambda }}_1^{\ast },\dots ,{\mathit{\lambda }}_G^{\ast }\right]$ indicate the factor loading matrix, where ${\mathit{\lambda }}_i^{\ast }=({\lambda }_{i1}^{\ast },\dots ,{\lambda }_{\mathrm{ip}}^{\ast }{)}^T,i=0,1,\dots ,G.$ The set of all group factors are indicated by $H_1=\left\{1,\dots ,G\right\}$. The first column ${\mathit{\lambda }}_0^{\ast }$ is the general factor loading, and the other columns indicate the group factor loadings, thus the total number of factors is $q=G+1$. For an item index set $S\in B$, we define ${\mathbf{\Lambda }}^{\ast }\left[S\right]=\left[{\mathit{\lambda }}_0^{\ast }\left[S\right],{\mathit{\lambda }}_1^{\ast }\left[S\right],\dots ,{\mathit{\lambda }}_G^{\ast }\left[S\right]\right]$ where ${\mathit{\lambda }}_i^{\ast }\left[S\right]$ is a part of ${\mathit{\lambda }}_i^{\ast }$ corresponding to items in $S$. We define:

• $B_g=\left\{j|{\lambda }_{\mathrm{gj}}^{\ast }\ne 0\right\}$, i.e. the items in $B=\left\{1,\dots ,p\right\}$corresponding to the group factor $g\left(g=1,\dots ,G\right).$ Thus, $B_h\cap B_g=\mathrm{\varnothing }\left(h\ne g\right)$ and ${\cup }_{g=1}^G{B}_g=B.$

When the strictly bifactor structure is altered so that there are cross loadings in group factors, we express factor loading as $\mathbf{\Lambda }=\left[{\mathit{\lambda }}_0,\dots ,{\mathit{\lambda }}_G\right]$, and ${\mathit{\lambda }}_i=({\lambda }_{i1},\dots ,{\lambda }_{\mathrm{ip}}{)}^T$ without asterisks. The specific variance matrix is expressed as $\mathrm{\Phi }=\mathrm{diag}\left({\varphi }_1,\dots {\varphi }_p\right)$. We also have the following additional definitions:

• $B_{g,c}$: a set of item indexes in $B_g$ that now have cross loadings, i.e. $B_{\mathrm{gc}}=\left\{j|{\lambda }_{\mathrm{gj}}\ne 0\right\}\cap B_g$, where $B_g$ is defined above based on the standard bifactor model ${\mathbf{\Lambda }}^{\ast }$.

• $B_{g,\sim c}=B_g-B_{g,c}$

• $H_2$ is a set of group factors defined as $H_2=\{g|B_{g,\sim c}$ has a partition $S_1$ and $S_2$ where $\mathrm{rank}\left(\mathrm{\Lambda }\left[S_1\right]\right)\ge 2$ and $\mathrm{rank}\left(\mathrm{\Lambda }\left[S_2\right]\right)\ge 2$, $g=1,\dots ,G$}.

Consider a model with two group factors ( $G=2$ or $q=3$). The elements in $\mathrm{\Sigma }$is expressed as $\mathrm{\Sigma }={\mathit{\lambda }}_0{\mathit{\lambda }}_0^T+{\mathit{\lambda }}_1{\mathit{\lambda }}_1^T+{\mathit{\lambda }}_2{\mathit{\lambda }}_2^T+\mathrm{\Phi }$, and the non-identifiability is defined as ${\mathit{\lambda }}_0{\mathit{\lambda }}_0^T+{\mathit{\lambda }}_1{\mathit{\lambda }}_1^T+{\mathit{\lambda }}_2{\mathit{\lambda }}_2^T+\mathrm{\Phi }={\mathit{\lambda }}_0^{\mathrm{\prime }}{\mathit{\lambda }}_0^{\mathrm{\prime }T}+{\mathit{\lambda }}_1^{\mathrm{\prime }}{\mathit{\lambda }}_1^{\mathrm{\prime }T}+{\mathit{\lambda }}_2^{\mathrm{\prime }}{\mathit{\lambda }}_2^{\mathrm{\prime }T}+{\mathrm{\Phi }}^{\mathrm{\prime }}$ for ${\mathit{\lambda }}_i\ne {\mathit{\lambda }}_i^{\mathrm{\prime }}\left(i=0,1,2\right)$ and $\mathrm{\Phi }\ne {\mathrm{\Phi }}^{\mathrm{\prime }}$. The following result states the condition for identifiability with two group factor models.

Theorem 1.

Suppose $=2$, $|H_1|=2$ ( $H_1=\left\{1,2\right\}$), and the model satisfies the following conditions.

(C1) $|B_g|\ge 3,g\in \left\{1,2\right\}.$

(C2) $|B_{g,\sim c}|\ge 1,$ $g\in \left\{1,2\right\}$.

(C3) All elements in $\mathbf{\Lambda }$ have the same sign.

(C4) $|H_2|\ge 1$.

(C5) At least one $g$ ( $\in \left\{1,2\right\}$) satisfies $|B_{g,\sim c}|\ge 3$.

(C6) For $g$ in (C5), $\mathrm{rank}\left(\mathrm{\Lambda }\right[B_{g,\sim c}\left]\right)\ge 2.$

With conditions (C1) – (C6), $\mathbf{\Lambda }$ is identifiable.

The proofs of Theorem 1 and all the other results are provided in the Appendix. Theorem 1 states the minimum requirements for identifying a bifactor model altered by cross loadings comprising of two group factors. In addition, if there is a fixed value in general factor, the conditions can be further relaxed as shown below.

Corollary 1.

Suppose $|H_1|=2$ and one element in ${\mathit{\lambda }}_0\left[B_{g,\sim c}\right]$ is a fixed value for at least one $g\left(\in H_1\right)$. In addition, all conditions except (C4) in Theorem 1 is satisfied. Then, $\mathbf{\Lambda }$ is identifiable.

According to Theorem 1 and Corollary 1, we have a good amount of flexibility in choosing a model that includes cross loadings. Each group factor in the model corresponds to at least three items (C1), when all factor loadings have the same sign (C3). A strictly bifactor structure (i.e. an item is related to only one group factor besides the general factor) is only necessary for 3 items for one group factor (C5) and 1 item for the other factor (C2).

In the following, we state the identifiability conditions of the model with one general factor and more than two group factors, i.e. $G\ge 3$.

Theorem 2.

Suppose | $H_1|\ge 3$, and the model satisfies the following conditions.

(C1) $|B_g|\ge 3,g\in H_1.$

(C2) | $B_{g,\sim c}|\ge 2$ ( $g\in H_1$) for at least $G-1$ elements in $H_1.$

(C3) All elements in $\mathbf{\Lambda }$ have the same sign.

(C4) | $H_2|\ge 1$.

(C5) At least one $g$ ( $\in H_1$) satisfies | $B_{g,\sim c}|\ge 3$.

(C6) For $g$ in (C5), $\mathrm{rank}\left(\mathrm{\Lambda }\right[B_{g,\sim c}\left]\right)\ge 2$.

With conditions (C1) – (C6), $\mathbf{\Lambda }$ is identifiable.

In addition, if there is a fixed value in the general factor, the conditions shown in Theorem 2 can be relaxed further as follows.

Corollary 2.

Suppose $|H_1|\ge 3$ and one element in ${\mathit{\lambda }}_0\left[B_{g,\sim c}\right]$ is a fixed value for at least one $g\left(\in H_1\right)$. In addition, all conditions except (C4) in Theorem 2 is satisfied. Then $\mathbf{\Lambda }$ is identifiable.

Theorem 2 and Corollary 2 again give much flexibility to include cross loadings when there are more than two group factors. The stated model’s factors correspond to at least three items each (C1), similar to 2-factor models, when all factor loadings have the same sign (C3). Then, according to Corollary 2, only 3 items in one group factor need to have a strict bifactor structure (C5), items in another factor do not have to have bifactor structure, and the remaining factors can only have 2 items with a strict bifactor structure (C2).

3.3. Implementation of the spike and slab prior

A priori, there are a few predetermined factor loading structures of the model. For example, researchers may consider that some question items have clear bifactor structures, i.e. the items correspond to only one group factor besides the general factor. This prior belief or information is needed to be incorporated into the posterior distributions. Applying the spike and slab prior, each factor loading has a discrete mass at point 0 or a continuous distribution based on the posterior distribution. We put the following prior on the cross loadings of ${\lambda }_{\mathrm{jk}}\left(j=1,\dots ,p,k=2,..,q\right),$

$$\begin{array}{rl}{\lambda }_{\mathrm{jk}}& \sim \left\{\right(1-r_{\mathrm{jk}}){\delta }_0+r_{\mathrm{jk}}N({\lambda }_{0\mathrm{jk}},{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2\left)\right\}{I}_{\left[0,\mathrm{\infty }\right)}\left({\lambda }_{\mathrm{jk}}\right),\\ r_{\mathrm{jk}}& \sim \mathrm{Bernoulli}\left({\pi }_{\mathrm{jk}}\right),\end{array}$$

where $r_{\mathrm{jk}}\in \left\{0,1\right\}$ is a mixture weight, ${\psi }_j$ is the $j$-th diagonal elements of error variance, ${\delta }_0$ is the point mass function at 0 (the spike), ${\sigma }_{\lambda 0\mathrm{jk}}^2$ is the variance of the slab, ${\pi }_{\mathrm{jk}}$ is a hyperparameter between 0 and 1 reflecting a prior probability regarding the selection of the factor loading. The function $I_s\left({\lambda }_{\mathrm{jk}}\right)$ is the indicator function where the value is 1 if ${\lambda }_{\mathrm{jk}}\in s$ or 0 otherwise. We use $r_{\mathrm{jk}}$ is a binary latent variable commonly used in the formulation of mixture models, which indicates whether ${\lambda }_{\mathrm{jk}}$ should be set to zero or freed to deviate from zero. Without strong prior knowledge of the model, values such as ${\lambda }_{0\mathrm{jk}}=0$, ${\sigma }_{\lambda 0\mathrm{jk}}^2=1$, and ${\pi }_{\mathrm{jk}}=0.5$ will be used. The hyperparameter ${\sigma }_{\lambda 0\mathrm{jk}}^2=1$ signifies that the variance is solely controlled by the error variance of the data $Y_i$, without incorporating any prior knowledge. The value ${\pi }_{\mathrm{jk}}=0.5$ aligns with the common practice of not having a precise understanding of the factor structure concerning cross loadings during model building.

We define the matrix $\text{R}=\left[r_{\mathrm{jk}}\right]=\left[{\text{R}}_1,\dots ,{\text{R}}_q\right]=[{\text{r}}_1^T,\dots ,{\text{r}}_p^T{]}^T$, where ${\text{R}}_k$ is $k$-th column ( $k=1,\dots ,q$) of $\text{R}$ and ${\text{r}}_j$ is the $j$-th row ( $j=1,\dots ,p$) of $\text{R}$. We also define the matrix $\mathrm{\Pi }=\left[{\pi }_{\mathrm{jk}}\right]=\left[{\mathrm{\Pi }}_1,\dots ,{\mathrm{\Pi }}_q\right]$, where ${\mathrm{\Pi }}_k$ is $k$-th column of $\mathrm{\Pi }$. We note that the values composed of ${\text{R}}_1$ and ${\mathrm{\Pi }}_1$ are always 1, indicating the general factor is always estimated.

In CFA, we handle known factor loading structures as follows. If a factor loading is determined to be always nonzero a priori, the factor loading is generated based on the prior distribution

$${\lambda }_{\mathrm{jk}}\sim \left\{\left(1-r_{\mathrm{jk}}\right){\delta }_0+r_{\mathrm{jk}}N\left({\lambda }_{0\mathrm{jk}},{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2\right)\right\}{I}_{\left(0,\mathrm{\infty }\right)}\left({\lambda }_{\mathrm{jk}}\right).$$

When a factor loading is determined to be always zero a priori, the factor loading is generated based on the prior distribution

$${\lambda }_{\mathrm{jk}}\sim \left\{\left(1-r_{\mathrm{jk}}\right){\delta }_0+r_{\mathrm{jk}}N\left({\lambda }_{0\mathrm{jk}},{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2\right)\right\}{I}_{\left[0\right]}\left({\lambda }_{\mathrm{jk}}\right).$$

In the following notations, $p(.)$ indicates the density or probability mass function of a random variable. We obtain samples from $p\left(F,\mathrm{\Lambda },\text{R},\mathrm{\Phi },\mathrm{\Psi }|Y\right)$ using the Gibbs sampler [15]. Starting with initial values of $\left\{F^{\left(0\right)},{\mathrm{\Lambda }}^{\left(0\right)},{\text{R}}^{\left(0\right)},{\mathrm{\Phi }}^{\left(0\right)},{\mathrm{\Psi }}^{\left(0\right)}\right\}$, the sample at $t$-th interaction $\left\{F^{\left(t\right)},{\mathrm{\Lambda }}^{\left(\text{t}\right)},R,{\mathrm{\Phi }}^{\left(t\right)},{\mathrm{\Psi }}^{\left(t\right)}\right\}$ is obtained in the following manner using the full conditional distributions:

Step 1: Generate $F^{\left(t\right)}$ from $p\left(F|Y,{\mathrm{\Lambda }}^{\left(t-1\right)},{\mathrm{\Phi }}^{\left(t-1\right)},{\mathrm{\Psi }}^{\left(t-1\right)}\right)$.

Step 2: Generate ${\mathrm{\Lambda }}^{\left(t\right)}$ and ${\mathrm{\Psi }}^{\left(t\right)}$ from $p\left(\mathrm{\Lambda },\mathrm{\Psi }|Y,F^{\left(t\right)},R^{\left(t-1\right)}\right)$.

Step 3: Generate $R^{\left(t\right)}$ from $p\left(\mathrm{R}|F^{\left(t\right)},{\mathrm{\Lambda }}^{\left(t\right)},{\mathrm{\Psi }}^{\left(t\right)},{\mathrm{\Phi }}^{\left(t-1\right)}\right)$.

Step 4: Generate ${\mathrm{\Phi }}^{\left(t\right)}$ from $p\left(\mathrm{\Phi }|F^{\left(t\right)}\right)$.

The conditional distributions used in Steps 1, 2, and 4 are given in Equations (2) and (3). For step 3, we consider the joint conditional distribution of parameters $p(\mathrm{\Lambda },\text{R},\mathrm{\Psi }|Y,F)$,

$$\begin{array}{rl}p(\mathrm{\Lambda },\text{R},\mathrm{\Psi }|Y,F)& \propto p(Y|\mathrm{\Lambda },\mathrm{\Psi },F)p(\mathrm{\Lambda }|\text{R},\mathrm{\Psi })p\left(\mathrm{R}|\mathrm{\Pi })p\right(\mathrm{\Psi })\\ & \propto |2\pi \mathrm{\Psi }{|}^{-\frac{N}{2}}\exp \left[-\frac{1}{2}\sum _{i=1}^N{\left(Y_i-\mathrm{\Lambda }{f}_i\right)}^T{\mathrm{\Psi }}^{-1}\left(Y_i-\mathrm{\Lambda }{f}_i\right)\right]\\ & \times \prod _{j=1}^p{{\psi }_j}^{-\left({\alpha }_{1j}+1\right)}\exp \left(-\frac{{\alpha }_{2j}}{{\psi }_j}\right)\prod _{j=1}^p\prod _{k=1}^q\left\{{\pi }_{\mathrm{jk}}^{r_{\mathrm{jk}}}{\left(1-{\pi }_{\mathrm{jk}}\right)}^{1-r_{\mathrm{jk}}}\right\}\\ & \times \prod _{j=1}^p\prod _{k=1}^q({\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2{)}^{-\frac{1}{2}}\exp \left\{-\frac{{\left({\lambda }_{\mathrm{jk}}-{\lambda }_{0\mathrm{jk}}\right)}^2}{2{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right\}{I}_s\left(\mathrm{\Lambda }\right),\end{array}$$ (4)

where $I_s\left(\mathrm{\Lambda }\right)$ indicates predetermined constraints on $\mathrm{\Lambda }$. Let $Y_i^{\ast }=Y_i-\left(\mathrm{\Lambda }\circ {\text{M}}_{\text{Ik}}\right)f_i$, ${\text{M}}_{\text{Ik}}$ is a matrix that has 0 for the predetermined structure on $f_i$ and the element $\left(j,k\right)$, and 1 for the rest of elements, and $\circ$ indicates the Hadamard product. If $r_{\mathrm{jk}}=0$, based on Equation (4), we have a conditional distribution:

$$\begin{array}{rl}p\left({\mathrm{\Lambda }}_k,\text{R},\mathrm{\Psi }|Y,F,r_{\mathrm{jk}}=0\right)& \propto \exp \left\{-\frac{1}{2}\sum _{i=1}^N{\left(Y_i^{\ast }-{\mathrm{\Lambda }}_k^{\ast c}{f}_{\mathrm{ki}}\right)}^T{\mathrm{\Psi }}^{-1}\left(Y_i^{\ast }-{\mathrm{\Lambda }}_k^{\ast c}{f}_{\mathrm{ki}}\right)\right\}\\ & \times \prod _{j=1}^p{{\psi }_j}^{-\left({\alpha }_{1j}+1\right)}\exp \left(-\frac{{\alpha }_{2j}}{{\psi }_j}\right)\prod _{j=1}^p\prod _{k=1}^q\left\{{\pi }_{\mathrm{jk}}^{r_{\mathrm{jk}}}{\left(1-{\pi }_{\mathrm{jk}}\right)}^{1-r_{\mathrm{jk}}}\right\}\\ & \times \prod _{j=1}^p\prod _{k=1}^q({\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2{)}^{-\frac{1}{2}}\exp \left\{-\frac{{\left({\lambda }_{\mathrm{jk}}-{\lambda }_{0\mathrm{jk}}\right)}^2}{2{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right\},\end{array}$$ (5)

where ${\mathrm{\Lambda }}_k^{\ast c}$ is the factor loading corresponding to $f_i$ complementing to $k$-th column of ${\text{M}}_{\text{Ik}}$. Also, if $r_{\mathrm{jk}}=1$, we have

$$\begin{array}{rl}p\left({\mathrm{\Lambda }}_k,\text{R},\mathrm{\Psi }|Y,F,r_{\mathrm{jk}}=1\right)& \propto \exp \left\{-\frac{1}{2}\sum _{i=1}^N{\left(Y_i^{\ast }-{\mathrm{\Lambda }}_k^{\ast }{f}_{\mathrm{ki}}\right)}^T{\mathrm{\Psi }}^{-1}\left(Y_i^{\ast }-{\mathrm{\Lambda }}_k^{\ast }{f}_{\mathrm{ki}}\right)\right\}\\ & \times \prod _{j=1}^p{{\psi }_j}^{-\left({\alpha }_{1j}+1\right)}\exp \left(-\frac{{\alpha }_{2j}}{{\psi }_j}\right)\prod _{j=1}^p\prod _{k=1}^q\left\{{\pi }_{\mathrm{jk}}^{r_{\mathrm{jk}}}{\left(1-{\pi }_{\mathrm{jk}}\right)}^{1-r_{\mathrm{jk}}}\right\}\\ & \times \prod _{j=1}^p\prod _{k=1}^q({\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2{)}^{-\frac{1}{2}}\exp \left\{-\frac{{\left({\lambda }_{\mathrm{jk}}-{\lambda }_{0\mathrm{jk}}\right)}^2}{2{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right\},\end{array}$$ (6)

where ${\mathrm{\Lambda }}_k^{\ast }$is same as ${\mathrm{\Lambda }}_k^{\ast c}$ except that ${\lambda }_{\mathrm{jk}}$ is the value estimated in the previous iteration. After integrating out $\mathrm{\Lambda }$ in Equations (5) and (6), we have the posterior distribution of $r_{\mathrm{jk}}$ as the Bernoulli random variable with the conditional probability

$$\begin{array}{rl}& \frac{\int p\left({\mathrm{\Lambda }}_k,\text{R},\mathrm{\Psi }|Y,F,r_{\mathrm{jk}}=1\right)d{\mathrm{\Lambda }}_k}{\int p\left({\mathrm{\Lambda }}_k,\text{R},\mathrm{\Psi }|Y,F,r_{\mathrm{jk}}=1\right)d{\mathrm{\Lambda }}_k+\int p\left({\mathrm{\Lambda }}_k,\text{R},\mathrm{\Psi }|Y,F,r_{\mathrm{jk}}=0\right)d{\mathrm{\Lambda }}_k}\\ & =\frac{K_j^{-\frac{1}{2}}\sqrt{2\pi }\text{exp}\left[{\frac{L_j}{2K_j}}^2-\frac{1}{2}\left(\frac{{\lambda }_{0j}^2}{{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}+\sum _{i=1}^N\frac{{y_{i,j}^{\ast }}^2}{{\psi }_j}\right)\right]{\pi }_{\mathrm{jk}}}{\begin{array}{l}{K}_j^{-\frac{1}{2}}\sqrt{2\pi }\text{exp}\left[{\frac{L_j}{2K_j}}^2-\frac{1}{2}\left(\frac{{\lambda }_{0j}^2}{{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}+\sum _{i=1}^N\frac{{y_{i,j}^{\ast }}^2}{{\psi }_j}\right)\right]{\pi }_{\mathrm{jk}}\\ +\exp \left[-\frac{1}{2}\left(\sum _{i=1}^N\frac{{y_{\mathrm{ij}}^{\ast }}^2}{{\psi }_j}\right)\right]\left(1-{\pi }_{\mathrm{jk}}\right)\end{array}},\end{array}$$ (7)

where $y_{i,j}^{\ast }$ is the $j$-th element of $Y_i^{\ast }$, $K_j=\sum _{i=1}^N\frac{f_{\mathrm{ki}}^2}{{\psi }_j}+\frac{1}{{\psi }_j{\sigma }_{\lambda ,0\mathrm{jk}}^2}$, and $L_j=\frac{{\lambda }_{0\mathrm{jk}}}{{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}+\sum _{i=1}^N\frac{y_{\mathrm{ij}}^{\ast }{f}_{\mathrm{ki}}}{{\psi }_j}$.

Since the derivation of the probability (7) is based on the whole group factor, we call the factor loading selection based on (7) as the spike and slab method with factor membership (BSS-FM). Alternatively, we can also obtain the probability for an individual factor loading as follows. We consider the joint conditional distribution of parameters for each item [30]:

$$\begin{array}{rl}p\left({\mathrm{\Lambda }}_j,{\text{r}}_j,{\psi }_j|Y_j,F\right)& \propto ({\psi }_j{)}^{-\frac{N}{2}}\exp \left\{-\frac{1}{2{\psi }_j}\sum _{i=1}^N{\left(y_{\mathrm{ij}}-{\mathrm{\Lambda }}_j{f}_i\right)}^2\right\}{{\psi }_j}^{-\left({\alpha }_{1j}+1\right)}\exp \left(-\frac{{\alpha }_{2j}}{{\psi }_j}\right)\\ & \times \prod _{k=1}^q\left\{{\pi }_{\mathrm{jk}}^{r_{\mathrm{jk}}}{\left(1-{\pi }_{\mathrm{jk}}\right)}^{1-r_{\mathrm{jk}}}\right\}\prod _{k=1}^q({\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2{)}^{-\frac{1}{2}}\exp \left\{-\frac{{\left({\lambda }_{\mathrm{jk}}-{\lambda }_{0\mathrm{jk}}\right)}^2}{2{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right\},\end{array}$$ (8)

where ${\mathrm{\Lambda }}_j$ is the $j$-th row of $\mathrm{\Lambda }$. Based on Equation (8), if $r_{\mathrm{jk}}=0$, we have

$$\begin{array}{rl}p\left({\mathrm{\Lambda }}_{\mathrm{jk}},{\text{r}}_j,{\psi }_j|Y_j,F,r_{\mathrm{jk}}=0\right)& \propto ({\psi }_j{)}^{-\frac{N}{2}}\exp \left\{-\frac{1}{2{\psi }_j}\sum _{i=1}^N{\left(z_{\mathrm{ij}}\right)}^2\right\}{{\psi }_j}^{-\left({\alpha }_{1j}+1\right)}\exp \left(-\frac{{\alpha }_{2j}}{{\psi }_j}\right)\\ & \times \prod _{k=1}^q\left\{{\pi }_{\mathrm{jk}}^{r_{\mathrm{jk}}}{\left(1-{\pi }_{\mathrm{jk}}\right)}^{1-r_{\mathrm{jk}}}\right\}\prod _{k=1}^q({\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2{)}^{-\frac{1}{2}}\\ & \times \exp \left\{-\frac{{\left({\lambda }_{\mathrm{jk}}-{\lambda }_{0\mathrm{jk}}\right)}^2}{2{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right\},\end{array}$$

where $z_{\mathrm{ij}}=y_{\mathrm{ij}}-{\mathrm{\Lambda }}_{j,-k}{f}_i$ and ${\mathrm{\Lambda }}_{j,-k}$ is ${\mathrm{\Lambda }}_j$ with 0 for the $k$-th element. Also, if $r_{\mathrm{jk}}=1$,

$$\begin{array}{rl}& p\left({\mathrm{\Lambda }}_{\mathrm{jk}},{\text{r}}_j,{\psi }_j|Y_j,F,r_{\mathrm{jk}}=1\right)\\ & \propto ({\psi }_j{)}^{-\frac{N}{2}}\exp \left\{-\frac{1}{2{\psi }_j}\sum _{i=1}^N{\left(z_{\mathrm{ij}}-{\lambda }_{\mathrm{jk}}{f}_{\mathrm{ik}}\right)}^2\right\}({\psi }_j{)}^{-\frac{{\alpha }_{1j}+1}{2}}\exp \left(-\frac{{\alpha }_{2j}}{{\psi }_j}\right)\\ & \times \prod _{k=1}^q\left\{{\pi }_{\mathrm{jk}}^{r_{\mathrm{jk}}}{\left(1-{\pi }_{\mathrm{jk}}\right)}^{1-r_{\mathrm{jk}}}\right\}\prod _{k=1}^q({\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2{)}^{-\frac{1}{2}}\exp \left\{-\frac{{\left({\lambda }_{\mathrm{jk}}-{\lambda }_{0\mathrm{jk}}\right)}^2}{2{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right\},\end{array}$$

where $f_{\mathrm{ik}}$ is the $k$-th element of $f_i$. Then, the posterior distribution of $r_{\mathrm{jk}}$ is the Bernoulli random variable with the conditional probability

$$\begin{array}{rl}& \frac{\int p\left({\mathrm{\Lambda }}_{\mathrm{jk}},{\text{r}}_j,{\psi }_j|Y_j,F,r_{\mathrm{jk}}=1\right)d{\mathrm{\Lambda }}_{\mathrm{jk}}}{\int p\left({\mathrm{\Lambda }}_{\mathrm{jk}},{\text{r}}_j,{\psi }_j|Y_j,F,r_{\mathrm{jk}}=1\right)d{\mathrm{\Lambda }}_{\mathrm{jk}}+\int p\left({\mathrm{\Lambda }}_{\mathrm{jk}},{\text{r}}_j,{\psi }_j|Y_j,F,r_{\mathrm{jk}}=0\right)d{\mathrm{\Lambda }}_{\mathrm{jk}}}\\ & =\frac{\exp \left[\frac{1}{2}\left(\frac{L^2}{K}-\frac{{\lambda }_{0\mathrm{jk}}^2}{{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right)\right]K^{-\frac{1}{2}}\sqrt{2\pi }{\pi }_{\mathrm{jk}}}{\exp \left[\frac{1}{2}\left(\frac{L^2}{K}-\frac{{\lambda }_{0\mathrm{jk}}^2}{{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right)\right]K^{-\frac{1}{2}}\sqrt{2\pi }{\pi }_{\mathrm{jk}}+\exp \left\{-\frac{{{\lambda }_{0\mathrm{jk}}}^2}{2{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right\}\left(1-{\pi }_{\mathrm{jk}}\right)},\end{array}$$ (9)

where $K=\frac{1}{{\psi }_j}\sum _{i=1}^N{f}_{\mathrm{ik}}^2+\frac{1}{{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2},L=\frac{1}{{\psi }_j}\sum _{i=1}^N{f}_{\mathrm{ik}}{z}_{\mathrm{ij}}+\frac{{\lambda }_{0\mathrm{jk}}}{{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}$. As opposed to the probability derivation in (7), we call the factor loading selection based on (9) as the spike and slab method in an individual loading (BSS-IL).

We remark that the derivations of the slab probability for BSS-FM and BSS-IL are based on the marginal conditional distributions of whole group factors and individual factor loadings, respectively. This is different from the approach by Lu et al. [31], which considers each variate’s distribution separately. Also, in an application of the spike and slab prior, we use distinct priors for unknown loadings, known nonzero loadings and known zero loadings. This allows us to incorporate a priori knowledge about model structures in the model selection process.

4. Simulation study

We conduct a Monte Carlo study using various scenarios in terms of the number of items, factors, and cross loadings. We primarily assess the proposed methods based on their rate of correctly identifying the underlying factor structure. We consider four different factor models: (1) an 8-item model ( $p=8$) with one general factor and two group factors (each group factor corresponds to four non-overlapping items); (2) a 12-item model ( $p=12$) with one general factor and three group factors (each group factor corresponds to four non-overlapping items); (3) an 8-item model ( $p=8$) with one general factor and two group factors (each group factor has two cross loadings); and (4) a 12-item model ( $p=12$) with one general factor and three group factors (each group factor has two cross loadings). Models (1) and (2) have strict bifactor model structures and models (3) and (4) allow some cross loadings. We compare the performance of our method with the method by Lu et al. (BSEM-SSP) [31], ridge prior [39] and a post-processing approach [41]. We include the post-processing approach to show the performance of factor rotations with MCMC samples. We do not include ESEM approach since the method is not designed for factor loading selection. It is worth mentioning that the varimax method, often used for factor rotation, has limited success with bifactor models [27]. In this comparison, all non-zero factor loadings of the factor models are 1, and $\mathrm{\Psi }$ and $\mathrm{\Phi }$ are the identity matrix from the factor model described in Section 2. In the simulation, we initially assume that the factor number is known, but we also consider the cases with unknown factor numbers as indicated by ‘with extra group factors’ as shown in the results of Tables 1–4. For each scenario, we generate 200 data sets with sample sizes $N=$ 200 and 500 observations. For each data set, MCMC samples are obtained using 1000 iterations and a burn-in of 300 draws to assess the performance of the variable selections. We employ a semblance of hypothesis testing with factor loading 0 as the null hypothesis. We utilize symmetric confidence intervals with a 95% confidence level based on the estimation of factor loadings, where a non-zero factor loading is indicated by the confidence interval not including 0. We do not consider the highest density interval [50] since its lower boundary does not include 0 when we have the positive value constraint. Alternatively, similar to other authors’ approaches (Lu, Chow and Loken [31]), we also use the 50% threshold criterion, by which a factor loading is considered 0 if 50% of MCMC samples assign the factor loading as ‘spike’.

Table 1.

Percentage of correctly identified factor loadings in group factors when $p=8$ and $N=500$. In each cell, 1 indicates that the factor loading is correctly identified as zero or non-zero 100% of the times.

		p = 8
		1st group factor	2nd group factor	Extra factor
Bi-factor model	BSS-IL (95% CI)	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
	BSS-IL (50% rule)	1.00 1.00 1.00 1.00	1.00 0.98 0.98 0.97
		1.00 0.98 0.97 0.99	1.00 1.00 1.00 1.00
	BSS-FM (95% CI)	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
	BSS-FM (50% rule)	1.00 1.00 1.00 1.00	1.00 0.95 0.97 0.98
		1.00 0.98 0.99 0.99	1.00 1.00 1.00 1.00
	Ridge prior (95% CI)	0.93 0.76 0.78 0.77	1.00 0.93 0.89 0.93
		1.00 0.84 0.85 0.88	0.97 0.98 0.96 0.97
	BSEM-SSP (95% CI)	1.00 0.98 0.97 0.97	1.00 0.91 0.92 0.95
		0.95 0.92 0.94 0.95	0.65 0.42 0.47 0.46
	BSEM-SSP (50% rule)	1.00 0.67 0.66 0.67	1.00 0.74 0.78 0.75
		1.00 0.76 0.77 0.81	1.00 0.44 0.44 0.43
	Post processing (95% CI)	0.73 0.96 0.95 0.96	0.00 0.00 0.00 0.00
		0.22 0.12 0.12 0.12	1.00 1.00 1.00 1.00
Bifactor model with extra group factor	BSS-IL (95% CI)	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
	BSS-IL (50% rule)	1.00 1.00 1.00 1.00	1.00 0.98 0.98 0.99	1.00 0.90 0.91 0.91
		1.00 0.98 0.97 0.98	1.00 1.00 1.00 1.00	1.00 0.76 0.78 0.79
	BSS-FM (95% CI)	0.99 1.00 0.99 0.99	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	0.99 0.99 0.99 1.00	1.00 0.99 0.99 1.00
	BSS-FM (50% rule)	1.00 1.00 1.00 1.00	1.00 0.97 0.99 0.99	1.00 0.89 0.86 0.90
		1.00 0.99 0.98 0.97	1.00 1.00 1.00 1.00	1.00 0.79 0.75 0.74
	Ridge prior (95% CI)	0.92 0.61 0.62 0.64	1.00 0.93 0.96 0.94	1.00 0.82 0.85 0.84
		1.00 0.92 0.89 0.85	0.99 0.97 0.97 0.96	1.00 0.89 0.89 0.84
	BSEM-SSP (95% CI)	0.89 0.50 0.49 0.43	1.00 0.94 0.93 0.89	1.00 0.93 0.91 0.94
		1.00 0.90 0.92 0.95	0.63 0.39 0.39 0.37	1.00 0.96 0.92 0.93
	BSEM-SSP (50% rule)	1.00 0.57 0.63 0.57	1.00 0.70 0.80 0.77	1.00 0.83 0.77 0.72
		1.00 0.77 0.74 0.82	1.00 0.50 0.49 0.50	1.00 0.77 0.82 0.77
	Post processing (95% CI)	0.78 0.93 0.92 0.92	0.00 0.00 0.00 0.00	0.98 0.92 0.90 0.90
		0.16 0.12 0.12 0.14	1.00 1.00 1.00 1.00	0.80 0.92 0.94 0.94
Bifactor model with cross loading	BSS-IL (95% CI)	0.98 1.00 1.00 0.98	1.00 1.00 1.00 1.00
		1.00 0.99 1.00 1.00	1.00 0.99 1.00 1.00
	BSS-IL (50% rule)	1.00 1.00 1.00 1.00	1.00 1.00 0.95 0.96
		1.00 1.00 0.97 0.97	1.00 1.00 1.00 1.00
	BSS-FM (95% CI)	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
	BSS-FM (50% rule)	1.00 1.00 1.00 1.00	1.00 1.00 0.96 0.97
		1.00 1.00 0.94 0.95	1.00 1.00 1.00 1.00
	Ridge prior (95% CI)	0.88 0.81 0.77 0.78	1.00 0.88 0.96 0.95
		1.00 0.77 0.96 0.97	0.99 0.88 0.98 0.98
	BSEM-SSP (95% CI)	0.96 0.37 0.75 0.73	1.00 0.25 0.93 0.96
		1.00 0.36 0.88 0.88	0.70 0.23 0.44 0.45
	BSEM-SSP (50% rule)	1.00 0.42 0.81 0.83	1.00 0.39 0.76 0.80
		1.00 0.42 0.70 0.68	1.00 0.39 0.65 0.61
	Post processing (95% CI)	1.00 1.00 1.00 1.00	0.00 1.00 0.00 0.00
		0.00 1.00 0.00 0.00	1.00 1.00 1.00 1.00

Table 2.

Percentage of correctly identified factor loadings in group factors when $p=12$ and $N=500$. In each cell, 1 indicates that the factor loading is correctly identified as zero or non-zero 100% of the times.

		p = 12
		1st group factor	2nd group factor	3rd group factor	Extra factor
Bifactor model	BSS-IL (95% CI)	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	0.99 1.00 0.99 1.00
	BSS-IL (50% rule)	1.00 1.00 1.00 1.00	1.00 0.98 0.98 1.00	1.00 1.00 0.99 0.97
		1.00 0.99 0.99 0.99	1.00 1.00 1.00 1.00	1.00 0.99 0.99 0.98
		1.00 0.99 0.99 0.98	1.00 0.98 1.00 0.99	1.00 1.00 1.00 1.00
	BSS-FM (95% CI)	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	0.99 0.99 0.99 1.00
	BSS-FM (50% rule)	1.00 1.00 1.00 1.00	1.00 0.99 0.99 1.00	1.00 1.00 0.99 0.99
		1.00 0.99 0.99 0.99	1.00 1.00 1.00 1.00	1.00 0.98 0.97 0.98
		1.00 0.99 1.00 0.99	1.00 0.98 0.98 0.99	1.00 1.00 1.00 1.00
	Ridge prior (95% CI)	0.92 0.93 0.92 0.92	1.00 0.90 0.86 0.88	1.00 0.90 0.91 0.92
		1.00 0.86 0.83 0.86	0.96 0.96 0.96 0.96	1.00 0.87 0.90 0.88
		1.00 0.89 0.89 0.86	1.00 0.86 0.89 0.89	0.99 0.99 0.98 0.98
	BSEM-SSP (95% CI)	0.94 0.74 0.75 0.71	1.00 0.98 0.97 0.95	1.00 0.95 0.97 0.96
		1.00 0.93 0.96 0.94	0.81 0.60 0.56 0.58	1.00 0.94 0.94 0.96
		1.00 0.96 0.96 0.92	1.00 0.96 0.94 0.94	0.76 0.56 0.56 0.54
	BSEM-SSP (50% rule)	1.00 0.76 0.77 0.78	1.00 0.96 0.98 0.96	1.00 0.96 0.97 0.96
		1.00 0.96 0.98 0.94	1.00 0.52 0.53 0.51	1.00 0.98 0.96 0.96
		1.00 1.00 0.96 0.98	1.00 0.94 0.98 0.98	1.00 0.53 0.54 0.56
	Post processing(95% CI)	0.90 0.76 0.80 0.78	0.00 0.00 0.00 0.00	0.00 0.00 0.00 0.00
		0.26 0.34 0.34 0.34	1.00 1.00 1.00 1.00	0.04 0.00 0.01 0.00
		0.26 0.34 0.34 0.32	0.07 0.00 0.00 0.00	1.00 1.00 1.00 1.00
Bifactor model with extra group factor	BSS-IL (95% CI)	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
	BSS-IL (50% rule)	1.00 1.00 1.00 1.00	1.00 0.99 1.00 0.99	1.00 0.99 0.99 0.99	1.00 0.95 0.94 0.96
		1.00 1.00 1.00 0.99	1.00 1.00 1.00 1.00	1.00 0.99 0.99 0.98	1.00 0.92 0.90 0.91
		1.00 1.00 1.00 1.00	1.00 0.99 1.00 0.99	1.00 1.00 1.00 1.00	1.00 0.95 0.92 0.95
	BSS-FM (95% CI)	0.99 1.00 1.00 0.99	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	0.99 0.99 0.99 0.99	1.00 1.00 1.00 1.00	1.00 0.99 0.99 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
	BSS-FM (50% rule)	1.00 1.00 1.00 1.00	1.00 1.00 0.99 1.00	1.00 1.00 1.00 1.00	1.00 0.99 0.97 0.95
		1.00 1.00 0.98 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 0.99	1.00 0.92 0.92 0.94
		1.00 1.00 1.00 1.00	1.00 0.99 1.00 1.00	1.00 1.00 1.00 1.00	1.00 0.94 0.92 0.91
	Ridge prior (95% CI)	0.98 0.91 0.93 0.96	1.00 0.88 0.90 0.89	1.00 0.91 0.88 0.92	1.00 0.83 0.79 0.80
		1.00 0.89 0.94 0.89	0.99 0.99 0.99 0.98	1.00 0.92 0.86 0.90	1.00 0.84 0.86 0.86
		1.00 0.91 0.90 0.91	1.00 0.92 0.92 0.93	1.00 0.99 0.97 0.98	1.00 0.84 0.89 0.90
	BSEM-SSP (95% CI)	0.94 0.64 0.66 0.66	1.00 0.96 0.97 0.98	1.00 0.97 0.96 0.96	1.00 0.94 0.96 0.96
		1.00 0.98 0.98 0.98	0.69 0.52 0.52 0.52	1.00 0.99 0.96 0.98	1.00 0.94 0.96 0.94
		1.00 0.97 0.99 0.98	1.00 0.98 0.98 0.97	0.64 0.50 0.48 0.50	1.00 0.97 0.97 0.98
	BSEM-SSP (50% rule)	1.00 0.82 0.78 0.78	1.00 0.97 1.00 0.98	1.00 0.96 0.96 0.95	1.00 0.90 0.90 0.89
		1.00 0.96 0.96 0.96	1.00 0.57 0.56 0.56	1.00 0.96 0.93 0.93	1.00 0.94 0.94 0.94
		1.00 0.96 0.98 0.98	1.00 0.96 0.96 0.96	1.00 0.48 0.50 0.48	1.00 0.92 0.94 0.92
	Post processing(95% CI)	1.00 1.00 1.00 1.00	0.00 0.00 0.00 0.00	0.00 0.00 0.00 0.00	0.95 0.96 0.96 0.95
		0.00 0.00 0.00 0.00	1.00 1.00 1.00 1.00	0.04 0.00 0.00 0.00	0.88 0.98 0.99 0.98
		0.00 0.00 0.00 0.00	0.10 0.00 0.00 0.00	1.00 1.00 1.00 1.00	0.86 0.96 0.98 0.96
Bifactor model with cross loading	BSS-IL (95% CI)	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 0.99	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
	BSS-IL (50% rule)	1.00 1.00 1.00 1.00	1.00 0.98 0.99 1.00	1.00 0.93 0.93 0.98
		1.00 0.92 0.93 0.97	1.00 1.00 1.00 1.00	1.00 0.96 0.94 1.00
		1.00 0.95 0.96 0.97	1.00 0.99 0.97 0.98	1.00 1.00 1.00 1.00
	BSS-FM (95% CI)	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 0.99
	BSS-FM (50% rule)	1.00 1.00 1.00 1.00	1.00 0.97 0.97 1.00	1.00 0.96 0.96 0.96
		1.00 0.95 0.96 0.98	1.00 1.00 1.00 1.00	1.00 0.96 0.93 1.00
		1.00 0.98 0.99 0.97	1.00 0.96 0.98 0.97	1.00 1.00 1.00 1.00
	Ridge prior (95% CI)	0.98 0.91 0.92 0.74	1.00 0.94 0.97 0.95	1.00 0.82 0.85 0.92
		1.00 0.80 0.84 0.96	0.98 0.97 0.97 0.99	1.00 0.85 0.80 0.97
		1.00 0.72 0.67 0.74	1.00 0.93 0.84 0.85	0.97 0.95 0.95 0.95
	BSEM-SSP (95% CI)	0.98 0.91 0.92 0.74	1.00 0.94 0.97 0.95	1.00 0.82 0.85 0.92
		1.00 0.80 0.84 0.96	0.98 0.97 0.97 0.99	1.00 0.85 0.80 0.97
		1.00 0.72 0.67 0.74	1.00 0.93 0.84 0.85	0.97 0.95 0.95 0.95
	BSEM-SSP (50% rule)	0.98 0.88 0.86 0.82	1.00 0.99 0.99 0.60	1.00 0.96 0.96 0.94
		1.00 1.00 1.00 0.98	0.76 0.65 0.66 0.62	1.00 0.98 0.96 0.40
		1.00 0.98 0.98 0.98	1.00 0.96 0.96 0.96	0.53 0.38 0.38 0.38
	Post processing (95% CI)	1.00 0.89 0.89 0.84	1.00 0.98 0.98 0.64	1.00 0.96 0.96 0.84
		1.00 0.97 0.96 0.95	1.00 0.72 0.71 0.66	1.00 0.92 0.93 0.47
		1.00 0.98 0.94 0.98	1.00 0.98 0.98 0.96	1.00 0.42 0.42 0.41

Table 3.

Percentage of correctly identified factor loadings in group factors when $p=8$ and $N=200$. In each cell, 1 indicates that the factor loading is correctly identified as zero or non-zero 100% of the times.

		P = 8
		1st group factor	2nd group factor	Extra factor
Bi-factor model	BSS-IL (95% CI)	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
	BSS-IL (50% rule)	1.00 1.00 1.00 1.00	1.00 0.94 0.96 0.93
		1.00 0.97 0.95 0.90	1.00 1.00 1.00 1.00
	BSS-FM (95% CI)	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
	BSS-FM (50% rule)	1.00 1.00 1.00 1.00	1.00 0.95 0.93 0.93
		1.00 0.97 0.94 0.95	1.00 1.00 1.00 1.00
	Ridge prior (95% CI)	0.93 0.76 0.71 0.79	1.00 0.94 0.95 0.94
		1.00 0.92 0.87 0.88	0.93 0.93 0.93 0.93
	BSEM-SSP (95% CI)	0.86 0.39 0.45 0.48	1.00 0.82 0.84 0.85
		1.00 0.88 0.89 0.85	0.55 0.36 0.34 0.35
	BSEM-SSP (50% rule)	1.00 0.62 0.61 0.59	1.00 0.59 0.61 0.62
		1.00 0.68 0.72 0.71	1.00 0.53 0.50 0.47
	Post processing(95% CI)	0.76 0.96 0.94 0.96	0.02 0.02 0.03 0.03
		0.22 0.12 0.12 0.12	0.99 0.99 0.99 0.99
Bifactor model with extra group factor	BSS-IL (95% CI)	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 0.99 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 0.99
	BSS-IL (50% rule)	1.00 1.00 1.00 1.00	1.00 0.95 0.96 0.95	1.00 0.69 0.65 0.71
		1.00 0.97 0.96 0.97	1.00 1.00 1.00 1.00	1.00 0.63 0.65 0.58
	BSS-IL (50% rule)	1.00 1.00 1.00 1.00	1.00 0.95 0.96 0.95	1.00 0.69 0.65 0.71
		1.00 0.97 0.96 0.97	1.00 1.00 1.00 1.00	1.00 0.63 0.65 0.58
	BSS-FM (95% CI)	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 0.99 0.99 1.00
		1.00 1.00 1.00 1.00	0.99 0.99 0.99 1.00	1.00 1.00 1.00 1.00
	BSS-FM (50% rule)	1.00 1.00 1.00 1.00	1.00 0.94 0.95 0.97	1.00 0.73 0.73 0.74
		1.00 0.97 0.96 0.95	1.00 1.00 1.00 1.00	1.00 0.63 0.59 0.58
	Ridge prior (95% CI)	0.94 0.73 0.79 0.76	1.00 0.98 0.99 0.99	1.00 0.90 0.89 0.87
		1.00 0.94 0.97 0.96	0.92 0.91 0.88 0.90	1.00 0.91 0.91 0.92
	BSEM-SSP (95% CI)	0.85 0.32 0.27 0.26	1.00 0.91 0.90 0.94	1.00 0.90 0.93 0.93
		1.00 0.93 0.88 0.95	0.55 0.20 0.24 0.22	1.00 0.92 0.93 0.94
	BSEM-SSP (50% rule)	1.00 0.54 0.57 0.56	1.00 0.46 0.52 0.51	1.00 0.52 0.55 0.55
		1.00 0.63 0.59 0.59	1.00 0.39 0.42 0.40	1.00 0.63 0.61 0.57
	Post processing (95% CI)	0.86 0.96 0.97 0.96	0.01 0.04 0.02 0.03	0.95 0.80 0.81 0.83
		0.11 0.13 0.13 0.16	0.99 0.99 0.99 0.99	0.86 0.87 0.88 0.85
Bifactor model with cross loading	BSS-IL (95% CI)	1.00 1.00 1.00 1.00	1.00 0.99 1.00 1.00
		1.00 1.00 0.99 1.00	1.00 1.00 1.00 1.00
	BSS-IL (50% rule)	1.00 1.00 1.00 1.00	1.00 1.00 0.94 0.93
		1.00 1.00 0.91 0.94	1.00 1.00 1.00 1.00
	BSS-IL (50% rule)	1.00 1.00 1.00 1.00	1.00 1.00 0.94 0.93
		1.00 1.00 0.91 0.94	1.00 1.00 1.00 1.00
	BSS-FM (95% CI)	0.99 0.99 0.99 1.00	1.00 0.99 0.99 1.00
		1.00 1.00 0.99 1.00	1.00 0.99 1.00 1.00
	BSS-FM (50% rule)	1.00 1.00 1.00 1.00	1.00 1.00 0.93 0.91
		1.00 1.00 0.90 0.95	1.00 1.00 1.00 1.00
	Ridge prior (95% CI)	0.94 0.73 0.88 0.91	1.00 0.86 0.96 0.94
		1.00 0.69 0.99 0.95	0.98 0.91 0.95 0.98
	BSEM-SSP (95% CI)	0.97 0.30 0.56 0.57	1.00 0.30 0.85 0.86
		1.00 0.28 0.92 0.93	0.61 0.27 0.35 0.39
	BSEM-SSP (50% rule)	1.00 0.42 0.67 0.67	1.00 0.45 0.61 0.55
		1.00 0.41 0.65 0.63	1.00 0.44 0.54 0.54
	Post processing(95% CI)	0.99 1.00 1.00 1.00	0.02 1.00 0.02 0.01
		0.03 1.00 0.02 0.05	1.00 1.00 1.00 1.00

Table 4.

Percentage of correctly identified factor loadings in group factors when $p=12$ and $N=200$. In each cell, 1 indicates that the factor loading is correctly identified as zero or non-zero 100% of the times.

		p = 12
		1st group factor	2nd group factor	3rd group factor	Extra factor
Bifactor model	BSS-IL (95% CI)	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
	BSS-IL (50% rule)	1.00 1.00 1.00 1.00	1.00 0.96 0.96 0.94	1.00 0.96 0.96 0.94
		1.00 0.96 0.98 0.98	1.00 1.00 1.00 1.00	1.00 0.98 0.97 0.98
		1.00 0.94 0.98 0.92	1.00 0.98 0.94 0.96	1.00 1.00 1.00 1.00
	BSS-FM (95% CI)	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 0.99 1.00 1.00
		1.00 1.00 1.00 1.00	0.99 0.99 0.99 0.99	1.00 1.00 1.00 0.99
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
	BSS-FM (50% rule)	1.00 1.00 1.00 1.00	1.00 0.95 0.95 0.97	1.00 0.96 0.97 0.97
		1.00 0.94 0.96 0.94	1.00 1.00 1.00 1.00	1.00 0.96 0.93 0.96
		1.00 0.97 0.99 0.97	1.00 0.96 0.96 0.96	1.00 1.00 1.00 1.00
	Ridge prior (95% CI)	0.94 0.96 0.94 0.96	1.00 0.88 0.89 0.90	1.00 0.87 0.88 0.90
		1.00 0.90 0.89 0.91	0.97 0.96 0.98 0.96	1.00 0.91 0.92 0.92
		1.00 0.88 0.92 0.92	1.00 0.90 0.87 0.92	0.97 0.98 0.98 0.97
	BSEM-SSP (95% CI)	0.85 0.50 0.50 0.48	1.00 0.96 0.92 0.94	1.00 0.96 0.93 0.96
		1.00 0.92 0.93 0.94	0.71 0.44 0.43 0.45	1.00 0.95 0.95 0.94
		1.00 0.94 0.94 0.97	1.00 0.93 0.92 0.94	0.73 0.54 0.53 0.54
	BSEM-SSP (50% rule)	1.00 0.66 0.64 0.66	1.00 0.90 0.91 0.90	1.00 0.88 0.90 0.90
		1.00 0.90 0.92 0.94	1.00 0.62 0.63 0.66	1.00 0.88 0.91 0.90
		1.00 0.92 0.88 0.93	1.00 0.88 0.91 0.92	1.00 0.56 0.56 0.52
	Post processing (95% CI)	0.83 0.89 0.90 0.90	0.03 0.03 0.03 0.03	0.03 0.04 0.04 0.04
		0.18 0.16 0.15 0.17	0.98 0.98 0.98 0.98	0.20 0.14 0.09 0.14
		0.18 0.16 0.18 0.16	0.46 0.04 0.04 0.04	0.98 0.98 0.99 0.98
Bifactor model with extra group factor	BSS-IL (95% CI)	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
	BSS-IL (50% rule)	1.00 1.00 1.00 1.00	1.00 0.95 0.98 0.96	1.00 0.97 0.97 0.98	1.00 0.90 0.86 0.84
		1.00 0.98 0.98 0.98	1.00 1.00 1.00 1.00	1.00 0.97 1.00 0.98	1.00 0.86 0.84 0.84
		1.00 0.98 0.97 0.96	1.00 0.96 0.96 0.98	1.00 1.00 1.00 1.00	1.00 0.84 0.84 0.84
	BSS-FM (95% CI)	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
	BSS-FM (50% rule)	1.00 1.00 1.00 1.00	1.00 0.97 0.96 0.97	1.00 0.97 0.95 0.96	1.00 0.92 0.88 0.88
		1.00 0.96 0.99 0.98	1.00 1.00 1.00 1.00	1.00 0.98 0.98 0.98	1.00 0.78 0.87 0.80
		1.00 0.98 0.98 0.98	1.00 0.98 0.97 0.95	1.00 1.00 1.00 1.00	1.00 0.86 0.81 0.82
	Ridge prior (95% CI)	1.00 0.98 0.97 0.94	1.00 0.94 0.93 0.94	1.00 0.92 0.92 0.93	1.00 0.88 0.94 0.92
		1.00 0.94 0.91 0.92	0.98 0.98 0.97 0.98	1.00 0.93 0.92 0.90	1.00 0.90 0.90 0.89
		1.00 0.92 0.92 0.91	1.00 0.94 0.90 0.88	0.96 0.96 0.95 0.97	1.00 0.91 0.91 0.92
	BSEM-SSP (95% CI)	0.89 0.46 0.47 0.48	1.00 0.99 0.98 1.00	1.00 0.98 0.98 0.99	1.00 0.96 0.98 0.96
		1.00 0.99 0.99 0.99	0.64 0.35 0.32 0.34	1.00 0.98 0.98 0.98	1.00 0.98 0.98 0.99
		1.00 1.00 1.00 1.00	1.00 1.00 0.98 1.00	0.66 0.37 0.39 0.37	1.00 0.98 0.96 0.97
	BSEM-SSP (50% rule)	1.00 0.66 0.64 0.64	1.00 0.91 0.88 0.86	1.00 0.92 0.94 0.92	1.00 0.80 0.82 0.82
		1.00 0.94 0.92 0.92	1.00 0.57 0.56 0.56	1.00 0.92 0.92 0.94	1.00 0.86 0.90 0.82
		1.00 0.92 0.94 0.93	1.00 0.86 0.90 0.89	1.00 0.59 0.60 0.60	1.00 0.87 0.89 0.89
	Post processing (95% CI)	0.99 0.99 1.00 0.98	0.01 0.00 0.01 0.01	0.01 0.03 0.02 0.01	0.96 0.98 0.96 0.95
		0.03 0.03 0.01 0.01	1.00 1.00 1.00 1.00	0.30 0.10 0.09 0.12	0.94 0.98 0.99 0.99
		0.03 0.02 0.03 0.00	0.42 0.02 0.01 0.04	1.00 0.99 1.00 1.00	0.92 0.98 0.96 0.99
Bifactor model with cross loading	BSS-IL (95% CI)	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 0.99 1.00 1.00	1.00 1.00 1.00 1.00	1.00 0.99 0.98 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
	BSS-IL (50% rule)	1.00 1.00 1.00 1.00	1.00 0.94 0.96 1.00	1.00 0.88 0.88 0.98
		1.00 0.92 0.89 0.97	1.00 1.00 1.00 1.00	1.00 0.84 0.90 1.00
		1.00 0.92 0.92 0.92	1.00 0.96 0.95 0.98	1.00 1.00 1.00 1.00
	BSS-FM (95% CI)	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 0.99 0.99 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
	BSS-FM (50% rule)	1.00 1.00 1.00 1.00	1.00 0.96 0.97 1.00	1.00 0.96 0.97 1.00
		1.00 0.82 0.92 0.98	1.00 1.00 1.00 1.00	1.00 1.00 1.00 1.00
		1.00 0.93 0.90 0.94	1.00 0.96 0.95 0.96	1.00 0.96 0.95 0.96
	Ridge prior (95% CI)	0.97 0.95 0.95 0.90	1.00 0.94 0.95 0.97	1.00 0.85 0.90 0.97
		1.00 0.88 0.85 0.97	1.00 1.00 1.00 0.99	1.00 0.85 0.87 0.98
		1.00 0.75 0.83 0.84	1.00 0.85 0.89 0.92	0.96 0.97 0.97 0.98
	BSEM-SSP (95% CI)	0.94 0.76 0.74 0.66	1.00 1.00 0.99 0.38	1.00 0.97 0.96 0.90
		1.00 1.00 0.99 0.96	0.71 0.56 0.52 0.44	1.00 1.00 0.99 0.42
		1.00 0.98 0.98 0.98	1.00 0.96 0.96 0.96	0.60 0.40 0.37 0.41
	BSEM-SSP (50% rule)	1.00 0.77 0.75 0.72	1.00 0.95 0.92 0.60	1.00 0.92 0.89 0.80
		1.00 0.95 0.94 0.88	1.00 0.64 0.66 0.68	1.00 0.86 0.87 0.60
		1.00 0.90 0.94 0.92	1.00 0.92 0.90 0.92	1.00 0.54 0.56 0.54
	Post processing (95% CI)	0.92 0.96 0.96 0.96	0.04 0.03 0.03 0.98	0.04 0.04 0.06 0.57
		0.08 0.07 0.09 0.32	0.98 0.98 0.98 0.98	0.20 0.14 0.10 0.96
		0.09 0.08 0.07 0.07	0.51 0.07 0.04 0.06	0.98 0.96 0.96 0.96

Table 1 $(p=8)$ and 2 $\left(p=12\right)$ show the factor loading selections of group factors with $N=500$, and Table 3 $(p=8)$ and 4 $(p=12)$are for $N=200$. All tables include strictly bifactor models as well as models with cross loadings. Additionally, ‘extra factor’ indicates simulations with an assumption that we have one more group factor with an unknown number of group factors. As shown in the simulation study, regardless of sample sizes, our BSS-IL and BSS-FM with the symmetric confidence intervals criteria demonstrate 98-100% correctness of identifying zero and non-zero variables for each latent factor (Tables 1–4) throughout the scenarios of strict bifactor models, model identification with extra group factor, and bifactor models with cross loading. Bayesian SEM with ridge prior demonstrate an average of 92% of correctness when identifying the factor structure in the bifactor models and an average of 89% correctness when identifying the extra factor (i.e. correctly identifying as zero). In terms of models with cross loadings, it can identify an average of 91% of cross loading structures. BSEM-SSP shows better performance when identifying zero factor loadings compared to non-zero factor loadings in each latent factor for each scenario. The post-processing method shows reasonable performance in identifying non-zero factor loadings while failing to distinguish the zero factor loadings in the bifactor model. This suggests that while factor rotation may help identify non-zero factor loadings, it cannot effectively separate the zero factor loadings across group latent factors and scenarios.

If we use the 50% threshold criterion, the correctness is in general inferior to symmetric confidence interval criteria, but BSS-IL and BSS-FM still show a better performance than other approaches. Overall, our methods provide quantifiable measures of the uncertainties associated with cross loading inclusion and model choices, leading to more correctly identified results compared to other available methods.

It has been discussed that the parameters can be estimated based on the MCMC samples with various prior settings including SSP [31]. We observe that oftentimes MCMC samples do not provide satisfactory estimates and relevant inferences, when the majority of group factor loading structures are not determined a priori. Instead, we recommend that MCMC samples using SSP be primarily used for model selection, followed by the final model being checked for identifiability. If identifiability is satisfied, then the parameters can be estimated using the maximum likelihood approach, in which the factor loadings identified as zero in the MCMC step remain at zero. This strategy works well, as shown in Table 5, where we present simulation results for two versions of the model: one with 12 items and two group factors and the other with 12 items and three group factors. Each model includes one or two cross loadings. Due to a high chance of correctly identifying the factor structure, the results show that the coverage rate of 95% confidence intervals of all factor loadings is close to the target confidence level.

Table 5.

Coverage rates of nonzero factor loading estimates using the 95% confidence interval. There are 12 general factor loadings and 6 (two group factor model) or 4 (three group factor model) group factor loadings per factor to be estimated in addition to cross loadings (indicated by bold numbers).

		General factor	1st group factor	2nd group factor	3rd group factor
2 factor model	Bifactor model with 1 cross loading (bold)	0.94 0.94 0.96 0.95	0.94 0.94 0.95 0.95	0.94 0.95 0.93 0.92
		0.93 0.96 0.94 0.94	0.94 0.95 0.94	0.96 0.94
		0.95 0.94 0.96 0.97
	Bifactor model with 2cross loading (bold)	0.93 0.95 0.96 0.92	0.90 0.91 0.90 0.95	0.98 0.95 0.95 0.93
		0.94 0.95 0.94 0.94	0.90 0.97 0.94	0.96 0.94 0.96
		0.95 0.94 0.93 0.93
3 factor model	Bifactor model with 1 cross loading	0.94 0.93 0.95 0.95	0.93 0.90 0.95 0.95	0.97 0.94 0.94 0.91	0.95 0.96 0.94 0.95
		0.95 0.96 0.94 0.93		0.94
		0.96 0.93 0.97 0.94
	Bifactor model with 2cross loading	0.94 0.92 0.95 0.93	0.97 0.96 0.96 0.96	0.92 0.94 0.93 0.95	0.94 0.93 0.95 0.91
		0.94 0.93 0.96 0.93		0.96	0.94
		0.93 0.93 0.97 0.95

5. Application

For the application of the proposed methods, we use the SF-12 version 2 (SF-12v2) data [36] from the Household Component of the Medical Expenditure Panel Survey (MEPS-HC) carried out between 2015 and 2016 ( $N$ = 17,017). The SF-12v2 is a widely used health survey consisting of 12 questions designed to assess self-reported health-related quality of life. The SF-12v2 is divided into two areas, physical health and mental health [51] with corresponding scales for each area. The area on physical health focuses on participants’ general health, limitations in mobility, work, and other physical activities as well as limitations due to pain. The corresponding scales include general health (one item), physical functioning (two items), role physical (two items), and bodily pain (one item). The area on mental health encompasses limitations in social activity, emotional state, and level of distraction. The corresponding scales include a physical health and vitality summary (one item), social functioning (one item), role emotional (two items), and mental health (two items). In the data analysis, following common practice [18,28], each of the coded scores in the SF-12v2 is converted to a value ranging from 0 to 100, and inverse-coded scores are corrected so that 100 indicates the best health condition.

Figure 1 compares the bifactor model, the Bayesian factor selection with ridge prior, and our proposed Bayesian factor selection (BSS-IL and BSS-FM). We note that, when employing the Lasso prior [11], the bifactor model is chosen. We use ‘limited in moderate activity’ and ‘feel blue’ as the items with the known bifactor structure for BSS-IL and BSS-FM as well as the ridge prior. That is, we have a prior belief that ‘limited in moderate activity’ strictly corresponds to only physical health, while ‘feel blue’ only corresponds to mental health.

Figure 1.

Factor model and factor loading estimate from bifactor model, model selected from the ridge prior, and model selected from our BSS-IL and BSS-FM. The numbers under the items (e.g. ADDAYA2, etc.) are estimated general factor loadings. The numbers in the columns under F1 and F2 are the factor loadings for respective group factors. If a factor loading is small negative value, it is fixed to be 0.

The model selected by the ridge prior has seven cross loadings, while the models selected by BSS-IL and BSS-FM show the same results with six cross loadings. In the model selected by BSS-IL and BSS-FM, the items belonging to physical health area such as ‘health in general’ and ‘pain limited in normal work’ load on the mental health area too. A majority of items belonging to mental health area such as ‘social time’, ‘energy’, ‘work limited due to mental problem’ and ‘calm/peaceful’ load on physical health. This may indicate that mental health status is highly affected by the physical health status. Model selection using the ridge prior chooses ‘accomplished less by physical’ as cross loading, while our proposed model does not. The estimated magnitude of this extra factor loading is relatively small compared to other factor loadings. In addition, we compare the fit indexes of these models as shown in Table 6. Both ridge prior selection and our model have the same CFI value. The RMSEA shows that BSS-IL and BSS-FM have a slight improvement of fit over the ridge prior selection. The ridge prior selection has lower AIC and BIC values, but the difference is small. Since BSS-IL and BSS-FM provide a more parsimonious model, we consider that the model selected by BSS-IL and BSS-FM is preferable to the model from ridge prior selection, and thus it may be reasonable to ignore this cross loading in the overall analysis.

Table 6.

Comparison of fit indexes.

	Bi-factor model	Ridge prior	Our model
Comparative Fit Index (CFI)	0.939	0.965	0.965
Akaike information criterion (AIC)	1,235,608.197	1,233,286.928	1,233,301.043
Bayesian information criterion (BIC)	1,235,865.841	1,233,603.462	1,233,610.215
RMSEA	0.105	0.089	0.088

It may be of interest that the model selection is changed by using different items for known bifactor structures. When we change a known structure from ‘limited in moderate activity’ to ‘limits in climbing’, we obtain the same model not affected by the different choice of item.

We note that the CFI value is larger than that of the bifactor model by 0.026, as shown in Table 6. The magnitude of the difference in CFI values is typically interpreted using guidelines proposed by several SEM researchers, such as Cheung and Rensvold [12]. They state that a difference in CFI of 0.01 or more may indicate that the proposed model is a better fit to the data than the alternative model. It is also suggested that a criterion of a 0.01 change in CFI is paired with a change in RMSEA of 0.015 [10]. The higher value of the CFI and lower value of the RMSEA indicate that including cross loadings produces a better model fit compared to the bifactor model. In our models, these cross loadings enhance the validity and reliability of the factor model by capturing the complexity and multidimensionality of the construct being measured.

In addition, we carry out a sensitivity analysis of parameters for the spike and slap prior. For sensitivity analysis in the first scenario, we set the value of the mean as 0 for the spike and slab prior as a common practice (e.g. [31]), while the variance values ( ${\sigma }_{\lambda 0\mathrm{jk}}^2$) are varied between 0.5 and 10 (0.5, 1, 2, 5, 10). The value 1 is the hyperparameter that we originally use as a non-informative prior choice. In this scenario, the prior probability of the slab is fixed at 0.5. Overall, our proposed methods produce consistent results regardless of different prior parameters, with only two items’ loadings showing changes. In particular, for BSS-FM, there are no model structure changes regardless of different variances $\left({\sigma }_{\lambda 0\mathrm{jk}}^2\right)$between 1 and 5. More changes are observed for BSS-IL, but the loading changes are limited to two items. For BSS-FM with variances of 0.5 or 10 and BSS-IL with a variance other than 1, changes are identified in the loadings of the two items: ADMALS2 (accomplished less because of emotional problems) and ADCAPE2 (felt calm/peaceful). ADMALS2 changed from a mental loading to both mental and physical loadings, while ADCAPE2 changed from both loadings to mental loading only. As the name’s annotation suggests, ADMALS2 may be associated with both physical and emotional constructs. For ADCAPE2, it corresponds to the mental construct by the original intention of the scale development. Thus, these changes produce reasonable explanations. The altered model exhibits similar CFI and RMSEA values (0.964 and 0.088, respectively) for model fitting.

In the second scenario for the sensitivity analysis, if researchers have a strong understanding of potential cross loadings, they may use a higher slab probability $\left({\pi }_{\mathrm{jk}}\right)$ than 0.5. In this context, we conduct a sensitivity analysis based on the changes of the slab probability while keeping the mean and variance for the spike and slab prior in a noninformative way (0 and 1, respectively). We vary the slab probability between 0.2 and 0.8 (0.2, 0.4, 0.5, 0.6, 0.8). A value 0.5 indicates no informative prior, that we use for the original data analysis. Once again, the changes were limited only on the two items, ADMALS2 and ADCAPE2. For BSS-FM with ${\pi }_{\mathrm{jk}}$ of 0.2, 0.4, and 0.6 and for BSS-IL with ${\pi }_{\mathrm{jk}}$ of 0.2 and 0.4, the same model as our original analysis is selected. With BSS-IL with ${\pi }_{\mathrm{jk}}$ = 0.6, ADMALS2 and ADCAPE2 have the same loadings as the model described above with the variance changes. When BSS-FM and BSS-IL with ${\pi }_{\mathrm{jk}}=0.8$ which indicates a strong belief of the existence of cross loadings, ADMALS2 and ADCAPE2 have loadings on both constructs. This model also exhibits similar CFI and RMSEA values (0.965 and 0.089, respectively).

Overall, we can reliably suggest that our proposed methods can produce relatively robust results, while our observations indicate that BSS-FM is more robust to hyperparameter changes compared to BSS-IL. Some different choices of the hyperparameters may provide further interpretations of certain items with changed structures. Changing structures indicate that these items may have relatively less solid interpretations. As a rule of thumb, for the final model choices, unless strong prior belief exists regarding the hyperparameters, we recommend choosing the model that shows consistent results by both BSS-FM and BSS-IL, using common non-informative prior choices such as ${\sigma }_{\lambda 0\mathrm{jk}}^2=1$ and ${\pi }_{\mathrm{jk}}$ = 0.5.

6. Conclusion

In this paper, we presented a novel Bayesian variable-selection approach for factor loading selection in CFA. Our methods employed SSP for variable selection, and we discussed how SSP distributions can be used in model selection. In the proposed methods, the probability of including each factor loading was assessed through a marginal conditional distribution and variables were selected using the binomial distribution, where the assessment involved unobserved latent factor scores. This approach enabled us to assess the level of uncertainties related to cross loading inclusion and model selection. Our methods simplified the calculation of inclusion probabilities and showed an improvement in variable selection performance compared to other existing methods. In addition, to improve the model identification procedures and the numerical stability of the factor model, we proposed model identification constraints for a bifactor model altered by certain cross loadings, which offered ample flexibility in including those loadings. We conducted a Monte Carlo simulation study under various scenarios, including different numbers of items, factors, and cross loadings, where our primary focus was to examine the rate of correct model identification. Our proposed methods led to more correctly identified results when compared to other available methods. For real data analysis, we applied our proposed methods to the SF-12V2 dataset from MEPS. The BSS-IL and BSS-FM models showed that items in the physical health area were loaded on the mental health area and that the majority of items in the mental health area were also loaded on the physical health area. This suggested that the statuses of one’s physical health and mental health may be highly linked. Our models demonstrated that incorporating cross loadings can improve the validity and reliability of the factor model by accounting for the intricacy and multi-dimensionality of the constructs under investigation. In summary, our BSS-IL and BSS-FM methods outperformed other methods in factor model selection, and we demonstrated their effectiveness through simulation studies and real-world applications.

Appendices.

Appendix 1: Proofs

In this appendix, we provide the proofs of theorems and corollaries in Section 3. We state following lemmas first to simplify the proof of the theorems.

Lemma 1

Suppose a cyclic system of equations $\mathrm{xy}=x^{\mathrm{\prime }}{y}^{\mathrm{\prime }},\mathrm{xz}=x^{\mathrm{\prime }}{z}^{\mathrm{\prime }},\mathrm{yz}=y^{\mathrm{\prime }}{z}^{\mathrm{\prime }}$, where $x,y,z,x^{\mathrm{\prime }},y^{\mathrm{\prime }},z^{\mathrm{\prime }}$ are real numbers. Then $x=\pm x^{\mathrm{\prime }},y=\pm y^{\mathrm{\prime }},z=\pm z^{\mathrm{\prime }}$ where the signs of $x,y,z$ are same.

Proof: If $x^{\mathrm{\prime }}=\mathrm{cx}$ for a real number constant $c$, then $y^{\mathrm{\prime }}=\frac{1}{c}y$ and $z^{\mathrm{\prime }}=\frac{1}{c}z$. This leads to $y^{\mathrm{\prime }}{z}^{\mathrm{\prime }}=\frac{1}{c^2}\mathrm{yz}=\mathrm{yz}$. Thus, $c=\pm 1$.

Lemma 2

Suppose that $a_i$ $\left(i=1,\dots ,n<\mathrm{\infty }\right)$ are arbitrary real numbers, and $x_i$ and $x_i^{\mathrm{\prime }}$ are real constants. Then, $\sum _{i=1}^n{a}_i{x}_i=\sum _{i=1}^n{a}_i{x}_i^{\mathrm{\prime }}\iff x_i=x_i^{\mathrm{\prime }},i=1,\dots ,n$.

Proof: $\sum _{i=1}^n{a}_i{x}_i=\sum _{i=1}^n{a}_i{x}_i^{\mathrm{\prime }}\iff \sum _{i=1}^n{a}_i(x_i-x_i^{\mathrm{\prime }})=0$. We can let $x_1-x_1^{\mathrm{\prime }}=c_i\left(x_i-x_i^{\mathrm{\prime }}\right),i=1,\dots ,n$ for some constant $c_i$’s. Then, $\sum _{i=1}^n{a}_i(x_i-x_i^{\mathrm{\prime }})=0\iff (x_1-x_1^{\mathrm{\prime }})\sum _{i=1}^n{a}_i{c}_i=0$. Thus, $x_1=x_1^{\mathrm{\prime }}$. In turn, $x_i=x_i^{\mathrm{\prime }},i=1,\dots ,n$.

Proof Proof of Theorem 1 —

The elements in $\mathrm{\Sigma }$is expressed as $\mathrm{\Sigma }={\mathit{\lambda }}_0{\mathit{\lambda }}_0^T+{\mathit{\lambda }}_1{\mathit{\lambda }}_1^T+{\mathit{\lambda }}_2{\mathit{\lambda }}_2^T+\mathrm{\Phi },$ where $\mathrm{\Phi }$ affects only diagonal elements of $\mathrm{\Sigma }.$

(i) Without loss of generality, let us assume | $B_{1,\sim c}|\ge 3$ and $2\in H_2$ by condition (C5) and (C4), respectively. The elements in $\mathrm{\Sigma }$ corresponding to $B_{1,\sim c}$ and $B_{2,\sim c}$ exist by condition (C2) and form off-diagonal elements. Thus, to satisfy non-identifiability, we have ${\mathit{\lambda }}_0\left[B_{1,\sim c}\right]{\mathit{\lambda }}_0[B_{2,\sim c}{]}^T$ =  ${\mathit{\lambda }}_0^{\mathrm{\prime }}\left[B_{1,\sim c}\right]{\mathit{\lambda }}_0^{\mathrm{\prime }}[B_{2,\sim c}{]}^T$ since ${\mathit{\lambda }}_1\left[B_{2,\sim c}\right]=0$ and ${\mathit{\lambda }}_2\left[B_{1,\sim c}\right]=0$. This leads to

$${\mathit{\lambda }}_0^{\mathbf{\prime }}\left[B_{1,\sim c}\right]=k{\mathit{\lambda }}_0\left[B_{1,\sim c}\right]\text{and}{\mathit{\lambda }}_0^{\mathbf{\prime }}\left[B_{2,\sim c}\right]=\frac{1}{k}{\mathit{\lambda }}_0\left[B_{2,\sim c}\right].$$ (A.1)

Since $2\in H_2$by (C4), we have a disjoint union of $S_1$ and $S_2$ satisfying $B_{2,\sim c}=S_1\bigsqcup S_2$, and each subset has a rank at least 2. Then, for off-diagonal elements in $\mathrm{\Sigma }$ corresponding to $S_1$ and $S_2$, non-identifiability results in

$$\begin{array}{rl}{\mathit{\lambda }}_0\left[S_1\right]{\mathit{\lambda }}_0[S_2{]}^T+{\mathit{\lambda }}_2\left[S_1\right]{\mathit{\lambda }}_2[S_2{]}^T& ={\mathit{\lambda }}_0^{\mathrm{\prime }}\left[S_1\right]{\mathit{\lambda }}_0^{\mathrm{\prime }{\left[S_2\right]}^T}+{\mathit{\lambda }}_2^{\mathrm{\prime }}\left[S_1\right]{\mathit{\lambda }}_2^{\mathrm{\prime }{\left[S_2\right]}^T}\\ & \iff \left(1-\frac{1}{k^2}\right){\mathit{\lambda }}_0\left[S_1\right]{\mathit{\lambda }}_0[S_2{]}^T+{\mathit{\lambda }}_2\left[S_1\right]{\mathit{\lambda }}_2[S_2{]}^T={\mathit{\lambda }}_2^{\mathrm{\prime }}\left[S_1\right]{\mathit{\lambda }}_2^{\mathrm{\prime }{\left[S_2\right]}^T},\end{array}$$ (A.2)

by (A.1). To match the rank of left-hand side and the right-hand side of the equation (A.2), $k=\pm 1$. Let $k=1$ by (C3). Thus, by (A.1), we have

$${\mathit{\lambda }}_0\left[B_{1,\sim c}\right]={\mathit{\lambda }}_0^{\mathbf{\prime }}\left[B_{1,\sim c}\right]\text{ and }{\mathit{\lambda }}_0\left[B_{2,\sim c}\right]={\mathit{\lambda }}_0^{\mathbf{\prime }}\left[B_{2,\sim c}\right].$$ (A.3)

Now, for the off-diagonal elements in $\mathrm{\Sigma }$ corresponding to $B_{1,\sim c}$,

$$\begin{array}{rl}& \left({\mathit{\lambda }}_0\right[B_{1,\sim c}]{\mathit{\lambda }}_0[B_{1,\sim c}{]}^T+{\mathit{\lambda }}_1[B_{1,\sim c}\left]{\mathit{\lambda }}_1[B_{1,\sim c}{]}^T\right)\circ J\\ & ={{\mathit{\lambda }}_0}^{\mathrm{\prime }}\left[B_{1,\sim c}\right]{{\mathit{\lambda }}_0}^{\mathrm{\prime }}[B_{1,\sim c}{]}^T+{{\mathit{\lambda }}_1}^{\mathrm{\prime }}\left[B_{1,\sim c}\right]{{\mathit{\lambda }}_1}^{\mathrm{\prime }}[B_{1,\sim c}{]}^T)\circ J\\ & \iff \left({\mathit{\lambda }}_1\right[B_{1,\sim c}\left]{\mathit{\lambda }}_1[B_{1,\sim c}{]}^T\right)\circ J=\left({\mathit{\lambda }}_1^{\mathrm{\prime }}\left[B_{1,\sim c}\right]{{\mathit{\lambda }}_1}^{\mathrm{\prime }}{\left[B_{1,\sim c}\right]}^T\right)\circ J,\end{array}$$

by (A.3), where $J$ is a conformable matrix consisted of 0 for the multiplication of same indexes and 1 for the multiplication of different indexes in the operation of $\left[B_{1,\sim c}\right][B_{1,\sim c}{]}^T$ and $\circ$ indicates the Hadamard product. This satisfies a cyclic relationship in Lemma 1 since $|B_{1,\sim c}|\ge 3$ by (C5). Thus,

$${\mathit{\lambda }}_1\left[B_{1,\sim c}\right]={\mathit{\lambda }}_1^{\mathbf{\prime }}\left[B_{1,\sim c}\right].$$ (A.4)

For the non-identifiability of the off-diagonal elements in $\mathrm{\Sigma }$ corresponding to $B_{1,\sim c}$ and $B_{1,c}\cup$ $B_{2,c}$, we have

$$\begin{array}{rl}& {\mathit{\lambda }}_0\left[B_{1,\sim c}\right]{\mathit{\lambda }}_0[B_{1,c}\cup B_{2,c}{]}^T+{\mathit{\lambda }}_1\left[B_{1,\sim c}\right]{\mathit{\lambda }}_1[B_{1,c}\cup B_{2,c}{]}^T\\ & ={\mathit{\lambda }}_0^{\mathrm{\prime }}\left[B_{1,\sim c}\right]{{\mathit{\lambda }}_0}^{\mathrm{\prime }}[B_{1,c}\cup B_{2,c}{]}^T+{\mathit{\lambda }}_1^{\mathrm{\prime }}\left[B_{1,\sim c}\right]{{\mathit{\lambda }}_1}^{\mathrm{\prime }}[B_{1,c}\cup B_{2,c}{]}^T\\ & \iff {\mathit{\lambda }}_0\left[B_{1,\sim c}\right]{\mathit{\lambda }}_0[B_{1,c}\cup B_{2,c}{]}^T+{\mathit{\lambda }}_1\left[B_{1,\sim c}\right]{\mathit{\lambda }}_1[B_{1,c}\cup B_{2,c}{]}^T\\ & ={\mathit{\lambda }}_0\left[B_{1,\sim c}\right]{\mathit{\lambda }}_0^{\mathrm{\prime }}[B_{1,c}\cup B_{2,c}{]}^T+{\mathit{\lambda }}_1\left[B_{1,\sim c}\right]{\mathit{\lambda }}_1^{\mathrm{\prime }}[B_{1,c}\cup B_{2,c}{]}^T,\end{array}$$ (A.5)

by (A.3) and (A.4). But, by Lemma 2 and $\mathrm{rank}\left(\mathrm{\Lambda }\right[B_{1,\sim c}\left]\right)\ge 2$ by (C6), (A.5) leads to

$${\mathit{\lambda }}_0\left[B_{1,c}\cup B_{2,c}\right]={\mathit{\lambda }}_0^{\mathbf{\prime }}\left[B_{1,c}\cup B_{2,c}\right],\text{ and }{\mathit{\lambda }}_1\left[B_{1,c}\cup B_{2,c}\right]={\mathit{\lambda }}_1^{\mathbf{\prime }}\left[B_{1,c}\cup B_{2,c}\right].$$ (A.6)

(A.3), (A.4) and (A.6) indicate that all elements in ${\mathit{\lambda }}_0$ and ${\mathit{\lambda }}_1$ are identifiable. Finally, for the non-identifiability of the off-diagonal elements in $\mathrm{\Sigma }$ corresponding to $B_{1,c}$ or $B_2$, we have

$$\left({\mathit{\lambda }}_2\left[B_2\cup B_{1,c}\right]{\mathit{\lambda }}_2{\left[B_2\cup B_{1,c}\right]}^T\right)\circ J=\left({\mathit{\lambda }}_2^{\mathrm{\prime }}\left[B_2\cup B_{1,c}\right]{{\mathit{\lambda }}_2}^{\mathrm{\prime }}{\left[B_2\cup B_{1,c}\right]}^T\right)\circ J,$$

where $J$ is a conformable matrix consisted of 0 for the multiplication of same indexes and 1 for the multiplication of different indexes in the operation of $\left[B_2\cup B_{1,c}\right][B_2\cup B_{1,c}{]}^T$. This satisfies cyclic relationship in Lemma 2 if $|B_2|\ge 3$ by (C1) regardless of the existence of $B_{1,c}$. Thus, all elements in ${\mathit{\lambda }}_2$ are identifiable.

(ii) Without loss of generality, let us assume $|B_{1,\sim c}|\ge 3$ and $1\in H_2$ by (C4) and (C5). The elements in $\mathrm{\Sigma }$ corresponding to $B_{1,\sim c}$ and $B_{2,\sim c}$ are off-diagonal elements, thus we have the relationship (A.1) similar to the proof part (i). Since $1\in H_2$ by (C4), we have a disjoint union of $S_1$ and $S_2$ satisfying $B_{1,\sim c}=S_1\bigsqcup S_2$, and each subset has rank at least 2. Then, for off-diagonal elements in $\mathrm{\Sigma }$ corresponding to $S_1$ and $S_2$, non-identifiability results in

$$\begin{array}{rl}{\mathit{\lambda }}_0\left[S_1\right]{\mathit{\lambda }}_0[S_2{]}^T+{\mathit{\lambda }}_1\left[S_1\right]{\mathit{\lambda }}_1[S_2{]}^T& ={\mathit{\lambda }}_0^{\mathrm{\prime }}\left[S_1\right]{{\mathit{\lambda }}_0}^{\mathrm{\prime }}[S_2{]}^T+{\mathit{\lambda }}_1^{\mathrm{\prime }}\left[S_1\right]{{\mathit{\lambda }}_1}^{\mathrm{\prime }}[S_2{]}^T,\\ & \iff \left(1-k^2\right){\mathit{\lambda }}_0\left[S_1\right]{\mathit{\lambda }}_0[S_2{]}^T+{\mathit{\lambda }}_1\left[S_1\right]{\mathit{\lambda }}_1[S_2{]}^T={\mathit{\lambda }}_1^{\mathrm{\prime }}\left[S_1\right]{{\mathit{\lambda }}_1}^{\mathrm{\prime }}[S_2{]}^T.\end{array}$$

To match the rank of left-hand side and the right-hand side of the equation (A.2), $k=\pm 1$. The rest of the proof is same as the proof part (i).

Proof Proof of Corollary 1 —

Without loss of generality, let us assume $|B_{1,\sim c}|\ge 3$and $|B_{2,\sim c}|\ge 1$ by (C5) and (C2) in Theorem 1. The elements in $\mathrm{\Sigma }$ corresponding to $B_{1,\sim c}$ and $B_{2,\sim c}$ are off-diagonal elements and thus satisfy non-identifiability if ${\mathit{\lambda }}_0\left[B_{1,\sim c}\right]{\mathit{\lambda }}_0[B_{2,\sim c}{]}^T={\mathit{\lambda }}_0^{\mathrm{\prime }}\left[B_{1,\sim c}\right]{{\mathit{\lambda }}_0}^{\mathrm{\prime }}[B_{2,\sim c}{]}^T$ since ${\mathit{\lambda }}_1\left[B_{2,\sim c}\right]=0$and ${\mathit{\lambda }}_2\left[B_{1,\sim c}\right]=0$. Thus, we have

$${\mathit{\lambda }}_0\left[B_{1,\sim c}\right]=k{\mathit{\lambda }}_0^{\mathbf{\prime }}\left[B_{1,\sim c}\right]\text{ and }{\mathit{\lambda }}_0\left[B_{2,\sim c}\right]=\frac{1}{k}{\mathit{\lambda }}_0^{\mathbf{\prime }}\left[B_{2,\sim c}\right].$$

However, one element in ${\mathit{\lambda }}_0\left[B_{1,\sim c}\right]$ is fixed, $k=1$. Thus, we have

$${\mathit{\lambda }}_0\left[B_{1,\sim c}\right]={\mathit{\lambda }}_0^{\mathbf{\prime }}\left[B_{1,\sim c}\right]\text{ and }{\mathit{\lambda }}_0\left[B_{2,\sim c}\right]={\mathit{\lambda }}_0^{\mathbf{\prime }}\left[B_{2,\sim c}\right].$$

This is the same conclusion in (A.3). Now, similar arguments used in the proof of Theorem 1 leads to the identifiable $\mathbf{\Lambda }$.

Proof Proof of Theorem 2 —

Without the loss of generality, let $|B_{1,\sim c}|\ge 3$ that satisfies (C5) and $2\in H_2$ by (C4). The elements in $\mathrm{\Sigma }$ corresponding to $B_{1,\sim c}$ and $B_{2,\sim c}$ exist by (C2) and form off-diagonal elements. Based on (C3), the same arguments in (C1), (C2) and (C3) in Theorem 1 lead to

$${\mathit{\lambda }}_0\left[B_{1,\sim c}\right]={\mathit{\lambda }}_0^{\mathbf{\prime }}\left[B_{1,\sim c}\right]\text{ and }{\mathit{\lambda }}_0\left[B_{2,\sim c}\right]={\mathit{\lambda }}_0^{\mathbf{\prime }}\left[B_{2,\sim c}\right].$$ (A.7)

Similar to the part (ii) in the proof of Theorem 1, the same conclusion above is made for let $|B_{1,\sim c}|\ge 3$ that satisfies (C5) and $1\in H_2$ by (C4). Now, for the off-diagonal elements in $\mathrm{\Sigma }$ corresponding to $B_{1,\sim c}$, the same argument used in (C4) establishes ${\mathit{\lambda }}_1\left[B_{1,\sim c}\right]={\mathit{\lambda }}_1^{\mathbf{\prime }}\left[B_{1,\sim c}\right]$.

For the off-diagonal elements in $\mathrm{\Sigma }$ corresponding to $B_{1,\sim c}$ and $B_{1,c}\cup B_2\cup \dots$ $\cup B_g$, the same argument in Theorem 1 establishes

$$\begin{array}{rl}{\mathit{\lambda }}_0\left[B_{1,c}\cup B_2\cup \dots \cup B_g\right]& ={\mathit{\lambda }}_0^{\mathbf{\prime }}\left[B_{1,c}\cup B_2\cup \dots \cup B_g\right]\text{ and }{\mathit{\lambda }}_1\left[B_{1,c}\cup B_2\cup \dots \cup B_g\right]\\ & ={\mathit{\lambda }}_1^{\mathbf{\prime }}\left[B_{1,c}\cup B_2\cup \dots \cup B_g\right],\end{array}$$

where the argument requires to use (C6). So far, we identify all elements in ${\mathit{\lambda }}_0$ and ${\mathit{\lambda }}_1$.

Now, for $B_{2,\sim c}$, $|B_{2,\sim c}|\ge 2$ by (C2) results in the cyclic system of equations. For example, assume $B_{2,\sim c}=\left\{a,b\right\}$, where $a$ and $b$ indicate the item index corresponding to the factor $g=2$. Since all elements in ${\mathit{\lambda }}_0$ and ${\mathit{\lambda }}_1$ are identified and ${\lambda }_{\mathrm{ka}}={\lambda }_{\mathrm{kb}}=0$ for all $k\ge 3$, this leads the non-identifiability relationship

$${\lambda }_{0a}{\lambda }_{0b}+{\lambda }_{1a}{\lambda }_{1b}+{\lambda }_{2a}{\lambda }_{2b}+\dots ={\lambda }_{0a}{\lambda }_{0b}+{\lambda }_{1a}{\lambda }_{1b}+{\lambda }_{2a}^{\mathrm{\prime }}{\lambda }_{2b}^{\mathrm{\prime }}+\dots \iff {\lambda }_{2a}{\lambda }_{2b}={\lambda }_{2a}^{\mathrm{\prime }}{\lambda }_{2b.}^{\mathrm{\prime }}$$

Likewise, we have ${\lambda }_{2a}{\lambda }_{2i}={\lambda }_{2a}^{\mathrm{\prime }}{\lambda }_{2i}^{\mathrm{\prime }}$ and ${\lambda }_{2b}{\lambda }_{2i}={\lambda }_{2b}^{\mathrm{\prime }}{\lambda }_{2i}^{\mathrm{\prime }}$, for any $i\in B$, which form a cyclic system of equations. This identifies all ${\mathit{\lambda }}_2.$ Similar arguments can be made for $B_{g,\sim c},|B_{g,\sim c}|\ge 2$, leading to the identification of all ${\mathit{\lambda }}_g$, $g\in H_1$ satisfying (C2).

Now, without the loss of generality, assume $g=G$ does not satisfy (C2). Since all ${\mathit{\lambda }}_0,\dots ,{\mathit{\lambda }}_{g-1}$ are identified, for the off-diagonal elements in $\mathrm{\Sigma }$ corresponding to $B_g$, the non-identifiability leads to

$${\lambda }_{\mathrm{Gi}}{\lambda }_{\mathrm{Gj}}={\lambda }_{\mathrm{Gi}}^{\mathrm{\prime }}{\lambda }_{\mathrm{Gj}}^{\mathrm{\prime }},i\ne j,j\in B_G.$$

If $|B_G|\ge 3$ by (C1), we have the system of cyclic equations, leading to the identification of ${\mathit{\lambda }}_g\left[B_G\right]$. This leads to the identification of all ${\mathit{\lambda }}_g$ by observing the relationship ${\lambda }_{\mathrm{gi}}{\lambda }_{\mathrm{gj}}={\lambda }_{\mathrm{gi}}^{\mathrm{\prime }}{\lambda }_{\mathrm{gj}}^{\mathrm{\prime }},i\in B-B_G,j\in B_G$. We note that, if there are crossloadings in $B-B_G$, can be less than 3 by forming the system of cyclic equations as ${\lambda }_{\mathrm{gi}}{\lambda }_{\mathrm{Gj}}={\lambda }_{\mathrm{gi}}^{\mathrm{\prime }}{\lambda }_{\mathrm{Gj}}^{\mathrm{\prime }},g\ne G,j\in B_G$; however, $|B_G|$ needs to be at least 3, if no crossloading exists in $B-B_G$.

Proof Proof of Corollary 2 —

Without loss of generality, let us assume $|B_{1,\sim c}|\ge 3$and $|B_{2,\sim c}|\ge 2$ by (C5) and (C2) in Theorem 2. The elements in $\mathrm{\Sigma }$ corresponding to $B_{1,\sim c}$ and $B_{2,\sim c}$ are off-diagonal elements and thus satisfy non-identifiability if ${\mathit{\lambda }}_0\left[B_{1,\sim c}\right]{\mathit{\lambda }}_0[B_{2,\sim c}{]}^T={\mathit{\lambda }}_0^{\mathrm{\prime }}\left[B_{1,\sim c}\right]{{\mathit{\lambda }}_0}^{\mathrm{\prime }}[B_{2,\sim c}{]}^T$ similar to the argument in the proof of Theorem 2. Thus, we have

$${\mathit{\lambda }}_0\left[B_{1,\sim c}\right]=k{\mathit{\lambda }}_0^{\mathbf{\prime }}\left[B_{1,\sim c}\right]\text{ and }{\mathit{\lambda }}_0\left[B_{2,\sim c}\right]=\frac{1}{k}{\mathit{\lambda }}_0^{\mathbf{\prime }}\left[B_{2,\sim c}\right].$$

However, one element in ${\mathit{\lambda }}_0\left[B_{1,\sim c}\right]$ is fixed, $k=1$. Thus, we have

$${\mathit{\lambda }}_0\left[B_{1,\sim c}\right]={\mathit{\lambda }}_0^{\mathbf{\prime }}\left[B_{1,\sim c}\right]\text{ and }{\mathit{\lambda }}_0\left[B_{2,\sim c}\right]={\mathit{\lambda }}_0^{\mathbf{\prime }}\left[B_{2,\sim c}\right].$$

This is the same conclusion in (A.7). Now, similar arguments used in the proof of Theorem 1 leads to the identifiable $\mathbf{\Lambda }$.

Appendix 2: Derivation of (7) and (9)

For $r_{\mathrm{jk}}=0$case, in the right-hand side of Equation (7), we can show

$$\begin{array}{rl}& \exp \left\{-\frac{1}{2}\sum _{i=1}^N{\left(Y_i^{\ast }-{\mathrm{\Lambda }}_k^{\ast c}{f}_{\mathrm{ki}}\right)}^T{\mathrm{\Psi }}^{-1}\left(Y_i^{\ast }-{\mathrm{\Lambda }}_k^{\ast c}{f}_{\mathrm{ki}}\right)\right\}\\ & =\exp \left\{-\frac{1}{2}\sum _{i=1}^N\left({Y_i^{\ast }}^T{\mathrm{\Psi }}^{-1}{Y}_i^{\ast }-{2Y_i^{\ast }}^T{\mathrm{\Psi }}^{-1}{\mathrm{\Lambda }}_k^{\ast c}{f}_{\mathrm{ki}}+f_{\mathrm{ki}}^T{\mathrm{\Lambda }}_l^{\ast c^T}{\mathrm{\Psi }}^{-1}{\mathrm{\Lambda }}_k^{\ast c}{f}_{\mathrm{ki}}\right)\right\}.\end{array}$$ (B.1)

Without loss of generality, let us assume that ${\lambda }_{1k},\dots ,{\lambda }_{j-1,k}$ have the predetermined structure. Then,

$$\begin{array}{rl}{Y}_i^{\ast T}{\mathrm{\Psi }}^{-1}{Y}_i^{\ast }& =\left(y_{i1}^{\ast },\dots ,y_{\mathrm{ip}}^{\ast }\right)\left(\begin{array}{ccc}{\psi }_1^{-1}& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & {\psi }_p^{-1}\end{array}\right)(y_{i1}^{\ast },\dots ,y_{\mathrm{ip}}^{\ast }{)}^T=\frac{{y_{i1}^{\ast }}^2}{{\psi }_1}+\dots +\frac{{y_{\mathrm{ip}}^{\ast }}^2}{{\psi }_p},\\ Y_i^{\ast T}{\mathrm{\Psi }}^{-1}{\mathrm{\Lambda }}_k^{\ast c}{f}_{\mathrm{ki}}& =\left(y_{i1}^{\ast },\dots ,y_{\mathrm{ip}}^{\ast }\right)\left(\begin{array}{ccc}{\psi }_1^{-1}& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & {\psi }_p^{-1}\end{array}\right)\left(\begin{array}{c}{\lambda }_{1k}\\ ⋮\\ {\lambda }_{\mathrm{jk}}\left(=0\right)\\ ⋮\\ 0\end{array}\right)f_{\mathrm{ki}}\\ & =\frac{y_{i1}^{\ast }{\lambda }_{1k}{f}_{\mathrm{ki}}}{{\psi }_1}+\dots +\frac{y_{i,j-1}^{\ast }{\lambda }_{j-1,k}{f}_{\mathrm{ki}}}{{\psi }_{j-1}}+0\dots +0,\\ f_{\mathrm{ki}}^T{\mathrm{\Lambda }}_k^{\ast c^T}{\mathrm{\Psi }}^{-1}{\mathrm{\Lambda }}_k^{\ast c}{f}_{\mathrm{ki}}& =\frac{{\lambda }_{1k}^2{f}_{\mathrm{ki}}^2}{{\psi }_1}+\dots +\frac{{\lambda }_{j-1,k}^2{f}_{\mathrm{ki}}^2}{{\psi }_{j-1}}+0+\dots +0.\end{array}$$

Thus, the orginal Equation (5) is expressed as

$$\begin{array}{rl}& p\left({\mathrm{\Lambda }}_k,\text{ R},\mathrm{\Psi }|Y,F,r_{\mathrm{jk}}=0\right)\\ & \propto \text{exp}\left\{-\frac{1}{2}\left[\begin{array}{rl}& \sum _{i=1}^N\left(\frac{{y_{i1}^{\ast }}^2}{{\psi }_1}-2\frac{y_{i1}^{\ast }{\lambda }_{1k}{f}_{\mathrm{ki}}}{{\psi }_1}+\frac{{\lambda }_{1k}^2{f}_{\mathrm{ki}}^2}{{\psi }_1}\right)+\frac{{({\lambda }_{1k}-{\lambda }_{01k})}^2}{{\psi }_1{\sigma }_{\lambda 01k}^2}+\dots \\ & +\sum _{i=1}^N\left(\frac{{y_{\mathrm{ij}-1}^{\ast }}^2}{{\psi }_{j-1}}-2\frac{y_{\mathrm{ij}-1}^{\ast }{\lambda }_{j-1,k}{f}_{\mathrm{li}}}{{\psi }_{j-1}}+\frac{{\lambda }_{j-1,k}^2{f}_{\mathrm{ki}}^2}{{\psi }_{j-1}}\right)+\frac{{({\lambda }_{j-1,k}-{\lambda }_{0,j-1,k})}^2}{{\psi }_{j-1}{\sigma }_{\lambda 0,j-1,k}^2}\\ & +\sum _{i=1}^N\frac{{y_{\mathrm{ij}}^{\ast }}^2}{{\psi }_j}+\dots +\sum _{i=1}^N\frac{{y_{\mathrm{ip}}^{\ast }}^2}{{\psi }_p}\end{array}\right]\right\}\end{array}$$ (B.2)

For the first term of the exponent in (B.2),

$$\begin{array}{rl}& \exp \left\{-\frac{1}{2}\left[\sum _{i=1}^N\left(\frac{{y_{i1}^{\ast }}^2}{{\psi }_1}-2\frac{y_{i1}^{\ast }{\lambda }_{1k}{f}_{\mathrm{ki}}}{{\psi }_1}+\frac{{\lambda }_{1k}^2{f}_{\mathrm{ki}}^2}{{\psi }_1}\right)+\frac{{({\lambda }_{1k}-{\lambda }_{01k})}^2}{{\psi }_1{\sigma }_{\lambda 01k}^2}\right]\right\}\\ & =\exp \{-\frac{1}{2}[{{\lambda }_{1k}}^2\sum _{i=1}^N\left(\frac{f_{\mathrm{ki}}^2}{{\psi }_1}+\frac{1}{{\psi }_1{\sigma }_{\lambda 01k}^2}\right)-2{\lambda }_{1k}\left(\frac{{\lambda }_{01k}}{{\psi }_1{\sigma }_{\lambda 01k}^2}+\sum _{i=1}^N\frac{y_{i1}^{\ast }{f}_{\mathrm{ki}}}{{\psi }_1}\right)\\ & +\frac{{\lambda }_{01}^2}{{\psi }_1{\sigma }_{\lambda 01k}^2}+\sum _{i=1}^N\frac{{y_{i1}^{\ast }}^2}{{\psi }_1}]\}\\ & =\exp \left[-\frac{1}{2}{K}_1\left({{\lambda }_{1k}}^2-2\frac{L_1}{K_1}{\lambda }_{1k}\right)-\frac{1}{2}\left(\frac{{\lambda }_{01k}^2}{{\psi }_1{\sigma }_{\lambda 01k}^2}+\sum _{i=1}^N\frac{{y_{i1}^{\ast }}^2}{{\psi }_1}\right)\right]\\ & =\exp \left[-\frac{1}{2}{K}_1\left({{\lambda }_{1k}}^2-2\frac{L_1}{K_1}{\lambda }_{1k}+\frac{L_1^2}{K_1^2}\right)+{\left(\frac{L_1}{K_1}\right)}^2\frac{K_1}{2}-\frac{1}{2}\left(\frac{{\lambda }_{01k}^2}{{\psi }_1{\sigma }_{\lambda 01k}^2}+\sum _{i=1}^N\frac{{y_{i1}^{\ast }}^2}{{\psi }_1}\right)\right]\\ & =\exp \left[-\frac{1}{2}{K}_1{\left({\lambda }_{1k}-\frac{L_1}{K_1}\right)}^2\right]\exp \left[{\frac{L_1}{2K_1}}^2-\frac{1}{2}\left(\frac{{\lambda }_{01k}^2}{{\psi }_1{\sigma }_{\lambda 01k}^2}+\sum _{i=1}^N\frac{{y_{i1}^{\ast }}^2}{{\psi }_1}\right)\right]\\ & =\frac{1}{K_1^{-\frac{1}{2}}\sqrt{2\pi }}\exp \left[-\frac{1}{2}{K}_1{\left({\lambda }_{1k}-\frac{L_1}{K_1}\right)}^2\right]K_1^{-\frac{1}{2}}\sqrt{2\pi }\exp \left[{\frac{L_1}{2K_1}}^2-\frac{1}{2}\left(\frac{{\lambda }_{01k}^2}{{\psi }_1{\sigma }_{\lambda 01k}^2}+\sum _{i=1}^N\frac{{y_{i1}^{\ast }}^2}{{\psi }_1}\right)\right],\end{array}$$ (B.3)

where $K_1=\sum _{i=1}^N\frac{f_{\mathrm{ki}}^2}{{\psi }_1}+\frac{1}{{\psi }_1{\sigma }_{\lambda 01k}^2},L_1=\frac{{\lambda }_{01k}}{{\psi }_1{\sigma }_{\lambda 01k}^2}+\sum _{i=1}^N\frac{y_{i1}^{\ast }{f}_{\mathrm{ki}}}{{\psi }_1}$. In (B.3), $\frac{1}{K_1^{-\frac{1}{2}}\sqrt{2\pi }}\exp [\phantom{{\left(\frac{L_1}{K_1}\right)}^2}-\frac{1}{2}{K}_1({\lambda }_{1k}-{\frac{L_1}{K_1})}^2]$ can be integreted out as $\int \mathrm{dF}\left({\lambda }_{1k}\right)=1$.

In this way, from (B.2), we obtain

$$\begin{array}{rl}{p}_0& =K_1^{-\frac{1}{2}}\sqrt{2\pi }\exp \left[{\frac{L_1}{2K_1}}^2-\frac{1}{2}\left(\frac{{\lambda }_{01k}^2}{{\psi }_1{\sigma }_{\lambda 01k}^2}+\sum _{i=1}^N\frac{{y_{i1}^{\ast }}^2}{{\psi }_1}\right)\right]\dots \\ & \times K_{j-1}^{-\frac{1}{2}}\sqrt{2\pi }\exp \left[{\frac{L_{j-1}}{2K_{j-1}}}^2-\frac{1}{2}\left(\frac{{\lambda }_{0,j-1,k}^2}{{\psi }_{j-1}{\sigma }_{\lambda 0,j-1,k}^2}+\sum _{i=1}^N\frac{{y_{i,j-1}^{\ast }}^2}{{\psi }_{j-1}}\right)\right]\\ & \times \exp \left[\frac{1}{2}\left(\sum _{i=1}^N\frac{{y_{\mathrm{ij}}^{\ast }}^2}{{\psi }_j}+\dots \sum _{i=1}^N\frac{{y_{\mathrm{ip}}^{\ast }}^2}{{\psi }_p}\right)\right]{\pi }_{1k}\dots {\pi }_{j-1,k}\left(1-{\pi }_{\mathrm{jk}}\right),\end{array}$$

where $K_j=\sum _{i=1}^N\frac{f_{\mathrm{ki}}^2}{{\psi }_j}+\frac{1}{{\psi }_j{\sigma }_{\lambda ,0,j,k}^2},L_j=\frac{{\lambda }_{0,j,k}}{{\psi }_j{\sigma }_{\lambda ,0,j,k}^2}+\sum _{i=1}^N\frac{y_{\mathrm{ij}}^{\ast }{f}_{\mathrm{ki}}}{{\psi }_j}.$

For Equation (6) ( $r_{\mathrm{jk}}=1$ case), a similar derivation above gives rise to

$$\begin{array}{rl}{p}_1& =K_1^{-\frac{1}{2}}\sqrt{2\pi }\exp \left[{\frac{L_1}{2K_1}}^2-\frac{1}{2}\left(\frac{{\lambda }_{01k}^2}{{\psi }_1{\sigma }_{\lambda 01k}^2}+\sum _{i=1}^N\frac{{y_{i1}^{\ast }}^2}{{\psi }_1}\right)\right]\dots \\ & \times K_j^{-\frac{1}{2}}\sqrt{2\pi }\exp \left[{\frac{L_j}{2K_j}}^2-\frac{1}{2}\left(\frac{{\lambda }_{0\mathrm{jk}}^2}{{\psi }_j{\sigma }_{\lambda 0,j,k}^2}+\sum _{i=1}^N\frac{{y_{i,j}^{\ast }}^2}{{\psi }_j}\right)\right]\\ & \times \exp \left[\frac{1}{2}\left(\sum _{i=1}^N\frac{{y_{\mathrm{ij}+1}^{\ast }}^2}{{\psi }_{j+1}}+\dots +\sum _{i=1}^N\frac{{y_{\mathrm{ip}}^{\ast }}^2}{{\psi }_p}\right)\right]{\pi }_{1k}{\pi }_{2k}\dots {\pi }_{\mathrm{jk}}.\end{array}$$

Thus, we have the posterior distribution of $r_{\mathrm{jk}}$ as the Bernoulli random variable with the conditional probability

$$\begin{array}{rl}& \frac{p_1}{p_1+p_0}\\ & =\frac{K_j^{-\frac{1}{2}}\sqrt{2\pi }\exp \left[{\frac{L_j}{2K_j}}^2-\frac{1}{2}\left(\frac{{\lambda }_{0\mathrm{jk}}^2}{{\psi }_j{\sigma }_{\lambda 0,j,2}^2}+\sum _{i=1}^N\frac{{y_{i,j}^{\ast }}^2}{{\psi }_j}\right)\right]{\pi }_{\mathrm{jk}}}{K_j^{-\frac{1}{2}}\sqrt{2\pi }\exp \left[{\frac{L_j}{2K_j}}^2-\frac{1}{2}\left(\frac{{\lambda }_{0\mathrm{jk}}^2}{{\psi }_j{\sigma }_{\lambda 0,j,k}^2}+\sum _{i=1}^N\frac{{y_{i,j}^{\ast }}^2}{{\psi }_j}\right)\right]p_{\mathrm{jl}}+\exp \left[-\frac{1}{2}\left(\sum _{i=1}^N\frac{{y_{\mathrm{ij}}^{\ast }}^2}{{\psi }_j}\right)\right]\left(1-{\pi }_{\mathrm{jk}}\right)}\end{array}$$

For the derivation of (9), we have the following calculations. Let $z_{\mathrm{ij}}=y_{\mathrm{ij}}-{\mathrm{\Lambda }}_{j,-k}{f}_i$, for $r_{\mathrm{jk}}=0$ case,

$$p_0=\exp \left\{-\frac{1}{2{\psi }_j}\sum _{i=1}^N{\left(z_{\mathrm{ij}}\right)}^2\right\}\prod _{k=1}^q({\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2{)}^{-\frac{1}{2}}\exp \left\{-\frac{{{\lambda }_{0\mathrm{jk}}}^2}{2{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right\}\left(1-{\pi }_{\mathrm{jk}}\right).$$

For $r_{\mathrm{jk}}=1$ case,

$$\begin{array}{rl}& p\left({\mathrm{\Lambda }}_{\mathrm{jk}},{\text{r}}_j,{\psi }_j|Y_j,F,r_{\mathrm{jk}}=1\right)\\ & \propto \exp \left[-\frac{1}{2{\psi }_j}\sum _{i=1}^N{\left(z_{\mathrm{ij}}-{\lambda }_{\mathrm{jk}}{f}_{\mathrm{ik}}\right)}^2\right]\prod _{k=1}^q({\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2{)}^{-\frac{1}{2}}\exp \left[-\frac{{\left({\lambda }_{\mathrm{jk}}-{\lambda }_{0\mathrm{jk}}\right)}^2}{2{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right]{\pi }_{\mathrm{jk}}\\ & \propto \exp \left[-\frac{1}{2{\psi }_j}\sum _{i=1}^N\left(z_{\mathrm{ij}}^2-2{\lambda }_{\mathrm{jk}}{f}_{\mathrm{ki}}{z}_{\mathrm{ij}}+{\lambda }_{\mathrm{jk}}^2{f}_{\mathrm{ik}}^2\right)\right]\prod _{k=1}^q({\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2{)}^{-\frac{1}{2}}\exp \left[-\frac{{\left({\lambda }_{\mathrm{jk}}-{\lambda }_{0\mathrm{jk}}\right)}^2}{2{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right]{\pi }_{\mathrm{jk}}\\ & \propto \exp \left(-\frac{1}{2{\psi }_j}\sum _{i=1}^N{z}_{\mathrm{ij}}^2\right)\exp \left[-\frac{1}{2{\psi }_j}\left(\sum _{i=1}^N-2{\lambda }_{\mathrm{jk}}{f}_{\mathrm{ik}}{z}_{\mathrm{ij}}+\sum _{i=1}^N{\lambda }_{\mathrm{jk}}^2{f}_{\mathrm{ik}}^2\right)\right]\\ & \times \prod _{k=1}^q({\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2{)}^{-\frac{1}{2}}\exp \left\{-\frac{{\lambda }_{\mathrm{jk}}^2-2{\lambda }_{\mathrm{jk}}{\lambda }_{0\mathrm{jk}}+{\lambda }_{0\mathrm{jk}}^2}{2{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right\}{\pi }_{\mathrm{jk}}\\ & \propto \exp \left\{-\frac{1}{2{\psi }_j}\sum _{i=1}^N{z}_{\mathrm{ij}}^2\right\}\exp [\left(-\frac{1}{2{\psi }_j}\sum _{i=1}^N{f}_{\mathrm{ik}}^2-\frac{1}{2{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right){\lambda }_{\mathrm{jk}}^2\\ & -\left(-\frac{1}{2{\psi }_j}\sum _{i=1}^{N}2f_{\mathrm{ik}}{z}_{\mathrm{ij}}-\frac{2{\lambda }_{0\mathrm{jk}}}{2{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right){\lambda }_{\mathrm{jk}}-\frac{{\lambda }_{0\mathrm{jk}}^2}{2{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}]{\pi }_{\mathrm{jk}}\prod _{k=1}^q({\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2{)}^{-\frac{1}{2}}\\ & \propto \exp \left\{-\frac{1}{2}\left[\left(\frac{1}{{\psi }_j}\sum _{i=1}^N{f}_{\mathrm{ik}}^2+\frac{1}{{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right){\lambda }_{\mathrm{jk}}^2-2\left(\frac{1}{{\psi }_j}\sum _{i=1}^N{f}_{\mathrm{ik}}{z}_{\mathrm{ij}}+\frac{{\lambda }_{0\mathrm{jk}}}{{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right){\lambda }_{\mathrm{jk}}+\frac{{\lambda }_{0\mathrm{jk}}^2}{{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right]\right\}\\ & \times \prod _{k=1}^q({\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2{)}^{-\frac{1}{2}}\exp \left\{-\frac{1}{2{\psi }_j}\sum _{i=1}^N{z}_{\mathrm{ij}}^2\right\}{\pi }_{\mathrm{jk}}.\end{array}$$ (B.4)

Define $K=\frac{1}{{\psi }_j}\sum _{i=1}^N{f}_{\mathrm{ik}}^2+\frac{1}{{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2},L=\frac{1}{{\psi }_j}\sum _{i=1}^N{f}_{\mathrm{ik}}{z}_{\mathrm{ij}}+\frac{{\lambda }_{0\mathrm{jk}}}{{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}$. Then, (B.4) is

$$\begin{array}{rl}& p\left({\mathrm{\Lambda }}_{\mathrm{jk}},{\text{r}}_j,{\psi }_j|Y_j,F,r_{\mathrm{jk}}=1\right)\\ & \propto \exp \left(-\frac{1}{2{\psi }_j}\sum _{i=1}^N{z}_{\mathrm{ij}}^2\right)\exp \left[-\frac{1}{2}\left(K{\lambda }_{\mathrm{jk}}^2-2L{\lambda }_{\mathrm{jk}}+\frac{{\lambda }_{0\mathrm{jk}}^2}{{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right)\right]{\pi }_{\mathrm{jk}}\\ & \propto \exp \left(-\frac{1}{2{\psi }_j}\sum _{i=1}^N{z}_{\mathrm{ij}}^2\right)\exp \left[-\frac{1}{2}K\left({\lambda }_{\mathrm{jk}}^2-2\frac{L}{K}{\lambda }_{\mathrm{jk}}+\frac{L^2}{K^2}-\frac{L^2}{K^2}+\frac{{\lambda }_{0\mathrm{jk}}^2}{K{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right)\right]{\pi }_{\mathrm{jk}}.\end{array}$$ (B.5)

Now, we integrate out with respect to ${\lambda }_{\mathrm{jk}}$, (B.5) becomes

$$\begin{array}{rl}{p}_1& =\exp \left(-\frac{1}{2{\psi }_j}\sum _{i=1}^N{z}_{\mathrm{ij}}^2\right){\pi }_{\mathrm{jk}}\exp \left[\frac{1}{2}K\left(\frac{L^2}{K^2}-\frac{{\lambda }_{0\mathrm{jk}}^2}{K{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right)\right]K^{-\frac{1}{2}}\sqrt{2\pi }\\ & \propto \exp \left(-\frac{1}{2{\psi }_j}\sum _{i=1}^N{z}_{\mathrm{ij}}^2\right){\pi }_{\mathrm{jk}}\exp \left[\frac{1}{2}\left(\frac{L^2}{K}-\frac{{\lambda }_{0\mathrm{jk}}^2}{{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right)\right]K^{-\frac{1}{2}}\sqrt{2\pi }.\end{array}$$

We have the posterior distribution of $r_{\mathrm{jk}}$ as the Bernoulli random variable with the conditional probability

$$\frac{p_1}{p_1+p_0}=\frac{\exp \left[\frac{1}{2}\left(\frac{L^2}{K}-\frac{{\lambda }_{0\mathrm{jk}}^2}{{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right)\right]K^{-\frac{1}{2}}\sqrt{2\pi }{\pi }_{\mathrm{jk}}}{\exp \left[\frac{1}{2}\left(\frac{L^2}{K}-\frac{{\lambda }_{0\mathrm{jk}}^2}{{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right)\right]K^{-\frac{1}{2}}\sqrt{2\pi }{\pi }_{\mathrm{jk}}+\exp \left\{-\frac{{{\lambda }_{0\mathrm{jk}}}^2}{2{\psi }_j{\sigma }_{\lambda 0\mathrm{jk}}^2}\right\}\left(1-{\pi }_{\mathrm{jk}}\right)}.$$

Disclosure statement

No potential conflict of interest was reported by the author(s).