The Improved EMS Algorithm for Latent Variable Selection in M3PL Model

Abstract

One of the main concerns in multidimensional item response theory (MIRT) is to detect the relationship between items and latent traits, which can be treated as a latent variable selection problem. An attractive method for latent variable selection in multidimensional 2-parameter logistic (M2PL) model is to minimize the observed Bayesian information criterion (BIC) by the expectation model selection (EMS) algorithm. The EMS algorithm extends the EM algorithm and allows the updates of the model (e.g., the loading structure in MIRT) in the iterations along with the parameters under the model. As an extension of the M2PL model, the multidimensional 3-parameter logistic (M3PL) model introduces an additional guessing parameter which makes the latent variable selection more challenging. In this paper, a well-designed EMS algorithm, named improved EMS (IEMS), is proposed to accurately and efficiently detect the underlying true loading structure in the M3PL model, which also works for the M2PL model. In simulation studies, we compare the IEMS algorithm with several state-of-art methods and the IEMS is of competitiveness in terms of model recovery, estimation precision, and computational efficiency. The IEMS algorithm is illustrated by its application to two real data sets.

Introduction

In the fields of psychology and pedagogy, tests involving items with dichotomous or polytomous responses are usually designed to evaluate individuals’ multiple latent traits or abilities. Multidimensional item response theory (MIRT; Reckase, 2009) is widely used in this scenario as it models the relationship between observed item responses and multiple latent traits by specifying a probability function. Two commonly used MIRT models are multidimensional 2-parameter logistic (M2PL) model and multidimensional 3-parameter logistic (M3PL) model. The M2PL model incorporates the item discrimination and difficulty parameters, whereas the M3PL model introduces an additional guessing parameter alongside these. Compared to the M2PL model, the M3PL model is more suitable to the multiple-choice data, especially in a low-stakes test where the subjects may use the guessing strategy to improve the performance. Traditional researches for MIRT are confirmatory in which the item-trait relationships are prespecified by prior knowledge (Janssen & De Boeck, 1999; Mckinley, 1989). It aims at testing substantive theory or scaling individuals along multiple latent traits, by parameter estimation, model comparison, and assessment of model goodness-of-fit (Chen & Zhang, 2021).

Another type of researches in MIRT is to explore the item-trait relationship (i.e., the loading structure) underlying the observed item responses. The commonly used method is exploratory item factor analysis (EIFA) (Bock et al., 1988; Cai, 2010; Chen et al., 2019). As opposed to the confirmatory method, the EIFA freely estimate the entire item-trait relationships (also called item factor loadings) only with some constraints on the location and scale of the latent traits. The estimation methods include, but are not limited to, marginal maximum likelihood (Bock et al., 1988), joint maximum likelihood (Chen et al., 2019), and Bayesian estimation (Béguin & Glas, 2001). The EIFA leads to a rotation invariant factor loading matrix, which may be dense and hence difficult to interpret. To obtain a simpler structure of the loading matrix for interpretability, some factor rotation techniques can be applied to produce a nearly sparse loading, and then followed by an arbitrary cut-off (Cho et al., 2024). There are a variety of rotation techniques available including varimax (Kaiser, 1958), promax (Hendrickson & White, 1964), and CF-quartimax rotation (Browne, 2001). However, different rotation techniques may yield very different or even conflicting loading matrices and thus selecting an appropriate rotation technique can be difficult (Sass & Schmitt, 2010). In addition, the settings of the cut-off are subjective and the loadings may vary with different choices of the thresholds (Jin et al., 2018).

In recent researches, the problem of estimating the loading structure in MIRT has been considered as a latent variable selection. Therefore, the sparsity of the loading structure can be obtained by optimizing a penalized likelihood (Cho et al., 2024; Shang et al., 2023, 2024; Sun et al., 2016; Xu et al., 2022; Zhang & Chen, 2022). These regularization methods derive the loading structure from the observed item responses with no subjective settings and thus avoid the limitations caused by the rotation techniques and cut-offs in the EIFA. Sun et al. (2016) proposed to obtain a sparse estimator for the loading structure by maximizing the $L_1$ -penalized likelihood (Tibshirani, 1996) with the expectation maximization (EM) algorithm (Dempster et al., 1977). The main challenge is to calculate the integral in the conditional expectation of the log-likelihood function that does not have a closed form due to the latent traits. Sun et al. (2016) approximated the integral numerically using a set of grid points and optimized the weighted log-likelihood of the augmented data by the coordinate descent algorithm (Friedman et al., 2010). Sun et al. (2016)’s method suffers from a very high computational cost and requires a few hours for the case involving five-dimensional latent traits (Zhang & Chen, 2022). To reduce the computational cost, Zhang and Chen (2022) proposed a quasi-Newton stochastic proximal algorithm for optimizing the $L_1$ -penalized likelihood; Cho et al. (2024) imposed an adaptive $L_1$ penalty to the variational lower bound of the likelihood and proposed a regularized Gaussian variational EM algorithm; and Shang et al. (2023) employed the coordinate descent algorithm to optimize a novel $L_1$ -penalized weighted log-likelihood derived from the “artificial data.”

In fact, the latent variable selection for MIRT models can be subsumed within the statistical model selection in the presence of missing data. A commonly used method to identify the optimal model is minimizing the information criteria including the Bayesian information criterion (BIC; Schwarz, 1978) and the Akaike information criterion (AIC; Akaike, 1974). Xu et al. (2022) optimized the observed BIC with the expectation model selection (EMS; Jiang et al., 2015) algorithm for latent variable selection problem in MIRT. The EMS algorithm is a new development for model selection with incomplete data that extends the EM algorithm by updating the model and the parameters under the model simultaneously in each iteration where an expectation (E-) step and a model selection (MS-) step are included. In the framework of EMS, Xu et al. (2022) computed the conditional expectation of the BIC of complete data for each candidate model in the E-step, and minimized the expected BIC over all candidate models to obtain the optimal model (i.e., the loading structure) with its parameter estimates (i.e., item parameters and the covariance matrix of latent traits). Under mild assumptions, the convergence of the EMS algorithm for latent variable selection in the M2PL models were proved. Although Xu et al. (2022) showed that the EMS algorithm performs better than the EM-based method in Sun et al. (2016), its implementation is still computationally expensive. As shown in Xu et al. (2022), EMS takes about 25 hours in the latent variable selection of M2PL model with five-dimensional latent traits under the sample size of 1000. To improve the efficiency of the EMS, Shang et al. (2024) developed a generalized EMS (GEMS) algorithm which seeks only a decrease in the expected BIC in the MS-step instead of a minimization of the expected BIC. The GEMS includes the EMS as a special case and the numerical convergence results of GEMS imply the convergence of EMS in Xu et al. (2022).

However, to the best of our knowledge, only two works mentioned above concern the latent variable selection in both the M2PL and M3PL models, that is, (Cho et al., 2024; Sun et al., 2016). Compared to the M2PL model, the latent variable selection for the M3PL model is more challenging due to the introduction of the guessing parameter. Sun et al. (2016) assumed that the guessing parameter in the M3PL model for all items are known, which might not be feasible in the practical scenario. Instead, Cho et al., (2024) treated the guessing parameter as unknown and estimated it by adopting an equivalent hierarchical model of the M3PL model with an auxiliary latent variable which indicates whether the subject answered the item based on the latent abilities or guessed it correctly. Both (Cho et al., 2024; Sun et al., 2016) adopted the $L_1$ -type penalized likelihood methods and required the selection of the optimal tuning parameter from a series of prespecified parameter values. The improper settings of the candidate tuning parameters may lead to a loading structure that inadequately represent the data.

In this paper, we extend the EMS algorithm to the latent variable selection for the M3PL model. Compared to the $L_1$ -type penalized likelihood methods, the observed BIC minimization method under the EMS framework (Xu et al., 2022) avoids the tuning parameter selection and selects the optimal loading structure from all possible structures. The current study differs from Xu et al. (2022)’s in the following aspects: (1) we give an implementation of the EMS algorithm for the M3PL model, which covers the latent variable selection for the M2PL model; (2) In the E-step, we approximate the posterior expectation using the Gauss-Hermite (GH) quadrature (Davis & Rabinowitz, 2014), which is widely used in the marginal maximum likelihood estimation for the item parameters (Bock & Aitkin, 1981; Bock et al., 1988); in the MS-step, the Newton’s method with quadratic convergence rate instead of the coordinate descent algorithm is applied for solving the maximum likelihood estimates. These changes made to the implementation lead to a computationally efficient EMS algorithm, see Table 1 in simulation studies. To differentiate it from the EMS described in Xu et al. (2022), this well-designed EMS is named improved EMS (IEMS).

Table 1.

The Average CPU Time in Seconds of IEMS, EML1, ConEIFA and Rotation (i.e., Quartimin and Geomin) for M3PL Models With $K=\mathrm{3,4,5}$ Under the Sample Size $N=\mathrm{500,1000,2000,4000}$ .

	$K=3$				$K=4$				$K=5$
	$N=500$	$N=1000$	$N=2000$	$N=4000$	$N=500$	$N=1000$	$N=2000$	$N=4000$	$N=500$	$N=1000$	$N=2000$	$N=4000$
IEMS	0.23	0.30	0.43	0.68	4.04	4.69	6.22	7.86	15.90	17.08	21.66	26.32
EML1	2.85	4.07	6.53	10.68	24.58	37.65	47.26	97.10	47.68	71.66	133.92	239.34
ConEIFA	0.13	0.19	0.33	0.58	1.17	1.74	2.42	5.14	2.51	3.80	8.49	13.00
Rotation	0.21	0.34	0.54	0.97	1.78	2.73	4.98	7.55	5.18	7.80	13.82	24.26

The rest of the article is organized as follows. In the The M3PL Models section, we introduce the M3PL models and the constraints imposed on the model parameters for identification. In the EMS for Latent Variable Selection in M3PL Models section, we give the detailed implementation of the IEMS algorithm for M3PL model using the GH quadrature and the Newton’s method. In the Simulation Studies section, simulations are conducted to measure the performance of the designed algorithm in terms of the model recovery, estimation precision, and computational efficiency. In the Real Data Analysis section, we illustrate the proposed method by its application to two real data sets. A detailed discussion is provided in the Discussion section.

The M3PL Models

Consider a $J$ -item test that measures $K$ latent traits of $N$ subjects with dichotomous observed responses $Y={(y_1,\dots ,y_N)}^T$ , where $y_i={(y_{i1},\dots ,y_{iJ})}^T$ denotes all responses of subject $i$ , $i=1,\dots ,N$ . $y_{ij}=1$ represents the correct response of subject $i$ to item $j$ , and $y_{ij}=0$ wrong response, for $i=1,\dots ,N$ and $j=1,\dots ,J$ . Let ${\theta }_i={({\theta }_{i1},\dots ,{\theta }_{iK})}^T$ be the $K$ -dimensional latent traits to be measured for subject $i$ . The M3PL model describes the relationship between the $j$ th item response and the $K$ -dimensional latent traits for subject $i$ as follows,

$$P_j({\theta }_i)\equiv P(y_{ij}=1|{\theta }_i,a_j,b_j,c_j)=c_j+(1-c_j)F_j({\theta }_i),$$ (1)

where

$$F_j({\theta }_i)=\frac{\exp (a_j^T{\theta }_i+b_j)}{1+\exp (a_j^T{\theta }_i+b_j)},$$ (2)

$a_j={(a_{j1},\dots ,a_{jK})}^T$ , $b_j$ , $c_j$ are known as the discrimination, difficulty and guessing parameters for item $j$ , respectively. $P(y_{ij}=1|{\theta }_i,a_j,b_j,c_j)$ denotes the probability that subject $i$ correctly responds to item $j$ given the subject’s latent traits ${\theta }_i$ and item parameters $a_j$ , $b_j$ and $c_j$ . The non-zero elements in $a_j$ imply the association of item $j$ with corresponding latent traits. The guessing parameter $c_j$ is also known as a lower asymptote, which equals the probability of a correct response to item $j$ by the subject with low ability (i.e., $\theta \to -\infty$ ). Without considering $c_j$ (i.e., $c_j=0$ , $j=1,\dots ,J$ ), the response function for M3PL model in Equation (1) degenerates to that for M2PL model (i.e., the Equation (2)). We define the model parameters and latent traits by $A={(a_1,\dots ,a_J)}^T$ , $b={(b_1,\dots ,b_J)}^T$ , $c={(c_1,\dots ,c_J)}^T$ , and $\mathrm{\Theta }={({\theta }_1,\dots ,{\theta }_N)}^T$ .

Similar to the M2PL models, several general assumptions are adopted in the M3PL models. The latent traits ${\theta }_i$ , $i=1,\dots ,N$ , are assumed to be independent and identically distributed, and follow a $K$ -dimensional normal distribution $N(0,\mathrm{\Sigma })$ with zero mean vector and covariance matrix $\mathrm{\Sigma }={({\sigma }_{k{k}^{\prime }})}_{K\times K}$ . Furthermore, the local independence assumption is assumed, that is, given the latent traits ${\theta }_i$ , $y_{i1},\dots ,y_{iJ}$ are conditionally independent.

Due to the addition of the guessing parameter $c_j$ , the parameter estimation for M3PL model is more complex than M2PL model. Instead of directly working on (1), we follow the mixture modeling approach proposed for the 3-parameter IRT model in Guo and Zheng (2019) and Béguin and Glas (2001). Under the mixture model framework, the M3PL model in (1) can be re-written as

$$P_j({\theta }_i)=F_j({\theta }_i)\times 1+(1-F_j({\theta }_i))\times c_j,$$ (3)

where $F_j({\theta }_i)$ and $(1-F_j({\theta }_i))$ are treated as the mixture weights of two subpopulations on which 1 and $c_j$ are the component densities. A latent indicate matrix $V={(v_{ij})}_{N\times J}$ is introduced, where

$$v_{ij}=\left\{\begin{array}{cc}1\hfill & \mathrm{i}\mathrm{f}\mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}i\mathrm{h}\mathrm{a}\mathrm{s}\mathrm{a}\mathrm{b}\mathrm{i}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}\mathrm{t}\mathrm{o}\mathrm{a}\mathrm{n}\mathrm{s}\mathrm{w}\mathrm{e}\mathrm{r}\mathrm{i}\mathrm{t}\mathrm{e}\mathrm{m}j\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{l}\mathrm{y},\hfill \\ 0\hfill & \mathrm{i}\mathrm{f}\mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}i\mathrm{d}\mathrm{o}\mathrm{e}\mathrm{s}\mathrm{n}\mathrm{o}\mathrm{t}\mathrm{h}\mathrm{a}\mathrm{v}\mathrm{e}\mathrm{a}\mathrm{b}\mathrm{i}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}\mathrm{t}\mathrm{o}\mathrm{a}\mathrm{n}\mathrm{s}\mathrm{w}\mathrm{e}\mathrm{r}\mathrm{i}\mathrm{t}\mathrm{e}\mathrm{m}j\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{l}\mathrm{y},\hfill \end{array}\right.$$ (4)

and $v_{ij}$ follows a Bernoulli distribution with parameter $F_j({\theta }_i)$ . In the case of $v_{ij}=1$ , all the conditional probability mass of $y_{ij}$ is on a single point. To be specific,

$$P(y_{ij}=1|v_{ij}=1,{\theta }_i,a_j,b_j,c_j)=1,P(y_{ij}=0|v_{ij}=1,{\theta }_i,a_j,b_j,c_j)=0.$$ (5)

For $v_{ij}=0$ , the conditional probability of $y_{ij}$ is specified as follows,

$$P(y_{ij}=1|v_{ij}=0,{\theta }_i,a_j,b_j,c_j)=c_j,P(y_{ij}=0|v_{ij}=0,{\theta }_i,a_j,b_j,c_j)=1-c_j.$$ (6)

Hence, given the latent traits ${\theta }_i$ and the item parameters $a_j,b_j,c_j$ , the joint distribution of $(y_{ij},v_{ij})$ can be expressed as

$$\begin{array}{ll}\hfill P(y_{ij}=1,v_{ij}=1|{\theta }_i,a_j,b_j,c_j)& =F_j({\theta }_i),\hfill \\ \hfill P(y_{ij}=0,v_{ij}=1|{\theta }_i,a_j,b_j,c_j)& =0,\hfill \\ \hfill P(y_{ij}=1,v_{ij}=0|{\theta }_i,a_j,b_j,c_j)& =c_j(1-F_j({\theta }_i)),\hfill \\ \hfill P(y_{ij}=0,v_{ij}=0|{\theta }_i,a_j,b_j,c_j)& =(1-c_j)(1-F_j({\theta }_i)).\hfill \end{array}$$ (7)

Omitting the redundant term $P(y_{ij}=0,v_{ij}=1|{\theta }_i,a_j,b_j,c_j)$ in Equation (7), the joint distribution of $(y_{ij},v_{ij})$ can be equivalently written as

$$\begin{array}{ll}\hfill p& (y_{ij},v_{ij}|{\theta }_i,a_j,b_j,c_j)\hfill \\ \hfill & ={[F_j({\theta }_i)]}^{y_{ij}{v}_{ij}}\times {[c_j(1-F_j({\theta }_i))]}^{y_{ij}(1-v_{ij})}\times {[(1-c_j)(1-F_j({\theta }_i))]}^{(1-y_{ij})(1-v_{ij})}\hfill \\ \hfill & =F_j{({\theta }_i)}^{v_{ij}}{[1-F_j({\theta }_i)]}^{1-v_{ij}}{[c_j^{y_{ij}}{(1-c_j)}^{1-y_{ij}}]}^{1-v_{ij}}.\hfill \end{array}$$ (8)

Let $v_i={(v_{i1},\dots ,v_{iJ})}^T$ . Under the local independence assumption, the joint distribution of the complete data $(y_i,v_i,{\theta }_i)$ for subject $i$ is

$$p(y_i,v_i,{\theta }_i|A,b,c,\mathrm{\Sigma })=\phi ({\theta }_i|\mathrm{\Sigma })\prod _{j=1}^{J}p(y_{ij},v_{ij}|{\theta }_i,a_j,b_j,c_j),$$ (9)

where $\phi ({\theta }_i|\mathrm{\Sigma })$ is the density function of latent traits ${\theta }_i$ . Then the likelihood function of the new augmented complete data $(Y,V,\mathrm{\Theta })$ for M3PL model is given by

$$L(A,b,c,\mathrm{\Sigma }|Y,V,\mathrm{\Theta })=\prod _{i=1}^N\phi ({\theta }_i|\mathbf{\Sigma })\prod _{j=1}^J{F}_j{({\theta }_i)}^{v_{ij}}{[1-F_j({\theta }_i)]}^{1-v_{ij}}{[c_j^{y_{ij}}{(1-c_j)}^{1-y_{ij}}]}^{1-v_{ij}}.$$ (10)

Thus, we have the following log-likelihood of the observed data $Y$ .

$$l(A,b,c,\mathrm{\Sigma }|Y)=\log \left[{\int }_{(\mathrm{\Theta },V)}L(A,b,c,\mathrm{\Sigma }|Y,V,\mathrm{\Theta })\mathrm{d}\mathrm{\Theta }\mathrm{d}V\right].$$ (11)

To guarantee the parameter identification and resolve the rotational indeterminacy for M3PL models, some constraints should be imposed. To identify the scale of the latent traits, we assume the variances of all latent traits are unity, that is, ${\sigma }_{kk}=1$ for $k=1,\dots ,K$ . Dealing with the rotational indeterminacy issue requires additional constraints on the loading matrix $A$ , that is, each of the first $K$ items is associated with only one latent trait separately, that is, $a_{jj}\ne 0$ and $a_{jk}=0$ for $1\le j\ne k\le K$ . In practice, the constraint on $A$ should be determined according to priori knowledge of the item and the entire study. These identification constraints are the same as those used by (Cho et al., 2024; Shang et al., 2024; Sun et al., 2016; Xu et al., 2022) for M2PL and M3PL models.

EMS for Latent Variable Selection in M3PL Models

As is known, the response function for M2PL takes a logistic regression form, where $y_{ij}$ acts as the response, the latent traits ${\theta }_i$ as the covariates, $a_j$ and $b_j$ as the regression coefficients and intercept respectively. Of interest is to explore the subset of the latent traits related to each item, that is, to find all non-zero $a_{jk}$ s. This can be viewed as a variable selection problem or model selection problem in a statistical sense. For latent variable selection in the M2PL model, Xu et al. (2022) applied the EMS algorithm to find the optimal model and its parameter estimates which result in the smallest observed BIC value. Similarly, we extend the EMS algorithm to the case of the M3PL model.

The EMS algorithm (Jiang et al., 2015), as an extension of the EM algorithm, iterates the E-step and MS-step to deal with the model selection problem in the case of incomplete data. Different to EM, the EMS algorithm updates both the model and the parameters under the model in each iteration and thus has the updating power to detect the underlying true model. For latent variable selection in the M3PL model, let $M(A)={(I(a_{jk}\ne 0))}_{J\times K}$ be the model corresponding to the discrimination parameter $A$ and $\mathcal{M}$ be the model space containing all candidate models. Under the EMS framework, the aim is to find the optimal model (i.e., $M\in \mathcal{M}$ ) and its parameter estimates (i.e., $A,b,c,\mathrm{\Sigma }$ under $M$ ) which correspond to the smallest observed BIC value,

$$\mathrm{B}\mathrm{I}\mathrm{C}(M,A,b,c,\mathrm{\Sigma })=-2l(A,b,c,\mathrm{\Sigma }|Y)+\Vert A{\Vert }_0\log N,$$ (12)

where $\Vert A{\Vert }_0={\sum }_{j=1}^J{\sum }_{k=1}^{K}I(a_{jk}\ne 0)$ denotes the number of non-zero elements in $A$ . For the sake of simplicity, denote the model and parameters by $\mathrm{\Psi }=(M,A,b,c,\mathrm{\Sigma })$ . Given the current model and parameters ${\mathrm{\Psi }}^{(t)}=(M^{(t)},A^{(t)},b^{(t)},c^{(t)},{\mathrm{\Sigma }}^{(t)})$ obtained in the $t$ th EMS iteration, the $Q$ -function is defined as the conditional expectation of the BIC for complete data with respect to the posterior distribution $p(\mathrm{\Theta },V|Y,{\mathrm{\Psi }}^{(t)})$ ,

$$Q(\mathrm{\Psi }|{\mathrm{\Psi }}^{(t)})=E\{-2\log L(\mathrm{\Psi }|Y,V,\mathrm{\Theta })+\Vert A{\Vert }_0\log N|Y,{\mathrm{\Psi }}^{(t)}\}.$$ (13)

Using the EMS algorithm, we optimize the Equation (12) by iteratively proceeds

(1) E-step: compute $Q(\mathrm{\Psi }|{\mathrm{\Psi }}^{(t)})$ for each candidate model;

(2) MS-step: find a minimum point of $Q(\mathrm{\Psi }|{\mathrm{\Psi }}^{(t)})$ with respect to $\mathrm{\Psi }$ (denoted by ${\mathrm{\Psi }}^{(t+1)}$ ) as the new current model and parameters.

Similar to the M2PL model, the main difficulty of the EMS for M3PL lies in the computationally intractable integral (conditional expectation in the E-step) due to the existence of latent variables. For the M2PL model, Xu et al. (2022) numerically approximated the integral by a summation using a set of equally spaced grid points on the interval ${[-\mathrm{4,4}]}^K$ . Instead, we adopt the GH quadrature (Bock & Aitkin, 1981; Davis & Rabinowitz, 2014) in order to improve the E-step, which is an alternative numerical approximation method to the integral. Furthermore, to speed up the MS-step, we apply the Newton’s method to obtain the maximum likelihood estimates of $M,A,b$ instead of the coordinate descent algorithm used in Xu et al. (2022). Hence, our proposed EMS is named improved EMS (IEMS).

The Implementation of IEMS

The initial guess for the model and parameters ${\mathrm{\Psi }}^{(0)}$ has to satisfy the identification constraints to ensure the identifiability. Given the current model and parameters ${\mathrm{\Psi }}^{(t)}$ , the $(t+1)$ th iteration proceeds E-step and MS-step as follows.

E-step

In the E-step, we compute the $Q$ -function for each candidate model as follows,

$$\begin{array}{ll}\hfill Q(\mathrm{\Psi }|{\mathrm{\Psi }}^{(t)})& =E\{-2\log L(\mathrm{\Psi }|Y,V,\mathrm{\Theta })|Y,{\mathrm{\Psi }}^{(t)}\}+\Vert A{\Vert }_0\log N\hfill \\ \hfill & =-2\sum _{i=1}^{N}E\{\log p(y_i,v_i,{\theta }_i|\mathrm{\Psi })|y_i,{\mathrm{\Psi }}^{(t)}\}+\Vert A{\Vert }_0\log N,\hfill \end{array}$$ (14)

where the expectation in the summation is computed with respect to the posterior density of $({\theta }_i,v_i)$ , that is,

$$p({\theta }_i,v_i|y_i,{\mathrm{\Psi }}^{(t)})=\frac{p(y_i,v_i,{\theta }_i|{\mathrm{\Psi }}^{(t)})}{p(y_i|{\mathrm{\Psi }}^{(t)})}$$ (15)

in which the denominator refers to the marginal density of $y_i$ given ${\mathrm{\Psi }}^{(t)}$ ,

$$p(y_i|{\mathrm{\Psi }}^{(t)})={\int }_{{\theta }_i}p(y_i|{\theta }_i,M^{(t)},A^{(t)},b^{(t)},c^{(t)})\phi ({\theta }_i|{\mathrm{\Sigma }}^{(t)})\mathrm{d}{\theta }_i,$$ (16)

where $p(y_i|{\theta }_i,M^{(t)},A^{(t)},b^{(t)},c^{(t)})={\prod }_{j=1}^{J}p(y_{ij}|{\theta }_i,M^{(t)},A^{(t)},b^{(t)},c^{(t)})$ due to the local independence assumption. Then, for each $i=1,\dots ,N$ , the expectation in (14) can be expressed as

$$\frac{1}{p(y_i|{\mathrm{\Psi }}^{(t)})}{\int }_{{\theta }_i}\sum _{v_i\in {\{\mathrm{0,1}\}}^J}\log p(y_i,v_i,{\theta }_i|\mathrm{\Psi })\cdot p(y_i,v_i|{\theta }_i,M^{(t)},A^{(t)},b^{(t)},c^{(t)})\phi ({\theta }_i|{\mathrm{\Sigma }}^{(t)})\mathrm{d}{\theta }_i.$$ (17)

By applying the GH quadrature, the Equation (17) can be approximated by

$$\sum _{g=1}^G\sum _{v_i\in {\{\mathrm{0,1}\}}^J}\log p(y_i,v_i,x_g|\mathrm{\Psi })\cdot \tilde{p}(x_g,v_i|y_i,{\mathrm{\Psi }}^{(t)}),$$ (18)

with

$$\tilde{p}(x_g,v_i|y_i,{\mathrm{\Psi }}^{(t)})=\frac{p(y_i,v_i|x_g,M^{(t)},A^{(t)},b^{(t)},c^{(t)})w_g}{\sum _{g=1}^{G}p(y_i|x_g,M^{(t)},A^{(t)},b^{(t)},c^{(t)})w_g}$$ (19)

being the approximation of the posterior density. The summation ${\sum }_{g=1}^G={\sum }_{s_1=1}^S\cdots {\sum }_{s_K=1}^S$ , with $S$ denoting the number of quadrature nodes for each latent trait and $G=S^K$ . Since the latent traits are correlated, the Cholesky factorization of ${\mathrm{\Sigma }}^{(t)}=D^{(t)}{[D^{(t)}]}^T$ is required for the GH quadrature (Bianconcini, 2014). The quadrature nodes and weights are $x_g={(x_{s_1},\dots ,x_{s_K})}^T=\sqrt{2}{D}^{(t)}{(x_{s_1}^{*},\dots ,x_{s_K}^{*})}^T$ and $w_g={\pi }^{-\frac{K}{2}}{\prod }_{k=1}^K{w}_{s_k}$ respectively, with $x_{s_k}^{*}$ being the classical quadrature nodes and $w_{s_k}$ the corresponding weights for $k=1,\dots ,K$ . These classical quadrature nodes and weights are tabulated in Abramowitz and Stegun (1972), and also can be easily calculated by many modern softwares (e.g., the R-packages fastGHquad and mvQuad).

The approximation of $Q$ -function in (14) can be further factorized as the summation of various parts as follows,

$$\begin{array}{ll}\hfill \tilde{Q}(\mathrm{\Psi }|{\mathrm{\Psi }}^{(t)})& =-2\sum _{i=1}^N\sum _{g=1}^G\sum _{v_i\in {\{\mathrm{0,1}\}}^J}\log p(y_i,v_i,x_g|\mathrm{\Psi })\cdot \tilde{p}(x_g,v_i|y_i,{\mathrm{\Psi }}^{(t)})+\Vert A{\Vert }_0\log N\hfill \\ \hfill & ={\tilde{Q}}_0(\mathrm{\Sigma }|{\mathrm{\Psi }}^{(t)})+\sum _{j=1}^J{\tilde{Q}}_{1,j}(M_j,a_j,b_j|{\mathrm{\Psi }}^{(t)})+\sum _{j=1}^J{\tilde{Q}}_{2,j}(c_j|{\mathrm{\Psi }}^{(t)}),\hfill \end{array}$$ (20)

where ${\tilde{Q}}_0$ is

$$\begin{array}{ll}\hfill {\tilde{Q}}_0(\mathrm{\Sigma }|{\mathrm{\Psi }}^{(t)})& =-2\sum _{i=1}^N\sum _{g=1}^G\sum _{v_i\in {\{\mathrm{0,1}\}}^J}\log \phi (x_g|\mathrm{\Sigma })\cdot \tilde{p}(x_g,v_i|y_i,{\mathrm{\Psi }}^{(t)})\hfill \\ \hfill & =-2\sum _{g=1}^G\log \phi (x_g|\mathrm{\Sigma })\cdot {\tilde{f}}_g^{(t)}.\hfill \end{array}$$ (21)

For $j=1,\dots ,J$ , ${\tilde{Q}}_{1,j}$ is

$${\tilde{Q}}_{1,j}(M_j,a_j,b_j|{\mathrm{\Psi }}^{(t)})=-2{\tilde{E}}_j^{(t)}+\Vert a_j{\Vert }_0\log N,$$ (22)

where $\Vert a_j{\Vert }_0={\sum }_{k=1}^{K}I(a_{jk}\ne 0)$ , and ${\tilde{E}}_j^{(t)}$ is

$$\begin{array}{ll}\hfill {\tilde{E}}_j^{(t)}& =\sum _{i=1}^N\sum _{g=1}^G\sum _{v_i\in {\{\mathrm{0,1}\}}^J}[v_{ij}\log F_j(x_g)+(1-v_{ij})\log (1-F_j(x_g))]\cdot \tilde{p}(x_g,v_i|y_i,{\mathrm{\Psi }}^{(t)})\hfill \\ \hfill & =\sum _{g=1}^G{\tilde{r}}_{gj}^{(t)}(a_j^T{x}_g+b_j)-{\tilde{f}}_g^{(t)}\log (1+\exp (a_j^T{x}_g+b_j)).\hfill \end{array}$$ (23)

Note that, the ${\tilde{f}}_g^{(t)}$ and ${\tilde{r}}_{gj}^{(t)}$ in (21) and (23) are also known as the “pseudo-counts” in IRT literature. Denote

$$\tilde{p}(x_g|y_i,{\mathrm{\Psi }}^{(t)})=\sum _{v_i\in {\{\mathrm{0,1}\}}^J}\tilde{p}(x_g,v_i|y_i,{\mathrm{\Psi }}^{(t)})=\frac{p(y_i|x_g,M^{(t)},A^{(t)},b^{(t)},c^{(t)})w_g}{\sum _{g=1}^{G}p(y_i|x_g,M^{(t)},A^{(t)},b^{(t)},c^{(t)})w_g},$$ (24)

and $\tilde{p}(x_g,v_{ij}|y_i,{\mathrm{\Psi }}^{(t)})=\tilde{p}(x_g|y_i,{\mathrm{\Psi }}^{(t)})p(v_{ij}|y_{ij},x_g,{\mathrm{\Psi }}^{(t)})$ , where $p(v_{ij}|y_{ij},x_g,{\mathrm{\Psi }}^{(t)})$ can be derived from Equations (1) and (7). Then, ${\tilde{f}}_g^{(t)}={\sum }_{i=1}^N\tilde{p}(x_g|y_i,{\mathrm{\Psi }}^{(t)})$ denotes the expected number of examinees with ability $x_g$ , and ${\tilde{r}}_{gj}^{(t)}={\sum }_{i=1}^N\tilde{p}(x_g,v_{ij}=1|y_i,{\mathrm{\Psi }}^{(t)})$ is the expected number of examinees with $x_g$ who has ability to correctly answer the item $j$ .

Additionally, for $j=1,\dots ,J$ , ${\tilde{Q}}_{2,j}$ is

$$\begin{array}{ll}\hfill & {\tilde{Q}}_{2,j}(c_j|{\mathrm{\Psi }}^{(t)})\hfill \\ \hfill & =-2\sum _{i=1}^N\sum _{g=1}^G\sum _{v_i\in {\{\mathrm{0,1}\}}^J}(1-v_{ij})[y_{ij}\log c_j+(1-y_{ij})\log (1-c_j)]\cdot \tilde{p}(x_g,v_i|y_i,{\mathrm{\Psi }}^{(t)})\hfill \\ \hfill & =-2\log c_j\times \sum _{i=1}^N{y}_{ij}\tilde{p}(v_{ij}=0|y_i,{\mathrm{\Psi }}^{(t)})-2\log (1-c_j)\times \sum _{i=1}^N(1-y_{ij})\tilde{p}(v_{ij}=0|y_i,{\mathrm{\Psi }}^{(t)}),\hfill \end{array}$$ (25)

where $\tilde{p}(v_{ij}=0|y_i,{\mathrm{\Psi }}^{(t)})={\sum }_{g=1}^G\tilde{p}(x_g,v_{ij}=0|y_i,{\mathrm{\Psi }}^{(t)})$ . The terms ${\sum }_{i=1}^N{y}_{ij}\tilde{p}(v_{ij}=0|y_i,{\mathrm{\Psi }}^{(t)})$ and ${\sum }_{i=1}^N(1-y_{ij})\tilde{p}(v_{ij}=0|y_i,{\mathrm{\Psi }}^{(t)})$ represent the expected number of examinees who, lacking the ability to correctly answer item $j$ , guess the answer correctly or incorrectly, respectively.

MS-step

In the MS-step, we update the model and parameters by minimizing the approximation of $Q$ -function obtained in the E-step, that is,

$${\mathrm{\Psi }}^{(t+1)}=(M^{(t+1)},A^{(t+1)},b^{(t+1)},c^{(t+1)},{\mathrm{\Sigma }}^{(t+1)})=\underset{\mathrm{\Psi }}{\arg \min }\tilde{Q}(\mathrm{\Psi }|{\mathrm{\Psi }}^{(t)}),$$ (26)

subject to $\mathrm{\Sigma }\succ 0$ and $\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}(\mathrm{\Sigma })=1$ , where $\mathrm{\Sigma }\succ 0$ denotes that $\mathrm{\Sigma }$ is a positive definite matrix, and $\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}(\mathrm{\Sigma })=1$ denotes that all the diagonal entries of $\mathrm{\Sigma }$ are unity. As expressed in the Equation (20), $\tilde{Q}$ is factorized as the summation of ${\tilde{Q}}_0$ involving $\mathrm{\Sigma }$ , ${\tilde{Q}}_{1,j}$ involving $(M_j,a_j,b_j)$ and ${\tilde{Q}}_{2,j}$ involving $c_j$ . Thus, the minimization problem in Equation (26) can be decomposed to minimizing ${\tilde{Q}}_0$ , ${\tilde{Q}}_{1,j}$ and ${\tilde{Q}}_{2,j}$ , $j=1,\dots ,J$ , separately, that is,

$${\mathrm{\Sigma }}^{(t+1)}=\underset{\mathrm{\Sigma }}{\arg \min }{\tilde{Q}}_0(\mathrm{\Sigma }|{\mathrm{\Psi }}^{(t)})\text{s.t.}\mathrm{\Sigma }\succ 0\text{and}\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}(\mathrm{\Sigma })=1,$$ (27)

and for $j=1,\dots ,J$ ,

$$(M_j^{(t+1)},a_j^{(t+1)},b_j^{(t+1)})=\underset{M_j,a_j,b_j}{\arg \min }{\tilde{Q}}_{1,j}(M_j,a_j,b_j|{\mathrm{\Psi }}^{(t)}),$$ (28)

$$c_j^{(t+1)}=\underset{c_j}{\arg \min }{\tilde{Q}}_{2,j}(c_j|{\mathrm{\Psi }}^{(t)}).$$ (29)

The minimization problem (28) intends to select the best subset of covariates (i.e., latent traits) in logistic regression models. Specifically, we need to find a sub-model $M_j^{(t+1)}$ over $2^K$ possible combinations of latent traits and the parameter estimates $a_j^{(t+1)}$ and $b_j^{(t+1)}$ under $M_j^{(t+1)}$ resulting in the smallest expected BIC value for item $j$ . Given a specific sub-model, we obtain the parameter estimates for $(a_j,b_j)$ by the Newton’s method with quadratic convergence rate. The detailed solutions to problems (27), (28), and (29) are given in the Supplemental Material.

After solving (27), (28), and (29), it is straightforward to obtain the model $M^{(t+1)}=(M_1^{(t+1)},\dots ,M_J^{(t+1)})$ and its parameter estimates of ${\mathrm{\Sigma }}^{(t+1)}$ , $A^{(t+1)}={(a_1^{(t+1)},\dots ,a_J^{(t+1)})}^T$ , $b^{(t+1)}={(b_1^{(t+1)},\dots ,b_J^{(t+1)})}^T$ and $c^{(t+1)}={(c_1^{(t+1)},\dots ,c_J^{(t+1)})}^T$ for the next iteration.

Remark 1

In fact, the implementation of IEMS for latent variable selection in the M3PL model described in this section can also be effectively applied to the M2PL model by initializing the guessing parameter $c^{(0)}=0$ . This adjustment enables the use of M3PL-oriented IEMS for M2PL analysis, bypassing the need for a separate implementation for the M2PL model. That is because, the estimate of $c^{(t+1)}$ will always be 0 if $c^{(t)}=0$ . Besides, the “pseudo-counts” ${\tilde{f}}_g^{(t)}$ and ${\tilde{r}}_{gj}^{(t)}$ involved in (21) and (23) will be naturally reduced to those commonly used in the M2PL model (e.g., Chen and Wang (2021)) if $c^{(t)}=0$ .

Simulation Studies

Design

In this section, we conduct simulation studies to evaluate the performance of the proposed IEMS in comparison to five methods under various settings in the M2PL and M3PL models, including the accelerated version of EMS (denoted by EMS90) in Xu et al. (2022), the EM based $L_1$ -penalized method (denoted by EML1) in Sun et al. (2016), the constrained EIFA (denoted by ConEIFA), the traditional EIFA followed by the Quartimin rotation technique (denoted by Quartimin) and the traditional EIFA followed by the Geomin rotation technique (denoted by Geomin). Note that EMS90 is applicable exclusively to the M2PL model.

For a fair comparison, the EML1 is extended to the case with unknown guessing parameter. For EML1, we consider 10 penalty parameters for each dataset, and select the estimation result corresponding to the best penalty parameter with the smallest observed BIC. For the ConEIFA method, the parameters are estimated via the EM algorithm with the same identification constraints as IEMS, and then followed by an arbitrary cutoff for the loadings. The entries are set to zero when they are lower than the given threshold. For the Rotation methods including Quartimin and Geomin, no constraints are imposed on the loading matrix $A$ and the covariance of the latent traits $\mathrm{\Sigma }$ is fixed as the identity matrix. The parameters are estimated via EM and then the oblique rotation technique implemented by the R package GPArotation (Bernaards & Jennrich, 2005) is adopted to rotate the factors and obtain a nearly sparse loading matrix. By setting a threshold to cut off the rotated $A$ , we can get a sparse estimator. In the simulations, several thresholds, i.e., $0.2,0.25,\dots ,0.5$ , are used. All analyses are conducted in R software. To accelerate the computation, parts of the codes are developed in C++ language by the R package Rcpp (Eddelbuettel & Francois, 2011). All codes are available at https://github.com/Laixu3107/EMS_for_M3PL.

Additionally, we evaluate the performance of the IEMS with different number of quadrature nodes in M2PL and M3PL models so as to give a suggestion about the selection of nodes for latent variable selection. Through the simulations given in the Supplemental Material, it is found that 5 quadrature nodes are adequate per dimension for IEMS to produce a satisfactory performance. We further conduct simulations on ConEIFA, Quartimin and Geomin using varying numbers of quadrature nodes in the E-step of EM to approximate the intractable integral. The results are similar, demonstrating that 5 quadrature nodes are sufficient. Therefore, 5 quadrature nodes are used for all these methods. For the sake of brevity, the results are not presented in this paper. Since the EMS90 is computationally expensive when using 11 grid points for each dimension, see Table 4 in Xu et al. (2022), we also use 5 grid points in our simulations. Nonetheless, EMS90 still requires substantial computational costs and takes about 2.6 hours in the case of $K=5,N=4000$ , see Table S15 in the supplemental material.

In the simulations, three M3PL models corresponding to $K=\mathrm{3,4,5}$ , respectively, are considered with the item number $J$ fixed to be 40. The true discrimination parameter matrices, difficulty parameters, and guessing parameters with $K=\mathrm{3,4,5}$ are denoted, respectively, by $(A_1,b_1,c_1),(A_2,b_2,c_2)$ , and $(A_3,b_3,c_3)$ , which are listed in the Supplemental Material. For the sake of simplicity, true parameters in the M2PL models with $K=\mathrm{3,4,5}$ are set as $(A_1,b_1),(A_2,b_2)$ and $(A_3,b_3)$ . The non-zero elements in $A$ -matrices are identically and independently generated from the uniform distribution $U(0.5,2)$ , the components in $b$ -vectors are sampled from the standard normal distribution and the entries in $c$ -vectors are drawn from $U(\mathrm{0,0.2})$ . For the true covariance matrix $\mathrm{\Sigma }$ of latent traits $\theta$ , the diagonal and off-diagonal elements are set to be 1 and 0.1, respectively. Four sample size levels $N=\mathrm{500,1000,2000,4000}$ are considered for each model. For each setting of $(K,N)$ , we draw 100 independent data sets.

The same identification constraints are used to resolve the rotational indeterminacy issue for all cases except for Quartimin and Geomin, that is, we fix a $K\times K$ sub-matrix of the loading structure to be an identity matrix. To be specific, items 1, 10, and 19 are constrained to relate only to latent traits 1, 2, and 3, respectively, for $K=3$ , that is, the structure of ${(a_1,a_{10},a_{19})}^T$ is fixed as identity matrix in each IEMS iteration. Similarly, the structures of ${(a_1,a_7,a_{13},a_{19})}^T$ and ${(a_1,a_5,a_9,a_{13},a_{17})}^T$ are set to be identity matrices for $K=\mathrm{4,5}$ , respectively.

Results

The performance is evaluated in terms of the correct rate (CR) of the latent variable selection, the mean squared error (MSE) of parameter estimates and the computational efficiency. The CR for latent variable selection is defined as follows,

$$\mathrm{C}\mathrm{R}=\frac{1}{K(J-K)}\sum _{\begin{array}{c}1\le j\le J,1\le k\le K,\\ a_{jk}\text{is\hspace{0.17em}not\hspace{0.17em}fixed\hspace{0.17em}for\hspace{0.17em}identification}\end{array}}I({\widehat{M}}_{jk}=M_{jk}),$$

where $\widehat{M}$ is an estimate of the true model $M$ . The MSE measures the average of the squares of the errors and is expressed for each $a_{jk}$ as

$$\mathrm{M}\mathrm{S}\mathrm{E}(a_{jk})=\frac{1}{Z}\sum _{z=1}^Z{({\widehat{a}}_{jk}^z-a_{jk})}^2,$$

where ${\widehat{a}}_{jk}^z$ denotes the estimate of $a_{jk}$ obtained from the $z$ th replication, and $Z=100$ is the number of data sets. For each $b_j$ in $b$ and ${\sigma }_{k{k}^{\prime }}$ in $\mathrm{\Sigma }$ , the MSE is calculated similarly to that of $a_{jk}$ .

Figures 1–5 present the boxplots of the CRs of the estimated model and boxplots of the MSEs of the estimated $A$ , $b$ , $c$ , and $\mathrm{\Sigma }$ for all settings of M3PL models. As we expect, the CRs of the model and MSEs of $A$ obtained by ConEIFA, Quartimin, and Geomin vary with different choices of the thresholds, which illustrates the subjectiveness of the cut-off. Inappropriate setting of the threshold may lead to inferior performance in the latent variable selection. It is also worth mentioning that selecting an optimal threshold can be challenging. For example, a threshold of 0.35 yields the highest CR when $K=3,N=500$ , 0.45 is optimal when $K=5,N=500$ , and 0.25 and 0.3 work best when $K=5,N=4000$ . A possible trend is that, with a given $K$ , the optimal threshold value tends to decrease as the sample size increases. As shown in Figure 1, except for the case of $K=3,N=4000$ , IEMS performs better than EML1 and is similar to the best performance among various thresholds of ConEIFA, Quartimin, and Geomin in all settings. Figure 2 demonstrates the superiority of the IEMS in the estimation of loadings. From Figures 3 and 4, it is revealed that all methods show similar performance in recovering $b$ and $c$ , except for EML1 which exhibits the worst performance. However, Rotation methods like Quartimin and Geomin exhibit comparatively lower performance in recovering $\mathrm{\Sigma }$ when compared to other methods, see Figure 5. Overall, IEMS is of competitiveness among the range of methods considered within the M3PL models.

Figure 1.

Boxplots of the CRs of the model $M$ obtained by IEMS, EML1, ConEIFA, Quartimin, and Geomin, respectively, for the M3PL models with the dimension of the traits $K=3$ (row 1), 4 (row 2) and 5 (row 3) under the sample size $N=500$ (column 1), 1000 (column 2), 2000 (column 3), and 4000 (column 4). For ConEIFA, Quartimin, and Geomin, 7 boxes are displayed corresponding to thresholds $0.2,0.25,\dots ,0.5$ from left to right, respectively.

Figure 2.

Boxplots of the MSEs of the estimated $A$ obtained by IEMS, EML1, ConEIFA, Quartimin, and Geomin, respectively, for the M3PL models with the dimension of the traits $K=3$ (row 1), 4 (row 2) and 5 (row 3) under the sample size $N=500$ (column 1), 1000 (column 2), 2000 (column 3), and 4000 (column 4). For ConEIFA, Quartimin, and Geomin, 7 boxes are displayed corresponding to thresholds $0.2,0.25,\dots ,0.5$ from left to right, respectively.

Figure 3.

Boxplots of the MSEs of the estimated $b$ obtained by IEMS, EML1, ConEIFA, Quartimin, and Geomin for the M3PL models with the dimension of the traits $K=3$ (row 1), 4 (row 2), and 5 (row 3) under the sample size $N=500$ (column 1), 1000 (column 2), 2000 (column 3), and 4000 (column 4).

Figure 4.

Boxplots of the MSEs of the estimated $c$ obtained by IEMS, EML1, ConEIFA, Quartimin, and Geomin for the M3PL models with the dimension of the traits $K=3$ (row 1), 4 (row 2), and 5 (row 3) under the sample size $N=500$ (column 1), 1000 (column 2), 2000 (column 3), and 4000 (column 4).

Figure 5.

Boxplots of the MSEs of the estimated $\mathrm{\Sigma }$ obtained by IEMS, EML1, ConEIFA, Quartimin, and Geomin for the M3PL models with the dimension of the traits $K=3$ (row 1), 4 (row 2), and 5 (row 3) under the sample size $N=500$ (column 1), 1000 (column 2), 2000 (column 3), and 4000 (column 4).

Given a fixed number of latent traits $K$ , as the sample size increases, the CRs of $M$ obtained by all methods generally improve, except for IEMS in the case of $K=3,N=4000$ . Additionally, the MSEs of $A$ decrease as the sample size increases, and the MSEs of $\mathrm{\Sigma }$ gets lower except for Rotation methods. The MSEs of $b$ and $c$ remain similar across different sample sizes. In general, the performance of all these methods tends to improve with the increasing sample size under a specific $K$ . The average CPU time in seconds of all methods for M3PL models are given in Table 1. For the Rotation methods including Quartimin and Geomin, only the CPU time of the traditional EIFA procedure is recorded for reference. It is obvious that IEMS is computationally more efficient than EML1 in all cases, especially when $K$ and $N$ are large. However, the Rotation and ConEIFA methods require less time than IEMS. This is because the EM involved in the Rotation and ConEIFA only updates parameters for the full model while the IEMS needs to estimate the parameters under all candidate models in order to update the model and its parameters in each iteration.

Additionally, we conduct further simulations with larger $c$ -vectors that sampled from $U(0.05,0.25)$ , yielding similar results. Due to the space constraints in the paper, the detailed simulation results and analysis are provided in the supplemental material, see Figures S3–S7. Compared to the results with smaller $c$ -vectors, all these methods exhibit slightly poorer performance in estimating $b$ and $c$ , but demonstrate similar performance in model selection, as well as in estimating $A$ and $\mathrm{\Sigma }$ . Furthermore, the results and analysis for the M2PL models that obtained by IEMS, EMS90, EML1, ConEIFA, Quartimin and Geomin are available in the Supplemental Material, see Table S15 and Figures S8–S11. It is obvious that IEMS improves EMS90 in terms of model recovery, parameter estimation and computation efficiency. Different to the results for the M3PL model, IEMS outperforms other methods in terms of model recovery when the sample size is small (i.e., $N=\mathrm{500,1000}$ ), but is surpassed by EML1 and the best results of ConEIFA, Quartimin, and Geomin as the sample size increases.

Real Data Analysis

In this section, we apply the IEMS algorithm to two real data sets in order to illustrate its validity: the Eysenck Personality Questionnaire data set and the Programme for International Student Assessment (PISA) 2012 data set.

Eysenck Personality Questionnaire Data Set

In this example, IEMS is used to analyze a data set involving the Eysenck Personality Questionnaire given in Eysenck and Barrett (2013) under the M2PL model. After eliminating subjects with missing data, the data set includes 754 Canadian females’ responses to 69 dichotomous items, where items 1–25, 26–46, and 47–69 measure the trait disposition of psychoticism (P), extraversion (E) and neuroticism (N), respectively. Like Xu et al. (2022), only the items whose corrected item-total correlation values greater than 0.2 (Kline, 1986) are used to analyze, in order to guarantee the psychometric properties of the items. The selected items and the original indices are listed in Table S16, from which we can see that there are 10, 19, and 23 items corresponding to P, E, and N, respectively. Items marked by asterisk refer to negatively worded items whose original scores have been reversed.

To ensure the identifiability, we designate one item for each trait according to the meaning of the items, and this prespecified item is only related to that trait. To be specific, we assign item 1, 11, and 30 to P, E, and N, respectively, which is the same as Xu et al. (2022). In this analysis, five quadrature nodes are used for each dimension in the E-step. The latent variable selection result (i.e., the loading structure) is displayed as a heat map in Figure 6. The parameter estimates of $A$ and $b$ are given in Table S17, and the estimate of $\mathrm{\Sigma }$ is

$$\widehat{\mathrm{\Sigma }}=\left(\begin{array}{ccc}\hfill 1.000\hfill & \hfill 0.143\hfill & \hfill -0.023\hfill \\ \hfill 0.143\hfill & \hfill 1.000\hfill & \hfill -0.179\hfill \\ \hfill -0.023\hfill & \hfill -0.179\hfill & \hfill 1.000\hfill \end{array}\right).$$

Figure 6.

Heat map of the loading matrix estimated by IEMS for the real data set involving Eysenck Personality Questionnaire.

The results presented in Figure 6 and Table S17 illustrate a predominant pattern where the majority of items are associated with only one single trait. However, a notable exception emerges, as eight items demonstrate connections with two distinct traits: extraversion and neuroticism. Specifically, items 15, 19, and 25, initially intended to measure extraversion, unexpectedly reveal slight associations with neuroticism based on small loadings. Conversely, items 34, 44, 47, 49, and 50, designed to assess neuroticism, surprisingly exhibit subtle links with extraversion. These intriguing findings suggest a nuanced interplay between these traits within the measurement of these particular items. In fact, these findings are sensible. For example, item 15 (“Do you tend to keep in the background on social occasions?”) and item 19 (“Would you call yourself happy-go-lucky?”) designed for extraversion are also related to neuroticism which reflects individuals’ emotional stability and negative feelings like anxiety, worry, and depression. Item 47 (“Do you worry too long after an embarrassing experience?”) and item 49 (“Do you often feel lonely?”) measuring neuroticism is also related to extraversion whose characteristics are enjoying going out, talking and socializing.

Additionally, we also run EML1, ConEIFA, Quartimin, and Geomin on this data set. To facilitate comparison, we use the results of the artificial recognition, that is, the originally designed item-trait relationships, as the benchmark so as to compute the correct recognition rate of the estimated laoding structure, that is, CR. The CRs of $M$ obtained by IEMS, EML1, ConEIFA, Quartimin, and Geomin are 0.946, 0.707, 0.707, 0.821, and 0.840, respectively. The heat maps displaying the corresponding estimated loading structures can be found in the Supplemental Material. It is evident that among all methods, our proposed IEMS with the highest CR selects the model that is closest to the artificial recognition. The heat maps for the other methods show denser loading structures that are more challenging to interpret, especially for EML1 and ConEIFA which have the lowest CR values. The eight items identified by IEMS as associated with both extraversion and neuroticism are also recognized as such by several other methods, particularly items 15, 19, and 44. All methods consider these three items to be associated with both traits. This further demonstrates the validity of our proposed IEMS.

PISA 2012 Data Set

In this subsection, we apply the IEMS to analyze the PISA 2012 data set under both the M2PL and M3PL models. The data set can be downloaded from https://www.oecd.org/pisa/. As an international large-scale assessment of student achievement developed by OECD, PISA 2012 aims to assess the competencies of 15-year-old students in reading, mathematics, and science in 65 countries and economies. The students took a paper-based test with a mixture of open-ended and multiple-choice questions that lasted 2 hours.

Given the extensive sample size, the analysis focuses solely on the cognitive scored item responses of 1636 Italian students who completed the items within booklet 12. The booklet 12 contains 61 items whose labels and description are presented in Supplemental Material, in which items 1–11, 12–26, and 27–61 measure students’ literacy in mathematics, reading, and science, respectively. The majority of the items are scored as 0 and 1, with 0 indicating no credit, missing and invalid (i.e., incorrect response) and 1 indicating full credit (i.e., correct response). For some items scored as 0, 1, and 2 where 1 indicates partial credit and 2 full credit, the original responses of 0–1 are recorded as 0, and the responses of 2 are recorded as 1. Table S18 also displays the frequency and percent of the students who correctly respond to the specific item.

To guarantee the identifiability, we designate two items for each trait based on the prior information. To be specific, we assign items 1, 2 to mathematical literacy, items 12, 13 to reading literacy, and items 27, 28 to science literacy, respectively. The IEMS algorithm yields the observed BIC value of 107014.6 under the M2PL model and 106922.7 under the M3PL model, which suggests a better fit of the M3PL model to this particular data set. The loading structures are displayed as heat maps in Figures 7 and 8. The parameter estimates of $(A,b)$ and $(A,b,c)$ are given in Tables S19 and S20 in the supplemental material, and the estimates of $\mathrm{\Sigma }$ are

$${\widehat{\mathrm{\Sigma }}}_{\mathit{M2PL}}=\left(\begin{array}{ccc}\hfill 1.000\hfill & \hfill 0.549\hfill & \hfill 0.694\hfill \\ \hfill 0.549\hfill & \hfill 1.000\hfill & \hfill 0.688\hfill \\ \hfill 0.694\hfill & \hfill 0.688\hfill & \hfill 1.000\hfill \end{array}\right),{\widehat{\mathrm{\Sigma }}}_{\mathit{M3PL}}=\left(\begin{array}{ccc}\hfill 1.000\hfill & \hfill 0.588\hfill & \hfill 0.721\hfill \\ \hfill 0.588\hfill & \hfill 1.000\hfill & \hfill 0.744\hfill \\ \hfill 0.721\hfill & \hfill 0.744\hfill & \hfill 1.000\hfill \end{array}\right).$$

Figure 7.

Heat map of the loading matrix estimated by IEMS for the PISA 2012 data set under the M2PL model.

Figure 8.

Heat map of the loading matrix estimated by IEMS for the PISA 2012 data set under the M3PL model.

We can see that correlations between the latent traits are high. Despite each item being intended to measure a specific literacy, it appears that the correct response might stem from the interplay between multiple literacies. This suggests that the skills or abilities assessed by these items might not be isolated but interconnected, influencing each other in determining the correct response. For instance, items 23–26 in the same testlet (READ-P2009 Chocolate and Health) are found to be associated with both reading and science. Besides, the estimated difficulty parameter $b$ aligns with the frequency of students with correct response. Items with lower correct response frequencies tend to exhibit lower $b$ values. For example, the frequency 265 of item 4 corresponds to the $b$ value −3.015 under the M3PL model, which means that this item is difficult for students to answer.

Similarly, EML1, ConEIFA, Quartimin and Geomin are used to analyze PISA 2012 data set under the M3PL model, and the resulting heat maps of the loading structures are provided in the Supplemental Material. While designed item-literacy relationships are considered as the benchmark, it’s important to note that multiple literacies may interact when responding to items. By comparing these methods with the benchmark, the CRs of IEMS, EML1, ConEIFA, Quartimin, and Geomin are 0.776, 0.570, 0.679, 0.667, and 0.732, respectively. IEMS achieves the highest CRs and produces the sparsest loading structure.

Discussion

In this paper, we apply the IEMS algorithm to the latent variable selection for M3PL models, and give an efficient implementation by using the GH quadrature in the E-step and the Newton’s method in the MS-step. The whole procedure can be easily adapted to M2PL models. Compared to five state-of-art methods, IEMS produces a satisfactory performance in terms of model recovery, parameter estimation precision and computational efficiency in the latent variable selection for both M2PL and M3PL models.

Specifically, we compare the IEMS with EML1, ConEIFA, Quartimin, and Geomin in the latent variable selection in M3PL models with both smaller and larger $c$ -vectors. Besides, simulation comparisons are also conducted in the case of M2PL models with EMS90 taken into account. The simulations demonstrate that IEMS generally outperforms EML1 and achieves performance similar to the best results among various thresholds of ConEIFA, Quartimin, and Geomin in recovering M3PL models. Meanwhile, IEMS shows its superiority in estimating loadings $A$ . This holds true for both small and large $c$ -vectors in M3PL models. As $c$ -vectors increase, the performance of all these methods in estimating $b$ and $c$ deteriorates. In contrast to M3PL models, IEMS outperforms other methods in recovering M2PL models with small sample size. However, as the sample size increases, IEMS is inferior to EML1 and the best results of ConEIFA, Quartimin, and Geomin, although IEMS still demonstrates a satisfactory performance. In general, the performance of these methods tends to improve with the increasing sample size given a fixed number of latent traits, except for EMS90.

Among these methods, ConEIFA, Quartimin, and Geomin have subjectivity in cut-off, which is verified by the simulations. The selection of an appropriate threshold can be challenging. Compared to other methods, EMS90 exhibits inferior performance in both model recovery and parameter estimation. This is due to EMS90’s trade-off of approximation precision for computation efficiency, despite EMS90 still being computationally expensive. IEMS improves EMS90 significantly in terms of model recovery, parameter estimation and computation efficiency. Meanwhile, IEMS offers an objective approach to model selection, overcoming the subjectivity associated with ConEIFA and Rotation methods. However, our proposed IEMS still has some limitations. It requires that each latent trait has an item exclusively associated with it to meet identifiability conditions, which may not be satisfied in practical research. This limitation is also present when using EMS90, EML1, and ConEIFA. Besides, IEMS cannot deal with the high-dimensional latent variable selection problems since GH quadrature is used for numerical approximation in the E-step. The cardinality of the set of selected points grows exponentially with the number of latent traits. Therefore, E-step needs further improvement in the future. It is important to note that, when using the IEMS, careful selection of items that satisfy the identifiability conditions according to the priori knowledge is necessary. Misspecifying these items can result in incorrect model selection.

The proposed method can be extended along the following directions for future research. First, we will extend the GEMS algorithm (Shang et al., 2024) to the latent variable selection for M3PL models. In the MS-step, GEMS seeks only a decrease in the $Q$ function value instead of minimization as in EMS, which can further improve the computational efficiency. Second, to improve IEMS, we can resort to stochastic approximation methods such as Metropolis-Hastings Robbins-Monro algorithm (MH-RM; Cai, 2010) that is proposed for parameter estimation in the high-dimensional item factor analysis. In contrast to IEMS, MH-RM avoids numerical integration in the E-step by employing the MH sampling and stochastic approximation. MH-RM also does not require iterative parameter updates in the RM update step. Moreover, the deterministic approximation scheme such as the variational inference method (Cho et al., 2021, 2024) can also be adapted to IEMS. The variational method approximates the conditional expectation by a closed-form lower bound, which can be hardly influenced by the dimension of latent traits in the E-step. These two methods significantly improve the computational efficiency and are feasible for high-dimensional problems. Third, we will investigate the application of EMS to the latent variable selection for the bi-factor model (Gibbons & Hedeker, 1992; Jennrich & Bentler, 2011), which can be viewed as a special type of MIRT model. Fourth, we will further explore the performance of various methods for latent variable selection in a wider range of settings to offer guidance for selecting the appropriate method for practical research. The settings worthy of consideration include between-item and within-item models, small and large correlations between latent traits, small and large loadings, varying numbers of items, and more.

ORCID iD

Ping-Feng Xu https://orcid.org/0000-0002-4721-9996