Bayesian Modal Estimation for the One-Parameter Logistic Ability-Based Guessing (1PL-AG) Model

Abstract

The calibration of the one-parameter logistic ability-based guessing (1PL-AG) model in item response theory (IRT) with a modest sample size remains a challenge for its implausible estimates and difficulty in obtaining standard errors of estimates. This article proposes an alternative Bayesian modal estimation (BME) method, the Bayesian Expectation-Maximization-Maximization (BEMM) method, which is developed by combining an augmented variable formulation of the 1PL-AG model and a mixture model conceptualization of the three-parameter logistic model (3PLM). By comparing with marginal maximum likelihood estimation (MMLE) and Markov Chain Monte Carlo (MCMC) in JAGS, the simulation shows that BEMM can produce stable and accurate estimates in the modest sample size. A real data example and the MATLAB codes of BEMM are also provided.

In educational measurement, student’s ability is always considered to be unobservable or latent. With the introduction of the item response theory (IRT), the issue has been properly addressed by collecting students’ responses to several items and provides a measurable result of students’ proficiency level. The Rasch model (Lord & Novick, 1968; Rasch, 1960) has become a widely used and highly reliable model based only on students’ ability and the item difficulty.

Despite of its popularity, the Rasch model has to keep evolving to meet changing needs from testing practice. One prominent example is how to model guessing behavior which might be even prevalent due to the dominant item format, multiple-choice items, in the testing industry. To solve this problem, the one-parameter logistic guessing (1PL-G) IRT model and its ability-based guessing version (1PL-AG) are proposed (San Martín et al., 2006, 2013). Increasingly, applications of the 1PL-AG model have appeared in the literature as supporting evidence of its usefulness (Fariña et al., 2019; San Martín et al., 2013; Wang & Huang, 2011).

One daunting challenge regarding the 1PL-AG model pertains to the difficulty in estimating item parameters. The lack of a user-friendly implementation of the 1PL-AG model remains a major barrier to its widespread application in practice. Few studies to date have addressed the item calibration issue for the 1PL-AG model. All the methods fall into three approaches: PROC NLMIXED in SAS (San Martín et al., 2006), Markov Chain Monte Carlo (MCMC) via general-purpose Bayesian tools like WinBUGS (San Martín et al., 2013) and marginal maximum likelihood estimation with expectation-maximization (MMLE/EM) algorithm (Park et al., 2015). For the PROC NLMIXED and BUGS methods, although they can provide ease of programming and accurate item estimates, both of them are general implementation tools and, thus, inherently time-consuming (Park et al., 2015) which motivates us to find an alternative implementation.

Compared with PROC NLMIXED and MCMC, MMLE/EM is a more efficient procedure to obtain the estimates of the 1PL-AG model. However, just as the essential difficulty of estimating the three-parameter logistic model (3PLM), the sparse data for the guessing parameters maybe cause extremely small diagonal elements in the negative Hessian matrix when running Newton–Raphson iterations, which finally leads to an unstable and implausible MLE (Mislevy, 1986) and low convergence rates (Waller & Feuerstahler, 2017). In this case, MMLE/EM usually requires a relatively large sample size to obtain stable MLEs (Park et al., 2015).

In this article, the authors propose an alternative Bayesian modal estimation approach that they term as Bayesian Expectation-Maximization-Maximization(BEMM) method, which is developed by using a latent variable augmentation approach for the 1PL-AG model in conjunction with the mixture modeling approach for the 3PLM (Zheng et al., 2018). Furthermore, this kind of methods can be also applied for estimating the four parameter logistic IRT model (Zhang, 2018; Zhang et al., 2018). The following article is organized as follows: First, MMLE/EM method and identification issues for the 1PL-AG model are summarized. Then, the derivation of the BEMM algorithm for the 1PL-AG is presented. Next, a simulation study and an empirical example are reported. The last section is a brief discussion and future directions.

Marginalized Maximum Likelihood Estimation for the 1PL-AG Model

The basic IRT formulation of the 1PL-AG model is

$$P_j\left({\mathrm{\theta }}_i\right)\equiv P(Y_{\mathrm{ij}}=1|{\mathrm{\theta }}_i,{\mathrm{\beta }}_j,{\mathrm{\gamma }}_j,\mathrm{\alpha })=P_j^{*}\left({\mathrm{\theta }}_i\right)+(1-P_j^{*}\left({\mathrm{\theta }}_i\right))G_j\left({\mathrm{\theta }}_i\right),$$ (1)

with

$$P_j^{*}\left({\mathrm{\theta }}_i\right)\equiv P(Y_{\mathrm{ij}}=1|{\mathrm{\theta }}_i,{\mathrm{\beta }}_j)=\frac{1}{1+\exp (-({\mathrm{\theta }}_i-{\mathrm{\beta }}_j))},$$ (2)

$$G_j\left({\mathrm{\theta }}_i\right)\equiv P(Y_{\mathrm{ij}}=1|{\mathrm{\theta }}_i,{\mathrm{\gamma }}_j,\mathrm{\alpha })=\frac{1}{1+\exp (-(\mathrm{\alpha }{\mathrm{\theta }}_i+{\mathrm{\gamma }}_j))},$$ (3)

where $\mathrm{\alpha }$ is the weighted guessing component on the latent ability for all items in one test, ${\mathrm{\beta }}_j$ and ${\mathrm{\gamma }}_j$ are difficulty and logistic scaled guessing parameter of item $j(j=1,2,3...,J)$, respectively. ${\mathrm{\theta }}_i\in \mathrm{\theta }$ is the ability parameter of examinee $i(i=1,2,3...,N)$. Furthermore, the 1PL-AG model can degenerate to the 1PL-G model by fixing $\mathrm{\alpha }=0$ in Equation 3.

Let ${\mathrm{\psi }}_j$ be the arbitrary item parameter in ${\mathbf{\Psi }}_j$, and ${\mathbf{\Psi }}_j=({\mathrm{\beta }}_j,{\mathrm{\gamma }}_j,\mathrm{\alpha }{)}^T$ be the $j$ th column vector of the matrix $\mathbf{\Psi }$ collecting the item parameters for all $J$ items. Similarly, let $y_{\mathrm{ij}}\in \mathbf{Y}$ be the matrix collecting all item responses of $N$ persons on $J$ items. Then, the marginal likelihood function should be

$$\begin{array}{c}L(\mathbf{Y}|\mathbf{\Psi })=\underset{i=1}{\overset{N}{\Pi }}P({\mathit{y}}_{\mathit{i}}|\mathbf{\Psi })\hfill \\ =\underset{i=1}{\overset{N}{\Pi }}{\int }_{{\mathrm{\theta }}_i}P({\mathit{y}}_{\mathit{i}}|\mathbf{\Psi },{\mathrm{\theta }}_i)g({\mathrm{\theta }}_i|\mathbf{\tau })d{\mathrm{\theta }}_i\hfill \\ =\underset{i=1}{\overset{N}{\Pi }}{\int }_{{\mathrm{\theta }}_i}\underset{j=1}{\overset{J}{\Pi }}\left[P_j{\left({\mathrm{\theta }}_i\right)}^{Y_{\mathrm{ij}}}{(1-P_j\left({\mathrm{\theta }}_i\right))}^{1-Y_{\mathrm{ij}}}\right]g({\mathrm{\theta }}_i|\mathbf{\tau })d{\mathrm{\theta }}_i\hfill \end{array}$$ (4)

with

$$P({\mathit{y}}_{\mathit{i}}|\mathbf{\Psi })={\int }_{{\mathrm{\theta }}_i}P({\mathit{y}}_{\mathit{i}}|\mathbf{\Psi },{\mathrm{\theta }}_i)g({\mathrm{\theta }}_i|\mathbf{\tau })d{\mathrm{\theta }}_i,$$ (5)

$$P({\mathit{y}}_{\mathit{i}}|\mathbf{\Psi },{\mathrm{\theta }}_i)=\underset{j=1}{\overset{J}{\Pi }}{P}_j{\left({\mathrm{\theta }}_i\right)}^{Y_{\mathrm{ij}}}{(1-P_j\left({\mathrm{\theta }}_i\right))}^{1-Y_{\mathrm{ij}}},$$ (6)

where $g({\mathrm{\theta }}_i|\mathbf{\tau })$ is a density function for each examinee $i$, and $\mathbf{\tau }$ is the vector containing the parameters of the examinee population ability distribution. Park et al. (2015) suggested using EM algorithm with Newton–Raphson method to solve this function, then the maximum likelihood estimates (MLEs) for each parameter can be obtained.

The Bayesian Modal Estimation for the 1PL-AG Model

Model Identification

The IRT models with guessing effects (e.g., 3PLM) suffer from the identification issue which causes estimation difficulty (Lord, 1980), and the 1PL-AG model is no exception (San Martín et al., 2015). However, several studies have been done to address this issue (Fariña et al., 2019; San Martín et al., 2006, 2013, 2015). Especially, San Martín et al. (2006) have shown that the identification issue of the 1PL-AG model can be avoided if we assume the latent variable $\mathrm{\theta }$ follows a normal distribution with a fixed mean of zero. Following the MMLE/EM algorithm (Park et al., 2015), the BEMM algorithm also uses a standard normal distribution to integrate $\mathrm{\theta }$ to guarantee the 1PL-AG model is identified.

Mixture-Modeling Reformulation

The 1PL-AG model is derived from the two-process interpretation of the 3PL model: The p-process and the g-process (Hutchinson, 1991). The p-process is the regular process that examinee tries to solve items depends only on the latent ability. The g-process, apart from the regular process, consists of a general weighted guessing component parameter of the examinee’s ability. Thus, it can be determined that the 1PL-AG model can be regarded as a latent mixture model. First, note that

$$P_j\left({\mathrm{\theta }}_i\right)=P_j^{*}\left({\mathrm{\theta }}_i\right)+(1-P_j^{*}\left({\mathrm{\theta }}_i\right))G_j\left({\mathrm{\theta }}_i\right)=P_j^{*}\left({\mathrm{\theta }}_i\right)\times \left[1\right]+(1-P_j^{*}\left({\mathrm{\theta }}_i\right))\times \left[G_j\left({\mathrm{\theta }}_i\right)\right].$$ (7)

Next, a latent indicator variable $W_{\mathrm{ij}}$ is defined to indicate whether examinee $i$ knows the answer of item $j$ only using the p-process (Béguin & Glas, 2001; Culpepper, 2015; Guo & Zheng, 2019). Specifically, let

$$W_{\mathrm{ij}}=\{\begin{array}{cc}1,\hfill & \mathrm{if}\mathrm{he}\mathrm{knows}\mathrm{the}\mathrm{answer}\mathrm{of}\mathrm{item}j\mathrm{only}\mathrm{using}\mathrm{the}\mathrm{p}-\mathrm{process},\hfill \\ \multicolumn{1}{c}{0,}& \mathrm{else}.\hfill \end{array}$$

Then, the authors model $W_{\mathrm{ij}}$ probabilistically as $W_{\mathrm{ij}}~\mathrm{Bernoulli}(P_j^{*}({\mathrm{\theta }}_i)),$ meaning that $P(W_{\mathrm{ij}}=1)=P_j^{*}\left({\mathrm{\theta }}_i\right)$.

From the mixture-modeling perspective, depending on the value of $W_{\mathrm{ij}}$, the 1PL-AG model can be decomposed into two irrelevant parts: 1 and $G_j\left({\mathrm{\theta }}_i\right)$. The conditional probabilities $P(Y_{\mathrm{ij}}|W_{\mathrm{ij}},{\mathrm{\theta }}_i,{\mathit{\psi }}_j)$ can be further obtained as

$$\begin{array}{c}P(Y_{\mathrm{ij}}=1|W_{\mathrm{ij}}=1,{\mathrm{\theta }}_i,{\mathit{\psi }}_j)=1,P(Y_{\mathrm{ij}}=1|W_{\mathrm{ij}}=0,{\mathrm{\theta }}_i,{\mathit{\psi }}_j)=G_j\left({\mathrm{\theta }}_i\right),\hfill \\ P(Y_{\mathrm{ij}}=0|W_{\mathrm{ij}}=1,{\mathrm{\theta }}_i,{\mathit{\psi }}_j)=0,P(Y_{\mathrm{ij}}=0|W_{\mathrm{ij}}=0,{\mathrm{\theta }}_i,{\mathit{\psi }}_j)=1-G_j\left({\mathrm{\theta }}_i\right).\hfill \end{array}$$ (8)

Thus, the joint distribution of $Y_{\mathrm{ij}}$ and $W_{\mathrm{ij}}$ (conditional on item and person parameters) can be found as follows:

$$P(Y_{\mathrm{ij}},W_{\mathrm{ij}}|{\mathrm{\theta }}_i,{\mathrm{\psi }}_j)=\{\begin{array}{cc}{P}_j^{*}\left({\mathrm{\theta }}_i\right),\hfill & \mathrm{for}{Y}_{\mathrm{ij}}=1,W_{\mathrm{ij}}=1\hfill \\ \multicolumn{1}{c}{0,}& \mathrm{for}{Y}_{\mathrm{ij}}=0,W_{\mathrm{ij}}=1\hfill \\ \multicolumn{1}{c}{G_j\left({\mathrm{\theta }}_i\right)(1-P_j^{*}\left({\mathrm{\theta }}_i\right)),}& \mathrm{for}{Y}_{\mathrm{ij}}=1,W_{\mathrm{ij}}=0\hfill \\ \multicolumn{1}{c}{(1-G_j\left({\mathrm{\theta }}_i\right))(1-P_j^{*}\left({\mathrm{\theta }}_i\right)),}& \mathrm{for}{Y}_{\mathrm{ij}}=0,W_{\mathrm{ij}}=0\hfill \end{array},$$ (9)

Since $P_j^{*}\left({\mathrm{\theta }}_i\right)+G_j\left({\mathrm{\theta }}_i\right)(1-P_j^{*}\left({\mathrm{\theta }}_i\right))+(1-G_j\left({\mathrm{\theta }}_i\right))(1-P_j^{*}\left({\mathrm{\theta }}_i\right))=1$, $P(Y_{\mathrm{ij}}=0,W_{\mathrm{ij}}=1|{\mathrm{\theta }}_i,{\mathrm{\psi }}_j)=0$ is unnecessary in the joint probability function. Let ${\mathit{w}}_i=(W_{i1},W_{i2},...,W_{\mathrm{iJ}})$ be the latent indicator vector for examinee $i$. Then the joint distribution for the new augmented complete data $(\mathbf{Y},\mathit{W},\mathrm{\theta })$ is

$$P({\mathit{y}}_i,{\mathit{w}}_i,{\mathrm{\theta }}_i|\mathbf{\Psi },\mathbf{\tau })=P({\mathit{y}}_i,{\mathit{w}}_i|\mathbf{\Psi },{\mathrm{\theta }}_i)g({\mathrm{\theta }}_i|\mathbf{\tau })$$ (10)

where

$$\begin{array}{c}P({\mathit{y}}_i,{\mathit{w}}_i|\mathbf{\Psi },{\mathrm{\theta }}_i)=\underset{j=1}{\overset{J}{\Pi }}{\left[P_j^{*}\left({\mathrm{\theta }}_i\right)\right]}^{Y_{\mathrm{ij}}{W}_{\mathrm{ij}}}{\left[G_j\left({\mathrm{\theta }}_i\right)(1-P_j^{*}\left({\mathrm{\theta }}_i\right))\right]}^{Y_{\mathrm{ij}}(1-W_{\mathrm{ij}})}\hfill \\ \times {\left[(1-G_j\left({\mathrm{\theta }}_i\right))(1-P_j^{*}\left({\mathrm{\theta }}_i\right))\right]}^{(1-Y_{\mathrm{ij}})(1-W_{\mathrm{ij}})}.\hfill \end{array}$$ (11)

Just like MMLE/EM algorithm (Bock & Aitkin, 1981), the BEMM algorithm also marginalizes out the latent ability variable, so the likelihood function of BEMM is

$$\begin{array}{c}L(\mathbf{Y},\mathbf{W}|\mathbf{\Psi })=\underset{i=1}{\overset{N}{\Pi }}P({\mathit{y}}_i,{\mathit{w}}_i|\mathbf{\Psi })\hfill \\ =\underset{i=1}{\overset{N}{\Pi }}{\int }_{{\mathrm{\theta }}_i}P({\mathit{y}}_i,{\mathit{w}}_i,{\mathrm{\theta }}_i|\mathbf{\Psi },\mathbf{\tau })d{\mathrm{\theta }}_i\hfill \\ =\underset{i=1}{\overset{N}{\Pi }}{\int }_{{\mathrm{\theta }}_i}P({\mathit{y}}_i,{\mathit{w}}_i|\mathbf{\Psi },{\mathrm{\theta }}_i)g({\mathrm{\theta }}_i|\mathbf{\tau })d{\mathrm{\theta }}_i.\hfill \end{array}$$ (12)

This approach follows the basic premise of an EM algorithm—that is, from a full likelihood of the data, iteratively taking expectations of the log-likelihood over the incomplete portion of the data, defining “artificial data,” and maximizing the likelihood—and proposes to split the maximization step into two parts: first maximizing over ${\mathrm{\beta }}_j$ and ${\mathrm{\gamma }}_j$, and then maximizing over $\mathrm{\alpha }$. Prior distributions for the item parameters are further used to obtain the Bayes estimates. As such, this approach is called the Bayesian Expectation-Maximization-Maximization (BEMM) method.

Follow the EM algorithm for IRT models (Bock & Aitkin, 1981; Park et al., 2015), the first derivative of the log-likelihood function is calculated for the arbitrary item parameter ${\mathrm{\psi }}_j$. Let $X_k$, $k=1,2,\dots ,q$, be nodes on the ability scale with associated weights $A\left(X_k\right)$, the integral can be approximated by using the numerical quadrature method. Then, we have (please refer to Online Appendix A for a detailed derivation):

$$0=\frac{\partial \ln L(\mathbf{Y},\mathbf{W}|\mathbf{\Psi })}{\partial {\mathrm{\psi }}_j}=\sum _{i=1}^N\sum _{k=1}^q\left[\frac{\partial \ln P({\mathit{y}}_i,{\mathit{w}}_i|\mathbf{\Psi },X_k)}{\partial {\mathrm{\psi }}_j}\right]f(X_k|{\mathit{y}}_i,\mathbf{\tau },\mathbf{\Psi })$$ (13)

with

$$f(X_k|{\mathit{y}}_i,\mathbf{\tau },\mathbf{\Psi })=f(X_k|{\mathit{y}}_i,{\mathit{w}}_i,\mathbf{\tau },\mathbf{\Psi })=\frac{P({\mathit{y}}_i|\mathbf{\Psi },X_k)A\left(X_k\right)}{\sum _{k=1}^{q}P({\mathit{y}}_i|\mathbf{\Psi },X_k)A\left(X_k\right)},$$ (14)

$$P({\mathit{y}}_{\mathit{i}}|\mathbf{\Psi },X_k)=\underset{j=1}{\overset{J}{\Pi }}{P}_j{\left(X_k\right)}^{Y_{\mathrm{ij}}}{(1-P_j\left(X_k\right))}^{1-Y_{\mathrm{ij}}},$$ (15)

$$A\left(X_k\right)=\frac{\Phi \left(X_k\right)}{{\sum }_{k=1}^q\Phi \left(X_k\right)},$$ (16)

where $f(X_k|{\mathit{y}}_i,\mathbf{\tau },\mathbf{\Psi })$ is the posterior density of ${\mathrm{\theta }}_i$ at quadrature point $X_k$ and $\Phi \left(X_k\right)$ is the density function of the standard normal distribution given $X_k$.

Expectation Step and Prior Distribution

In the expectation step, the expectation of the first derivative of the log-likelihood is taken for each item parameter over the incomplete portion of the data, the latent variables ${\mathrm{\theta }}_i$ and $W_{\mathrm{ij}}$. Based on the general equation of the Bayesian model estimation given by Mislevy (1986), the next maximization step is equivalent to finding the zero of the sum of the expected first derivative of log-likelihood and the first derivative of the prior as

$$0=E\left(\frac{\partial \ln L(\mathbf{Y},\mathbf{W}|\mathbf{\Psi })}{\partial {\mathrm{\psi }}_j}\right)+\frac{\partial \ln g({\mathrm{\psi }}_j|{\mathbf{\eta }}_j)}{\partial {\mathrm{\psi }}_j},$$ (17)

where $g({\mathrm{\psi }}_j|{\mathbf{\eta }}_j)$ is the density function of the prior distribution of ${\mathrm{\psi }}_j$ with hyperparameter ${\mathbf{\eta }}_j$. Notice that the above expression includes the latent discrete variable $W_{\mathrm{ij}}$ so when the expectation of the first derivatives is taken, it will be necessary to calculate $E(W_{\mathrm{ij}}|Y_{\mathrm{ij}},X_k,{\mathbf{\Psi }}_j)$. By using Bayes’ rule, the conditional distribution of $W_{\mathrm{ij}}$ is

$$P(W_{\mathrm{ij}}=1|Y_{\mathrm{ij}}=1,X_k,{\mathit{\psi }}_j)=\frac{P_j^{*}\left(X_k\right)}{P_j^{*}\left(X_k\right)+G_j\left(X_k\right)(1-P_j^{*}\left(X_k\right))}=\frac{P_j^{*}\left(X_k\right)}{P_j\left(X_k\right)},$$ (18)

$$P(W_{\mathrm{ij}}=1|Y_{\mathrm{ij}}=0,X_k,{\mathit{\psi }}_j)=\frac{0}{0+(1-G_j\left(X_k\right))(1-P_j^{*}\left(X_k\right))}=0.$$ (19)

Thus, the conditional expectation of $W_{\mathrm{ij}}$ is found to be

$$E(W_{\mathrm{ij}}|Y_{\mathrm{ij}},X_k,{\mathbf{\Psi }}_j)=\frac{P_j^{*}\left(X_k\right)}{P_j\left(X_k\right)}\times Y_{\mathrm{ij}}+0\times (1-Y_{\mathrm{ij}}).$$ (20)

Several “artificial data” can be defined for the algorithm (Bock & Aitkin, 1981):

$${\overline{f}}_k=\sum _{i=1}^{N}f(X_k|{\mathit{y}}_i,\mathbf{\tau },\mathbf{\Psi }),{\overline{r}}_{\mathrm{jk}}=\sum _{i=1}^N{Y}_{\mathrm{ij}}f(X_k|{\mathit{y}}_i,\mathbf{\tau },\mathbf{\Psi }),$$ (21)

and

$${\overline{f}}_{\mathrm{jk}}^{\left(W\right)}=\sum _{i=1}^{N}E(W_{\mathrm{ij}}|Y_{\mathrm{ij}},X_k,\mathbf{\Psi })f(X_k|{\mathit{y}}_i,\mathbf{\tau },\mathbf{\Psi }),{\overline{r}}_{\mathrm{jk}}^{\left(W\right)}=\sum _{i=1}^N{Y}_{\mathrm{ij}}E(W_{\mathrm{ij}}|Y_{\mathrm{ij}},X_k,\mathbf{\Psi })f(X_k|{\mathit{y}}_i,\mathbf{\tau },\mathbf{\Psi }).$$ (22)

We find that these expressions have very intuitive interpretations:

1. ${\overline{f}}_k$ is the expected number of persons (out of $N$) with ability $X_k$;

2. ${\overline{r}}_{\mathrm{jk}}$ is the expected number of persons with ability $X_k$ to answer item $j$ correctly $(Y_{\mathrm{ij}}=1)$;

3. ${\overline{f}}_{\mathrm{jk}}^{\left(W\right)}$ is the expected number of persons with ability $X_k$ that only using the p-process to know the answer of item $j$$(W_{\mathrm{ij}}=1)$;

4. ${\overline{r}}_{\mathrm{jk}}^{\left(W\right)}$ is the expected number of persons with ability $X_k$ that answering item $j$ correctly $(Y_{\mathrm{ij}}=1)$ and only using the p-process to know the answer of item $j$$(W_{\mathrm{ij}}=1)$. Due to the definition of the conditional expectation $E(W_{\mathrm{ij}}|Y_{\mathrm{ij}},X_k,{\mathbf{\Psi }}_j)$, ${\overline{f}}_{\mathrm{jk}}^{\left(W\right)}$ mathematically equals ${\overline{r}}_{\mathrm{jk}}^{\left(W\right)}$.

Since the $\mathrm{\alpha }$ parameter should be larger than 0 (San Martín et al., 2006), a log transformation for $\mathrm{\alpha }$ to facilitate estimation is made (Mislevy, 1986). Let ${\mathrm{\alpha }}^{*}=\ln (\mathrm{\alpha })$, and then a normal distribution prior can be imposed on the arbitrary parameter ${\mathrm{\psi }}_j^{*}\in ({\mathrm{\beta }}_j,{\mathrm{\gamma }}_j,{\mathrm{\alpha }}^{*}{)}^T$. Thus, instead of directly estimating $\mathrm{\alpha }$, first ${\mathrm{\alpha }}^{*}$ is obtained and its exponential value is taken as the estimates of $\mathrm{\alpha }$. Let $g({\mathrm{\psi }}_j^{*}|{\mathrm{\mu }}_j,{\sigma }_j)$ be the general form of the density function of the normal distribution with mean ${\mathrm{\mu }}_j$ and variance ${\sigma }_j$. By plugging the artificial data and priors, Equation 17 can be derived as

$$\begin{array}{c}0=E\left(\frac{\partial \ln L(\mathbf{Y},\mathbf{W}|\mathbf{\Psi })}{\partial {\mathrm{\psi }}_j^{*}}\right)+\frac{\partial \ln g({\mathrm{\psi }}_j^{*}|{\mathbf{\eta }}_j)}{\partial {\mathrm{\psi }}_j^{*}}\hfill \\ \Rightarrow {\left(\begin{array}{c}\sum _{k=1}^q\left[\frac{{\overline{r}}_{\mathrm{jk}}^{\left(W\right)}-{\overline{f}}_k{P}_j^{*}\left(X_k\right)}{P_j^{*}\left(X_k\right)[1-P_j^{*}\left(X_k\right)]}\frac{\partial P_j^{*}\left(X_k\right)}{\partial {\mathrm{\beta }}_j}\right]+\frac{\partial \ln g({\mathrm{\beta }}_j|{\mathrm{\mu }}_j^{\left(\mathrm{\beta }\right)},{\sigma }_j^{\left(\mathrm{\beta }\right)})}{\partial {\mathrm{\beta }}_j},\hfill \\ \multicolumn{1}{c}{\sum _{k=1}^q\left[\frac{{\overline{r}}_{\mathrm{jk}}-{\overline{r}}_{\mathrm{jk}}^{\left(W\right)}-({\overline{f}}_k-{\overline{f}}_{\mathrm{jk}}^{\left(W\right)})G_j^{*}\left(X_k\right)}{G_j^{*}\left(X_k\right)[1-G_j^{*}\left(X_k\right)]}\frac{\partial G_j^{*}\left(X_k\right)}{\partial {\mathrm{\gamma }}_j}\right]+\frac{\partial \ln g({\mathrm{\gamma }}_j|{\mathrm{\mu }}_j^{\left(\mathrm{\gamma }\right)},{\sigma }_j^{\left(\mathrm{\gamma }\right)})}{\partial {\mathrm{\gamma }}_j},}\\ \multicolumn{1}{c}{\sum _{k=1}^q\sum _{j=1}^J\left[\frac{{\overline{r}}_{\mathrm{jk}}-{\overline{r}}_{\mathrm{jk}}^{\left(W\right)}-({\overline{f}}_k-{\overline{f}}_{\mathrm{jk}}^{\left(W\right)})G_j^{*}\left(X_k\right)}{G_j^{*}\left(X_k\right)[1-G_j^{*}\left(X_k\right)]}\frac{\partial G_j^{*}\left(X_k\right)}{\partial {\mathrm{\alpha }}^{*}}\right]+\frac{\partial \ln g({\mathrm{\alpha }}^{*}|{\mathrm{\mu }}^{\left({\mathrm{\alpha }}^{*}\right)},{\sigma }^{\left({\mathrm{\alpha }}^{*}\right)})}{\partial {\mathrm{\alpha }}^{*}}}\end{array}\right)}^T,\hfill \end{array}$$ (23)

where

$$G_j^{*}\left(X_k\right)=\frac{1}{1+\exp (-(\exp \left({\mathrm{\alpha }}^{*}\right)X_k+{\mathrm{\gamma }}_j))}.$$ (24)

Maximization Step 1: $({\mathrm{\beta }}_{\mathit{j}},{\mathrm{\gamma }}_{\mathit{j}})$ Parameters

Let ${\mathrm{\lambda }}_{{\mathrm{\beta }}_j}$ and ${\mathrm{\lambda }}_{{\mathrm{\gamma }}_j}$ be the part of Equation 23 with respect to ${\mathrm{\beta }}_j$, ${\mathrm{\gamma }}_j$. Then,

$${\mathrm{\lambda }}_{{\mathrm{\beta }}_j}=-\sum _{k=1}^q[{\overline{r}}_{\mathrm{jk}}^{\left(W\right)}-{\overline{f}}_k{P}_j^{*}\left(X_k\right)]-\frac{{\mathrm{\beta }}_j-{\mathrm{\mu }}_j^{\left(\mathrm{\beta }\right)}}{{\sigma }_j^{\left(\mathrm{\beta }\right)}},$$ (25)

$${\mathrm{\lambda }}_{{\mathrm{\gamma }}_j}=\sum _{k=1}^q[{\overline{r}}_{\mathrm{jk}}-{\overline{r}}_{\mathrm{jk}}^{\left(W\right)}-({\overline{f}}_k-{\overline{f}}_{\mathrm{jk}}^{\left(W\right)})G_j^{*}\left(X_k\right)]-\frac{{\mathrm{\gamma }}_j-{\mathrm{\mu }}_j^{\left(\mathrm{\gamma }\right)}}{{\sigma }_j^{\left(\mathrm{\gamma }\right)}}.$$ (26)

Since ${\mathrm{\beta }}_j$, ${\mathrm{\gamma }}_j$ and $\mathrm{\alpha }$ parameters do not have closed-form solutions, it is necessary to find them numerically. The method used here is the Newton–Raphson (N-R) algorithm for ${(t+1)}^{\mathrm{th}}$ iteration:

$$\left[\begin{array}{c}{\mathrm{\beta }}_j^{(t+1)}\hfill \\ \multicolumn{1}{c}{{\mathrm{\gamma }}_j^{(t+1)}}\end{array}\right]=\left[\begin{array}{c}{\mathrm{\beta }}_j^{\left(t\right)}\hfill \\ \multicolumn{1}{c}{{\mathrm{\gamma }}_j^{\left(t\right)}}\end{array}\right]-{\left[\begin{array}{cc}{\mathrm{\delta }}_{\mathrm{\beta }{\mathrm{\beta }}_j}^{\left(t\right)}\hfill & {\mathrm{\delta }}_{\mathrm{\beta }{\mathrm{\gamma }}_j}^{\left(t\right)}\hfill \\ \multicolumn{1}{c}{{\mathrm{\delta }}_{\mathrm{\beta }{\mathrm{\gamma }}_j}^{\left(t\right)}}& {\mathrm{\delta }}_{\mathrm{\gamma }{\mathrm{\gamma }}_j}^{\left(t\right)}\hfill \end{array}\right]}^{-1}\left[\begin{array}{c}{\mathrm{\lambda }}_{{\mathrm{\beta }}_j}^{\left(t\right)}\hfill \\ \multicolumn{1}{c}{{\mathrm{\lambda }}_{{\mathrm{\gamma }}_j}^{\left(t\right)}}\end{array}\right],$$ (27)

where ${\mathrm{\delta }}_j$ is the second derivatives of the expected log-likelihood function for parameters listed in the subscript:

$${\mathrm{\delta }}_{\mathrm{\beta }{\mathrm{\beta }}_j}=\frac{\partial {\mathrm{\lambda }}_{{\mathrm{\beta }}_j}}{\partial {\mathrm{\beta }}_j}=-\sum _{k=1}^q\left[{\overline{f}}_k{P}_j^{*}\left(X_k\right)(1-P_j^{*}\left(X_k\right))\right]-\frac{1}{{\sigma }_j^{\left(\mathrm{\beta }\right)}},$$ (28)

$${\mathrm{\delta }}_{\mathrm{\gamma }{\mathrm{\gamma }}_j}=\frac{\partial {\mathrm{\lambda }}_{{\mathrm{\gamma }}_j}}{\partial {\mathrm{\gamma }}_j}=-\sum _{k=1}^q\left[({\overline{f}}_k-{\overline{f}}_{\mathrm{jk}}^{\left(W\right)})G_j^{*}\left(X_k\right)(1-G_j^{*}\left(X_k\right))\right]-\frac{1}{{\sigma }_j^{\left(\mathrm{\gamma }\right)}}.$$ (29)

$${\mathrm{\delta }}_{\mathrm{\beta }{\mathrm{\gamma }}_j}=\frac{\partial {\mathrm{\lambda }}_{{\mathrm{\beta }}_j}}{\partial {\mathrm{\gamma }}_j}=0.$$ (30)

Although $\mathrm{\beta }$ and $\mathrm{\gamma }$ parameters are maximized in the same N-R cycle, it can be regarded as two independent procedures because ${\mathrm{\delta }}_{\mathrm{\beta }{\mathrm{\gamma }}_j}=0$. Thus, Equation 27 can be rewritten as

$$\{\begin{array}{c}{\mathrm{\beta }}_j^{(t+1)}={\mathrm{\beta }}_j^{\left(t\right)}-{\left[{\mathrm{\delta }}_{\mathrm{\beta }{\mathrm{\beta }}_j}^{\left(t\right)}\right]}^{-1}\times {\mathrm{\lambda }}_{{\mathrm{\beta }}_j}^{\left(t\right)}\hfill \\ \multicolumn{1}{c}{{\mathrm{\gamma }}_j^{(t+1)}={\mathrm{\gamma }}_j^{\left(t\right)}-{\left[{\mathrm{\delta }}_{\mathrm{\gamma }{\mathrm{\gamma }}_j}^{\left(t\right)}\right]}^{-1}\times {\mathrm{\lambda }}_{{\mathrm{\gamma }}_j}^{\left(t\right)}}\end{array}.$$ (31)

In addition, the above-mentioned procedure can be also applied to estimate the 1PL-G model with some minor modifications.

Maximization Step 2: $\mathrm{\alpha }$-Parameters

Unlike $\mathrm{\beta }$ and $\mathrm{\gamma }$ parameters, the first and second derivative of ${\mathrm{\alpha }}^{*}$ involves the entire set of items, so it is impossible to maximize ${\mathrm{\alpha }}^{*}$ with $({\mathrm{\beta }}_j,{\mathrm{\gamma }}_j)$ in one Hessian matrix in the family of EM algorithm. In this case, following Park et al. (2015), the solution of $\mathrm{\beta }$ and $\mathrm{\gamma }$ for each item is found, and then maximize ${\mathrm{\alpha }}^{*}$ parameter. Let ${\mathrm{\lambda }}_{{\mathrm{\alpha }}^{*}}$ and ${\mathrm{\delta }}_{{\mathrm{\alpha }}^{*}{\mathrm{\alpha }}^{*}}$ be the part of Equation 23 and corresponding second derivative with respect to ${\mathrm{\alpha }}^{*}$, the $(t+1{)}^{\mathrm{th}}$ estimates in N-R iteration can be obtained from

$${\mathrm{\alpha }}_{(t+1)}^{*}={\mathrm{\alpha }}_{\left(t\right)}^{*}-{\left[{\mathrm{\delta }}_{{\mathrm{\alpha }}^{*}{\mathrm{\alpha }}^{*}}^{\left(t\right)}\right]}^{-1}\times {\mathrm{\lambda }}_{{\mathrm{\alpha }}^{*}}^{\left(t\right)},$$ (32)

where

$${\mathrm{\lambda }}_{{\mathrm{\alpha }}^{*}}=\left[\sum _{k=1}^q\sum _{j=1}^J\exp \left({\mathrm{\alpha }}^{*}\right)X_k({\overline{r}}_{\mathrm{jk}}-{\overline{r}}_{\mathrm{jk}}^{\left(W\right)}-({\overline{f}}_k-{\overline{f}}_{\mathrm{jk}}^{\left(W\right)})G_j^{*}\left(X_k\right))\right]-\frac{{\mathrm{\alpha }}^{*}-{\mathrm{\mu }}_j^{\left({\mathrm{\alpha }}^{*}\right)}}{{\sigma }^{\left({\mathrm{\alpha }}^{*}\right)}},$$ (33)

$${\mathrm{\delta }}_{{\mathrm{\alpha }}^{*}{\mathrm{\alpha }}^{*}}=-\sum _{k=1}^q\sum _{j=1}^J\exp \left(2{\mathrm{\alpha }}^{*}\right)X_k^2\left[({\overline{f}}_k-{\overline{f}}_{\mathrm{jk}}^{\left(W\right)})G_j\left(X_k\right)(1-G_j\left(X_k\right))\right]-\frac{1}{{\sigma }^{\left({\mathrm{\alpha }}^{*}\right)}}.$$ (34)

Once the procedure of BEMM converged, the estimate of $\mathrm{\alpha }$ is equal to $\exp ({\mathrm{\alpha }}^{*})$.

Standard Errors of Parameter Estimation in the BEMM

An important index of estimation quality is the standard error (SE). However, one major criticism of the EM algorithm is that parameter estimate SEs are not a natural product of the algorithm, and so other methods have to be devised (McLachlan & Krishnan, 2007). The BEMM falls prey to this criticism, just as all members of the EM algorithm family do. As such, it is necessary to compute SEs separately from the estimation algorithm itself.

Although the inverse of the negative expected value of the matrix of second derivatives of the log-likelihood (i.e., the inverse of the information matrix) is the simplest way to obtain the covariances, it can lead to serious bias and underestimate SEs (Kendall & Stuart, 1961). Cai and Lee (2009) proposed the Supplemented EM (SEM) method to offer a robust covariance matrix of estimates in the EM algorithm, and Tian et al. (2013) further speeded up that process and proposed the Updated SEM (USEM) method. Thus, the USEM method is used to obtain SEs, which can provide accurate and robust SEs for IRT model estimation.

Based on the general large-sample covariance matrix of MLEs proposed by Cai and Lee (2009), the standard error of BMEs in the 1PL-AG model should be:

$$\begin{array}{c}(S{E}_{{\mathrm{\beta }}_j},S{E}_{{\mathrm{\gamma }}_j})=\sqrt{\mathrm{diag}\left({\left[\begin{array}{cc}-{\mathrm{\delta }}_{\mathrm{\beta }{\mathrm{\beta }}_j}\hfill & 0\hfill \\ \multicolumn{1}{c}{0}& -{\mathrm{\delta }}_{\mathrm{\gamma }{\mathrm{\gamma }}_j}\hfill \end{array}\right]}^{-1}{[I_2-\mathrm{\Delta }(\mathrm{\beta },\mathrm{\gamma })]}^{-1}\right)},\hfill \\ S{E}_{{\mathrm{\alpha }}^{*}}=\sqrt{-{\mathrm{\delta }}_{{\mathrm{\alpha }}^{*}{\mathrm{\alpha }}^{*}}^{-1}\times {(1-\mathrm{\Delta }\left({\mathrm{\alpha }}^{*}\right))}^{-1}},S{E}_{\mathrm{\alpha }}=\sqrt{\exp \left(2{\mathrm{\alpha }}^{*}\right){\mathrm{SE}}_{{\mathrm{\alpha }}^{*}}^2}.\hfill \end{array}$$ (35)

where $I_2$ is a 2×2 identity matrix, $\mathrm{\Delta }({\mathrm{\psi }}^{*})$ is the rate function with respect to the estimated parameter ${\mathrm{\psi }}^{*}$, $S{E}_{\mathrm{\alpha }}$ is transformed from $S{E}_{{\mathrm{\alpha }}^{*}}$ by using Delta method (Oehlert, 1992). The calculation of $\mathrm{\Delta }({\mathrm{\psi }}^{*})$ not only requires the temporary estimates in each BEMM iteration history but also needs dozens of additional BEMM cycles. Please refer to Cai and Lee (2009) for the details of $\mathrm{\Delta }({\mathrm{\psi }}^{*})$, Tian et al. (2013) for the USEM method, and Online Appendix B for the specific implementation.

Simulation Studies

This section presents Monte Carlo results regarding the accuracy of the developed BEMM procedure for estimating 1PL-AG model item parameters, compared with MMLE/EM (Park et al., 2015) and MCMC in JAGS (Plummer, 2003). The goal of the simulation studies was to assess the influence of various design features, including the sizes of the weighted guessing component on ability, the sample sizes, and the priors. To achieve this goal, two implementations were programmed for BEMM and MMLE/EM algorithms in the MATLAB environment.

Item and Person Parameter Generation

The item parameters were generated from the same distribution as in Park et al. (2015). Specifically, item parameters for 20 items were generated from $\mathrm{\beta }~N(0.641,1.021)$ and $\mathrm{\gamma }~N(-1.223,0.644)$, and these item parameters were kept consistent to facilitate comparison (the generated parameters are listed in Online Appendix C). Since the Bayesian methods are more powerful in the modest or relatively small sample sizes, the ability parameters were drawn from the standard normal distribution with the sample size of $(500,1000,2500)$ across three weighted guessing component conditions $\mathrm{\alpha }\in (0.065,0.265,0.365)$.

Priors

Two types of priors are applied in the simulation study, the true priors and the commonly-used priors. The true priors for $\mathrm{\beta }$ and $\mathrm{\gamma }$ are the same as the true parameter generation distribution given in the last section, which can reflect the parameter recovery in the ideal situation. However, there are no true priors in the real-world environment, so the commonly-used priors are provided here. Based on the recommendation of BILOG-MG (du Toit, 2003) for $\mathrm{\beta }$ parameters and flexMIRT (Houts & Cai, 2015) for the logistic scaled guessing parameters of 3PLM, the priors for $\mathrm{\beta }$ and $\mathrm{\gamma }$ parameters were set as follows: $\mathrm{\beta }~N(0,2^2)$ and $\mathrm{\gamma }~N(-1.39,{0.5}^2)$. Besides, ${\mathrm{\alpha }}^{*}=\ln (\mathrm{\alpha })~N(-1.90,1)$ was set as the prior of ${\mathrm{\alpha }}^{*}$ to cover the reasonable range of $\mathrm{\alpha }$, which is usually distributed between $[0,0.707)$(San Martín et al., 2006).

The Setting of BEMM and MMLE/EM in MATLAB

In both BEMM and MMLE/EM, 61 quadrature points were set to approximate the Gaussian distribution from −6 to +6 in all simulation studies and real-data examples, and use the same starting values for each type of parameters: ${\mathrm{\alpha }}^{*}=\ln (0.15)=-1.90$, $\mathrm{\beta }=0$ and $\mathrm{\gamma }=-1.39$. Both algorithms use the USEM method to obtain the standard errors of parameter estimates. The convergence criteria for the E-step and M-step in BEMM and MMLE/EM algorithms are 0.0001 and 0.01, respectively. In addition, both types of priors are employed in BEMM.

The Setting of JAGS

Four chains were run parallelly with 10,000 iterations (the first 5,000 were discarded for burn-in) per chain for JAGS. The convergence criterion for all following conditions is the Gelman-Rubin potential scale reduction factor $\hat{R}<1.1$ (Gelman & Rubin, 1992), which can be computed by multiple independent chains with random starting values. Since the MCMC presented here is to be a reference as another Bayesian method, only true priors are employed for JAGS to obtain Bayesian estimates in an ideal situation.

In sum, 36 conditions were simulated: 4 estimation method conditions (BEMM with true priors [TP], BEMM with commonly-used priors [CP], MMLE/EM, MCMC with TP) each crossed with three examinee sample size conditions (500, 1,000, and 2,500) and three weighted guessing component parameters (0.065, 0.265, and 0.365). For each condition, 100 replications were run to reduce sampling error. Each condition was assessed by its accuracy in recovering item parameters as defined by bias, the root mean squared error (RMSE), and the average of standard error (ASE) across the 100 replications, where

$$\mathrm{bias}=\frac{{\sum }_{s=1}^{S=100}({\widehat{\mathit{ѱ}}}_j^{\left(s\right)}-{\mathbf{\Psi }}_j)}{S},\mathrm{RMSE}=\sqrt{\frac{{\sum }_{s=1}^{S=100}{({\widehat{\mathit{ѱ}}}_j^{\left(s\right)}-{\mathbf{\Psi }}_j)}^2}{S}},\mathrm{ASE}=\frac{{\sum }_{s=1}^{S=100}{\mathrm{SE}}_{{\widehat{\mathit{ѱ}}}_j}^{\left(s\right)}}{S}.$$ (36)

Results

Figures 1, 2, and 3 present of RMSEs for each item in three $\mathrm{\alpha }$ conditions (0.065, 0.265, 0.365), respectively. Table 1 summarizes the mean of the bias, RMSEs, and ASEs across 100 replications in 36 conditions.

Figure 1.

The RMSEs of item parameter recovery for 500, 1,000, and 2,500 examinees when α = 0.065.

Note. RMSE = root mean squared error.

Figure 2.

The RMSEs of item parameter recovery for 500, 1,000, and 2,500 examinees when α = 0.265.

Note. RMSE = root mean squared error.

Figure 3.

The RMSEs of item parameter recovery for 500, 1,000, and 2,500 examinees when α = 0.365.

Note. RMSE = root mean squared error.

Table 1.

The Mean of the Bias, RMSEs, and ASEs Across 100 Replications in 36 Conditions.

Conditions			α			β			γ
Indicator	True α	Methods	500	1,000	2,500	500	1,000	2,500	500	1,000	2,500
Bias	.065	BEMM-TP	.01	.01	.01	.01	.02	.02	.05	.06	.06
		BEMM-CP	−.00	−.00	.00	−.04	−.02	.00	.11	.09	.07
		MCMC-TP	.03	.02	.02	.03	.03	.03	.07	.07	.06
		MMLE/EM	.00	.00	−.01	.12	.06	.02	−.21	−.16	−.11
	.265	BEMM-TP	−.09	−.07	−.06	−.08	−.07	−.06	.02	.01	.00
		BEMM-CP	−.11	−.09	−.07	−.12	−.11	−.09	−.04	−.04	−.03
		MCMC-TP	−.02	−.02	−.01	−.10	−.08	.05	.03	.03	.03
		MMLE/EM	−.11	−.10	−.08	−.16	−.14	−.11	−.32	−.24	−.22
	.365	BEMM-TP	−.11	−.08	−.06	−.09	−.08	−.07	.01	.01	.01
		BEMM-CP	−.11	−.08	−.07	−.14	−.13	−.10	−.06	−.05	−.03
		MCMC-TP	−.03	−.02	−.02	−.10	−.08	−.08	.02	.02	.03
		MMLE/EM	−.16	−.14	−.11	−.19	−.18	−.15	−.38	−.27	−.24
RMSE	.065	BEMM-TP	.02	.02	.02	.31	.24	.19	.38	.31	.25
		BEMM-CP	.01	.02	.02	.32	.25	.19	.35	.28	.23
		MCMC-TP	.04	.04	.03	.32	.25	.19	.36	.29	.24
		MMLE/EM	.08	.07	.05	.51	.36	.21	.86	.64	.43
	.265	BEMM-TP	.10	.08	.07	.34	.28	.23	.42	.36	.30
		BEMM-CP	.13	.10	.08	.36	.29	.23	.39	.33	.28
		MCMC-TP	.06	.05	.04	.34	.28	.22	.40	.34	.28
		MMLE/EM	.15	.14	.12	.70	.54	.37	1.05	.81	.64
	.365	BEMM-TP	.13	.10	.08	.36	.31	.25	.45	.39	.33
		BEMM-CP	.13	.11	.09	.38	.31	.25	.42	.36	.30
		MCMC-TP	.07	.05	.04	.37	.30	.24	.42	.37	.31
		MMLE/EM	.18	.16	.14	.86	.73	.49	1.19	.96	.78
ASE	.065	BEMM-TP	.04	.03	.02	.33	.26	.19	.40	.33	.26
		BEMM-CP	.03	.03	.02	.30	.24	.18	.34	.29	.23
		MCMC-TP	.06	.05	.04	.33	.26	.20	.37	.31	.26
		MMLE/EM	.04	.03	.02	.32	.23	.16	.41	.32	.25
	.265	BEMM-TP	.05	.04	.02	.33	.27	.21	.44	.39	.32
		BEMM-CP	.05	.04	.03	.29	.24	.19	.37	.32	.28
		MCMC-TP	.07	.06	.05	.34	.28	.22	.41	.36	.30
		MMLE/EM	.05	.04	.03	.37	.27	.20	.44	.36	.29
	.365	BEMM-TP	.05	.04	.03	.34	.29	.23	.47	.41	.34
		BEMM-CP	.06	.04	.03	.30	.25	.22	.39	.35	.31
		MCMC-TP	.08	.06	.05	.36	.30	.24	.43	.38	.33
		MMLE/EM	.05	.04	.03	.38	.29	.22	.46	.38	.31

Note. RMSE = root mean squared error; ASE = average of standard error; BEMM-TP = Bayesian expectation-maximization-maximization-true priors; BEMM-CP = Bayesian expectation-maximization-maximization-commonly-used priors; MCMC-TP = Markov Chain Monte Carlo-true priors; MMLE/EM = marginal maximum likelihood estimation/expectation-maximization.

In general, with the increasing of the sample size and the decreasing of the weighted guessing component, the absolute values of bias and RMSEs of BEMM, MCMC, and MMLE/EM gradually decrease, although the differences between $\mathrm{\alpha }=0.365$ and $\mathrm{\alpha }=0.265$ are relatively small in some conditions. The bias and RMSEs of BEMM and MCMC are obviously better than MMLE/EM, especially for the $\mathrm{\gamma }$ parameters in the sample size of 500 and 1,000. Indicated by the figures, some RMSEs of MMLE/EM for $\mathrm{\gamma }$ are quite large.

Among the three Bayesian estimation methods, the MCMC with true priors recovers $\mathrm{\alpha }$ parameter better than BEMM with true priors in the condition of $\mathrm{\alpha }=0.265$ and $\mathrm{\alpha }=0.365$, but their other estimates are comparable in the same condition. Compared with BEMM with commonly-used priors, BEMM with true priors has relatively better performance in estimating $\mathrm{\beta }$. Furthermore, the item recovery of $\mathrm{\gamma }$ shows a mixed result: The bias from true priors is smaller than those from commonly-used priors while the situation of RMSEs is opposite. Despite this, both priors can offer acceptable bias and RMSEs for $(\mathrm{\beta },\mathrm{\gamma },\mathrm{\alpha })$.

As can be seen, the estimated standard errors among BEMM, MMLE, and MCMC are decreasing with the increase of the sample size, and they are similar in the same condition. For MMLE/EM, all ASEs are smaller than their corresponding RMSEs, but this relationship is unstable for the Bayesian method due to the interference of priors. In addition, the running time for three methods is also summarized in Table 2. It can be seen that the running time of BEMM is very close to MMLE/EM, and both of them are obviously less than MCMC in JAGS.

Table 2.

The Mean of the Running Time (Minute) Across 100 Replications in 36 Conditions.

Methods	0.065			0.265			0.365
	500	1,000	2,000	500	1,000	2,000	500	1,000	2,000
BEMM-TP	.05	.09	.14	.05	.12	.18	.05	.09	.22
BEMM-CP	.04	.09	.15	.05	.12	.19	.06	.11	.21
MCMC-TP	40.30	118.13	234.12	42.48	121.81	241.87	41.55	119.44	244.29
MMLE/EM	.03	.02	.09	.06	.08	.11	.06	.08	.15

Note. BEMM-TP = Bayesian expectation-maximization-maximization-true priors; BEMM-CP = Bayesian expectation-maximization-maximization-commonly-used priors; MCMC-TP = Markov Chain Monte Carlo-true priors; MMLE/EM = marginal maximum likelihood estimation/expectation-maximization.

Based on these results, several conclusions can be drawn: (a) BEMM only requires 1,000 examinees to obtain relatively stable estimates (500 examinees are barely acceptable if necessary) while MMLE/EM requires at least 2,500 examinees. (b) The item recovery of $\mathrm{\gamma }$ in MMLE/EM is obviously worse than $\mathrm{\beta }$’s, and MMLE/EM cannot recovery $\mathrm{\gamma }$ well in the small sample size. (c) The estimated standard errors yielded by the USEM method are reasonable and sensitive to the change of the sample size. (d) The commonly-used priors in this study work well in calibrating the item parameters in the 1PL-AG model. (e) The bias and RMSEs of MMLE/EM in the sample size of 2,500 replicate the results of Park et al. (2015) and serve as the baseline condition in the simulation.

An Empirical Example

The 1PL-AG model is applied to the flexMIRT example data set “g341-19.dat” which consists of the responses to 12 items from 2,844 examinees. In a prior study, Guo and Zheng (2019) calibrated the items using the 3PLM and found the difficulty of the whole test is relatively low. In this case, the authors believe this example can be a necessary complement to our simulation studies which are designed for a high difficulty situation. To further demonstrate the feasibility of BEMM in the 1PL-AG model calibration, the authors randomly sample 500 and 1,000 examinees from the full data set. In sum, the point estimates and the estimated standard errors yielded by BEMM, MCMC in JAGS, and MMLE/EM will be compared among the sample size of 500, 1,000 and 2,844 examinees. The commonly-used priors in the last section will be used in BEMM and JAGS.

As can be seen from Figure 4, both Bayesian methods (BEMM and JAGS) have very similar point estimates and corresponding SEs under the same conditions. Furthermore, most of the point estimates of $\mathrm{\alpha }$ and $\mathrm{\beta }$ are also similar among BEMM, JAGS, and MMLE/EM across the three sample sizes. However, the point estimates for γs of MMLE/EM show an obvious fluctuation with the increasing of the sample size while those estimates yielded by BEMM and JAGS are more stable and very closed to the results of the complete data set. Meanwhile, the estimated standard error generally tends to decrease with the increase of sample size among the three methods. These results are consistent with the simulation studies. In addition, compared with BEMM and JAGS, MMLE/EM has larger SEs in $\mathrm{\beta }$ and $\mathrm{\gamma }$ but smaller SEs in $\mathrm{\alpha }$.

Figure 4.

The point estimates and corresponding SEs for α, β, and γ.

To further demonstrate the usefulness of 1PL-AG model, a comparison between 1PL-AG and 1PL-G models is made for the full data set. It can be seen from Table 3 that the 1PL-AG model has smaller Akaike information criterion (AIC) and Bayesian information criterion (BIC) than 1PL-G model, so the 1PL-AG model fits the data better. Furthermore, the significant result of the likelihood ratio (LR) test actually indicates that the $\mathrm{\alpha }$ parameter is far more than 0, which offers a substantively interesting interpretation that correctly guessing is related to ability. In this case, the 1PL-G model does not fit these data well.

Table 3.

The Model Fits Index and Likelihood Ratio Test.

Indicator	Model		LR test between two models
	1PL-G	1PL-AG	χ2	df	Significance
−2loglikehood	33508.75	33456.62	52.14	1	<.001
AIC	33556.75	33506.62
BIC	33699.63	33655.44

Note. 1PL-G = one-parameter logistic guessing; 1PL-AG = one-parameter logistic ability-based guessing; LR = likelihood ratio; AIC = Akaike information criterion; BIC = Bayesian information criterion.

Discussion

In recent years, interest in the 1PL-AG model has been on the rise in measurement research. The current study presents a new Bayesian formulation of the 1PL-AG model and shows an approach to the estimation that has the potential to open the gates for its wide application among applied researchers. This section summarizes the current study’s contributions and discusses future research directions.

First, this study offers a fast and stable Bayesian modal estimation for the 1PL-AG model in a modest sample size. One major barrier to a wide application in educational and psychological research is a user-friendly package for the 1PL-AG model. Through the simulation studies and a real data analysis, the authors demonstrated that BEMM could provide stable estimates in a modest sample size. To facilitate calibration, the authors have made available a MATLAB package online (see Online Appendix B). The authors believe this package offers a better alternative for 1PL-AG model calibration, helping to promote the use of the 1PL-AG model in educational and psychological research, especially in the large-scale assessment (Beghetto, 2019).

Second, this study offers a mixture modeling reformulation of the 1PL-AG model, aligning the 1PL-AG model in a general framework in IRT literature like those given for the 3PLM (Zheng et al., 2018), the 4PLM (Zhang et al., 2018), the three-parameter normal ogive model (3PNO; Béguin & Glas, 2001) and the 4PNO (Culpepper, 2015). This unified framework connects the IRT models with guessing effects with a well-known framework in statistics—namely, the mixture model—which can potentially serve to spur on new possibilities for modeling and parameter estimation methods within IRT. For instance, more complexities in modeling guessing and slipping effects can be easily incorporated within this framework. From a broader perspective, the 1PL-AG model is a special case of the hybrid model (von Davier & Rost, 2006; Yamamoto, 1982), which can be seen if weights are selected as the guessing parameters, and densities chosen to be the degenerate density and the 1PLM, showing that this mixture modeling perspective can indeed be extended easily. On the parameter estimation side, the most widely used algorithm in mixture modeling is the EM algorithm, serving as the main inspiration for the current study.

In contrast to the BEMM algorithm for 3PLM (Guo & Zheng, 2019), there is an obvious difference between two algorithms, even though both of them belong to the mixture-modeling framework, share the similar E-step and the same name. In the 3PLM, the first M-step is to calculate the guessing parameter via a closed-form solution, and then maximize the difficulty and discrimination parameters in the second M-step. However, in the 1PL-AG model, the first M-step is designed for the difficulty and logistic-scaled guessing parameters (although their estimation is independent of each other), and the second M-step is to obtain the $\mathrm{\alpha }$ parameter, which cannot be absorbed into the first M-step since the $\mathrm{\alpha }$ parameter is for all items.

Finally, several potential directions can be studied in future: (a) It is interesting to investigate the effects of different priors on the accuracy of estimation; (2) it is possible to extend BEMM to the 2PL-AG or ability-based slipping model; (3) it is a potential direction to compare BEMM with other Bayesian methods like Hamiltonian Monte Carlo.