Approximating Bifactor IRT True-Score Equating With a Projective Item Response Model

Abstract

Item response theory (IRT) true-score equating for the bifactor model is often conducted by first numerically integrating out specific factors from the item response function and then applying the unidimensional IRT true-score equating method to the marginalized bifactor model. However, an alternative procedure for obtaining the marginalized bifactor model is through projecting the nuisance dimensions of the bifactor model onto the dominant dimension. Projection, which can be viewed as an approximation to numerical integration, has an advantage over numerical integration in providing item parameters for the marginalized bifactor model; therefore, projection could be used with existing equating software packages that require item parameters. In this paper, IRT true-score equating results obtained with projection are compared to those obtained with numerical integration. Simulation results show that the two procedures provide very similar equating results.

In item response theory (IRT) true-score equating, the true score (i.e., expected score) associated with a given $\theta -\mathrm{value}$ on the new form is equated to the true score associated with the same $\theta -\mathrm{value}$ on the old form. For unidimensional IRT (UIRT) models, the true-score equating procedure is quite straightforward to implement because there is a one-to-one correspondence between the values of $\theta$ and true scores. By contrast, multiple $\mathbf{\theta }-\mathrm{vectors}$ are related to the same true score for multidimensional IRT (MIRT) models, which makes it difficult to apply the true-score equating procedure to MIRT models. For the bifactor model, however, assuming that specific factors are nuisance variables, one common approach that has been used to overcome the aforementioned difficulty is to integrate out the specific factors from the item response function and equate two test forms based solely on the general factor (e.g., Lee et al., 2015). For each item, the specific factor ${\theta }_s$ is integrated out from the item response function as

$$\begin{array}{c}P(U_j=1|{\theta }_0)=\underset{-\infty }{\overset{\infty }{\int }}P(U_j=1|{\theta }_0,{\theta }_s)h\left({\theta }_s\right)\mathrm{d}{\theta }_s,\hfill \end{array}$$ (1)

where ${\theta }_0$ is the general factor, $P(U_j=1|{\theta }_0,{\theta }_s)$ is the item response function for the bifactor model, and $h({\theta }_s)$ is the probability density function of {{\theta }_s}}. This procedure for obtaining $P(U_j=1|{\theta }_0)$ will hereinafter be referred to as the “exact” method. A numerical method can be used to solve Equation 1. Although a unidimensional test characteristic curve (TCC) can be obtained by adding $P(U_j=1|{\theta }_0)$ across all items, it is difficult to use the resulting TCC for equating with existing IRT equating software packages because most packages require item parameters, not the TCC itself.

An alternative approach to obtain $P(U_j=1|{\theta }_0)$ is through projecting the nuisance dimensions of the bifactor model onto the dominant dimension (see Ip & Chen, 2012, for more details about projection). Such a projection results in a locally dependent UIRT model that is empirically indistinguishable from the original MIRT model (Ip, 2010). Projecting the specific factors onto the dimension of the general factor yields

$$\begin{array}{c}P(U_j=1|{\theta }_0)=c_j^{*}+\frac{1-c_j^{*}}{1+\exp (a_j^{*}{\theta }_0+d_j^{*})},\hfill \end{array}$$ (2)

where

$$\begin{array}{c}{a}_j^{*}=\lambda (a_{j0}+\frac{a_{\mathrm{js}}\rho {\mathrm{\sigma }}_s}{{\sigma }_0}),d_j^{*}=\lambda d_j,c_j^{*}=c_j,\hfill \end{array}$$ (3)

with $\mathrm{\lambda }=[k^2{a}_{\mathrm{js}}^2(1-{\rho }^2){\sigma }_s^2+1]$ and $k=16\sqrt{3}/(15\pi )$. In Equation 3, $a_{j0}$ and $a_{\mathrm{js}}$ are the general factor and specific factor slopes for item $j$, respectively; $c_j$ is the pseudo-guessing parameter for item $j$; $d_j$ is the intercept parameter for item $j$; ${\sigma }_0$ and ${\sigma }_s$ are the standard deviations of the general and specific factors, respectively; and $\rho$ is the correlation between the general and specific factors. More details of the aforementioned formulae are available in Ip and Chen (2012). Under the standard bifactor assumption of complete independence, the correlation between the general and specific factors is zero (i.e., $\rho =0$), and the standard deviations are set equal to 1 for the purpose of fixing the unit of measurement along each dimension (i.e., ${\sigma }_0={\sigma }_s=1$). As this procedure can be viewed as an approximation to Equation 1, it will hereinafter be referred to as the “approximation” method. Although the local independence assumption does not hold for the projected model, true-score equating could still be conducted due to the fact that the expected value of the total score $\sum U_j$—in other words, the TCC—is equal to $\sum P(U_j=1|{\theta }_0)$ regardless of whether the item responses are independent or not. The approximation method provides item parameters for a marginalized bifactor item response function and therefore enables the use of existing standalone equating software packages, such as PIE (Hanson & Zeng, 2004), POLYEQUATE (Kolen, 2004), and IRTEQ (Han, 2009), and an R package equateIRT (Battauz, 2015).

Simulation Example

Under the random groups design, equating results obtained with the exact and approximation methods were compared through simulation. Both methods were implemented with a program written in R (R Core Team, 2019) by the authors of this study. Readers interested in obtaining a copy of the IRT equating R package may do so by contacting the authors by e-mail at k_kim9@uncg.edu. As the bifactor model seems appropriate for fitting testlet-based items, two 40-item test forms with five testlets $(S=5)$ of eight items were generated. The group of eight items that belong to a common testlet was set to load on the same specific factor to take account of the residual dependence that exists above and beyond the general dimension. Each of the five testlets was crossed with two levels of local item dependence (LID): low and moderate. The general factor slopes were sampled from a N(1, 0.3) distribution; specific factor slopes were sampled from a U(0.3, 0.6) distribution for the low-LID condition and a U(0.7, 1.0) distribution for the moderate-LID condition; multidimensional item difficulty parameters (MDIFF) for the old and new forms were sampled from N(0, 1) and N(0.1, 1) distributions, respectively; and guessing parameters were sampled from a U(0.05, 0.35) distribution. The MDIFF parameters were converted to intercept parameters adopting Reckase’s (2009, p. 90) formula:

$$\begin{array}{c}\mathrm{MDIFF}=-\frac{d}{\sqrt{a_0^2+a_s^2}},\hfill \end{array}$$ (4)

where $a_0$ is the general factor slope, $a_s$ is the specific factor slope, and $d$ is the intercept parameter.

For both groups, $\theta -\mathrm{vectors}$ for 3,000 simulees were sampled from a $(S+1)-\mathrm{variate}$ standard normal distribution. One hundred data sets were generated for the two LID conditions by fixing the item parameters but sampling different sets of $\theta -\mathrm{vectors}$. For the two test forms, average Cronbach’s alpha across the 100 data sets was approximately .89 for the low-LID condition and approximately .87 for the moderate-LID condition. The mean absolute difference (MAD) between the exact and approximation methods was calculated for each number-correct score over the 100 replications (hereinafter referred to as conditional MAD), which was defined by

$$\begin{array}{c}\mathrm{MAD}\left(x\right)=\frac{1}{100}\sum _{r=1}^{100}|{\hat{e}}_{Y_E}^{\left(r\right)}\left(x\right)-{\hat{e}}_{Y_A}^{\left(r\right)}\left(x\right)|,\hfill \end{array}$$ (5)

where $E$ and $A$ denote the exact and approximation methods, respectively, and ${\hat{e}}_Y(x)$ is the old form equivalent of number-correct score $x$ on the new form.

The conditional MAD plots for the two LID conditions are depicted in Figure 1. The reason that the conditional MADs were zero for scores outside the range of true scores (scores between 0 and 7 and score 40) was because the ad hoc procedure used to equate these scores produced the same equating results for the exact and approximation methods (see, for example, Kolen & Brennan, 2014, for more details about the ad hoc procedure). In general, although the average and maximum MAD increased with increasing LID, the conditional MADs were smaller than .1 across the entire score scale for both LID conditions; specifically, the maximum MAD for the low- and moderate-LID conditions was .0270 and .0856, respectively (see Table 1). Note that number-correct score differences smaller than .5 are typically regarded as practically nonsignificant according to the difference that matters criterion (Dorans & Feigenbaum, 1994).

Figure 1.

Conditional MAD between the exact and approximation methods.

Note. MAD = mean absolute difference.

Table 1.

Average and Maximum MAD Between the Exact and Approximation Methods.

	Average		Maximum
Evaluation measure	Low LID	Moderate LID	Low LID	Moderate LID
MAD	.0135	.0349	.0270	.0856

Note. Average MAD was calculated over scores within the range of true scores. MAD = mean absolute difference; LID = local item dependence.