Obtaining Fixed Effects for Between-Within Designs in Explanatory Longitudinal Item Response Models Using Mplus

Abstract

Between-within designs that include a person group (i.e., a between-subjects factor) and repeated measures of binary responses over time (i.e., a within-subjects factor) are common in educational and psychological research. This software note describes how explanatory item response models can be specified to analyze longitudinal item-level data to detect fixed effects in Mplus for between-within designs. In particular, a necessary parameter transformation is illustrated in detail to obtain the fixed effects in Mplus.

Description

Explanatory item response models (EIRM; De Boeck & Wilson, 2004) can be used when the interest is in the comparison of means between groups on the latent true score scale for longitudinal item-level data. The aim of this note is to present how fixed effects in the between-within design (i.e., interaction effects between the within-subjects factor and the between-subjects factor or the main effect of the between-subjects factor or the main effect of the within-subjects factor) can be estimated in a latent true score scale using Mplus (Muthén & Muthén, 1998-2016). The authors focus on a design that has one between-subjects factor and one within-subjects factor, and a unidimensional test is assumed at each time point.

Motivation: Software Comparisons

In fitting EIRM, the fixed effects for the between-within design can be specified directly in the SAS NLMIXED procedure (e.g., Nandakumar & Hotchkiss, 2012, for item response theory [IRT] parameter estimation), the glmer function in the lme4 package (Bates., Mächler, Bolker, & Walker, 2015), and the FLIRT R package (Jeon, Rijmen, & Rabe-Hesketh, 2014). However, parameter transformation is required to obtain these effects in Mplus because they cannot be estimated directly. Compared with other software packages, Mplus provides flexibility in measurement model specification, estimation method options, and adjustment for stratification (see the supplementary note for details). To take advantage of these flexibilities in Mplus, it may be instructive to show how Mplus parameters can be transformed to fixed effects in between-within designs.

EIRM Specification for Binary Responses

EIRM

A measurement model in EIRM is a four-parameter version of the longitudinal item response model (Anderson, 1985), described as follows:

$$P\left(y_{jgti}=1\mathrm{|}{\theta }_{jgt}\right)=c_i+\left(d_i-c_i\right)\frac{1}{1+\exp \left[-\left(a_i{\theta }_{jgt}-b_i\right)\right]},$$

where y represents the binary item response, ${\theta }_{jgt}$ is a latent variable for person j in a group g at a time point t, $a_i$ is a time-invariant item discrimination for item i, $b_i$ is a time-invariant item location for item i, $c_i$ is a time-invariant lower asymptote item parameter for item i, and $d_i$ is a time-invariant upper asymptote item parameter for item i. Three-parameter EIRM can be obtained with $d_i=0$ in Equation 1. Two-parameter EIRM can be specified with $c_i=0$ and $d_i=0$ in Equation 1. The structural model of EIRM for ${\mathrm{\theta }}_{\mathrm{jgt}}$ includes the effects of the between-subjects factor ($A_g$), the within-subjects factor ($W_t$), and their interaction ($A_g{W}_t$):

$${\theta }_{jgt}={\gamma }_0+{\gamma }_{1g}·A_g+{\gamma }_{2t}·W_t+{\gamma }_{3gt}·A_g{W}_t+{\epsilon }_{jgt},$$

where ${\gamma }_0$ is the fixed parameter for the intercept or the grand mean, ${\mathrm{\gamma }}_{1g}$ is the fixed parameter for the between-subjects factor, ${\mathrm{\gamma }}_{2t}$ is the fixed parameter for the within-subjects factor, ${\mathrm{\gamma }}_{3\mathrm{gt}}$ is the fixed parameter for the interaction between the within-subjects factor and between-subjects factor, and ${\epsilon }_{\mathrm{jgt}}$ is the random residual at each time point, following a multivariate normal (MVN) distribution (${\mathrm{\epsilon }}_{\mathrm{jg}}=[{\epsilon }_{\mathrm{jg}1},\dots ,{\epsilon }_{\mathrm{jgT}}]\prime ~\mathrm{MVN}(0_{(T\times 1)},{\mathrm{\Sigma }}_{(T\times T)})$). When the item location parameter ($b_i$ in Equation 1) is a fixed effect, the grand mean (${\mathrm{\gamma }}_0$ in Equation 2) cannot be estimated. The grand mean is the average of the item location parameter estimates across items. Thus, it can be calculated as ${\mathrm{\gamma }}_0=\sum _{i=1}^I\widehat{b_i}/I$.

Specifying fixed effects in EIRM

With the unweighted effect codes of $A_g$, $W_t$, and $A_g{W}_t$ (an example of the unweighted effect codes is provided in the supplementary note), the fixed effects (${\mathrm{\gamma }}_{1g}$, ${\mathrm{\gamma }}_{2t}$, and ${\mathrm{\gamma }}_{3\mathrm{gt}}$) can be directly obtained. EIRM is identified with the fixed effects using effect codes: $\sum _{g=1}^G{\mathrm{\gamma }}_{1g}=0$, $\sum _{t=1}^T{\mathrm{\gamma }}_{2t}=0$, $\sum _{g=1}^G{\mathrm{\gamma }}_{3\mathrm{gt}}=0$ for all $t=1,\dots ,T$, and $\sum _{t=1}^T{\mathrm{\gamma }}_{3\mathrm{gt}}=0$ for all $g=1,\dots ,G$.

In addition to the constraints, an additional model identification constraint on item discriminations is required for two-parameter or three-parameter or four-parameter EIRM. In this study, item discrimination for one of the items (e.g., the first item) is fixed to 1.

Implementation in Mplus

The measurement model specified in Equation 1 can be specified in Mplus. The structural model in Mplus is first described and then the parameter transformation is specified to obtain the fixed effects, ${\mathrm{\gamma }}_{1g}$, ${\mathrm{\gamma }}_{2t}$, and ${\mathrm{\gamma }}_{3\mathrm{gt}}$, in Equation 2 based on the parameters of the structural model in Mplus, as described below.

In the structural model, a covariate, $A_g$ (i.e., the between-subjects factor), can be added to explain ${\mathrm{\theta }}_{\mathrm{jgt}}$:

$${\theta }_{jgt}={\mu }_t+{\omega }_{gt}·A_g+{\epsilon }_{jgt},$$

where ${\mu }_t$ is a mean at each time point, ${\mathrm{\omega }}_{\mathrm{gt}}$ is the fixed parameter for $A_g$ at each time point, and ${\epsilon }_{\mathrm{jgt}}$ is the random residual at each time point.

Specifying fixed effects in Mplus

The following three constraints can be set to obtain ${\mathrm{\gamma }}_{1g}$, ${\mathrm{\gamma }}_{2t}$, and ${\mathrm{\gamma }}_{3\mathrm{gt}}$ in Equation 2, using parameters in the structural model with Mplus, ${\mathrm{\mu }}_t$ and ${\mathrm{\omega }}_{\mathrm{gt}}$ in Equation 3: (a) $\sum _{t=1}^T{\mathrm{\omega }}_{\mathrm{gt}}/T={\mathrm{\gamma }}_{1g}$, (b) ${\mathrm{\mu }}_t={\mathrm{\gamma }}_{2t}$ with a constraint, ${\mathrm{\mu }}_T=-\sum _{t=1}^{T-1}{\mathrm{\mu }}_t$, and (c) in unweighted effect coding, ${\mathrm{\gamma }}_{3\mathrm{gt}}$ represents the discrepancy of the corresponding group mean (${\mathrm{\omega }}_{\mathrm{gt}}$) from the unweighted average group mean (${\mathrm{\gamma }}_{1g}$): ${\mathrm{\omega }}_{\mathrm{gt}}={\mathrm{\gamma }}_{3\mathrm{gt}}+{\mathrm{\gamma }}_{1g}\leftrightarrow {\mathrm{\gamma }}_{3\mathrm{gt}}={\mathrm{\omega }}_{\mathrm{gt}}-{\mathrm{\gamma }}_{1g}$.

Testing fixed effects

To determine whether the fixed effects in EIRM are statistically significant, null hypothesis significance testing for the effect has been used in most EIRM applications (De Boeck & Wilson, 2004). Test statistics (i.e., the z-statistic in the current study) and corresponding p values are used to test whether the population value of the effect differs from a specified value (generally 0). Standard errors of the derived estimates for ${\mathrm{\gamma }}_{1g}$, ${\mathrm{\gamma }}_{2t}$, and ${\mathrm{\gamma }}_{3\mathrm{gt}}$ are calculated using a delta method in Mplus (support@statmodel.com, personal communication, April 19, 2016).

Availability

A supplementary note including an illustrative example, Mplus code, its detailed description, and an example dataset is available at no charge by sending a request by emailing a request to sj.cho@vanderbilt.edu.