Pairwise likelihood estimation and limited‐information goodness‐of‐fit test statistics for binary factor analysis models under complex survey sampling

Abstract

This paper discusses estimation and limited‐information goodness‐of‐fit test statistics in factor models for binary data using pairwise likelihood estimation and sampling weights. The paper extends the applicability of pairwise likelihood estimation for factor models with binary data to accommodate complex sampling designs. Additionally, it introduces two key limited‐information test statistics: the Pearson chi‐squared test and the Wald test. To enhance computational efficiency, the paper introduces modifications to both test statistics. The performance of the estimation and the proposed test statistics under simple random sampling and unequal probability sampling is evaluated using simulated data.

1. INTRODUCTION

Latent variable models, such as factor analysis, are widely used in social sciences to measure abilities, attitudes, and behaviour through observed categorical (binary, ordinal) or metric variables, also known as items or indicators. There are two main approaches for modelling categorical observed variables with latent variables, namely the complete information maximum likelihood (FIML) approach used in item response theory (e.g. Bartholomew et al., ²⁰¹¹; Skrondal & Rabe‐Hesketh, ²⁰⁰⁴) and the limited‐information approach used in structural equation modelling (SEM) (e.g. Jöreskog, ¹⁹⁹⁰; Muthén, ¹⁹⁸⁴). The latter uses first‐ and second‐order statistics in univariate and bivariate likelihood functions. The limited‐information approach and, in particular, pairwise maximum likelihood (PML) are adopted here (Jöreskog & Moustaki, 2001; Katsikatsou, ²⁰¹³; Katsikatsou et al., ²⁰¹²; Liu, ²⁰⁰⁷; Xi, ²⁰¹¹). PML is a special case of composite maximum likelihood (CML) estimation (Lindsay, 1988; Varin, ²⁰⁰⁸; Varin et al., ²⁰¹¹). CML estimators have the desired properties of being asymptotically consistent and normally distributed. Specifically, in the case of latent variable models, PML maximizes the sum of all bivariate log‐likelihoods, which contains all the information needed for all model parameters. This approach offers a clear advantage over FIML estimation since it only requires evaluating up to two‐dimensional integrals, each integrating over a different pair of underlying variables, irrespective of the number of observed or latent variables.

Pairwise likelihood ratio tests for overall goodness of fit (GOF), nested models, and model selection criteria have been proposed by Katsikatsou and Moustaki (2016). However, due to data sparseness, overall GOF test statistics pose computational and theoretical challenges in high‐dimensional settings. To address the sparseness issues, limited‐information test statistics have been proposed. These test statistics are defined as quadratic forms of first‐ and second‐order residuals and utilize information from first‐ and second‐order marginals (Bartholomew & Leung, 2002; Cagnone & Mignani, ²⁰⁰⁷; Cai et al., ²⁰⁰⁶; Maydeu‐Olivares & Joe, ^{2005, 2006}; Reiser, ^{1996, 2008}). Salomaa (1990) also showed that the parameters of the factor model could be inferred from the first‐ and second‐order marginal distributions of observed variables. Pairwise likelihood estimation uses this property to estimate all model parameters from the bivariate likelihoods. Furthermore, lower‐order residuals offer valuable evidence regarding model fit. Previous research on limited‐information tests mostly used a FIML estimator and simple random sampling.

In this paper, we first extend the PML to incorporate sampling weights, thereby rendering it suitable for complex and informative sampling designs. Subsequently, we introduce a limited‐information Pearson chi‐squared test statistic alongside three variants of the Wald test statistic. These test statistics are designed to apply to simple random and complex sampling designs with unequal selection probabilities.

In complex sampling designs, the pairwise likelihood components are appropriately weighted using the provided sampling weights (Skinner, 1989). Skinner (1989) and Muthén and Satorra (1995), in the context of structural equation modelling, made a distinction between an aggregated approach, where the definitions of the model parameters take no account of complex population features such as stratification or clustering used in sampling and a disaggregated approach in which the complex population features are taken into account in the specification of the model of interest, for example, via fixed or random effect terms. Here, we focus on the aggregated approach. Skinner (2019) provides an overview of categorical data analysis for complex surveys.

The paper is organized as follows. Section 2 presents the model framework. Section 3 gives the pairwise likelihood estimation for latent variable models for categorical responses under simple random and complex survey sampling. Section 4 proposes limited‐information GOF tests for complex sampling designs. Section 5 presents the results of the simulation studies, and Section 6 concludes the paper.

2. FACTOR MODELS FOR BINARY DATA

The basic idea of latent variable analysis is as follows. For a given set of response variables $y_1,\text{…},y_p$, one wants to find a set of latent factors ${\eta }_1,\text{…},{\eta }_q$, fewer in number than the observed variables that contain essentially the same information. The latent factors are supposed to account for the dependencies of the response variables in the sense that if the factors are held fixed, the observed variables would be independent. If both the response variables and the latent factors are normally distributed with zero means and unit variances, this leads to the model (Jöreskog, 1979)

$$\begin{array}{l}\hfill E(y_i|{\eta }_1,\text{…},{\eta }_q)={\lambda }_{i1}{\eta }_1+\text{⋯}+{\lambda }_{iq}{\eta }_q,\end{array}$$ (1)

$$\begin{array}{l}\hfill E(y_i{y}_j|{\eta }_1,\text{…},{\eta }_q)=0,i\ne j.\end{array}$$ (2)

If the factors are independent, it follows that the correlation between $y_i$ and $y_j$ is ${\sum }_{l=1}^q{\lambda }_{il}{\lambda }_{jl}$.

Equation (1) is a suitable representation of the factor analysis model if the response variables $y_i$ are continuous variables measured on an interval or ratio scale. However, it cannot be used if the response variables $y_i$ are categorical. In those cases, one must instead specify the probability of each response pattern as a function of ${\eta }_1,\text{…},{\eta }_q$:

$$\pi =\mathrm{Pr}(y_1=c_1,\text{…},y_p=c_p|{\eta }_1,\text{…},{\eta }_q)=f({\eta }_1,\text{…},{\eta }_q),$$ (3)

where $c_1,\text{…},c_p$ represent the different response categories of $y_1,\text{…},y_p$ respectively. In this paper, we consider the case of binary response variables ($c_i\in \{0,1\}$) and continuous latent variables. The methodology presented can easily be extended to ordinal variables.

Let $\mathbf{y}={\left(y_1,\text{…},y_p\right)}^{\prime }$ denote the vector of $p$ binary observed variables. Thus, there are $R=2^p$ possible response patterns of the form ${\mathbf{y}}_r={\left(c_1,c_2,\text{…},c_p\right)}^{\prime }$, $c_i\in \{0,1\}$. For a random sample $\mathrm{34𝒮}={\{{\mathbf{y}}^{(h)}\}}_{h=1}^n$ of size $n$, where ${\mathbf{y}}^{(h)}$ denotes the value of $\mathbf{y}$ for unit $h$, the log‐likelihood is

$$\log L(\mathit{\theta };\mathrm{34𝒮})=\sum _{r=1}^R{n}_r\log {\pi }_r(\mathit{\theta }),$$ (4)

where $n_r$ and ${\pi }_r(\mathit{\theta })$ are the observed frequency and the probability under the model, respectively, for the response pattern $r$, for some parameter vector $\mathit{\theta }$. As such, ${\sum }_{r=1}^R{n}_r=n$ and ${\sum }_{r=1}^R{\pi }_r(\mathit{\theta })=1$.

In the case of complex survey designs where the sample is drawn from a finite population of size $N$, we maximize the weighted log‐likelihood

$$\log L(\mathit{\theta };\mathrm{34𝒮})=\sum _{r=1}^R{\widehat{n}}_r\ln {\pi }_r(\mathit{\theta }),$$ (5)

where ${\widehat{n}}_r$ is the sum of survey weights across sample units with response pattern $r$. If we denote the survey weight for unit $h$ in $\mathrm{34𝒮}$ by $w_h$, we can write ${\widehat{n}}_r={\sum }_{h\in \mathrm{34𝒮}}{w}_h[{\mathbf{y}}^{(h)}={\mathbf{y}}_r]$. Here, the notation $[·]$ denotes the Iverson bracket: $[A]=1$ if proposition $A$ is true, and $[A]=0$ otherwise. Skinner (1989) showed that the parameter estimates from the weighted log‐likelihood (pseudo‐likelihood) are consistent under any sampling scheme.

We adopt the underlying response variable approach (UV) to model the binary variables. Each binary variable $y_i$ is taken to be a manifestation of an underlying continuous and normally distributed random variable $y_i^{\ast }$. The connection between the observed $y_i$ and the underlying variable $y_i^{\ast }$ is as follows:

$$y_i=\left\{\begin{array}{cc}1& y_i^{\ast }>{\tau }_i\\ 0& y_i^{\ast }\le {\tau }_i,\end{array}\right.$$

where ${\tau }_i$ is a threshold associated with the underlying variable $y_i^{\ast }$. For convenience, the distribution of $y_i^{\ast }$ is assumed to be standard normal.

The factor model is of the form

$${\mathbf{y}}^{\star }=\mathbf{\Lambda }\mathit{\eta }+\mathit{ϵ},$$ (6)

where ${\mathbf{y}}^{\star }={(y_1^{\ast },\text{⋯},y_p^{\ast })}^{\prime }$ is the $p$‐dimensional vector of the underlying variables, $\mathbf{\Lambda }$ is the $p\times q$ matrix of loadings, and $\mathit{ϵ}$ is the $p$‐dimensional vector of unique variables. In addition, it is assumed that $\mathit{\eta }\sim {\text{N}}_q(\mathbf{0},\mathbf{\Psi })$, where $\mathbf{\Psi }$ contains 1s on its main diagonal. In this way, $\mathbf{\Psi }$ is the correlation matrix of latent factors. Further, it is assumed that $\mathit{ϵ}\sim {\text{N}}_p(\mathbf{0},{\mathbf{\Theta }}_{ϵ})$, with ${\mathbf{\Theta }}_{ϵ}$ a diagonal matrix defined by ${\mathbf{\Theta }}_{ϵ}=\mathbf{I}-\text{diag}(\mathbf{\Lambda }\mathbf{\Psi }{\mathbf{\Lambda }}^{\prime })$, and that $\text{Cov}({ϵ}_i,{\eta }_j)=0$ for each combination of $i=1,\text{…},p$ and $j=1,\text{…},q$.

The parameter vector ${\mathit{\theta }}^{\prime }=\left({\mathit{\lambda }}^{\prime },{\mathit{\rho }}^{\prime },{\mathit{\tau }}^{\prime }\right)$ contains $\mathit{\lambda }$ and $\mathit{\rho }$, which are the vectors of the free non‐redundant parameters in matrices $\mathbf{\Lambda }$ and $\mathbf{\Psi }$ respectively, as well as $\mathit{\tau }$, which is the vector of all free thresholds. Under the UV framework, the probability of a response pattern $r$ is written

$${\pi }_r(\mathit{\theta })=\mathrm{Pr}\left(y_1=c_1,\text{…},y_p=c_p;\mathit{\theta }\right)=\int \; ···\int {\varphi }_p({\mathbf{y}}^{\star }|\mathbf{0},{\mathbf{\Sigma }}_{{\mathbf{y}}^{\star }})\text{d}{\mathbf{y}}^{\star },$$ (7)

where ${\varphi }_p(·|\mathit{\mu },\mathbf{\Sigma })$ is the $p$‐dimensional Gaussian probability density function with mean vector $\mathit{\mu }$ and variance‐covariance matrix $\mathbf{\Sigma }$. The variance‐covariance matrix of the underlying variables is the correlation matrix ${\mathbf{\Sigma }}_{{\mathbf{y}}^{\star }}=\mathbf{\Lambda }\mathbf{\Psi }{\mathbf{\Lambda }}^{\prime }+{\mathbf{\Theta }}_{ϵ}$. The $p$‐dimensional integral (7) is evaluated on the domain of ${\mathbf{y}}^{\ast }$ constrained accordingly using the thresholds ${\tau }_i$, so that the manifest variables correspond to $\mathbf{c}=(c_1,\text{…},c_p)$.

3. PAIRWISE LIKELIHOOD ESTIMATION

To mazimize the log‐likelihood function in (4) over the parameter vector $\mathit{\theta }$, we need to evaluate (7), which cannot be expressed in a closed form. As a result, limited‐information estimation methods have been developed and are now widely available in commercial software. The most popular method is the three‐stage weighted least‐squares estimation (Jöreskog, 1990, 1994; Muthén, ¹⁹⁸⁴). However, we will be using a different method known as PML as proposed in Katsikatsou et al. (2012) and implemented in the R package lavaan (Rosseel, 2012). The method is briefly explained in what follows.

To define the pairwise likelihood, we assume that ${\pi }_{c_i{c}_j}^{(ij)}(\mathit{\theta })$ is the probability that both $y_i$ and $y_j$ will take on values $c_i$ and $c_j$ respectively under a particular model. For a random sample of size $n$, the pairwise log‐likelihood is

$$\sum _{i<j}\sum _{c_i=0,1}\sum _{c_j=0,1}{n}_{c_i{c}_j}^{(ij)}\log {\pi }_{c_i{c}_j}^{(ij)}(\mathit{\theta }),$$ (8)

where $n_{c_i{c}_j}^{(ij)}$ is the observed frequency of sample units with $y_i=c_i$ and $y_j=c_j$. To account for complex sampling, we use the weighted pairwise log‐likelihood instead:

$${\ell }_{\text{P}}(\mathit{\theta };\mathrm{34𝒮})=\sum _{i<j}\sum _{c_i=0,1}\sum _{c_j=0,1}{p}_{c_i{c}_j}^{(ij)}\log {\pi }_{c_i{c}_j}^{(ij)}(\mathit{\theta }),$$ (9)

where $p_{c_i{c}_j}^{(ij)}={\sum }_{h\in \mathrm{34𝒮}}{w}_h[y_i^{(h)}=c_i,y_j^{(h)}=c_j]/{\sum }_{h\in s}{w}_h,$ and $y_i^{(h)}$ refers to the $i$th item for sampling unit $h$ in $\mathrm{34𝒮}$. We have introduced survey weights $w_h$ and scaled them by the constant factor $1/{\sum }_{h\in s}{w}_h$. The pairwise likelihood only requires the calculation of up to two‐dimensional normal probabilities, regardless of the number of observed or latent variables. In this way, it is always computationally feasible.

The maximum pairwise likelihood estimator (MPLE) ${\widehat{\mathit{\theta }}}_{\text{PL}}$ solves the estimating equations ${ℝ}^m\ni \nabla {\ell }_{\text{P}}(\mathit{\theta };\mathrm{34𝒮})=\mathbf{0}$, where the entries of this score vector are

$$\frac{\partial {\ell }_{\text{P}}(\mathit{\theta };\mathrm{34𝒮})}{\partial {\theta }_k}=\sum _{i<j}\sum _{c_i=0,1}\sum _{c_j=0,1}\frac{p_{c_i{c}_j}^{(ij)}}{{\pi }_{c_i{c}_j}^{(ij)}(\mathit{\theta })}\frac{\partial {\pi }_{c_i{c}_j}^{(ij)}(\mathit{\theta })}{\partial {\theta }_k},k=1,\text{…},m.$$ (10)

The $(k,l)$ element of the $m\times m$ Hessian matrix, ${\nabla }^2{\ell }_{\text{P}}(\mathit{\theta };\mathbf{\text{y}})$, is

$$\frac{\partial {\ell }_{\text{P}}(\mathit{\theta };\mathrm{34𝒮})}{\partial {\theta }_k\partial {\theta }_l}=\sum _{i<j}\sum _{c_i=0,1}\sum _{c_j=0,1}\frac{p_{c_i{c}_j}^{(ij)}}{{\pi }_{c_i{c}_j}^{(ij)}(\mathit{\theta })}\left\{\frac{{\partial }^2{\pi }_{c_i{c}_j}^{(ij)}(\mathit{\theta })}{\partial {\theta }_k\partial {\theta }_l}-\frac{1}{{\pi }_{c_i{c}_j}^{(ij)}(\mathit{\theta })}\frac{\partial {\pi }_{c_i{c}_j}^{(ij)}(\mathit{\theta })}{\partial {\theta }_k}\frac{\partial {\pi }_{c_i{c}_j}^{(ij)}(\mathit{\theta })}{\partial {\theta }_l}\right\}.$$ (11)

Using Taylor expansion, we may write

$${\widehat{\mathit{\theta }}}_{\text{PL}}-\mathit{\theta }={\{{\nabla }^2{\ell }_{\text{P}}(\mathit{\theta };\mathrm{34𝒮})\}}^{-1}\nabla {\ell }_{\text{P}}(\mathit{\theta };\mathrm{34𝒮})+o_p(n^{-1/2}).$$ (12)

It follows that

$$\sqrt{n}({\widehat{\mathit{\theta }}}_{\text{PL}}-\mathit{\theta })\stackrel{\text{d}}{\to }{\text{N}}_m(\mathbf{0},\mathbf{H}{(\mathit{\theta })}^{-1}\mathbf{J}(\mathit{\theta })\mathbf{H}{(\mathit{\theta })}^{-1}),$$ (13)

where $m$ is the dimension of $\mathit{\theta }$, $\mathbf{H}(\mathit{\theta })=\text{E}\{-{\nabla }^2{\ell }_{\text{P}}(\mathit{\theta };\mathbf{\text{y}})\}$ is the sensitivity matrix, and $\mathbf{J}(\mathit{\theta })=\mathrm{Var}\left\{\nabla {\ell }_{\text{P}}(\mathit{\theta };\mathbf{y})\right\}$ is the variability matrix. $\mathbf{G}=\mathbf{H}(\mathit{\theta })\mathbf{J}{(\mathit{\theta })}^{-1}\mathbf{H}(\mathit{\theta })$ is known as the Godambe information (Godambe, 1960) or sandwich information matrix, and it indicates the loss of efficiency compared to full information maximum likelihood estimation. If ${\ell }_{\text{P}}$ is a true log‐likelihood function, then the Godambe, sensitivity, and Fisher information matrices are all identical. In practice, we can estimate the $\mathbf{H}$ and $\mathbf{J}$ matrices using (Varin et al., 2011; Zhao & Joe, ²⁰⁰⁵)

$$\begin{array}{l}\hfill \widehat{\mathbf{H}}({\widehat{\mathit{\theta }}}_{\text{PL}})=-{\left.\frac{1}{{\sum }_{h\in \mathrm{34𝒮}}{w}_h}\sum _{h\in \mathrm{34𝒮𝒮}}{\nabla }^2{\ell }_{\text{P}}(\mathit{\theta };{\mathbf{y}}^{(h)})\right|}_{\mathit{\theta }={\widehat{\mathit{\theta }}}_{\text{PL}}}\end{array},$$ (14)

and

$$\begin{array}{l}\hfill \widehat{\mathbf{J}}({\widehat{\mathit{\theta }}}_{\text{PL}})={\left.\frac{1}{{\sum }_{h\in \mathrm{34𝒮}}{w}_h}\sum _{h\in \mathrm{34𝒮𝒮}}\nabla {\ell }_{\text{P}}(\mathit{\theta };{\mathbf{y}}_h)\nabla {\ell }_{\text{P}}{(\mathit{\theta };{\mathbf{y}}^{(h)})}^{\prime }\right|}_{\mathit{\theta }={\widehat{\mathit{\theta }}}_{\text{PL}}}.\end{array}$$ (15)

The estimator for the variability matrix $\mathbf{J}$ assumes independent observations. However, in cluster sampling, the observations are no longer independent. The variance estimator for the weighted pairwise estimates should be adjusted to account for the dependencies and stratification (see Asparouhov, ²⁰⁰⁵; Binder, ¹⁹⁸³; Wolter, ²⁰⁰⁷). This gives

$$\widehat{\mathbf{J}}({\widehat{\mathit{\theta }}}_{\text{PL}})=\frac{1}{n}\sum _a\frac{n_a}{n_a-1}\sum _b(z_{ab}-\overline{z_a}){(z_{ab}-\overline{z_a})}^{\prime },$$ (16)

where $n_a$ is the number of sampled clusters from stratum $a$, $z_{ab}={\sum }_{h\in {\mathrm{34𝒮}}_{ab}}\nabla {\ell }_{\text{P}}(\mathit{\theta };{\mathbf{y}}^{(h)})$ is the total value of the score for all individuals $h$ in cluster $b$ in stratum $a$, and $\overline{z_a}$ is the average of $z_{ab}$. In addition, Varin (2008) discusses alternative empirical estimators for the variability matrix in the presence of clustering in a composite likelihood framework.

4. GOODNESS OF FIT

The overall GOF of the model can be tested by constructing a likelihood ratio test using the full log‐likelihood given in Equation (5) for testing $H_0:{\pi }_r={\pi }_r(\mathit{\theta })$ against $H_1:{\pi }_r\ne {\pi }_r(\mathit{\theta })$ for all $r=1,\text{…},R$, where $\mathit{\theta }$ is a vector of all free parameters and $r$ runs over all possible response patterns.

Let ${\widehat{\pi }}_r={\pi }_r(\widehat{\mathit{\theta }})$, where $\widehat{\mathit{\theta }}$ denotes the FIML estimates, and let ${\widehat{p}}_r=n_r/n$ denote the sample proportion of response pattern $r$. The maximum log‐likelihood value under $H_0$ is $n{\sum }_{r=1}^R{p}_r\log {\widehat{\pi }}_r$, while the maximum log‐likelihood value under $H_1$ is $n{\sum }_{r=1}^R{p}_r\log p_r$. The likelihood ratio (LR) test statistic is

$$X_{\text{LR}}^2=2n\sum _{r=1}^R{p}_r\log (p_r/{\widehat{\pi }}_r).$$ (17)

Under $H_0$, (17) is distributed approximately as ${\chi }^2$, with degrees of freedom equal to the number of independent response patterns minus one minus the number of elements in $\mathit{\theta }$. For the $q$‐factor model, the degrees of freedom is $2^p-p(q+1)-1$. Alternatively, one can use the Pearson GOF test statistic, written

$$X_{\text{GF}}^2=n\sum _{r=1}^R{(p_r-{\widehat{\pi }}_r)}^2/{\widehat{\pi }}_r.$$ (18)

Under $H_0$, both test statistics (17) and (18) have the same asymptotic distribution.

In principle, these tests are possible to use with FIML. They cannot be used with the UV approach because this does not maximize an overall likelihood function, so the ${\widehat{\pi }}_r$ are not computed. In practice, when dealing with sparse data characterised by zero or small frequencies $n_r$, the approximation to the chi‐squared distribution may fail and lead to unreliable results (see e.g. Reiser & VandenBerg, ¹⁹⁹⁴).

To address the sparseness issue, limited‐information test statistics focus on how well the model fits first‐ and second‐order marginals (marginal proportions) such as univariate, bivariate, and trivariate rather than the entire response pattern. In the following sections, we introduce limited‐information chi‐squared test statistics for evaluating the adequacy of a parametric model under simple random sampling and complex sampling designs and pairwise likelihood estimation.

4.1. Definitions

Define ${\dot{\pi }}_i=\mathrm{Pr}(y_i=1)$ as the first‐order marginal proportion of a positive response for the $i$th binary variable $i=1,\text{…},p$, and let ${\dot{\mathit{\pi }}}_1={({\dot{\pi }}_1,\text{…},{\dot{\pi }}_p)}^{\prime }$ be the $p\times 1$ vector containing all such first‐order moments. Further define with ${\dot{\pi }}_{ij}=\mathrm{Pr}(y_i=1,y_j=1)$ the second‐order marginal proportion for a pair of variables $i,j=1,\text{…},p$, and collect these proportions into the $\left(\genfrac{}{}{0.0pt}{}{p}{2}\right)$‐vector ${\dot{\mathit{\pi }}}_2={\left({\dot{\pi }}_{ij}\right)}_{i<j}$. Finally, let ${\mathit{\pi }}_2=\left(\begin{array}{c}{\dot{\mathit{\pi }}}_1\\ {\dot{\mathit{\pi }}}_2\end{array}\right)$be the vector that contains both these first‐ and second‐order marginal proportions with dimension $S=p+\left(\genfrac{}{}{0.0pt}{}{p}{2}\right)=p(p+1)/2$.

The vectors ${\dot{\mathit{\pi }}}_1$ and ${\dot{\mathit{\pi }}}_2$ are formed by summing the joint proportions of the response patterns $\mathit{\pi }$ that satisfy specific criteria. For example, the first‐order moment ${\dot{\pi }}_i$ for a specific variable $i$ is obtained by summing the joint proportions ${\pi }_r$, where the response pattern $r$ takes the value 1 in the $i$th position. Similarly, the second‐order moment ${\dot{\pi }}_{ij}$ for a pair of variables $i$ and $j$ is obtained by summing the values of ${\pi }_r$ for which the response patterns $r$ take on a value of 1 at both locations $i$ and $j$ simultaneously. In other words, there exists a transformation matrix ${\mathbf{T}}_2:{ℝ}^R\to {ℝ}^S$ that maps the joint proportions $\mathit{\pi }$ to the transformed vectors ${\mathit{\pi }}_2$, i.e. ${\mathit{\pi }}_2={\mathbf{T}}_2\mathit{\pi }$. This matrix consists of 0s and 1s and has a rank of $S$, where $S$ represents the dimension of the transformed space. A more detailed treatment of this transformation matrix can be found in Reiser (1996) or Maydeu‐Olivares and Joe (2005).

Let $\mathbf{\text{p}}$ denote the $R\times 1$ vector of sample proportions based on a sample of size $n$, which estimates the population proportions vector $\mathit{\pi }$. Assuming independent and identically distributed (iid) observations, it is known that

$$\sqrt{n}(\mathbf{\text{p}}-\mathit{\pi })\stackrel{\text{d}}{\to }{\text{N}}_R(\mathbf{0},\mathbf{\Sigma }),$$ (19)

where $\mathbf{\Sigma }$ is the multinomial covariance matrix given by $\mathbf{\Sigma }=\text{diag}(\mathit{\pi })-\mathit{\pi }{\mathit{\pi }}^{\prime }$. In the case of a complex sampling design, the vector $\mathbf{p}$ becomes the weighted vector of proportions with elements ${\sum }_{h\in \mathrm{34𝒮}}{w}_h[{\mathbf{y}}^{(h)}={\mathbf{y}}_r]/{\sum }_{h\in \mathrm{34𝒮}}{w}_h$. The vector $\mathit{\pi }$ may now be either the vector of proportions in the finite population or a superpopulation. Under suitable conditions (e.g. Fuller, ²⁰⁰⁹ sec. 1.3.2), we still have a central limit theorem as in (19), where the covariance matrix $\mathbf{\Sigma }$ does not take a multinomial form. In either case, the central limit theorem in (19) readily transforms to the limited‐information case. Consider the $S\times 1$ vectors ${\mathit{\pi }}_2={\mathbf{T}}_2\mathit{\pi }$ and ${\mathbf{\text{p}}}_2={\mathbf{T}}_2\mathbf{\text{p}}$, whose meanings are self apparent. Then

$$\sqrt{n}({\mathbf{p}}_2-{\mathit{\pi }}_2)\stackrel{\text{d}}{\to }{\text{N}}_S(\mathbf{0},{\mathbf{\Sigma }}_2),$$ (20)

where ${\mathbf{\Sigma }}_2={\mathbf{T}}_2\mathbf{\Sigma }{\mathbf{T}}_2^{\prime }$. Because ${\mathbf{T}}_2$ is of full column rank $S$, ${\mathbf{\Sigma }}_2$ is also of full rank $S$.

4.2. Test statistics for simple and composite hypotheses

In the case of a simple null hypothesis $H_0:\mathit{\pi }={\mathit{\pi }}_0$ and iid observations, Maydeu‐Olivares and Joe (2005) proposed the test statistic

$$X^2=n{({\mathbf{p}}_2-{\mathit{\pi }}_{20})}^{\prime }{\mathbf{\Sigma }}_{20}^{-1}({\mathbf{p}}_2-{\mathit{\pi }}_{20}),$$ (21)

where ${\mathit{\pi }}_{20}$ and ${\mathbf{\Sigma }}_{20}$ are the values of ${\mathit{\pi }}_2$ and $\text{diag}({\mathit{\pi }}_2)-{\mathit{\pi }}_2{\mathit{\pi }}_2^{\prime }$, respectively, when $\mathit{\pi }={\mathit{\pi }}_0$. As $n$ increases, the test statistic tends towards a ${\chi }^2$ distribution with $S$ degrees of freedom.

To evaluate the adequacy of the parametric model (composite hypothesis), we define the null hypothesis $H_0:\mathit{\pi }=\mathit{\pi }(\mathit{\theta })$ for some $\mathit{\theta }$ against an alternative hypothesis $H_1:\mathit{\pi }\ne \mathit{\pi }(\mathit{\theta })$ for any $\mathit{\theta }$. To construct the test statistics, we consider the residual vector ${\widehat{\mathbf{e}}}_2={\mathbf{p}}_2-{\mathit{\pi }}_2({\widehat{\mathit{\theta }}}_{\text{PL}})$, where ${\mathbf{p}}_2$ represent the first‐ and second‐order observed proportions, while ${\mathit{\pi }}_2({\widehat{\mathit{\theta }}}_{\text{PL}})$ represent the same predicted by the parametric model when evaluated at PML estimate ${\widehat{\mathit{\theta }}}_{\text{PL}}$.

We first derive the asymptotic distribution of ${\widehat{\mathbf{\text{e}}}}_2$. Noting that ${\mathit{\pi }}_2(\mathit{\theta })={\mathbf{T}}_2\mathit{\pi }(\mathit{\theta })$, a Taylor series expansion about the MPLE gives

$${\mathit{\pi }}_2({\widehat{\mathit{\theta }}}_{\text{PL}})={\mathit{\pi }}_2(\mathit{\theta })+{\mathbf{T}}_2\mathbf{\Delta }({\widehat{\mathit{\theta }}}_{\text{PL}}-\mathit{\theta })+o_p(|{\widehat{\mathit{\theta }}}_{\text{PL}}-\mathit{\theta }|),$$ (22)

where $\mathbf{\Delta }=\frac{\partial \mathit{\pi }(\mathit{\theta })}{\partial \mathit{\theta }}$. Let ${\mathbf{\Delta }}_2$ be defined as ${\mathbf{\Delta }}_2=\frac{\partial {\mathit{\pi }}_2(\mathit{\theta })}{\partial \mathit{\theta }}={\mathbf{T}}_2\mathbf{\Delta }$. Using the earlier Taylor expansion of the score vector in (12), we have that

$$\begin{array}{l}\hfill {\widehat{\mathbf{e}}}_2={\mathbf{p}}_2-{\mathit{\pi }}_2(\mathit{\theta })-{\mathbf{\Delta }}_2{\{{\nabla }^2{\ell }_{\text{P}}(\mathit{\theta })\}}^{-1}\nabla {\ell }_{\text{P}}(\mathit{\theta })+o_p(|{\widehat{\mathit{\theta }}}_{\text{PL}}-\mathit{\theta }|).\end{array}$$ (23)

To express $\nabla {\ell }_{\text{P}}(\mathit{\theta })$ in terms of ${\mathbf{p}}_2-{\mathit{\pi }}_2(\mathit{\theta })$, we use

(24)

as a means of expressing the $k$th component of the score vector in (10). The cancellation in the last line above follows from the fact that for all $k=1,\text{…},m$ and any pairwise proportions with index $i,j\in \{1,\text{…},p\}$, $i<j$,

$$\sum _{c_i=0,1}\sum _{c_j=0,1}{\pi }_{c_i{c}_j}^{(ij)}(\mathit{\theta })=1\Rightarrow \sum _{c_i=0,1}\sum _{c_j=0,1}\frac{\partial {\pi }_{c_i{c}_j}^{(ij)}(\mathit{\theta })}{\partial {\theta }_k}=0.$$ (25)

Furthermore, for any pair $(i,j)$ we require a $1\times S$ design vector ${\mathbf{B}}_{c_i{c}_j}^{(ij)}$ such that

$$p_{c_i{c}_j}^{(ij)}-{\pi }_{c_i{c}_j}^{(ij)}(\mathit{\theta })={\mathbf{B}}_{c_i{c}_j}^{(ij)}({\mathbf{p}}_2-{\mathit{\pi }}_2(\mathit{\theta })),$$ (26)

since the pairwise proportions may be obtained through a linear combination of the first‐ and second‐order proportions. By substituting (26) into (24) and amalgamating the preceding terms in (24) with the design vector ${\mathbf{B}}_{c_i{c}_j}^{(ij)}$, we find an $m\times S$ matrix $\mathbf{B}(\mathit{\theta })$ that depends on the parameters $\mathit{\theta }$, which satisfies

$$\nabla {\ell }_{\text{P}}(\mathit{\theta })=\mathbf{B}(\mathit{\theta })({\mathbf{p}}_2-{\mathit{\pi }}_2(\mathit{\theta })).$$ (27)

The intermediate steps from Equations (26) to (27) are given in Appendix 1. Hence, from (23), and multiplying through by $\sqrt{n}$, we get

(28)

So from (20), the consistency of the Hessian, and using Slutzky's theorem, we have the limiting distribution of the lower order residuals under $H_0$. It is $\sqrt{n}{\widehat{\mathbf{e}}}_2\stackrel{\text{d}}{\to }{\text{N}}_S(\mathbf{0},{\mathbf{\Omega }}_2)$, where

$${\mathbf{\Omega }}_2=\left(\mathbf{I}-{\mathbf{\Delta }}_2\mathbf{H}{(\mathit{\theta })}^{-1}\mathbf{B}(\mathit{\theta })\right){\mathbf{\Sigma }}_2{\left(\mathbf{I}-{\mathbf{\Delta }}_2\mathbf{H}{(\mathit{\theta })}^{-1}\mathbf{B}(\mathit{\theta })\right)}^{\prime }.$$ (29)

To calculate test statistics, it is necessary to use an estimator for the asymptotic covariance matrix of ${\widehat{\mathbf{e}}}_2$. Evaluate $\frac{\partial \mathit{\pi }(\mathit{\theta })}{\partial \mathit{\theta }}$ at the PL estimate ${\widehat{\mathit{\theta }}}_{\text{PL}}$ to obtain ${\widehat{\mathbf{\Delta }}}_2={\mathbf{T}}_2\mathbf{\Delta }{|}_{\mathit{\theta }={\widehat{\mathit{\theta }}}_{\text{PL}}}$, and set

$${\widehat{\mathbf{\Omega }}}_2=\left(\mathbf{I}-{\widehat{\mathbf{\Delta }}}_2\widehat{\mathbf{H}}{({\widehat{\mathit{\theta }}}_{\text{PL}})}^{-1}\mathbf{B}({\widehat{\mathit{\theta }}}_{\text{PL}})\right){\widehat{\mathbf{\Sigma }}}_2{\left(\mathbf{I}-{\widehat{\mathbf{\Delta }}}_2\widehat{\mathbf{H}}{({\widehat{\mathit{\theta }}}_{\text{PL}})}^{-1}\mathbf{B}({\widehat{\mathit{\theta }}}_{\text{PL}})\right)}^{\prime },$$ (30)

with ${\widehat{\mathbf{\Sigma }}}_2={\mathbf{T}}_2\widehat{\mathbf{\Sigma }}{\mathbf{T}}_2^{\prime }$. The construction of the estimator $\widehat{\mathbf{\Sigma }}$ is discussed in Section 4.3. In the case of iid observations with a multinomial covariance matrix, we may set $\widehat{\mathbf{\Sigma }}=\text{diag}(\mathit{\pi }({\widehat{\mathit{\theta }}}_{\text{PL}}))-\mathit{\pi }({\widehat{\mathit{\theta }}}_{\text{PL}})\mathit{\pi }{({\widehat{\mathit{\theta }}}_{\text{PL}})}^{\prime }$.

Subsequently, the aim is to build limited‐information GOF test statistics taking the quadratic form

$$X^2=n{\widehat{\mathbf{e}}}_2^{\prime }\widehat{\mathbf{\Xi }}{\widehat{\mathbf{e}}}_2,$$ (31)

such that $\widehat{\mathbf{\Xi }}\stackrel{\text{P}}{\to }\mathbf{\Xi }$ as $n\to \infty$, where $\mathbf{\Xi }$ is some $S\times S$ weight matrix. Generally, $p$‐values relating to $X^2$ will be compared against a reference chi‐squared distribution. This is because the quadratic form in (31) converges in distribution to ${\sum }_{s=1}^S{\delta }_s{U}_s$ as $n\to \infty$, where each $U_s\hspace{0.17em}\stackrel{\text{iid}}{\sim }\hspace{0.17em}{\chi }_1^2$ and ${\delta }_1,\text{…},{\delta }_S$ are the eigenvalues of ${\mathbf{\Omega }}_2^{1/2}\mathbf{\Xi }{\mathbf{\Omega }}_2^{1/2}$ (or of ${\mathbf{L}}^{\prime }\mathbf{\Xi }\mathbf{L}$, where $\mathbf{L}$ is the lower triangular matrix from the Choleski decomposition of ${\mathbf{\Omega }}_2$). An exact asymptotic chi‐squared distribution arises if $\mathbf{\Xi }$ is chosen such that the eigenvalues ${\delta }_s$ are either 0 or 1. If not, a sum of scaled chi‐squared variates arises. A chi‐squared distribution can approximate this mixture of chi‐squared variates, but its degrees of freedom must be estimated. We describe in Appendix 2 the moment matching procedure to estimate the degrees of freedom of $X^2$ in such cases.

In what follows, we discuss Wald‐type, Pearson, and other test statistics that utilize different forms for the weight matrix $\mathbf{\Xi }$.

4.2.1. Wald‐type tests

A Wald test statistic can be constructed by selecting the inverse of ${\mathbf{\Omega }}_2$ as the weight matrix $\mathbf{\Xi }$. This choice ensures that the resulting test statistic follows an exact asymptotic chi‐squared distribution under the null hypothesis $H_0$. Due to numerical instabilities, the rank of ${\mathbf{\Omega }}_2$ may be deficient, causing problems for calculating the inverse. In our implementation, we adopt the Moore‐Penrose inverse $\widehat{\mathbf{\Xi }}={\widehat{\mathbf{\Omega }}}_2^{+}$, as is done in Reiser (1996). Under the null hypothesis ($H_0$), this test statistic is asymptotically chi‐squared distributed with degrees of freedom equal to the rank of ${\widehat{\mathbf{\Omega }}}_2$, which falls between $S-m$ and $S$ (Maydeu‐Olivares & Joe, 2005). Here $m$ represents all the free model parameters. One drawback is that the degrees of freedom are initially unknown and can only be estimated after estimating ${\widehat{\mathbf{\Omega }}}_2$ by inspecting the magnitude of its eigenvalues. As a result, the $p$‐value depends on the assessment of which eigenvalues are greater than zero. Additionally, inverting ${\widehat{\mathbf{\Omega }}}_2$ can be computationally challenging when its dimensions are large.

A different version of the Wald test is also considered, referred to as the diagonal Wald test, where $\widehat{\mathbf{\Xi }}=\text{diag}{({\widehat{\mathbf{\Omega }}}_2)}^{-1}$ is used instead of the pseudo‐inverse of ${\mathbf{\Omega }}_2$. The motivation behind this choice is enhanced numerical stability and computational simplicity. Inverting a diagonal matrix is straightforward compared to inverting a full matrix, especially as the number of items $p$ increases. However, the diagonal Wald test statistic converges to a sum of scaled chi‐squared values, as mentioned earlier and, in finite samples, can be approximated by an appropriate chi‐squared distribution.

The drawbacks of ${\mathbf{\Omega }}_2$ disscused above led to Maydeu‐Olivares and Joe (2005, 2006) suggesting the use of a weight matrix $\mathbf{\Xi }$ such that ${\mathbf{\Omega }}_2$ is a generalized inverse of $\mathbf{\Xi }$, i.e. $\mathbf{\Xi }=\mathbf{\Xi }{\mathbf{\Omega }}_2\mathbf{\Xi }$. Let ${\mathbf{\Delta }}_2^{\perp }$ be an $S\times (S-m)$ orthogonal complement to ${\mathbf{\Delta }}_2$, i.e. it satisfies ${({\mathbf{\Delta }}_2^{\perp })}^{\prime }{\mathbf{\Delta }}_2=\mathbf{0}$. Then, due to the asymptotic normality of ${\widehat{\mathbf{e}}}_2$, we see that as $n\to \infty$,

$$\sqrt{n}{({\mathbf{\Delta }}_2^{\perp })}^{\prime }{\widehat{\mathbf{e}}}_2\stackrel{\text{d}}{\to }{\text{N}}_{S-m}(\mathbf{0},{({\mathbf{\Delta }}_2^{\perp })}^{\prime }{\mathbf{\Omega }}_2{\mathbf{\Delta }}_2^{\perp }).$$ (32)

Because of (29), the asymptotic covariance matrix may be written

$${({\mathbf{\Delta }}_2^{\perp })}^{\prime }{\mathbf{\Omega }}_2{\mathbf{\Delta }}_2^{\perp }={({\mathbf{\Delta }}_2^{\perp })}^{\prime }{\mathbf{\Sigma }}_2{\mathbf{\Delta }}_2^{\perp },$$

since all the multiplications of ${\mathbf{\Delta }}_2$ with its orthogonal complement cancels out. By letting

$$\mathbf{\Xi }={\mathbf{\Delta }}_2^{\perp }{\left({({\mathbf{\Delta }}_2^{\perp })}^{\prime }{\mathbf{\Sigma }}_2{\mathbf{\Delta }}_2^{\perp }\right)}^{-1}{({\mathbf{\Delta }}_2^{\perp })}^{\prime },$$

we can then verify $\mathbf{\Xi }=\mathbf{\Xi }{\mathbf{\Omega }}_2\mathbf{\Xi }$, that is, ${\mathbf{\Omega }}_2$ is a generalized inverse of $\mathbf{\Xi }$. Let $\widehat{\mathbf{\Xi }}$ be an appropriate estimate of $\mathbf{\Xi }$, e.g. by replacing the foregoing matrices with their corresponding hat versions. The implication here is that

$$X^2=n{\widehat{\mathbf{e}}}_2^{\prime }\widehat{\mathbf{\Xi }}{\widehat{\mathbf{e}}}_2=n{\widehat{\mathbf{e}}}_2^{\prime }{\widehat{\mathbf{\Delta }}}_2^{\perp }{\left({({\widehat{\mathbf{\Delta }}}_2^{\perp })}^{\prime }{\widehat{\mathbf{\Sigma }}}_2{\widehat{\mathbf{\Delta }}}_2^{\perp }\right)}^{-1}{({\widehat{\mathbf{\Delta }}}_2^{\perp })}^{\prime }{\widehat{\mathbf{e}}}_2,$$ (33)

converges in distribution to a ${\chi }_{S-m}^2$ variate as $n\to \infty$ due to (32) and Slutsky's theorem. The degrees of freedom are as such because ${\mathbf{\Delta }}_2^{\perp }$ is of full column rank $S-m$ and hence $\mathbf{\Xi }$ is also of rank $S-m$, since the dimensions of a vector space and its orthogonal complement always add up to the dimension of the whole space. We refer to this test as the variance‐covariance free (VCF) Wald test.

4.2.2. Pearson test statistic

To construct a Pearson test statistic, let ${\widehat{\mathbf{\Xi }}}^{-1}={\widehat{\mathbf{D}}}_2=\text{diag}({\mathit{\pi }}_2({\widehat{\mathit{\theta }}}_{\text{PL}}))$, which converges in probability to ${\mathbf{D}}_2=\text{diag}({\mathit{\pi }}_2(\mathit{\theta }))$ as $n\to \infty$. Then we can see that

$$X^2=n{\widehat{\mathbf{e}}}_2^{\prime }{\widehat{\mathbf{D}}}_2^{-1}{\widehat{\mathbf{e}}}_2=n\sum _{r=1}^R\frac{{({\dot{p}}_r-{\dot{\pi }}_r(\widehat{\mathit{\theta }}))}^2}{{\dot{\pi }}_r(\widehat{\mathit{\theta }})}+n\sum _{r<s}\frac{{({\dot{p}}_{rs}-{\dot{\pi }}_{rs}(\widehat{\mathit{\theta }}))}^2}{{\dot{\pi }}_{rs}(\widehat{\mathit{\theta }})},$$ (34)

which resembles the traditional Pearson chi‐squared test statistic. Here, the ${\dot{p}}_r$ and ${\dot{p}}_{rs}$ are the first‐ and second‐order sample proportion estimates to ${\dot{\pi }}_r$ and ${\dot{\pi }}_{rs}$ respectively. When computing the Pearson test statistic, there is no need to invert ${\widehat{\mathbf{\Omega }}}_2$ as in the Wald test. However, unlike the traditional Pearson test statistic, this does not follow an asymptotic chi‐squared distribution because the independence requirement is violated in the second part of the foregoing sum. Instead, it converges to a sum of scaled chi‐squared values, whose distribution may be approximated by an appropriate chi‐squared distribution. Similarly, here, we employ the moment matching adjustment described in Appendix 2 to approximate the distribution of the Pearson test statistic.

4.2.3. Other test statistics

Two alternative weight matrices $\widehat{\mathbf{\Xi }}$ are proposed and selected for their ease of computation in determining the test statistic $X^2$. The first option is the identity matrix. This results in a chi‐squared value resembling the sum of squared residuals (RSS), which gives a pure measure of the discrepancy between the fitted model and the data. We refer to this as the RSS test.

The second option involves using multinomial weights ${\widehat{\mathbf{\Xi }}}^{-1}=\text{diag}({\widehat{\mathit{\pi }}}_2)-{\widehat{\mathit{\pi }}}_2{\widehat{\mathit{\pi }}}_2^{\prime }$. This matrix is a consistent estimator for the (inverse of) the multinomial covariance matrix ${\mathbf{\Sigma }}_2$ for the univariate and bivariate proportions ${\mathbf{p}}_2$. Bartholomew and Leung (2002) previously examined a variation of this test, focusing solely on the bivariate combinations and excluding the univariate ones.

In both choices mentioned here, the asymptotic distribution of the resulting test statistics is a sum of scaled chi‐squared values, so it is approximated by a chi‐squared whose degrees of freedom must be estimated using the data by employing the moment matching adjustment described in Appendix 2.

4.3. Estimation of covariance matrix under complex sampling

We now consider how to construct the covariance estimator of the proportion, $\widehat{\mathbf{\Sigma }}$ of $\mathbf{\Sigma }$, in (19) for complex sampling designs. We noted earlier that under iid assumptions, we may use the multinomial expression $\widehat{\mathbf{\Sigma }}=\text{diag}(\mathbf{p})-\mathbf{p}{\mathbf{p}}^{\prime }$ in the simple hypothesis case and $\widehat{\mathbf{\Sigma }}=\text{diag}(\mathit{\pi }({\widehat{\mathit{\theta }}}_{\text{PL}}))-\mathit{\pi }({\widehat{\mathit{\theta }}}_{\text{PL}})\mathit{\pi }{({\widehat{\mathit{\theta }}}_{\text{PL}})}^{\prime }$ in the composite hypothesis case. We now consider the case of complex sampling, in particular, stratified multistage sampling.

From (19) we may write

$$\mathbf{\Sigma }=\text{limvar}\{\sqrt{n}(\mathbf{p}-\mathit{\pi })\}=\text{limvar}\left\{\sqrt{n}\left(\frac{\sum _{h\in \mathrm{34𝒮}}{w}_h{\mathbf{y}}^{(h)}}{\sum _{h\in \mathrm{34𝒮}}{w}_h}-\mathit{\pi }\right)\right\},$$

where $\text{limvar}$ denotes the asymptotic covariance matrix, and $h$ is the index for the units in the sample $\mathrm{34𝒮}={\{{\mathbf{y}}^{(h)}\}}_{h=1}^n$. Using a usual linearization argument for a ratio, we may write

$$\mathbf{\Sigma }=\text{limvar}\left\{\sqrt{n}\frac{\sum _{h\in \mathrm{34𝒮}}{w}_h({\mathbf{y}}^{(h)}-\mathit{\pi })}{\text{E}(\sum _{h\in \mathrm{34𝒮}}{w}_h)}\right\}.$$

We consider a stratified multistage sampling scheme where the strata are labelled $a$ and the primary sampling units are labelled $b=1,\text{…},n_a$, where $n_a$ is the number of primary sampling units selected in stratum $a$.

Then we write

$$\frac{\sum _{h\in \mathrm{34𝒮}}{w}_h({\mathbf{y}}^{(h)}-\mathit{\pi })}{\text{E}(\sum _{h\in \mathrm{34𝒮}}{w}_h)}=\sum _{a=1}\sum _{b=1}{\tilde{\mathbf{u}}}_{ab},$$

where ${\tilde{\mathbf{u}}}_{ab}={\sum }_{h\in \mathrm{34𝒮}}{w}_h({\mathbf{y}}^{(h)}-\mathit{\pi })/\text{E}({\sum }_{h\in \mathrm{34𝒮}}{w}_h)$ and $\mathrm{34𝒮}$ is the set of sample units contained within primary sampling unit $b$ within stratum $a$. So

$$\mathbf{\Sigma }=\text{limvar}\left\{\sqrt{n}\sum _a\sum _b{\tilde{\mathbf{u}}}_{ab}\right\}.$$

A standard estimator of $n^{-1}\mathbf{\Sigma }$ is the between‐cluster variance estimator, which gives consistent variance estimates of linear and non‐linear statistics when the number of clusters grows large (see e.g. Asparouhov, ²⁰⁰⁵; Bieler & Williams, ¹⁹⁹⁵; Binder, ¹⁹⁸³; Skinner, ¹⁹⁸⁹; Williams, ²⁰⁰⁰; Wolter, ²⁰⁰⁷). This estimator is given by

$$n^{-1}\widehat{\mathbf{\Sigma }}=\sum _{a=1}^M\frac{n_a}{n_a-1}\sum _{b=1}^{n_a}({\mathbf{\text{u}}}_{ab}-{\overline{\mathbf{\text{u}}}}_a){({\mathbf{u}}_{ab}-{\overline{\mathbf{u}}}_a)}^{\prime },$$

where $n_a$ is the number of sampled clusters from stratum $a$, $M$ is the total number of strata, ${\mathbf{u}}_{ab}={\sum }_{h\in s}{w}_h({\mathbf{y}}^{(h)}-{\mathit{\pi }}_0)/{\sum }_{h\in s}{w}_h$ is the total for all individuals $h$ in cluster $b$ in stratum $a$, and ${\overline{\mathbf{\text{u}}}}_a=n_a^{-1}{\sum }_{b=1}^{n_a}{\mathbf{u}}_{ab}$ is the average of ${\mathbf{u}}_{ab}$. Here, ${\mathit{\pi }}_0$ refers to the value of $\mathit{\pi }$ under a simple null hypothesis or estimates of the model‐implied proportions $\mathit{\pi }(\mathit{\theta })$ under a composite null hypothesis.

In the case of simple random sampling, we have just one stratum, the clusters are elements, and the weights are constant so that $n_a=n$, ${\mathbf{u}}_{ab}=({\mathbf{\text{y}}}^{(h)}-{\mathit{\pi }}_0)/n$, ${\overline{\mathbf{\text{u}}}}_a=\mathbf{0}$, and the estimator reduces to $\frac{n}{n-1}\left(\text{diag}({\mathit{\pi }}_0)-{\mathit{\pi }}_0{\mathit{\pi }}_0^{\prime }\right)$.

We only require ${\widehat{\mathbf{\Sigma }}}_2={\mathbf{T}}_2\widehat{\mathbf{\Sigma }}{\mathbf{T}}_2^{\prime }$ to compute the Wald and Pearson test statistics. We may write

$$n^{-1}{\widehat{\mathbf{\Sigma }}}_2=\sum _{a=1}^M\frac{n_a}{n_a-1}\sum _{b=1}^{n_a}({\mathbf{\text{v}}}_{ab}-{\overline{\mathbf{\text{v}}}}_a){({\mathbf{\text{v}}}_{ab}-{\overline{\mathbf{\text{v}}}}_a)}^{\prime },$$

where ${\mathbf{\text{v}}}_{ab}={\sum }_{h\in \mathrm{34𝒮}}{w}_h({\mathbf{\text{y}}}_2^{(h)}-{\mathit{\pi }}_{20})/({\sum }_{h\in \mathrm{34𝒮}}{w}_h)$, ${\overline{\mathbf{\text{v}}}}_a=n_a^{-1}{\sum }_{b=1}^{n_a}{\mathbf{\text{v}}}_{ab}$, and ${\mathbf{y}}_2^{(h)}={\mathbf{T}}_2{\mathbf{y}}^{(h)}$ is the $S\times 1$ indicator vector containing values $[y_i^{(h)}=1]$ and $[y_i^{(h)}=y_j^{(h)}=1]$ for different values of $i$ and $j$.

5. SIMULATION STUDY

Our simulation study comprises three parts. The first part (Part A) focuses on assessing the performance of the parameter estimates and their asymptotic standard errors in bias and mean squared error under an informative sampling scheme. These evaluations are carried out for the weighted pairwise likelihood, which accounts for unequal selection probabilities due to sampling design or informative sampling. The simulation's second and third parts (Parts B and C) study the performance of the proposed test statistics about Type I error and power for simple random sampling (SRS) and complex sampling respectively.

We made a deliberate choice to include simulation results for SRS. This was done to evaluate the performance of the proposed test statistics under limited‐information estimation methods. Research related to limited‐information GOF test statistics has primarily focused on full‐information maximum likelihood methods, as these produce the best asymptotically normal (BAN) estimators with the lowest asymptotic variance. In contrast, limited‐information estimation methods do not yield BAN estimators.

Additionally, we investigate the degree to which the null distribution of the test statistics approximate the assumed theoretical distribution as we increase the sample size and the number of observed variables.

5.1. Simulation Part A: Assessing parameter estimates and standard errors for the weighted pairwise likelihood

The data were generated as follows. A population of size $N$ was created under the one‐factor model using the specified true parameter values. The size of $N$ varied according to the size of the sample $n$ to keep the sample‐to‐population ratio small and constant at $n/N=1\%$ to avoid the need for any finite population correction factors. Each individual indexed by $h$ was assigned a probability of selection denoted by ${\pi }_h$, which was set at ${\pi }_h=1/(1+\exp (y_1^{\ast }))$ based on the first underlying variable $y_1^{\ast }$. Thus, a larger value of $y_1^{\ast }$ resulted in a smaller probability of selection. A similar sampling scheme was described in Asparouhov (2005). The one‐factor model was subsequently estimated using two different approaches, namely the unweighted PML given in (4) and the weighted PML (PMLW) given in (5). The sampling and estimation were replicated 1000 times and repeated for three sample sizes: $n=500,1000,5000$.

Table 1 shows the parameter bias for different sample sizes. When the weights from the informative sampling are considered (PMLW), the bias is minimal and close to zero. Ignoring the weights (PML) leads to a more significant bias for the threshold estimates.

TABLE 1.

Simulation A: Bias of estimated factor loadings and thresholds for one‐factor model under informative sampling scheme, $n=500,1000,5000$, unweighted PML and PMLW.

	True values	$n=500$		$n=1000$		$n=5000$
		PML	PMLW	PML	PMLW	PML	PMLW
Loadings
${\lambda }_1$	.80	−.026	−.002	−.030	−.006	−.017	.005
${\lambda }_2$	.70	−.026	−.007	−.020	−.002	−.029	−.005
${\lambda }_3$	.47	−.020	−.001	−.024	−.004	−.024	−.003
${\lambda }_4$	.38	−.017	−.001	−.019	−.002	−.017	.003
${\lambda }_5$	.34	−.003	.010	−.019	−.002	−.022	−.006
Thresholds
${\tau }_1$	−1.43	.309	−.004	.311	.000	.304	−.007
${\tau }_2$	−.55	.215	−.006	.218	.002	.213	−.008
${\tau }_3$	−.13	.145	−.006	.161	.009	.153	−.001
${\tau }_4$	−.72	.123	.003	.117	−.001	.120	−.002
${\tau }_5$	−1.13	.112	.009	.114	.009	.106	.002

Table 2 gives the coverage and ratio of the standard deviation of the estimated parameters across replications to the estimated asymptotic standard error. The latter was calculated using the Godambe information matrix. The coverage is .95 for most parameters regardless of the sample size under the PMLW estimation. Furthermore, the estimated asymptotic standard errors computed using the Godambe information matrix closely match the standard deviations computed across 1000 replications. Ignoring the weights during estimation impacts coverage and standard errors, leading to incorrect inferences.

TABLE 2.

Simulation A: Coverage rate and ratio of standard deviation across replications to estimated asymptotic standard error (SD/SE) for factor loadings and thresholds for one‐factor model under an informative sampling scheme, $n=500,1000,5000$, PML and PMLW.

	$n=500$				$n=1000$				$n=5000$
	Coverage		SD/SE		Coverage		SD/SE		Coverage		SD/SE
	PML	PMLW	PML	PMLW	PML	PMLW	PML	PMLW	PML	PMLW	PML	PMLW
Loadings
${\lambda }_1$	.95	.96	1.01	.99	.94	.95	1.04	1.01	.89	.95	1.22	1.01
${\lambda }_2$	.95	.95	1.03	.99	.93	.95	1.09	1.01	.85	.95	1.31	.99
${\lambda }_3$	.95	.96	1.02	.97	.94	.95	1.06	.97	.85	.95	1.35	.99
${\lambda }_4$	.94	.94	1.04	1.03	.94	.95	1.05	1.00	.88	.95	1.25	1.01
${\lambda }_5$	.95	.95	.99	.98	.95	.96	1.02	1.00	.90	.95	1.19	1.01
Thresholds
${\tau }_1$	.01	.96	4.38	.98	.00	.96	6.24	.97	.00	.96	13.81	.97
${\tau }_2$	.04	.95	3.91	1.05	.00	.94	5.47	1.01	.00	.94	12.03	1.06
${\tau }_3$	.22	.96	2.93	1.02	.03	.94	4.03	1.02	.00	.96	8.76	.96
${\tau }_4$	.49	.94	2.22	1.03	.20	.95	2.97	1.03	.00	.95	6.40	1.04
${\tau }_5$	.61	.94	1.88	1.02	.42	.95	2.39	1.01	.00	.95	5.13	1.04

5.2. Simulation Part B: Assessing performance of limited‐information test statistics under simple random sampling

We evaluate the performance of the six test statistics outlined in Table 4 under simple random sampling (SRS) and PML. We study five different models described in Table 3 with varying numbers of items ($p$) and factors ($q$) and five sample sizes: $n=500,1000,2500,5000$, and 10,000. This results in 25 simulation conditions. We conducted 1000 replications for each simulation condition.

TABLE 4.

Summary of test statistics evaluated in Simulations B and C.

	Name	Weight matrix ($\mathbf{\Xi }$)	df
1	Wald	${\mathbf{\Omega }}_2^{+}$	$S-m,S$
2	VCF Wald	$\mathbf{\Xi }{\mathbf{\Omega }}_2\mathbf{\Xi }$	$S-m$
3	Wald diagonal	$\text{diag}{({\mathbf{\Omega }}_2)}^{-1}$	Est.
4	Pearson	$\text{diag}{({\mathit{\pi }}_2)}^{-1}$	Est.
5	RSS	$\mathbf{I}$	Est.
6	Multinomial	${[\text{diag}({\mathit{\pi }}_2)-{\mathit{\pi }}_2{\mathit{\pi }}_2^{\prime }]}^{-1}$	Est.

TABLE 3.

Simulations B and C: Models used for generating the data.

Model	Label	Number of variables ($p$) and factors ($q$)
1	1F 5V	$p=5$ and $q=1$
2	1F 8V	$p=8$ and $q=1$
3	1F 15V	$p=15$ and $q=1$
4	2F 10V	$p=10$ and $q=2$, 5 indicators per factor
5	3F 15V	$p=15$ and $q=3$, 5 indicators per factor

In Model 1, the true factor loadings ${\mathit{\lambda }}^{\prime }$ are $(0.8,0.7,0.47,0.38,0.34)$. The true threshold values ${\mathit{\tau }}^{\prime }$ are $(-1.43,-0.55,-0.13,-0.72,-1.13)$. For Model 2, the factor loading values are identical to those in Model 1 for the first five items. Additionally, the first three factor loadings (.8, .7, .47) from Model 1 are repeated to set the values for the last three items in Model 2. The threshold values follow the same pattern. The same mechanism is used to set the true values for Model 3. Models 4 and 5 are confirmatory factor analysis models. The true factor loadings for each factor are used in Model 1. The same goes for the thresholds. For Models 4 (two factors) and 5 (three factors), the factor correlations are $\rho =0.3$ and ${\mathit{\rho }}^{\prime }=(0.2,0.3,0.4)$ respectively.

In each replication within each condition, we compute the GOF test statistic using Equation (31), and the weight matrices used are summarized in Table 4. The moment matching adjustment described in Appendix 2 has been applied to all tests but the Wald and VCF Wald. The degrees of freedom for the Wald test statistics are in the range $(S-m,S)$. We found that $S-m$ gave the best results in approximating the test statistic distribution, which we used in all our simulations. Our comparison focuses on the Wald, VCF Wald, and Pearson test statistics. The study also includes the performance of simpler versions of the tests (WaldDiag, RSS, and multinomial). However, we recommend using these simpler versions independently only after further investigations to determine their suitability in any model setting.

To conduct a power analysis, we introduced an additional latent variable $z\sim \text{N}(0,1)$ independent of the latent variables $\mathit{\eta }$. This extra variable was included in the data‐generating model to capture together with $\mathit{\eta }$ the associations among the $y^{\ast }$ variables. The loadings of variable $z$ in the data‐generating model closely resembled the loadings of the true latent factor $\mathit{\eta }$. For Models 1 and 2, the extra latent variable $z$ loads onto all items except two ($y_2$ and $y_6$). For Model 3, $z$ loads onto all items except three ($y_2,y_6,y_{14}$). For Models 4 and 5, $z$ loads onto all items. This additional latent variable in the data‐generation process results in a misspecified model, and the power of the test correctly detects the misspecification.

Figure 1 gives the Type I error rates for the six test statistics across all simulation conditions. The Wald diagonal test statistic exhibited the poorest performance. In contrast, Pearson and the Wald‐type test statistics demonstrated satisfactory performance across all three significance levels $\alpha =10\%,5\%$, and $1\%$, with improvement noted as the sample size increased. The power of all tests, as shown in Figure 2, increased with sample size, although it remained at lower levels in the case of the two‐ and three‐factor models. In addition, as the complexity of the model increased, the power of the Wald‐type test statistics consistently decreased compared to the Pearson test statistic. This could be attributed to the instability in inverting the ${\widehat{\mathrm{\Omega }}}_2$ matrix and potential issues with its estimation arising from possible sparsity in the lower‐order margins and smaller sample sizes.

FIGURE 1.

Simulation B: Type I error rates at significance levels $\alpha =(10\%,5\%,1\%)$ for the six test statistics in Table 4 across the five models in Table 3 and $n=500$, 1000, 2500, 5000, 10,000, SRS. The reported intervals are the 95% confidence intervals for each rejection proportion.

FIGURE 2.

Simulation B: Power analysis at significance levels $\alpha =(10\%,5\%,1\%)$ for the six test statistics in Table 4 across the five models in Table 3, and $n=500$, 1000, 2500, 5000, 10,000, SRS.

5.3. Simulation C: Assessing performance of limited‐information test statistics under complex sampling

We generated data for an entire population, mirroring a commonly employed sampling approach in educational surveys. The population comprised 2000 schools, acting as primary sampling units (PSUs). These schools were stratified into three types: A (400 units), B (1000 units), and C (600 units).

The schools were of different sizes so that an unequal probability sample might be conducted for the clustered sample scenario based on each school's size. The number of students allocated to each school followed a normal distribution, with a mean of 500 and a standard deviation of 125. Each school's assigned students was then rounded to the nearest whole number.

Additionally, students within each school type were randomly distributed among classes, where the average class sizes were set at 15, 25, and 20 for school Types A, B, and C respectively. The simulated population comprised approximately one million students (clustered within classrooms and classrooms clustered within schools). The population statistics, encompassing the count of schools, school types, and average class sizes, are given in Table 5.

TABLE 5.

Simulation C: Population statistics for simulated school population.

Schools		Students					Class size
Type	$N$	$N$	Avg.	SD	Min.	Max.	Avg.	SD	Min.	Max.
A	400	200,670	501.7	96.9	253	824	15.2	3.9	3	36
B	1000	501,326	501.3	100.2	219	839	25.7	5.0	6	48
C	600	303,861	506.4	101.9	195	829	20.4	4.4	6	39

The item responses were generated based on the five factor models outlined in Table 3. The PSUs (schools) were stratified into distinct strata based on the latent vector $\mathit{\eta }$, representing the abilities of each student. The observation units were ordered in descending order based on their latent variable value for the first factor ${\eta }_{1h}$. Subsequently, they were allocated to school Types A, B, or C. If the vector of observed variables $\mathbf{\text{y}}$ represents test items, this categorization implies that school Type A comprises ‘high‐ability’ students, Type B ‘medium‐ability’ students, and Type C ‘low‐ability’ students. This data‐generating scheme resulted in varying intraclass correlations (ICC) for response items across PSUs, ranging from .05 to .6. Of note, Hedges and Hedberg (2007) found that a typical value for the ICC in an educational study was about .16, adjusted for covariates.

To obtain a sample from this simulated population, we explored two multistage probability sampling designs: (1) two‐stage cluster sampling and (2) stratified cluster sampling. The following discussion details the process of selecting a sample of (approximate) size $n$ from each design. Table 6 shows the number of clusters (PSUs) sampled using each sampling design discussed below.

1. Two‐stage cluster sampling

   Let $n_c=\lfloor \frac{n}{21.5}\rfloor$, where 21.5 represents the average class size across all types of schools in the population. The value of $n_c$ approximates the number of schools (PSUs). Schools are selected using probability proportional to size (PPS) sampling, where the size variable is the number of students in each school. Next, one class is selected using simple random sampling from each selected PSU. All students in the selected class are included in the sample. The probability of selecting a student from school $b$ is

   $$\mathrm{Pr}(\text{selection})=\frac{z_b}{\sum _b{z}_b}\times \frac{1}{M_b},$$

   where $z_b$ is the number of students in school $b$, ${\sum }_b{z}_b$ is the total number of students in all schools in the population, and $M_b$ is the number of classes in school $b$.

   The total sample size varies from sample to sample due to varying school and class sizes. However, on average, it is expected to be close to $n=n_c\times 21.5$, where 21.5 is the average number of students per class.

2. Stratified cluster sampling

   Select a fixed number of schools (PSUs) denoted by $n_c$ using SRS from each stratum. The quantity $n_c$ is determined using the formula $n_c=\lfloor n/(15+20+25)\rfloor$, since we want the sample size $n$ to be close to $15n_c+20n_c+25n_c$, where the numbers 15, 20, and 25 represent the average class size in each stratum. One class is selected from each selected school using SRS. All the students from the selected class are included in the sample.

   The inclusion probability for a student from school, $b$ within stratum $a$, is calculated to be

   $$\mathrm{Pr}(\text{selection})=\frac{n_a}{N_a}\times \frac{n_{ab}}{N_{ab}}\times \frac{n_{abc}}{N_{abc}},$$

   where $n_a$ ($N_a$) is the sample (population) number of schools in stratum $a$, $n_{ab}$ ($N_{ab}$) is the sample (population) number of classes in school $b$ within stratum $a$, and $n_{abc}$ ($N_{abc}$) is the sample (population) number of students in class $c$, school $b$ within stratum $a$. In our experiment, $n_a$ was chosen to be the same in each stratum (equal allocation) denoted by $n_c$, and all students were selected from the selected classes, $n_{abc}=N_{abc}$.

TABLE 6.

Number of clusters (PSUs) sampled with each sampling method.

Sampling method	$n=500$	$n=1000$	$n=2500$	$n=5000$	$n$ = 10,000
Cluster	23	47	116	233	465
Stratified cluster	24	51	126	249	501

For the two‐stage cluster sampling design, we provide additional results in Appendix 3. Table A1 shows the coverage and the SD/SE ratio for each estimated parameter obtained from 1000 replications over the estimated asymptotic standard error for all model parameters. This is for the one‐factor model with five items, using weighted pairwise likelihood and considering clustering in estimating the variability matrix $\mathbf{J}$ (16) in the Godambe information matrix. The results show that when clustering is considered in estimating the Godambe information matrix, the standard deviation to standard error ratio is much closer to one than when it is not accounted for.

Figure 3 displays the Type I error and power rates for the six test statistics under the two complex sampling designs across the five models outlined in Table 3. All test statistics performed well under both complex designs, except for the WaldDiag, which required larger sample sizes. Due to its poor performance even under SRS, it is not advisable to use the WaldDiag. The Wald and VCF Wald also exhibited poor performance in the larger models with smaller sample sizes. The Wald test is known to be unstable, mainly when used with data from complex sampling designs and has poor small‐sample behaviour (see e.g. Fay, ¹⁹⁸⁵; Lumley & Scott, ²⁰¹⁴). As Fay (1985) and Skinner (2019) have stated, the covariance matrix estimator of the estimated proportions is consistent, but in practice, it will typically be less precise than the multinomial analogues. This reduced precision seriously affects the inversion required in the Wald statistic. In particular, (Fay, 1985, p. 148) wrote that ‘the instability in the estimated inverse, in turn, inflates the rate of rejection under the null hypothesis, often enough to make the test unusable’.

FIGURE 3.

Simulation C: Type I error rates (top) and power analysis (bottom) for the two complex sampling designs at significance level $\alpha =5\%$ for the six test statistics in Table 4 across the five models in Table 3 and $n=500,1000,2500,5000,$10,000. The reported intervals are the 95% confidence intervals for each rejection proportion.

Similar to SRS, the power of the test statistics increased with sample size across all models. Notably, the 2F10V model required a sample size of 5000 to exhibit larger power levels in both sampling designs. The 3F15V model also needed a large sample size under the two‐stage stratified cluster design. For sample sizes of 500 and 1000, the test statistics exhibited low power. It is important to note that in those cases, the Pearson test statistic exhibited power higher than or similar to that of the Wald‐type tests except for the 1F8V model under both designs. The Wald‐type test statistics are possibly affected by the instability of computing and inverting the ${\widehat{\mathbf{\Omega }}}_2$ matrix and the likely sparsity even in the lower order margins in the bigger models and small sample sizes.

The asymptotic distribution of the proposed test statistics is checked using QQ plots. In Appendix 4, Figures A1, A2, A3 display QQ plots across the 1000 replications for all five models, five sample sizes and simple random sampling, two‐stage cluster sampling, and two‐stage stratified cluster sampling respectively. The plots reveal a generally close alignment between the theoretical distribution of each test and its Monte Carlo distribution. The largest deviations from the asymptotic distribution were found in the WaldDiag test.

6. DISCUSSION

This paper expands the pairwise likelihood estimation and limited information test statistics to accommodate complex and informative sampling designs. Specifically, sampling and informative weights are incorporated into the pairwise likelihood to consider the complex design during model estimation and the estimation of asymptotic standard errors. Additionally, the paper introduces limited‐information test statistics of Wald and Pearson's types tailored for complex sampling within the pairwise likelihood estimation framework.

The weighted pairwise likelihood produced unbiased estimates and robustly estimated standard errors. The limited‐information test statistics implemented using pairwise likelihood estimates showed good performance in terms of Type I error and power under simple random and two complex sampling designs, namely, two‐stage cluster and stratified cluster sampling. Although we studied both limited information Wald‐ and Pearson‐type test statistics that work satisfactorily in most cases, the Wald test statistic, for the reason we discussed above, can be unstable, mainly when used to data from complex sampling designs and has poor small‐sample behaviour. Therefore, we recommend using the Pearson test statistic with the pairwise likelihood estimator and the moment matching adjustment. Also, the Pearson test statistic does not require to invert the ${\mathbf{\Omega }}_2$ matrix.

One must also consider that sparseness may occur in the lower‐order margins for complex models and small sample sizes. In this case, further investigation is needed to study their performance.

Further research can extend the tests to ordinal responses and investigate resampling methods for estimating the proportions' variance under multistage designs. A Bayesian pairwise approach can also be helpful here for obtaining the test statistics distribution via posterior sampling without relying on the large sample theory presented in this paper.

AUTHOR CONTRIBUTIONS

Haziq Jamil: investigation; writing – review and editing; methodology; software; visualization. Irini Moustaki: conceptualization; methodology; validation; formal analysis; writing – original draft. Chris Skinner: methodology; writing – review and editing.

APPENDIX 1. PROOF

From (27) we wrote that the score vector can be expressed as a transformation of the $({\mathbf{p}}_2-{\mathit{\pi }}_2(\mathit{\theta }))$ vector, which we prove here. Extending (24) to vector form, we have

$$\begin{array}{ll}\hfill {ℝ}^m\ni \nabla {\ell }_{\text{P}}(\mathit{\theta })& =n{\left(\frac{\partial {\pi }_{c_i{c}_j}^{(ij)}(\mathit{\theta })}{\partial {\theta }_k}\right)}_{\genfrac{}{}{0ex}{}{i<j}{k=1,\text{…},m}}^{\prime }\text{diag}{\left({\pi }_{c_i{c}_j}^{(ij)}(\mathit{\theta })\right)}^{-1}{\left(p_{c_i{c}_j}^{(ij)}-{\pi }_{c_i{c}_j}^{(ij)}(\mathit{\theta })\right)}_{i<j}\\ \hfill & =n{\left(\frac{\partial {\pi }_{c_i{c}_j}^{(ij)}(\mathit{\theta })}{\partial {\theta }_k}\right)}_{\genfrac{}{}{0ex}{}{i<j}{k=1,\text{…},m}}^{\prime }\text{diag}{\left({\pi }_{c_i{c}_j}^{(ij)}(\mathit{\theta })\right)}^{-1}{({\mathbf{B}}_{c_i{c}_j}^{(ij)})}_{i<j}({\mathbf{p}}_2-{\mathit{\pi }}_2(\mathit{\theta })),\end{array}$$ (A1)

where ${({\mathbf{B}}_{c_i{c}_j}^{(ij)})}_{i<j}$ is the $4p(p-1)/2\times S$ matrix containing stacked $1\times S$ row vectors ${\mathbf{B}}_{c_i{c}_j}^{(ij)}$ satisfying (26). That is, for any $i,j\in \{1,\text{…},p\}$ and $i<j$,

$$\left(\begin{array}{c}{\mathbf{B}}_{00}^{(ij)}\\ {\mathbf{B}}_{10}^{(ij)}\\ {\mathbf{B}}_{01}^{(ij)}\\ {\mathbf{B}}_{11}^{(ij)}\end{array}\right)({\mathbf{p}}_2-{\mathit{\pi }}_2(\mathit{\theta }))=\left(\begin{array}{c}{p}_{00}^{(ij)}-{\pi }_{00}^{(ij)}(\mathit{\theta })\\ p_{10}^{(ij)}-{\pi }_{10}^{(ij)}(\mathit{\theta })\\ p_{01}^{(ij)}-{\pi }_{01}^{(ij)}(\mathit{\theta })\\ p_{11}^{(ij)}-{\pi }_{11}^{(ij)}(\mathit{\theta })\end{array}\right).$$

The design should be obvious and simple for the patterns $c_i{c}_j=10,01,11$. As for the pattern ‘00’, we may use the fact that the sum of the pairwise proportions is 1:

Combining this design matrix with the preceding terms in (A1), we get

$$\begin{array}{l}\hfill \mathbf{B}(\mathit{\theta }):={\left(\frac{\partial {\pi }_{c_i{c}_j}^{(ij)}(\mathit{\theta })}{\partial {\theta }_k}\right)}_{\genfrac{}{}{0ex}{}{i<j}{k=1,\text{…},m}}^{\prime }\text{diag}{\left({\pi }_{c_i{c}_j}^{(ij)}(\mathit{\theta })\right)}^{-1}{({\mathbf{B}}_{c_i{c}_j}^{(ij)})}_{i<j},\end{array}$$ (A2)

an $m\times S$ matrix dependent on the parameter vector $\mathit{\theta }$. Equation (27) then follows directly.

APPENDIX 2. ESTIMATION OF DEGREES OF FREEDOM

We discuss the moment matching procedure used to estimate the degrees of freedom of $X^2=n{\widehat{\mathbf{e}}}_2^{\prime }\widehat{\mathbf{\Xi }}{\widehat{\mathbf{e}}}_2$. This statistic has a limiting distribution of ${\sum }_{s=1}^S{\delta }_s{U}_s$, where each $U_s\hspace{0.17em}\stackrel{\text{iid}}{\sim }\hspace{0.17em}{\chi }_1^2$ and ${\delta }_s$ represents the eigenvalues of $\mathbf{M}={\mathbf{\Omega }}_2^{1/2}\mathbf{\Xi }{\mathbf{\Omega }}_2^{1/2}$.

The eigenvalues, which are invariant under cyclic permutations, can be more efficiently computed as the eigenvalues of $\mathbf{\Xi }{\mathbf{\Omega }}_2$, especially when $\mathbf{\Xi }$ is diagonal and ${\mathbf{\Omega }}_2$ is dense. In our context, these eigenvalues are estimated using the hat versions of ${\mathbf{\Omega }}_2$ and $\mathbf{\Xi }$. For the Wald and VCF Wald test, $\mathbf{M}$ becomes idempotent, resulting in an exact chi‐squared distribution for $X^2$. Consequently, these estimation procedures are specifically applied to the diagonal Wald and the Pearson test statistics.

Suppose $Y$ follows a ${\chi }_c^2$ distribution. We assume $X^2$ can be approximately represented as a linear transformation of this chi‐squared random variate:

$$X^2\approx a+bY.$$ (A3)

The first three moments of $X^2$ are expressed as (Mathai & Provost, 1992, theorem 3.2b.2, p. 53)

$${\mu }_1(X^2)=\text{tr}(\mathbf{\Xi }{\mathbf{\Omega }}_2),{\mu }_2(X^2)=2\text{tr}\left({(\mathbf{\Xi }{\mathbf{\Omega }}_2)}^2\right),{\mu }_3(X^2)=8\text{tr}\left({(\mathbf{\Xi }{\mathbf{\Omega }}_2)}^3\right).$$ (A4)

The moments of the approximating random variable $a+bY$ are denoted as

$${\mu }_1(a+bY)=a+bc,{\mu }_2(a+bY)=2b^{2}c,{\mu }_3(a+bY)=8b^{3}c.$$ (A5)

The formulas for the raw moments of a chi‐squared random variable $Y$ with $c$ degrees of freedom are $\text{E}(Y^k)=c(c+2)(c+4)\text{⋯}(c+2k-2)$. If we equate these moment expressions and solve for the parameters $a$, $b$, and $c$ in the three‐moment adjustment equation, we obtain

$$b=\frac{{\mu }_3(X^2)}{4{\mu }_2(X^2)},c=\frac{{\mu }_2(X^2)}{2b^2},a=\text{E}(X^2)-bc.$$

We can also use two‐moment and one‐moment matching techniques similarly. For the two‐moment adjustment, if we assume $X^2\approx bY$, where $Y\sim {\chi }_c^2$ (i.e. set $a=0$), the formulae for $b$ and $c$ are as follows:

$$b=\frac{{\mu }_2(X^2)}{2{\mu }_1(X^2)}\text{and}c=\frac{{\mu }_1(X^2)}{b}.$$

On the other hand, in the one‐moment adjustment, let us set $c=S$, which represents the number of degrees of freedom available for testing. In this case, $b$ can be calculated as $b={\mu }_1(X^2)/c$.

In these cases, the tail probabilities can be approximated as

$$\mathrm{Pr}(X^2>x)\approx \mathrm{Pr}\left(Y>\left.\frac{x-a}{b}\right|Y\sim \underset{c}{\overset{2}{\chi }}\right).$$

The moment matching procedure is often referred to as the Satterwaithe approximation. The parameter estimates for $a,b,c$ are calculated using estimated versions of $\mathbf{\Xi }$ and ${\mathbf{\Omega }}_2$ in the given formulas. In this paper, we use the three‐moment procedure due to previous findings suggesting inaccuracies in $p$‐values obtained from the one‐moment procedure (e.g. Maydeu‐Olivares & Joe, ²⁰⁰⁸).

APPENDIX 3. TWO‐STAGE CLUSTER SAMPLING DESIGN, COVERAGE, AND STANDARD ERRORS FOR PARAMETER ESTIMATES

3.1.

TABLE A1.

Two‐stage cluster sampling: Coverage rate and the ratio of standard deviation across replications to the estimated asymptotic standard error (SD/SE) for factor loadings and thresholds for the one‐factor model, $n=500,1000,5000$, weighted and weighted adjusted for clustering pairwise maximum likelihood (PMLW and PMLW‐adj).

	Coverage		SD/SE		Coverage		SD/SE		Coverage		SD/SE
	$n=500$				$n=1000$				$n=5000$
	PMLW	PMLW‐adj	PMLW	PMLW‐adj	PMLW	PMLW‐adj	PMLW	PMLW‐adj	PMLW	PMLW‐adj	PMLW	PMLW‐adj
Loadings
${\lambda }_1$	.94	.92	1.08	1.10	.93	.92	1.09	1.08	.94	.96	1.04	1.00
${\lambda }_2$	.94	.93	1.07	1.04	.93	.94	1.11	1.05	.93	.95	1.06	.99
${\lambda }_3$	.93	.93	1.11	1.06	.92	.93	1.14	1.06	.92	.95	1.08	.99
${\lambda }_4$	.95	.93	1.03	1.03	.94	.94	1.07	1.03	.92	.94	1.08	1.02
${\lambda }_5$	.92	.90	1.08	1.10	.93	.92	1.08	1.07	.95	.95	1.02	1.00
Thresholds
${\tau }_1$	.64	.93	2.19	1.08	.67	.95	2.09	1.02	.66	.96	1.99	.97
${\tau }_2$	.56	.93	2.63	1.05	.57	.95	2.45	.97	.57	.97	2.41	.95
${\tau }_3$	.70	.94	1.85	1.01	.71	.95	1.86	1.01	.71	.96	1.84	.99
${\tau }_4$	.80	.94	1.57	1.04	.81	.95	1.45	.96	.80	.96	1.51	.99
${\tau }_5$	.84	.92	1.38	1.05	.87	.96	1.28	.97	.87	.96	1.31	.98

APPENDIX 4. QQ PLOTS OF TEST STATISTICS UNDER $H_0$

4.1.

FIGURE A1.

QQ plot for the six test statistics in Table 4 across the five models in Table 3 from simulated data for simple random sampling under the null hypothesis and their respective theoretical asymptotic distributions, $n=500,1000,2500,5000$, 10,000.

FIGURE A2.

QQ plot for the six test statistics in Table 4 across the five models in Table 3 from simulated data for the two‐stage cluster sampling under the null hypothesis and their respective theoretical asymptotic distributions, $n=500,1000,2500,5000$, 10,000.

FIGURE A3.

QQ plot for the six test statistics in Table 4 across the five models in Table 3 from simulated data for the two‐stage stratified cluster sampling under the null hypothesis and their respective theoretical asymptotic distributions, $n=500,1000,2500,5000$, 10,000.

Jamil, H. , Moustaki, I. , & Skinner, C. (2025). Pairwise likelihood estimation and limited‐information goodness‐of‐fit test statistics for binary factor analysis models under complex survey sampling. British Journal of Mathematical and Statistical Psychology, 78, 258–285. 10.1111/bmsp.12358

DATA AVAILABILITY STATEMENT

All analyses were conducted in R (R Core Team, ²⁰²⁴) using the accompanying lavaan.bingof (Jamil, 2024) package. The R scripts are available at https://osf.io/2d97y/, including tabulated Type I error rates and power for all simulations conducted.