General location multivariate latent variable models for mixed correlated bounded continuous, ordinal, and nominal responses with non-ignorable missing data

Abstract

Using a multivariate latent variable approach, this article proposes some new general models to analyze the correlated bounded continuous and categorical (nominal or/and ordinal) responses with and without non-ignorable missing values. First, we discuss regression methods for jointly analyzing continuous, nominal, and ordinal responses that we motivated by analyzing data from studies of toxicity development. Second, using the beta and Dirichlet distributions, we extend the models so that some bounded continuous responses are replaced for continuous responses. The joint distribution of the bounded continuous, nominal and ordinal variables is decomposed into a marginal multinomial distribution for the nominal variable and a conditional multivariate joint distribution for the bounded continuous and ordinal variables given the nominal variable. We estimate the regression parameters under the new general location models using the maximum-likelihood method. Sensitivity analysis is also performed to study the influence of small perturbations of the parameters of the missing mechanisms of the model on the maximal normal curvature. The proposed models are applied to two data sets: BMI, Steatosis and Osteoporosis data and Tehran household expenditure budgets.

1. Introduction

The percentages, proportions, and fractions are some examples for variables supported on the standard unit interval. Some examples of proportions include the proportion of household income spent on electronic devices, the proportion of homicides involving firearms, and the proportion of crude oil converted to gasoline after distillation, etc. Various models have been proposed to analyze such data. One of the mentioned models is beta regression model introduced by Kieschnick and McCullough [18] and then Ferrari and Cribari-Neto [15] amplified the use of the beta regression model with changing the parameterization of the beta distribution indexed by mean and dispersion parameters. This model studied by many researchers such as Ferrari and Pinheiro [16], Anholeto et al. [3], Barreto-Souza and Simas [4], and Tabrizi et al. [28,29]. There are many practical situations in which the dataset contains more than one response supported on the standard unit interval. Multivariate beta regression models using a copula function to construct the joint distribution of the responses are proposed by De Souza and Da Silva Moura [14]. Sometimes, the mentioned responses lying in a bounded interval sum up to a constant (called compositional responses). For example, consider a demography study in which the proportions of the population with some specific religions are the response variables. Let $Y_1$, $Y_2$, $Y_3$, and $Y_4$ denote the proportions of the people who are Christians, Muslims, Jews, and the remaining religions living in a town, respectively. It is clear that $\sum _{p=1}^4{Y}_p=1$. Two other examples for compositional data are the sediment composition in a lake in which the samples are taken and classified into sand, silt, and clay by their weights and the proportions of household income spent on housing and fuel, foodstuffs, health-care, and remaining expenditures. Dirichlet regression models studied by Maier [20] can be used to analyze compositional data.

Multivariate data containing mixtures of continuous and discrete responses are common. Specially, continuous and categorical correlated responses data are commonly collected in medical studies. For example, consider the data from a medical study where the correlated responses are the ordinal responses of the steatosis and osteoporosis of the spine and continuous response of body mass index with the possibility of non-ignorable values. Also, sometimes the continuous response has bounded support. As an example, again consider the data from a medical study such that the bounded continuous responses are the proportions of four serum protein components in blood samples (Albumin, PreAlbumin, Globulin A, and Globulin B) which affect a special disease and again the ordinal responses are the steatosis and osteoporosis of the spine. As regards, separate analysis of each response has some shortcomings and it gives biased estimates for the parameters and misleading inference [11]. A way out of this problem is to use methods that simultaneously allow the joint modelling of the mixed data considering non-ignorable missing mechanisms. Specifying the joint distribution of the mixed responses can be formalized in two different ways: (1) specifying the marginal distribution of the discrete variables and the conditional distribution of the continuous variables, given the discrete variables, or (2) specifying the marginal distribution of the continuous variables and the conditional distribution of the discrete variables, given the continuous variables [13]. According to the first approach, for joint modelling of some continuous and nominal variables, one method is to use the general location model (GLOM). Olkin and Tate [22] described a general location model based on a multinomial distribution for the nominal response and a multivariate Gaussian model for the continuous response conditional on the discrete response. In each level of the composition of the nominal variables, the continuous variables are assumed to have a multivariate normal distribution with constant covariance matrix and changeable mean without considering any covariate effects on the responses simultaneously. Note that the GLOM does not accommodate dependence between ordinal and nominal responses and this model is suitable for the continuous and nominal responses. Also, there is not any types of GLOM in the literature in which continuous response has bounded support. In contrast, based on second approach, Cox [8] described a model in which the marginal distribution of the continuous response is Gaussian and it is multiplied by a logistic representation for the conditional distribution of the binary response given the continuous outcome. More recently, Cox and Wermuth [9] compared a number of different models based on these two factorizations of the joint distribution. Another method uses the simultaneous modelling of the continuous and discrete responses by applying correlated errors in the model to take into account the correlation between the responses [17]. To model simultaneously two correlated continuous and ordinal responses, one method is to use the concept of a latent variable. In this situation, it is supposed that the ordinal response is obtained by partitioning the space of an unobservable continuous variable called latent variable into non-overlapping intervals [24]. The conditional grouped continuous model (CGCM) uses the latent variable and applies the second approach to model the mentioned responses such that CGCM considers the multivariate normal distribution as the distribution for the latent variables and the conditional distribution of the latent variables given the continuous responses [2,26]. Finally, to model simultaneously three correlated continuous, ordinal, and nominal responses, De Leon and Carriere [12] combined two models GLOM and CGCM to obtain a model which called the general mixed data model (GMDM). The joint distribution of the responses in GMDM is expressed as the product of the joint distribution of the nominal and continuous responses (GLOM) multiplied by the conditional distribution of the ordinal responses given the nominal and continuous responses (CGCM). Paleti et al. [23] extended the GMDM such that this model includes mixed continuous, nominal, ordinal, and count variables. Mirkamali and Ganjali [21] extended the GLOM using a joint model for analyzing zero-inflated outcomes and skew continuous outcomes. Amiri et al. [1] proposed a general location model with factor analyzer covariance matrix structure.

For data having missing values, traditional methods which may give biased and inconsistent estimates are not suitable since in any way they ignore the missing data mechanism. Rubin [27] and Little and Rubin [19] define a typology of incomplete-data models and made important distinctions between the various types of missing mechanism. They classified the missing data mechanism into three categories missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR). The missing data mechanism is MCAR if the probability of the observed missingness indicator is dependent neither on the observed responses nor on the missing responses, MAR if, given the observed responses, it is not dependent on the missing responses and MNAR if, given the observed responses, it is dependent on the missing responses. From the likelihood-based inferences, MCAR and MAR can be regarded as ignorable. MCAR is ignorable for both sampling-based and likelihood-based inferences. In MNAR, the mechanism is non-ignorable. So, the important factor for selection of an approach to missing values is missing data mechanism. There are different strategies for dealing with missing values in this paper, we use a famous technique in which a dummy variable for whether a variable is missing or not is determined.

For general location model with missing data, Belin et al. [5] extended the general location model with MAR assumption in a health study. Peng et al. [25] proposed an extension of the general location model for causal inferences with non compliance and missing data on the outcomes. Also, also they considered the effect of covariates. In the above research, the univariate variable approach for the discrete and continuous responses with MAR mechanism is assumed. Also, Cui et al. [10] considered other different methods like the Bayesian approach. They proposed a novel Bayesian Gaussian copula factor approach that is proven to be consistent with ignorable data (MCAR). In this approach, the problem of learning about parameters of latent variable models from mixed continuous and ordinal data with missing values is considered. Chen and Tang [6] consider non-parametric approach for analyzing mixed continuous and discrete explanatory variables. In this approach, they proposed a non-parametric regression with a mixture of continuous and discrete explanatory variables with missing response.

In this paper, we extend the model of De Leon and Carriere [12] along with the multivariate latent variable approach for the bounded continuous, continuous, ordinal, and nominal responses with non-ignorable missing data. Also, we proposed a general location model for the univariate and multivariate bounded responses using beta and Dirichlet distributions. This model has not been yet considered by other researchers. The joint distribution of the bounded continuous, nominal and ordinal responses is decomposed into a marginal multinomial distribution for the nominal variables and a conditional joint distribution for the bounded continuous and ordinal responses given the nominal responses. In each level of composition of the nominal responses, the bounded continuous variables with missing values are beta and Dirichlet distribution with changeable mean with considering any covariate effects on the responses simultaneously. We also apply the random effects and latent variables approach to take into account the correlation between responses.

In Section 2, first, we review a GMDM model for the mixed correlated nominal, continuous, and ordinal responses. Also, then extended this model to a new model in which there is the possibility of the non-ignorable missing values in the continuous responses. Second, we extended the general location models for mixed correlated nominal, bounded continuous and ordinal data with and without non-ignorable responses. Section 3 is devoted to a brief discussion of the identifiability issues associated with the proposed joint models. In Section 4, some simulation studies are conducted in order to assess the performance of the proposed models. We use two dataset, the data of body mass index, the osteoporosis of the spine, waist, job status and type of the accommodation of the patients in Taleghani hospital of Tehran. The data of the household expenditure budgets, the education level of the household head. Also, having a toddler at home status in Tehran, to illustrate the application of the proposed models in Section 5. In Section 6, a sensitivity analysis is performed to study the influence of a small perturbation of the parameters of the missing mechanism on the maximal normal curvature. Finally, the paper ended with some remarks in the last section.

2. Models and likelihoods

Let $U_i=(U_{i1},\dots ,U_{iD}{)}^{\prime }$ denote the $D\times 1$ vector of nominal variables where the dth component of $U_i$, $U_{id}$, having $J_d$ possible states such that $J_d\in \{1,2,3,\dots \}$ and therefore $u_{id}=1,2,\dots ,j_d,\dots ,J_d$. So the vector $U_i$ defines a contingency table with $S=\prod _{d=1}^D{J}_d$ cells. Now, $j=(j_1,\dots ,j_D)$ represents a specific cell in the mentioned contingency table. Suppose $Y_{ij}=(Y_{i1j},\dots ,Y_{iPj}{)}^{\prime }$ show the continuous or bounded continuous response vector of the ith individual in the cell j, where $Y_{ipj}$ is the pth component of the response vector $Y_{ij}$ for $i=1,2,\dots ,n_j$ and $p=1,\dots ,P$. Also, $Z_{ij}=(Z_{i1j},\dots ,Z_{iQj}{)}^{\prime }$ denote the $Q\times 1$ ordinal response vector defined as

$$\begin{array}{r}{Z}_{iqj}=\{\begin{array}{ll}1,& Y_{iqj}^{\ast }\le {\theta }_{1q}\\ c+1,& {\theta }_{cq}<Y_{iqj}^{\ast }\le {\theta }_{(c+1)q},c=1,2,\dots ,C-1\\ C,& Y_{iqj}^{\ast }>{\theta }_{(C-1)q},\end{array},\end{array}$$

where $i=1,2,\dots ,n_j$, $q=1,\dots ,Q$, ${\theta }_{1q},{\theta }_{2q},\dots ,{\theta }_{(C-1)q}$ are the cut-point parameters, and $Y_{ij}^{\ast }=(Y_{i1j}^{\ast },\dots ,Y_{iQj}^{\ast }{)}^{\prime }$ denote the underlying latent variables for ordinal responses. Suppose that the mentioned three responses are recorded for n individuals and the cell frequency for each cell j is denoted by $n_j$. So, it is clear that $\sum _{j\in J}{n}_j=n$, where $J=\{(j_1,\dots ,j_D)|j_d\in \{1,2,\dots ,J_d\},d=1,2,\dots ,D\}$ is the set of all possible states for j. Moreover, the expected probability of falling into cell j is denoted by ${\pi }_j$, such that $\sum _{j\in J}{\pi }_j=1$. In all models in this paper, like the general location model, $U_i$ are modelled by a multinomial distribution where ${\pi }_j=Pr(U_i=j)$ is the vector of state probabilities. The nominal variables make together a contingency table. As an example, Figure C1 shows the demographic characteristics of the modelling for D = 2, $J_1=J_2=2$, S = 4, $J=\{(1,1),(1,2),(2,1),(2,2)\}$, P = 1, Q = 2, $Y_{ij}=Y_{i1j}$, $Z_{ij}=(Z_{i1j},Z_{i2j})$, and $n_{(1,1)}+n_{(1,2)}+n_{(2,1)}+n_{(2,2)}=n$.

2.1. GMDM for mixed continuous, ordinal, and nominal data

In this subsection, suppose that the support of the distribution of the continuous responses is not bounded.

2.1.1. Complete data model

In GMDM, the joint distribution of $[U_i,Z_{ij},Y_{ij}]$ (the joint distribution of the jth cell of the contingency table) can be factorized as

$$[U_i=j,Z_{ij},Y_{ij}]=[Z_{ij},Y_{ij}|U_i=j][U_i=j]=[Z_{ij}|Y_{ij},U_i=j][Y_{ij}|U_i=j][U_i=j],$$

where $[U_i]$, $[Z_{ij}|Y_{ij},U_i]$ and $[Y_{ij}|U_i]$ denote the marginal distribution of $U_i$, the conditional distribution of $Z_{ij}$ given $Y_{ij}$ and $U_i$, and the conditional distribution of $Y_{ij}$ given $U_i$, respectively. The GMDM takes the form:

$$\begin{array}{rl}& Y_{ipj}|(U_i=j)={\gamma }_{pj}^{\prime }{G}_{ij}+{\epsilon }_{ipj}^{(1)},p=1,\dots ,P,\\ & Y_{iqj}^{\ast }|(U_i=j)={\beta }_{qj}^{\prime }{H}_{iqj}+{\epsilon }_{iqj}^{(2)},q=1,\dots ,Q,\end{array}$$ (1)

where $\gamma =\{{\gamma }_{pj},p=1,\dots ,P,j\in J\}$, $\beta =\{{\beta }_{qj},q=1,\dots ,Q,j\in J\}$, $\pi =({\pi }_j,j\in J{)}^{\prime }$, also $G_{ij}$ and $H_{iqj}$ are the vector of the explanatory variables with regard to $Y_{ipj}$ and $Y_{iqj}^{\ast }$, respectively, for the ith individual in the cell j. Also, $({\epsilon }_{ij}^{(1)},{\epsilon }_{ij}^{(2)}{)}^{\prime }$, where ${\epsilon }_{ij}^{(1)}=({\epsilon }_{i1j}^{(1)},\dots ,{\epsilon }_{iPj}^{(1)}{)}^{\prime }$, ${\epsilon }_{ij}^{(2)}=({\epsilon }_{i1j}^{(2)},\dots ,{\epsilon }_{iQj}^{(2)}{)}^{\prime }$, ${\mathrm{\Sigma }}_{11}=\mathrm{Var}({\epsilon }_{ij}^{(1)})$, ${\mathrm{\Sigma }}_{22}=\mathrm{var}({\epsilon }_{ij}^{(2)})$ and ${\mathrm{\Sigma }}_{12}={\mathrm{\Sigma }}_{21}^{\prime }=\mathrm{Cov}({\epsilon }_{ij}^{(1)},{\epsilon }_{ij}^{(2)})$ is $(P+Q)$-variate normal distribution with zero mean and covariance matrix

$$\mathrm{\Sigma }=\left(\begin{array}{cc}{\mathrm{\Sigma }}_{11}& {\mathrm{\Sigma }}_{12}\\ {\mathrm{\Sigma }}_{21}& {\mathrm{\Sigma }}_{22}\end{array}\right),$$

such that Σ shows the covariance matrix between two errors (or two responses). So, the dependence between continuous responses and continuous latent variables has been modelled simultaneously.

The Likelihood for this model, which has been given in the Appendix (the Subsection 1.1 of the supplementary materials), shows the simplification obtained by using the assumption of normality for errors of the model in the system of Equations (1).

2.1.2. Incomplete data model

Typically, when missing data occur in an outcome, assume $R_{y_{ij}}=(R_{y_{i1j}},\dots ,R_{y_{iPj}}{)}^{\prime }$ as the missingness indicator vector related to $Y_{ij}$ where $R_{y_{ipj}}$ is defined as

$$R_{y_{ipj}}=\{\begin{array}{ll}1,& \mathrm{if}{R}_{y_{ipj}}^{\ast }>0\\ 0,& \mathrm{Otherwise}.\end{array}$$

Also, suppose that $R_{z_{ij}}=(R_{z_{i1j}},\dots ,R_{z_{iQj}}{)}^{\prime }$ is the missingness indicator vector related to $Z_{ij}$ where $R_{z_{iqj}}$ is defined as

$$R_{z_{iqj}}=\{\begin{array}{ll}1,& \mathrm{if}{R}_{z_{iqj}}^{\ast }>0\\ 0,& \mathrm{Otherwise}.\end{array}$$

In the above definitions, $R_{y_{ipj}}^{\ast }$ and $R_{z_{iqj}}^{\ast }$ denote the underlying latent variables of the non-response mechanism for the continuous and ordinal variables, respectively. The GMDM takes the form:

$$\begin{array}{rl}& Y_{ipj}|(U_i=j)={\gamma }_{pj}^{\prime }{G}_{ij}+{\epsilon }_{ipj}^{(1)},p=1,\dots ,P,\\ & Y_{iqj}^{\ast }|(U_i=j)={\beta }_{qj}^{\prime }{H}_{iqj}+{\epsilon }_{iqj}^{(2)},q=1,\dots ,Q,\\ & R_{y_{ipj}}^{\ast }|(U_i=j)={\alpha }_{pj}^{\prime }{M}_{ipj}+{\epsilon }_{ipj}^{(3)},p=1,\dots ,P,\\ & R_{z_{iqj}}^{\ast }|(U_i=j)={\eta }_{qj}^{\prime }{N}_{iqj}+{\epsilon }_{iqj}^{(4)},q=1,\dots ,Q,\end{array}$$ (2)

where $\alpha =\{{\alpha }_{pj},p=1,\dots ,P,j\in J\}$, $\eta =\{{\eta }_{qj},q=1,\dots ,Q,j\in J\}$, $M_{ipj}$ and $N_{iqj}$ are the vector of the explanatory variables for the ith individual in the cell j. Let

$$({\epsilon }_{ij}^{(1{)}^{\prime }},{\epsilon }_{ij}^{(2{)}^{\prime }},{\epsilon }_{ij}^{(3{)}^{\prime }},{\epsilon }_{ij}^{(4{)}^{\prime }}{)}^{\prime }\stackrel{iid}{\sim }MVN(0,{\mathrm{\Sigma }}_{\epsilon }),$$

where ${\epsilon }_{ij}^{(3)}=({\epsilon }_{i1j}^{(3)},\dots ,{\epsilon }_{iPj}^{(3)}{)}^{\prime }$, ${\epsilon }_{ij}^{(4)}=({\epsilon }_{i1j}^{(4)},\dots ,{\epsilon }_{iQj}^{(4)}{)}^{\prime }$, and

$${\mathrm{\Sigma }}_{\epsilon }=\left(\begin{array}{cccc}{\mathrm{\Sigma }}_{11}& {\mathrm{\Sigma }}_{12}& {\mathrm{\Sigma }}_{13}& {\mathrm{\Sigma }}_{14}\\ {\mathrm{\Sigma }}_{21}& {\mathrm{\Sigma }}_{22}& {\mathrm{\Sigma }}_{23}& {\mathrm{\Sigma }}_{24}\\ {\mathrm{\Sigma }}_{31}& {\mathrm{\Sigma }}_{32}& {\mathrm{\Sigma }}_{33}& {\mathrm{\Sigma }}_{34}\\ {\mathrm{\Sigma }}_{41}& {\mathrm{\Sigma }}_{42}& {\mathrm{\Sigma }}_{43}& {\mathrm{\Sigma }}_{44}\end{array}\right),$$

where ${\mathrm{\Sigma }}_{uu}=\mathrm{Var}({\epsilon }_{ij}^{(u)})$, for u = 1, 2, 3, 4 and ${\mathrm{\Sigma }}_{uv}=\mathrm{Cov}({\epsilon }_{ij}^{(u)},{\epsilon }_{ij}^{(v)})$, u<v, u, v = 1, 2, 3, 4 and ${\mathrm{\Sigma }}_{uv}={\mathrm{\Sigma }}_{vu}^{\prime }$. Note that if one of the matrices ${\mathrm{\Sigma }}_{13},{\mathrm{\Sigma }}_{14},{\mathrm{\Sigma }}_{23},{\mathrm{\Sigma }}_{24}$ is not zero, then the missing data mechanism of the response is not missing completely at random.

The Likelihood for this model, which has been given in the Appendix (the Subsection 1.2 of the supplementary materials)., shows the simplification obtained by using the assumption of normality for errors of the model in the system of Equations (2).

2.2. GLOM for mixed bounded continuous, ordinal, and nominal data

Here, we want to extend the general location model for multivariate bounded continuous responses using the Dirichlet distribution. In this subsection, assume that $Y_{ipj}$ is bounded continuous for $i=1,\dots ,n_j$, $p=1,\dots ,P$, and $j\in J$ such that it is a recorded percentage or proportion. So, $0<Y_{ipj}<1$. Note that if $a<Y_{ipj}<b$ such that a and b ( $a,b\in R$) are the known real numbers, we can use the transformation $Y_{ipj}^{\ast }=(Y_{ipj}-a)/(b-a)$ ( $0<Y_{ipj}^{\ast }<1$) and then model the $Y_{ipj}^{\ast }$ by the method introduced in this article. Let $Y_{ipj}\sim \mathrm{Beta}(\varphi {\mu }_{ipj},\varphi (1-{\mu }_{ipj}))$, such that

$$f(y_{ipj};{\mu }_{ipj},\varphi )=\frac{\mathrm{\Gamma }(\varphi )}{\mathrm{\Gamma }(\varphi {\mu }_{ipj})\mathrm{\Gamma }(\varphi (1-{\mu }_{ipj}))}{y}_{ipj}^{\varphi {\mu }_{ipj}-1}(1-y_{ipj}{)}^{(1-{\mu }_{ipj})\varphi -1},y_{ipj}\in (0,1),$$ (3)

where $0<{\mu }_{ipj}<1$, $\varphi >0$, and $\mathrm{\Gamma }(.)$ is the gamma function. By the above parameterization of the beta distribution, $E(Y_{ipj})={\mu }_{ipj}$ and $\mathrm{Var}(Y_{ipj})={\mu }_{ipj}(1-{\mu }_{ipj})/(1+\varphi )$. Also, let the joint distribution of $Y_{ij}$ be the Dirichlet distribution ( $Y_{ij}\sim \mathcal{D}({\mu }_{ij},\varphi )$) of order P + 1, such that

$$\begin{array}{rl}f(y_{ij};{\mu }_{ij},\varphi )& =\frac{\mathrm{\Gamma }(\varphi )}{\mathrm{\Gamma }(\varphi (1-\sum _{p=1}^P{\mu }_{ipj}))\prod _{p=1}^P\mathrm{\Gamma }(\varphi {\mu }_{ipj})}\\ & \times {(1-\sum _{p=1}^P{y}_{ipj})}^{\varphi (1-\sum _{p=1}^P{\mu }_{ipj})}\prod _{p=1}^P{y}_{ipj}^{\varphi {\mu }_{ipj}-1},\end{array}$$ (4)

$y_{ipj}\in (0,1)$, $\sum _{p=1}^P{y}_{ipj}<1$, ${\mu }_{ij}=({\mu }_{i1j},\dots ,{\mu }_{iPj}{)}^{\prime }$, and $\mathrm{Cov}(y_{ipj},y_{i{p}^{\prime }j})=-{\mu }_{ipj}{\mu }_{i{p}^{\prime }j}/(1+\varphi )$ for $p\ne p^{\prime }$.

Note 2.1

Dirichlet regression models are commonly used to analyze a set of bounded continuous responses that sum up to a constant. Based on the above notation, this regression is usually used when $\sum _{p=1}^{P+1}{Y}_{ipj}=1$, where $Y_{i(P+1)j}=1-\sum _{p=1}^P{Y}_{ipj}$. In such situation, the joint distribution of $(Y_{ij}^{\prime },Y_{i(P+1)j})$ is as follows:

$$f(y_{ij},y_{i(P+1)j};{\mu }_{ij},\varphi )=\frac{\mathrm{\Gamma }(\varphi )}{\prod _{p=1}^{P+1}\mathrm{\Gamma }(\varphi {\mu }_{ipj})}\prod _{p=1}^{P+1}{y}_{ipj}^{\varphi {\mu }_{ipj}-1},y_{ipj}\in (0,1),\sum _{p=1}^{P+1}{y}_{ipj}=1.$$

It is suitable for modelling the compositional data.

2.2.1. Complete data model

An extended general location model for correlated $(0,1)$-supported, ordinal, and nominal responses take the form:

$$\begin{array}{rl}& \log \left(\frac{{\mu }_{ipj}}{1-\sum _{p=1}^P{\mu }_{ipj}}\right)|(U_i=j,b_{ij})={\gamma }_{pj}^{\prime }{G}_{ipj}+b_{ij},p=1,\dots ,P\\ & \mathrm{logit}(\mathrm{Pr}(Z_{iqj}\le c))|(U_i=j,b_{ij})={\beta }_{qjc}+{\beta }_{qj}^{\prime }{H}_{iqj}+b_{ij},q=1,\dots ,Q,\end{array}$$ (5)

For $c=1,\dots ,C-1$, with C−1 strictly increasing model intercept parameters ${\beta }_{qj1}<{\beta }_{qj2}<\cdots <{\beta }_{qj(C-1)}$ and the link function $\mathrm{logit}(a)=\log (a/(1-a))$. Also, the shared random effect is represented by $b_{ij}$ assumed to be distributed in the population as $N(0,{\sigma }_b^2)$ given $U_i=j$. Note that for $(i,j)\ne (i^{\prime },j^{\prime })$, $b_{ij}$ and $b_{i^{\prime }{j}^{\prime }}$ are independent. Moreover, ${\beta }_{qj}$ does not contain an intercept. It is noteworthy that under this model

$${\mu }_{ipj}|(U_i=j,b_{ij})=\frac{e^{{\gamma }_{pj}^{\prime }{G}_{ipj}+b_{ij}}}{1+\sum _{p=1}^P{e}^{{\gamma }_{pj}^{\prime }{G}_{ipj}+b_{ij}}},p=1,\dots ,P.$$

So, model (5) guarantees that $0<{\mu }_{ipj}<1$ for $p=1,\dots ,P$ and $\sum _{p=1}^P{\mu }_{ipj}<1$. The joint distribution of $[U_i=j,Z_{ij},Y_{ij}]$ can be factorized as

$$[U_i=j,Z_{ij},Y_{ij}]=[Z_{ij},Y_{ij}|U_i=j][U_i=j].$$

Note that $Y_{ij}|U_i=j,b_{ij}\sim \mathcal{D}({\mu }_{ij},\varphi )$ and the $Y_{ij}$ and $Z_{ij}$ given $b_{ij}$ are independent. The ${\sigma }_b$ is a parameter related to the dependency between $(0,1)$-supported and ordinal responses. The likelihood of model (5) is presented in the Appendix (the Subsection 1.3 of the supplementary materials).

2.2.2. Incomplete data model

Again, let $R_{y_{ipj}}$ and $R_{z_{iqj}}$ be the observed missingness indicators related to $Y_{ipj}$ and $Z_{iqj}$. To manage the missing mechanism issue, the latent variables $R_{y_{ipj}}^{\ast }$ and $R_{z_{iqj}}^{\ast }$ can be applied, but now we prefer to use another approach. Let $R_{y_{ipj}}$ and $R_{z_{iqj}}$ be distributed according to Bernoulli distribution. The incomplete joint model takes the form:

$$\begin{array}{rl}& \log \left(\frac{{\mu }_{ipj}}{1-\sum _{p=1}^P{\mu }_{ipj}}\right)|(U_i=j,b_{ij})={\gamma }_{pj}^{\prime }{G}_{ipj}+b_{ij},p=1,\dots ,P,\\ & \mathrm{logit}(\mathrm{Pr}(Z_{iqj}\le c))|(U_i=j,b_{ij})={\beta }_{qjc}+{\beta }_{qj}^{\prime }{H}_{iqj}+b_{ij},q=1,\dots ,Q,\\ & \mathrm{logit}(\mathrm{Pr}(R_{y_{ipj}}=1))|(U_i=j,y_{ipj})={\alpha }_{pj}^{\prime }{M}_{ipj}+{\omega }_{pj}{y}_{ipj},\\ & \mathrm{logit}(\mathrm{Pr}(R_{z_{iqj}}=1))|(U_i=j,z_{iqj})={\eta }_{qj}^{\prime }{N}_{iqj}+{\psi }_{qj}{z}_{iqj},\end{array}$$ (6)

For $c=1,\dots ,C-1$, where the statistical significance of ${\omega }_{pj}$ and ${\psi }_{qj}$ imply that the missingness mechanism is MNAR.

The Likelihood for this model has been given in the Appendix (the Subsection 1.4 of the supplementary materials). The likelihoods of the mentioned models (1) and (2) can be maximized by the function ‘nlminb’ in software R. This function uses the port routine optimization method given in ‘ http://netlib.bell-labs.com/cm/cs/cstr/153.pdf’. The function uses a sequential quadratic programming (SQP) method to minimize the requested function. Needless to say, many problems in statistics are of the form finding the values of the parameters that maximize the likelihood function in which some constraints are also imposed on the parameters. For example, in models (5) and (6) we know that ${\beta }_{qj1}<{\beta }_{qj2}<\cdots <{\beta }_{qj(C-1)}$. This adds a combinatorial layer to the problem, which makes it much harder to solve. There are a lot more packages available to solve optimization problems in R. ‘Rsolnp’ package can be applied to solve general non-linear optimization problems using augmented Lagrange multiplier method with an SQP interior algorithm. It is a reason to choose the ‘solnp’ function to maximize the likelihoods of models (5) and (6). Moreover, the observed Hessian matrix may be obtained by ‘nlminb’ function or maybe provided by function ‘fdHess’.

3. Identifiability conditions

The model identifiability issue have been discussed here. All proofs are given in the Appendix (the Section 2 of the supplementary materials).

Definition 3.1

The model is identifiable if for any two different values ${\theta }_1\ne {\theta }_2$ in Θ, the corresponding probability distributions $\mathrm{Pr}({\theta }_1)$ and $\mathrm{Pr}({\theta }_2)$ are different.

Now, first, we review three theorems which can be proven using the same argument as applied in [30]. We apply them to prove the other theorems presented in this section. In the following, we call the model of all $(Y_i^{\prime },Z_i^{\prime },U_i^{\prime }{)}^{\prime }$s or $(Y_i^{\prime },Z_i^{\prime },U_i^{\prime },R_{y_i},R_{z_i}{)}^{\prime }$s (response variables of all individuals) the joint model and the model of the ith response variables the individual model, where $Y_i=(Y_{ij}^{\prime };j\in J{)}^{\prime }$ and $Z_i=(Z_{ij}^{\prime };j\in J{)}^{\prime }$. In the following theorems, we use the non-identifiability definition of a covariance matrix discussed by Wang [31].

Theorem 3.2

The joint model is identifiable if and only if at least one individual model is identifiable.

The result of Theorem 3.2 to model (1) yields that $({\pi }_j;j\in J)$, $(\gamma ,{\mathrm{\Sigma }}_{11})$, $(\beta ,{\mathrm{\Sigma }}_{22},{\theta }_{1q},\dots ,,{\theta }_{(C-1)q},\dots ,{\theta }_{(C-1)Q})$, and ${\mathrm{\Sigma }}_{12}$ are identifiable if they are identifiable in the separate models for $U_i$, $Y_i|U_i$, $Z_i|U_i$, and $Z_i|Y_i,U_i$ responses, respectively. Also, in model (5), $({\pi }_j;j\in J)$, $({\gamma }_j,\varphi )$, and ${\beta }_{0qj}=({\beta }_{qj1},{\beta }_{qj2},\dots ,{\beta }_{qj(C-1)}{)}^{\prime }$. $({\beta }_{0qj},{\beta }_{qj})$ are identifiable if they are identifiable in the separate models for $U_i$, $Y_{ij}|U_i=j$, and $Z_{iqj}|U_i=j$ responses, respectively. Moreover, ${\sigma }_b$ is identifiable, if it is identifiable in at least one of the separate models for $Y_i|U_i$ and $Z_i|Y_i,U_i$ responses.

Proposition 3.3

model (1) is identifiable under the following conditions:

1. The parameter vector γ can include the intercept parameters but β should not include any intercept.

2. ${\mathrm{\Sigma }}_{22}$ is restricted to be a correlation matrix $R_{22}$ in which all diagonal elements equal one.

3. All $n_j\times \mathrm{length}(G_{ij})$ design matrices $G_j=[G_{1j},\dots ,G_{n_{j}j}{]}^{\prime }$ have full ranks.

4. $H_{iqj}$ contains at least one continuous covariate.

Proposition 3.4

The necessary conditions for identifiability of model (2) are:

1. The parameter vector γ, α, and η can include the intercept parameters but β should not include any intercept.

2. ${\mathrm{\Sigma }}_{22},$ ${\mathrm{\Sigma }}_{33},$ and ${\mathrm{\Sigma }}_{44}$ are restricted to be the correlation matrices $R_{22},$ $R_{33},$ and $R_{44}$ in which all diagonal elements equal one.

3. All $n_j\times \mathrm{length}(G_{ij})$ design matrices $G_j=[G_{1j},\dots ,G_{n_{j}j}{]}^{\prime }$ have full ranks.

Proposition 3.5

$(\pi ,\gamma ,\varphi )$ in model (5) is identifiable under the following conditions:

1. ${\gamma }_{pj}={\gamma }_j$ for $p=1,\dots ,P$.

2. Let γ= ( ${\gamma }_j,i\in J$). The parameter vector γ should not include any intercept.

3. There are at least two values for p (p and $p^{\prime }$) such that all the components in the covariate vectors $G_{ipj}$ and $G_{i{p}^{\prime }j}$ take all values in ${\mathcal{S}}_{G_p}$ and ${\mathcal{S}}_{G_{p^{\prime }}},$ where ${\mathcal{S}}_{G_p},{\mathcal{S}}_{G_{p^{\prime }}}\subseteq \mathcal{R}$ contains at least one interval and zero. For other covariates, it is enough that their ${\mathcal{S}}_G$ contain zero.

For example, ${\mathcal{S}}_{G_p}$ and ${\mathcal{S}}_{G_p^{\prime }}$ can be $\mathcal{R}$, $(-a,b)$, $(-a,b]\bigcup (-\mathrm{\infty },c)$, $[-a,+\mathrm{\infty })$, or $(a,b)\bigcup \{0,1,2\}$, where $a,b,c\in {\mathcal{R}}^{+}$.

4. Simulation study

Here, three different simulation studies are considered to assess the performances of models (1) and (5) for complete data and model (2) for incomplete data. Note that the optimization algorithms need to specify the initial values for the parameters. As in the simulation studies, the true values of the parameters are known, one may use values close to them as the initials to evaluate the performances of the models under study. However, these values are unknown in practice and the final estimation results might be affected by the initial values. Therefore, we should suggest at least one appropriate method for the specification of initial points. As a comparison, we estimates the parameters of model (5) in the second simulation study under two different sets of the initial values, the real parameter values and our suggested initial points. Then, the results are compared. Tables and Figures are relegated to the Appendix (the Section 3 of the supplementary materials).

4.1. Simulation study 1: GMDM for mixed continuous, ordinal, and nominal responses

In the first study, let D = 2, $J_1=J_2=2$, P = 1, Q = 2, and C = 3. So, we consider the case of two nominal variables $U_1$ and $U_2$ each with two states. The variables $U_1$ and $U_2$ are generated from a multinomial distribution with $\pi =(0.25,0.25,0.25,0.25{)}^{\prime }$. Also, the variables $Y_{1j}$, $Y_{1j}^{\ast }$, and $Y_{2j}^{\ast }$ are generated from a multivariate normal distribution with zero mean ( ${\gamma }_j={\beta }_{1j}={\beta }_{2j}=0$) and covariance matrix

$$\begin{array}{r}\mathrm{\Sigma }=\left(\begin{array}{ccc}1& 0.5& 0.5\\ & 1& 0.5\\ & & 1\end{array}\right).\end{array}$$

In fact, First, we want to study the model (1) with the covariates $G_{ij}$, $H_{i1j}$, and $H_{i2j}$ designed to take values in $\mathcal{R}$. So, we generate them from normal distributions such that

$$G_{ij}\stackrel{iid}{\sim }N(0,1),\left(\begin{array}{c}{H}_{i1j}\\ H_{i2j}\end{array}\right)\stackrel{iid}{\sim }{N}_2(\mathbf{0},\left[\begin{array}{cc}1& 0.5\\ 0.5& 1\end{array}\right]).$$

Let ${\theta }_{11}={\theta }_{12}=-1$ and ${\theta }_{21}={\theta }_{22}=1$. The ordinal variables $Z_{1j}$ and $Z_{2j}$ with three levels are defined as

$$Z_{i1j}=\{\begin{array}{ll}1,& Y_{i1j}^{\ast }\le {\theta }_{11}\\ 2,& {\theta }_{11}<Y_{i1j}^{\ast }\le {\theta }_{21},\\ 3,& Y_{i1j}^{\ast }>{\theta }_{21},\end{array}$$

and

$$Z_{i2j}=\{\begin{array}{ll}1,& Y_{i2j}^{\ast }\le {\theta }_{12}\\ 2,& {\theta }_{12}<Y_{i2j}^{\ast }\le {\theta }_{22},\\ 3,& Y_{i2j}^{\ast }>{\theta }_{22},\end{array}.$$

Three sample sizes $n=1000,5000,$ and 10, 000 are used for this model. Also, 1000 Monte Carlo replications are applied. In the first simulation study, for n = 1000, we have $n_{11}=250$, $n_{12}=301$, $n_{21}=332$, and $n_{22}=117$, for n = 5000, we have $n_{11}=1340$, $n_{12}=1421$, $n_{21}=527$, and $n_{22}=1712$, and for n = 10, 000, we have $n_{11}=2504$, $n_{12}=2415$, $n_{21}=2540$, and $n_{22}=2541$. Figure 1 of the Section 3 of the supplementary materials shows the demographic characteristics of the model for the first simulation study. So, the following simple model is the target model of the first simulation study

$$\begin{array}{rl}& Y_{i1j}|(U_i=j)={\mu }_{y_{i1j}}+{ϵ}_{i1j}^{(1)},\\ & Y_{i1j}^{\ast }|(U_i=j)={\mu }_{y_{i1j}^{\ast }}+{ϵ}_{i1j}^{(2)},\\ & Y_{i2j}^{\ast }|(U_i=j)={\mu }_{y_{i2j}^{\ast }}+{ϵ}_{i2j}^{(2)},\end{array}$$ (7)

where ${\mu }_{y_{i1j}}={\gamma }_j{G}_{ij}$, ${\mu }_{y_{i1j}^{\ast }}={\beta }_{1j}{H}_{i1j}$, and ${\mu }_{y_{i2j}^{\ast }}={\beta }_{2j}{H}_{i2j}$. The main results are presented in Table 1 of the Section 3 of the supplementary materials. Note that, it contains the average estimated parameters over all simulations and they are close to the true values. So, model (7) produces consistent estimates of the regression parameters.

In the second study, a missing data mechanism related to the continuous response, $Y_{1j}$, was added to the model. Let consider four continuous variables $Y_{1j}$, $R_{y_{1j}}^{\ast }$, $Y_{1j}^{\ast }$, and $Y_{2j}^{\ast }$. The variables are generated from a multivariate normal distribution with zero mean ( ${\alpha }_j=0$ and $M_{ij}\sim N(0,1)$) and covariance matrix

$$\mathrm{\Sigma }=\left(\begin{array}{cccc}1& 0.5& 0.5& 0.5\\ & 1& 0.5& 0.5\\ & & 1& 0.5\\ & & & 1\end{array}\right).$$

$R_{y_{i1j}}$ is defined as follows:

$$R_{y_{i1j}}=\{\begin{array}{ll}1,& \mathrm{if}{R}_{y_{i1j}}^{\ast }>0\\ 0,& \mathrm{Otherwise}.\end{array}$$

Figure 2 of the Section 3 of the supplementary materials shows the demographic characteristics of the model for the second simulation study and the rates of missingness in $Y_{1j}$. In this simulation study, we analyze the following model:

$$\begin{array}{rl}& Y_{i1j}|(U_i=j)={\mu }_{y_{i1j}}+{ϵ}_{i1j}^{(1)},\\ & Y_{i1j}^{\ast }|(U_i=j)={\mu }_{y_{i1j}^{\ast }}+{ϵ}_{i1j}^{(2)},\\ & Y_{i2j}^{\ast }|(U_i=j)={\mu }_{y_{i2j}^{\ast }}+{ϵ}_{i2j}^{(2)},\\ & R_{y_{1j}}^{\ast }|(U_i=j)={\mu }_{R_{y_{i1j}}^{\ast }}+{ϵ}_{i1j}^{(3)},\end{array}$$ (8)

where ${\mu }_{R_{y_{i1j}}^{\ast }}={\alpha }_j{M}_{ij}$. Estimation results are given in Table 2 of the Section 3 of the supplementary materials. These results show that the parameter estimates are close to the true values of the parameters. Here, the true values of parameters have been used as the initial values.

4.2. Simulation study 2: GLOM for mixed bounded continuous, ordinal, and nominal responses

Consider the model (5) and let P = 2, Q = 1, D = 2, $J_1=J_2=2$, and C = 3. Three effective sample sizes 100, 500, and 1000 are used in this subsection. The outcome model is as follows:

$$\begin{array}{rl}& \log \left(\frac{{\mu }_{i1j}}{1-{\mu }_{i1j}-{\mu }_{i2j}}\right)|(U_i=j,b_{ij})={\gamma }_j{G}_{i1j}+b_{ij},\\ & \log \left(\frac{{\mu }_{i2j}}{1-{\mu }_{i1j}-{\mu }_{i2j}}\right)|(U_i=j,b_{ij})={\gamma }_j{G}_{i2j}+b_{ij},\\ & \mathrm{logit}(\mathrm{Pr}(Z_{i1j}\le c))|(U_i=j,b_{ij})={\beta }_{1jc}+{\beta }_{1j}{H}_{i1j}+b_{ij},c=1,2,\end{array}$$ (9)

where $j\in \{(1,1),(1,2),(2,1),(2,2)\}$. We also choose ${\gamma }_{(1,1)}={\gamma }_{(1,2)}={\gamma }_{(2,1)}={\gamma }_{(2,2)}=1$, $\varphi =2$, ${\sigma }_b=0.5$, ${\pi }_{(1,1)}={\pi }_{(1,2)}={\pi }_{(2,1)}=0.25$, ${\beta }_{1(1,1)1}={\beta }_{1(1,2)1}={\beta }_{1(2,1)1}={\beta }_{1(2,2)1}=1$, $,{\beta }_{1(1,1)2}={\beta }_{1(1,2)2}={\beta }_{1(2,1)2}={\beta }_{1(2,2)2}=2$, and ${\beta }_{1(1,1)}={\beta }_{1(1,2)}={\beta }_{1(2,1)}={\beta }_{1(2,2)}=1$ to perform this simulation study. The covariates $G_{ipj}$ and $H_{ij}$ for p = 1, 2 are simulated independently as follows:

$$H_{ij}\stackrel{iid}{\sim }N(0,1),\left(\begin{array}{c}{G}_{i1j}\\ G_{i2j}\end{array}\right)\stackrel{iid}{\sim }{N}_2(\mathbf{0},\left[\begin{array}{cc}1& 0.5\\ 0.5& 1\end{array}\right]).$$

The ‘solnp’ function of ‘Rsolnp’ package in software ‘R’ is used to maximize the log-likelihood function in terms of parameters with the inequality constraints ${\beta }_{1(1,1)1}<{\beta }_{1(1,1)2}$, ${\beta }_{1(1,2)1}<{\beta }_{1(1,2)2}$, ${\beta }_{1(2,1)1}<{\beta }_{1(2,1)2}$, and ${\beta }_{1(2,2)1}<{\beta }_{1(2,2)2}$. Also, ‘fdHess’ function is used to gain observed information matrix. The results are summarized in Table 3 of the Section 3 of the supplementary materials. For n = 100, there are two columns for the estimates where the estimated values in the first column (Est.) are obtained when initial points are close to real values and the estimated values in the second column (Est.0) are obtained when the initial points are chosen as follow: we fit eight separate beta regression models for $Y_{i1(1,1)}$, $Y_{i1(1,2)}$, $Y_{i1(2,1)}$, $Y_{i1(2,2)}$, $Y_{i2(1,1)}$, $Y_{i2(1,2)}$, $Y_{i2(2,1)}$, and $Y_{i2(2,2)}$. It gives us some estimates for ${\gamma }_j$s. The means of two obtained estimates for each ${\gamma }_j$ from two models for $Y_{i1j}$, and $Y_{i2j}$ are our target initial values. Also, we use a Dirichlet regression for $Y_1$, $Y_2$, and $1-Y_1-Y_2$ (only intercept in the model) for finding an estimates for φ. ${\pi }_j$ is estimated by $n_j/n$. The initial values for β are found by fitting four separate cumulative logistic regression models for $Z_{i1j}$. Finally, in the above twelve models, we use a random sample generated from the standard normal distribution, $G_{ipj}$, and $H_{iqj}$ as the covariates. So, in these models, we gain some initial estimates for ${\sigma }_b$. The mean of all obtained estimates for ${\sigma }_b$ from twelve models is considered here. It is evident that the parameter estimates are close to the true values and the more the value of n the better the estimates and the smaller the standard errors. For n = 100, the estimates in the Est. column show, compared to the estimates in the Est.0 column, that the estimates given real parameter values as initial points and the estimates given our suggested initial values match, up to the first four decimal points. Figure 3 of the Section 3 of the supplementary materials presents the mean squared errors (MSE) varying with respect to n. As the number of subjects increases, the MSEs of estimated parameters are decreased.

5. Application

This section contains two applications of the proposed models in the second section. First, we use model (5) for the Tehran household expenditure budgets dataset described in the following subsection. Second, we apply model (2) for the medical dataset described in the Subsection 5.2.

5.1. Application 1: tehran household expenditure budgets data

The set of household expenditure data available to us is from a study of 37,962 Tehran households. This dataset, collected by the statistical centre of Iran at 2017, is available on the statistical centre of Iran website. From this set, we have for simplicity selected a random sample of 1000 households. For each household, data is available on the number of persons ( $G_{i1j}$), the total household income, the education level of the head of household ( $Z_{ij}$), having a toddler at home status ( $U_i$), the area of the house in square meters ( $G_{i2j}$), the smoking or drinking status of the head of household ( $G_{i3j}$), and the proportion of income spent on monthly expenditures in some commodity/service groups ( $Y_{ij}$) such as foodstuffs ( $Y_{i1j}$), housing and fuel ( $Y_{i2j}$), health-care ( $Y_{i3j}$), and so on. A brief description of the data is presented in Table 1. Also, Table 4 of the Section 3 of the supplementary materials shows the contingency table of the nominal and ordinal responses (U and Z). According to this Table, the most frequent type of the family in the dataset is the family with no toddler when the household head's education level is primary.

Table 1. Descriptive statistics for the household expenditure budgets data.

Dependent variable name	Notation	Level	Level notation	Mean or Percentage
Education	$Z_1$	Illiterated	0	0.188
		Primary	1	0.524
		Diploma	2	0.170
		Higher educated	3	0.118
Toddler status	$U_1$	Having a toddler	1	0.288
		No having toddler	0	0.712
Monthly income				1916004 Toman
Foodstuffs expenditure	$Y_1$			0.308
Housing and fuel expenditure	$Y_2$			0.228
Health-care expenditure	$Y_3$			0.064
Number of members	$G_1$			4
Area of house	$G_2$			100.425
Smoking or drinking status	$G_3$	Yes	1	0.243
		No	0	0.757

The box plot of the monthly income and the monthly expenditures on foodstuffs, housing and fuel, and health-care are presented in Figure 4 of the Section 3 of the supplementary materials. This Figure show that the most proportion of the income is spent on foodstuffs. Also, some observations seem to be outliers based on these box plots. As we know, an outlier is an observation that appears to deviate markedly from other observations in the sample. An outlier may indicate bad data. For example, the data may have been coded incorrectly or an experiment may not have been run correctly. If it can be determined that an outlying point is in fact erroneous, then the outlying value should be deleted from the analysis (or corrected if possible). Outliers may be due to random variation or may indicate something scientifically interesting. In any event, we should not simply delete the outlying observation before a through investigation. If the data contains significant outliers, we may need to consider the use of robust statistical techniques. Here, we do not think that the income data with large values have been coded incorrectly and we do not remove any observation in our analysis. A careful discussion about outliers detection in our regression model can be a new work. Then, maybe, a robust regression method is recommended if there are outliers in the data. In this paper, we do not focus on outliers detection methods or robust regression methods. In order to do a joint analysis, we consider the following general location model for $Y_1$, $Y_2$, $Y_3$, Z, and U responses as follows:

$$\begin{array}{rl}& \log \left(\frac{{\mu }_{i1j}}{1-{\mu }_{i1j}-{\mu }_{i2j}-{\mu }_{i3j}}\right)|(U_i=j,b_{ij})={\gamma }_{1j}{G}_{i1j}+b_{ij},\\ & \log \left(\frac{{\mu }_{i2j}}{1-{\mu }_{i1j}-{\mu }_{i2j}-{\mu }_{i3j}}\right)|(U_i=j,b_{ij})={\gamma }_{2j}{G}_{i2j}+b_{ij},\\ & \log \left(\frac{{\mu }_{i2j}}{1-{\mu }_{i1j}-{\mu }_{i2j}-{\mu }_{i3j}}\right)|(U_i=j,b_{ij})={\gamma }_{3j}{G}_{i3j}+b_{ij},\\ & \mathrm{logit}(\mathrm{Pr}(Z_{ij}\le c))|(U_i=j,b_{ij})={\beta }_{jc}+{\beta }_j{G}_{i3j}+b_{ij},c=0,1,2,\end{array}$$ (10)

where $j\in \{0,1\}$ and $b_{ij}\sim N(0,{\sigma }_b^2)$. To specify initial values for the parameters which is needed in the optimization algorithms, we fit three separate beta regression models for $Y_1$, $Y_2$, and $Y_3$. It gives us some estimates for ${\gamma }_{pj}$s. Also, we use a Dirichlet regression for $Y_1$, $Y_2$, $Y_3$, and $1-Y_1-Y_2-Y_3$ (considering only intercept in the model) for finding an estimates for φ. ${\pi }_1$ is estimated by the proportion of families having a toddler and finally the initial values for β are found by fitting a separate cumulative logistic regression model for Z. The mean of all obtained estimates for ${\sigma }_b$ from four models for $Y_1$, $Y_2$, $Y_3$, and Z is considered here.

5.1.1. Results

Results of using model (10) are given in Table 2. From Table 2, the response variables $Y_{ij}$ and $Z_{ij}$ are dependent according to the significance of the parameter ${\sigma }_b$. As we expect, the expenditure spent on foodstuff is affected by the number of members in a family. Also, the area of the house has significant effect on the expenditure spent on housing and fuel. However, the effect of smoking and drinking status is negligible on the level of education in families with no toddler, it affects the expenditure spent on health-care services and the level of education in families having at least one toddler in the home.

Table 2. Results of using model (10), (parameter estimates highlighted in bold are significant at 5 $\mathrm{\%}$ level).

Parameter	${\gamma }_{1(1)}$	${\gamma }_{1(2)}$	${\gamma }_{2(1)}$	${\gamma }_{2(2)}$	${\gamma }_{3(1)}$	${\gamma }_{3(2)}$	φ	${\sigma }_b$	${\pi }_1$
Est.	0.211	0.215	0.006	0.005	−0.584	−0.705	8.38	1.192	0.288
S.E.	0.011	0.01	0.001	0.00032	0.122	0.075	0.32	0.048	0.014
Parameter	${\beta }_{(1)0}$	${\beta }_{(1)1}$	${\beta }_{(1)2}$	${\beta }_{(1)}$	${\beta }_{(2)0}$	${\beta }_{(2)1}$	${\beta }_{(2)2}$	${\beta }_{(2)}$
Est.	−2.905	0.994	2.343	1.238	−0.564	1.98	3.144	−0.005
S.E.	0.313	0.155	0.184	0.324	0.106	0.105	0.139	0.1

5.2. Application 2: body mass index, steatosis, and osteoporosis data

The medical dataset is obtained from an observational study on women in the Taleghani hospital of Tehran, Iran. These data record status of osteoporosis of the spine as an ordinal response and BMI and waist as continuous responses for 163 patients. BMI is defined on a continuous scale. Nominal variables which affect these variables are: job status (employee or homemaker) and type of the accommodation (apartment or private). Osteoporosis, which literally means ‘porous bone’, is a disease in which the density and quality of bone are reduced. As the bones become more porous and fragile, the risk of fracture is greatly increased. The loss of bone occurs ‘silently’ and progressively. Often there are no symptoms until the first fracture occurs. The most common fractures associated with osteoporosis occur at the hip, spine, and wrist. The incidence of these fractures, particularly at the hip and spine, increases with age in both women and men. It most often is seen in postmenopausal women, particularly light-skinned, small-framed women with a family history of osteoporosis. The loss of calcium from bones is the major effect of aging on the skeletal system. The body mass index, or Quetelet index is defined as the individual's body weight divided by the square of his or her height. The formula universally used in medicine produce a unit measure of $\mathrm{kg}/{\mathrm{m}}^2$. $\mathrm{BMI}=\mathrm{Weight}(\mathrm{kg})/(\mathrm{height}(\mathrm{m}){)}^2$. Steatosis is fatty infiltration of the liver. When inflammation is associated with the fatty change, the term steatohepatitis is used. Steatosis is often but not exclusively an early histological feature of alcoholic liver disease (alcohol-related fatty liver) leading to alcohol-related steatohepatitis. The non alcohol-related cases are known as a non-alcoholic fatty liver disease (NAFLD) and non-alcoholic steatohepatitis (NASH).The osteoporosis and steatosis are defined our two ordinal variables ( $Y_1$ and $Y_2$) with three levels as

$$Y_1=\{\begin{array}{ll}1,& \mathrm{The}\mathrm{person}\mathrm{does}\mathrm{not}\mathrm{have}\mathrm{osteoporosis},\\ 2,& \mathrm{The}\mathrm{person}\mathrm{has}\mathrm{mediocre}\mathrm{osteoporosis},\\ 3,& \mathrm{The}\mathrm{person}\mathrm{has}\mathrm{many}\mathrm{osteoporosis}\end{array}$$

and

$$Y_2=\{\begin{array}{ll}1,& \mathrm{The}\mathrm{person}\mathrm{does}\mathrm{not}\mathrm{have}\mathrm{steatosis},\\ 2,& \mathrm{The}\mathrm{person}\mathrm{has}\mathrm{mediocre}\mathrm{steatosis},\\ 3,& \mathrm{The}\mathrm{person}\mathrm{has}\mathrm{many}\mathrm{steatosis}.\end{array}$$

An interesting question is whether there are associations between the osteoporosis, body mass index (BMI) and steatosis with considering any covariate effects on the responses simultaneously. Explanatory variables which affect these variables are: (1) amount of total body calcium (Ca), (2) job status (Job, employee, or housekeeper), (3) type of the accommodation (house or apartment) and (4) the systolic blood pressure (SBP) which is defined as the peak pressure in the arteries and occurs near the beginning of the cardiac cycle. The normal rate, in adult humans, for systolic is near but less than 120 mmHg. Descriptive statistics (mean and standard deviation for continuous response and frequency or percentage for ordinal responses) are given in Table 3. Figure 5 of the Section 3 of the supplementary materials shows demographic characteristics of modelling for BMI, steatosis and osteoporosis data.

Table 3. Descriptive statistics for medical data.

		No.	Mean	S.E.
BMI		143	29.357	10.806
Missing data		20
Osteoporosis of the spine	Levels	No.	Percentage
	None	59	0.362
	Mild	65	0.399
	Severe	39	0.239
Steatosis
	None	39	0.240
	Mild	68	0.421
	Severe	53	0.339

Table 3 shows less percentage for severe osteoporosis than those of none and mild levels. It also gives the frequency and percentage of different levels of steatosis. As it can be seen more than 50 percent of individuals have severe steatosis. The mean of BMI is 28.266 which is high for the sample in hand. In our application, the percentage of missing values of BMI is 20.000%. The model is

$$\begin{array}{rl}& {\mathrm{BMI}}_j|Ta,\mathrm{Job}={\gamma }_{0j}+{\gamma }_{11j}{\mathrm{SBP}}_j+{\gamma }_{12j}{\mathrm{Ca}}_j+{\epsilon }_{1j},\\ & R_{\mathrm{BMI}j}^{\ast }|Ta,\mathrm{Job}={\gamma }_{21j}{\mathrm{SBP}}_j+{\gamma }_{22j}{\mathrm{Ca}}_j+{\epsilon }_{2j},\\ & Y_{1j}^{\ast }|Ta,\mathrm{Job}={\gamma }_{31j}{\mathrm{SBP}}_j+{\gamma }_{32j}{\mathrm{Ca}}_j+{\epsilon }_{3j}\\ & Y_{2j}^{\ast }|Ta,\mathrm{Job}={\gamma }_{41j}{\mathrm{SBP}}_j+{\gamma }_{42j}{\mathrm{Ca}}_j+{\epsilon }_{4j}\end{array}$$

where $J=(J_1,J_2),J_d=1,2$. The covariance matrix of the vector of errors $({\epsilon }_{1j},{\epsilon }_{2j},{\epsilon }_{3j},{\epsilon }_{4j}{)}^{\prime }$, for this model is

$$\mathrm{\Sigma }=\left(\begin{array}{cccc}{\sigma }^2& \sigma {\rho }_{12}& \sigma {\rho }_{13}& \sigma {\rho }_{14}\\ & 1& {\rho }_{23}& {\rho }_{24}\\ & & 1& {\rho }_{34}\\ & & & 1\end{array}\right).$$

Here, a multivariate normal distribution with the correlation between four errors named as ${\rho }_{12}$, ${\rho }_{13}$, ${\rho }_{14}$, ${\rho }_{23}$, ${\rho }_{24}$, ${\rho }_{34},$ are assumed and these parameters should be also estimated.

5.2.1. Results

Results of using model (2) are given in Table 4. Model (2) shows a significant effect of SBP on BMI, significant effect of SBP on the missing indicator for BMI, a significant effect of SBP on the probability of the low value of steatosis and a significant effect of Ca on the probability of the low value of osteoporosis of the spine. From these effects, we can infer that the people who live in apartment have more BMI than that of people who live in a house and the more the amount of calcium in the body of the patient, the higher is the probability of the low value of osteoporosis of the spine.

Table 4. Results of using model (2), (parameter estimates highlighted in bold are significant at 5% level).

Parameter	Est.	S.E	Parameter	Est.	S.E.
BMI
${\gamma }_{0(1,1)}$	21.386	4.218	${\gamma }_{0(1,2)}$	22.423	4.528
${\gamma }_{0(2,1)}$	23.678	5.321	${\gamma }_{0(2,2)}$	19.789	4.860
${\gamma }_{11(1,1)}$	0.037	0.024	${\gamma }_{11(1,2)}$	0.043	0.027
${\gamma }_{11(2,1)}$	0.035	0.028	${\gamma }_{11(2,2)}$	0.041	0.028
${\gamma }_{12(1,1)}$	−0.252	0.514	${\gamma }_{12(1,2)}$	−0.276	0.443
${\gamma }_{12(2,1)}$	−0.243	0.421	${\gamma }_{(2,2)}$	−0.287	0.423
$R_{\mathrm{BMI}}^{\ast }$
${\gamma }_{21(1,1)}$	0.152	0.111	${\gamma }_{21(1,2)}$	0.169	0.123
${\gamma }_{21(2,1)}$	0.163	0.167	${\gamma }_{21(2,2)}$	0.148	0.101
${\gamma }_{22(1,1)}$	−0.001	0.005	${\gamma }_{22(1,2)}$	−0.006	0.008
${\gamma }_{22(2,1)}$	−0.007	0.009	${\gamma }_{22(2,2)}$	−0.004	0.006
$Y_1^{\ast }$
${\gamma }_{31(1,1)}$	0.213	0.123	${\gamma }_{31(1,2)}$	0.312	0.143
${\gamma }_{31(2,1)}$	0.067	0.059	${\gamma }_{31(2,2)}$	0.078	0.032
${\gamma }_{32(1,1)}$	0.044	0.020	${\gamma }_{32(1,2)}$	0.054	0.025
${\gamma }_{32(2,1)}$	0.018	0.154	${\gamma }_{32(2,2)}$	0.012	0.0149
$Y_2^{\ast }$
${\gamma }_{41(1,1)}$	0.010	0.173	${\gamma }_{41(1,2)}$	0.012	0.154
${\gamma }_{41(2,1)}$	0.056	0.023	${\gamma }_{41(2,2)}$	0.061	0.024
${\gamma }_{42(1,1)}$	0.012	0.169	${\gamma }_{42(1,2)}$	0.013	0.174
${\gamma }_{42(2,1)}$	0.012	0.169	${\gamma }_{42(2,2)}$	0.013	0.174
${\pi }_{(1,1)}$	0.031	0.013	${\pi }_{(1,2)}$	0.025	0.012
${\pi }_{(2,1)}$	0.030	0.013	${\pi }_{(2,2)}$	0.031	0.013
${\sigma }^2$	12.643	0.251	${\rho }_{12}$	0.315	0.038
${\rho }_{13}$	−0.212	0.086	${\eta }_2$	−1.787	1.295
${\rho }_{24}$	0.299	0.223	${\eta }_1$	−2.643	1.304
${\rho }_{14}$	0.559	0.113	${\xi }_1$	1.205	1.475
${\rho }_{23}$	0.109	0.089	${\xi }_2$	2.283	1.470
${\rho }_{34}$	0.019	0.011

For model (2) correlation parameters ${\rho }_{12}$, ${\rho }_{13}$, ${\rho }_{14}$, and ${\rho }_{24}$ are strongly significant. They show a positive correlation between BMI and the missing indicator for BMI ( ${\hat{\rho }}_{12}=0.315$), it shows a negative correlation between BMI and osteoporosis of the spine ( ${\hat{\rho }}_{13}=-0.212$) and a positive correlation between BMI and steatosis ( ${\hat{\rho }}_{14}=0.559$). By these results, we can conclude that the missing indicator for BMI is related to BMI but is not related to two ordinal responses. This leads to having a NMAR mechanism (a NMAR for BMI which means correlation between error terms of BMI and $R_{\mathrm{BMI}}^{\ast }$).

6. Sensitivity analysis

We used sensitivity analysis to study model output varies with changes in model inputs. Cook [7] presented a general method for assessing the local influence of minor perturbations of a statistical model.

Generally, one introduce perturbations into the model through the $q\times 1$ vector ω which is restricted to some open subset Ω of $R^q$ and θ is a $p\times 1$ vector of unknown parameters. Cook [7] has then shown that the normal curvature $C_l$ of the lifted line in the direction l can be easily calculated by

$$C_l=2\left|l^{\prime }{\mathrm{\Delta }}^{\prime }(\ddot{L}{)}^{-1}\mathrm{\Delta }l\right|,$$ (11)

where

$${\mathrm{\Delta }}_i=\frac{{\mathrm{\partial }}^2{l}_i(\theta |{\omega }_i)}{\mathrm{\partial }{\omega }_i\mathrm{\partial }\eta }{|}_{\theta =\hat{\theta },{\omega }_i=0}$$

and define Δ as the $p\times n$ matrix with ${\mathrm{\Delta }}_i$ as its ith column and $\ddot{L}$ denote the $p\times p$ matrix of the second-order derivatives of $l(\theta |{\omega }_0)$, where there is an ${\omega }_0$ in Ω, with respect to θ, also evaluated at $\theta =\hat{\theta }.$ Obviously, $C_l$ can be calculated for any direction l. One evident choice is the vector $l_i$ containing one in the ith position and zero elsewhere, corresponding to the perturbation of the ith weight only.

For finding the condition for MAR, let $W_j=(Z_j,Y_{1j},Y_{2j})=(W_{\mathrm{obs}j},W_{\mathrm{mis}j})$, $W_j^{\ast }=(Z_j,Y_{1j}^{\ast },Y_{2j}^{\ast })=(W_{\mathrm{obs}j}^{\ast },W_{\mathrm{mis}j}^{\ast })$ and $R_j^{\ast }=(R_{zj}^{\ast },R_{Y_{1j}}^{\ast }),$ where $W_{\mathrm{obs}j}^{\ast }$ is the vector of latend variables related to the observed part of $W_j=(Z_j,Y_{1j},Y_{2j})$, $W_{\mathrm{mis}j}^{\ast }$ is the vector of latend variables related to the missing part of $W_j=(Z_j,Y_{1j},Y_{2j})$. According to our joint model, the vector of responses along with the missing indicators $(W_j^{\ast },R_j^{\ast })=(W_{\mathrm{obs}j}^{\ast },W_{\mathrm{mis}j}^{\ast },R_j^{\ast })$ has a multivariate normal distribution with the following covariance structure:

$$\mathrm{\Sigma }=\left(\begin{array}{ccc}{\mathrm{\Sigma }}_{o,o}& {\mathrm{\Sigma }}_{o,m}& {\mathrm{\Sigma }}_{o,R^{\ast }}\\ {\mathrm{\Sigma }}_{m,o}& {\mathrm{\Sigma }}_{m,m}& {\mathrm{\Sigma }}_{m,R^{\ast }}\\ {\mathrm{\Sigma }}_{R^{\ast },o}& {\mathrm{\Sigma }}_{R^{\ast },m}& {\mathrm{\Sigma }}_{R^{\ast },R^{\ast }}\end{array}\right),$$

where

$$\begin{array}{rl}{\mathrm{\Sigma }}_{o,o}& =\mathrm{cov}(W_{\mathrm{obs}j}^{\ast },W_{\mathrm{obs}j}^{\ast }),\\ {\mathrm{\Sigma }}_{m,m}& =\mathrm{cov}(W_{\mathrm{mis}j}^{\ast },W_{\mathrm{mis}j}^{\ast }),\\ {\mathrm{\Sigma }}_{o,m}& =\mathrm{cov}(W_{\mathrm{obs}j}^{\ast },W_{\mathrm{mis}j}^{\ast }),\\ {\mathrm{\Sigma }}_{o,R^{\ast }}& =\mathrm{cov}(W_{\mathrm{obs}j}^{\ast },R_j^{\ast }),\\ {\mathrm{\Sigma }}_{R^{\ast },R^{\ast }}& =\mathrm{cov}(R_j^{\ast },R_j^{\ast }).\end{array}$$

The joint density function can also be partitioned as

$$f(W_j^{\ast },R_j^{\ast })=f(W_{\mathrm{mis}j}^{\ast },R_j^{\ast }|W_{\mathrm{obs}j}^{\ast })f(W_{\mathrm{obs}j}^{\ast }),$$

where $f(W_{\mathrm{mis}j}^{\ast },R_j^{\ast }|W_{\mathrm{obs}j}^{\ast })$ and $f(W_{\mathrm{obs}j}^{\ast })$ have, respectively, a conditional and a marginal normal distribution. According to the missing mechanism definitions, to have an MAR mechanism the covariance matrix of the above mentioned conditional normal distribution, i.e.

$$\begin{array}{rl}{\mathrm{\Sigma }}_{m,R^{\ast }|o}& =\mathrm{cov}(W_{\mathrm{mis}j}^{\ast },R_j^{\ast }|W_{\mathrm{obs}j}^{\ast }),\\ & =\left(\begin{array}{cc}{\mathrm{\Sigma }}_{m,m}& {\mathrm{\Sigma }}_{m,R^{\ast }}\\ {\mathrm{\Sigma }}_{R^{\ast },m}& {\mathrm{\Sigma }}_{R^{\ast },R^{\ast }}\end{array}\right)-{\left(\begin{array}{cc}{\mathrm{\Sigma }}_{m,o}& {\mathrm{\Sigma }}_{R^{\ast },o}\end{array}\right)}^{\mathrm{\prime }}{\mathrm{\Sigma }}_{o,o}^{-1}\left(\begin{array}{cc}{\mathrm{\Sigma }}_{m,o}& {\mathrm{\Sigma }}_{R^{\ast },o}\end{array}\right),\\ & =\left(\begin{array}{cc}{\mathrm{\Sigma }}_{m,m}-{\mathrm{\Sigma }}_{m,o}^{\mathrm{\prime }}{\mathrm{\Sigma }}_{o,o}^{-1}{\mathrm{\Sigma }}_{m,o}& {\mathrm{\Sigma }}_{m,R^{\ast }}-{\mathrm{\Sigma }}_{m,o}^{\mathrm{\prime }}{\mathrm{\Sigma }}_{o,o}^{-1}{\mathrm{\Sigma }}_{R^{\ast },o}\\ {\mathrm{\Sigma }}_{R^{\ast },m}-{\mathrm{\Sigma }}_{R^{\ast },o}^{\prime }{\mathrm{\Sigma }}_{o,o}^{-1}{\mathrm{\Sigma }}_{m,o}& {\mathrm{\Sigma }}_{R^{\ast },R^{\ast }}-{\mathrm{\Sigma }}_{R^{\ast },o}^{\mathrm{\prime }}{\mathrm{\Sigma }}_{o,o}^{-1}{\mathrm{\Sigma }}_{R^{\ast },o}\end{array}\right),\end{array}$$

should satisfy the following constraint:

$${\mathrm{\Sigma }}_{m,R^{\ast }}-{\mathrm{\Sigma }}_{m,o}^{\prime }{\mathrm{\Sigma }}_{o,o}^{-1}{\mathrm{\Sigma }}_{R^{\ast },o}=\mathbf{0}.$$ (12)

For our application (see section of application), we have missing values only for our continuous variable and we may have $W_{\mathrm{obs}}^{\ast }=(Y_1^{\ast },Y_2^{\ast }{)}^{\prime }$ and $W_{\mathrm{mis}s}^{\ast }=Z$ , ( $Y_1$ and $Y_2$ are ordinal responses and Z is the continuous response.). For missing mechanism we only need to define $R^{\ast }=R_z^{\ast }$, as we do not have any missing value for our continuous response, and

$$\begin{array}{rl}{\mathrm{\Sigma }}_{o,o}& =\left(\begin{array}{cc}1& {\rho }_{13}\\ {\rho }_{13}& 1\end{array}\right),{\mathrm{\Sigma }}_{m,R^{\ast }}=\sigma {\rho }_{12},{\mathrm{\Sigma }}_{R^{\ast },R^{\ast }}=1,\\ {\mathrm{\Sigma }}_{m,m}& ={\sigma }^2,{\mathrm{\Sigma }}_{m,o}=(\sigma {\rho }_{13},\sigma {\rho }_{14}{)}^{\prime },{\mathrm{\Sigma }}_{R^{\ast },o}=({\rho }_{23},\sigma {\rho }_{24}{)}^{\prime },\end{array}$$

so that the above constraint will be reduced to,

$${\omega }_{MAR}=1-[\frac{1}{1-{\rho }_{13}^2}(\sigma {\rho }_{13}{\rho }_{23}-\sigma {\rho }_{14}{\rho }_{13}{\rho }_{23}-\sigma {\rho }_{13}^2{\rho }_{24}+\sigma {\rho }_{14}{\rho }_{24})]=0.$$

The

$$\omega =1-[\frac{1}{1-{\rho }_{13}^2}(\sigma {\rho }_{13}{\rho }_{23}-\sigma {\rho }_{14}{\rho }_{13}{\rho }_{23}-\sigma {\rho }_{13}^2{\rho }_{24}+\sigma {\rho }_{14}{\rho }_{24})],$$

as weight defines the perturbation of the MAR model.

Also, For finding the condition for MAR and correlated responses, Let

$$\omega =({\omega }_{MAR},{\rho }_{12},{\rho }_{13},{\rho }_{14},{\rho }_{23},{\rho }_{24},{\rho }_{34}{)}^{\prime },$$

and ${\omega }_0=(0,0,0,0,0,0,0{)}^{\prime }$ to be the log-likelihood function that corresponds to MAR and correlated model.

This reflects the influence of the condition for MAR of the responses and correlated responses. The corresponding local influence measure, denoted by $C_l$, then becomes $C_l=2|{\mathrm{\Delta }}_l^{\prime }{\ddot{L}}^{-1}{\mathrm{\Delta }}_l|$. The corresponding local influence measure, denoted by $C_l$, then becomes $C_l=2|{\mathrm{\Delta }}_l^{\prime }{\ddot{L}}^{-1}{\mathrm{\Delta }}_l|.$ Another important direction is the direction $l_{max}$ of maximal normal curvature $C_{max}$. It shows how to perturb the condition for MAR of the responses to obtain the largest local changes in $C_{max}$. $C_{max}$ is the largest eigenvalue of ${\mathrm{\Delta }}_i^{\prime }{\ddot{L}}^{-1}{\mathrm{\Delta }}_i$ and $l_{max}$ is the corresponding eigenvector. To search for Sensitivity analysis we find $C_{max}$. This is confirmed by the obtained curvature $C_{max}=14.133$ computed from (3). This curvature indicates extreme local sensitivity.

7. Discussion

In this article, we proposed a general location multivariate latent variable model for mixed nominal, ordinal and continuous responses with and without non-ignorable responses. This procedure should be used when the specification of general location models of all the variables with non-ignorable responses are difficult. Work on extending our methodology to allow for clustering in the data is an on-going research on our part. We are also exploring generalizations of the model to the multivariate case of more than one discrete and more than one continuous outcome, including the incorporation of random effects. The challenge here lies in defining models that allow for different levels of association among outcomes, as in longitudinal studies, and always guarantees proper joint distributions.

Disclosure statement

No potential conflict of interest was reported by the author(s).