A multivariate Poisson regression model for count data

Abstract

We propose a new technique for the study of multivariate count data. The proposed model is applied to the study of the number of individuals several fossil species found in a set of geographical observation points. First, we are proposing a multivariate model based on the Poisson distributions, which allows positive and negative correlations between the components. We are extending the log-linear Poisson model in the multivariate case through the conditional distributions. For this model, we obtain the maximum likelihood estimates and compute several goodness of fit statistics. Finally we illustrate the application of the proposed method over data sets: various simulated data sets and a count data set of various fossil species.

1. Introduction

Count data has been the subject of an increasing number of proposals in several scientific areas. There are numerous examples in scientific literature, but models taking into account multivariate counts are still fairly rare. In investigations that are directed to the study of such variables as career interruptions, number of children, scores of soccer games or days in an hospital, the discrete Poisson distribution can be used. In general, for single counts, the application of this distribution provides good results in the modelling of the phenomena. On occasions, it is necessary to apply some modifications to avoid difficulties such as the under-dispersion or over-dispersion problem and/or the excess zeros problem. Besides, in count data analysis, if the interest is focused on the study of a single count variable in a regression context, the building of the model and its analysis are widely developed by means of the generalized linear model [21].

For multiple counts, however, the application of the Poisson distribution is not so clear. The literature reports good works using a bivariate Poisson distribution based on the models described by Johnson and Kotz [13], Kocherlakota and Kocherlakota [19] and Cameron and Johansson [3]. The disadvantage of these suggested distributions is that the correlation between count variables is restricted to being positive. This disadvantage is not strong since negative correlation for small counts is not realistic, it only occurs between high counts. Some works that study and use these multivariate versions are Jung and Winkelmann [15], Karlis [16], Karlis and Meligkotsidou [17] and Li et al. [20].

Other approaches to analyze multivariate count data have been proposed. Alfò and Tovato [1] used semi-parametric mixture models, Famoye [7] develops a bivariate negative binomial regression model, Hellstrom [11] applies a bivariate mixed Poisson log-normal model, Chib and Winkelmann [6] use the Poisson-lognormal distribution, Karlis and Meligkotsidou [18] examine finite mixtures of multivariate Poisson distributions. Cameron et al. [4], McHale and Scarf [22] and Nikoloulopoulos and Karlis [23] propose the use of copulas-based model. Famoye [8] proposes a multivariate generalized Poisson regression model based on a multivariate distribution with several parameters to model the overdispersion and several parameters to model the correlation between the count variables. Berkhout and Plug [2] introduce a bivariate model based on conditioning Poisson distributions. This model allows for positive as well as negative correlation. So, the authors introduce a model which allows for a more flexible correlation structure.

In many studies, the objective is to analyze the relationship between a set of explanatory variables and a set of count variables (count vector) with an internal dependency structure that follows the pattern: the kth component of count vector depends on the values obtained by the $(k-1)$ first components. This dependency structure between the count variables is not explicitly taking into account in the multivariate models proposed in the works cited above. These models do not include the specification of the effect of some count variables on others according to the ordering structure. And this order is present in many real data sets.

For example, in a model for studying the number of tourists (T), the number of visits (V) to a tourism point (amusement park, museum, historical building, etc.) and the number of official guide sales (S). the order should be $(T,V,S)$. In a model for studying the number of individuals of various species in a type of ecosystem, the order can be known by the nature relations between the species.

However, if there is not complete information beforehand about this order, one has to select the most adequate model between the possible models. For example, the study of the number of individuals several fossil species found in a set of geographical observation points. This generic pattern is followed by the work of Ruíz et al. [25] that analyzes the foraminifera collected in three estuaries (Guadiana, Piedras, Tinto-Odiel) of southwestern Spain. The foraminifera are a large group of amoeboid protists. Generally, the foraminifera are regarded as the most important groups of marine microfossils because they are very abundant in marine sediments. So, because of their diversity, abundance and complex morphology, fossil foraminiferal assemblages are useful for biostratigraphy and paleontology.

In this experience, the count variables considered as objective are number of individuals of three species of foraminifera. The researchers try to model these count variables through their intrinsic dependency structure and their relationship with various variables that characterize the physical environment of the observations (water salinity during high tide and tidal height about the lowest astronomical tide).

In this work, we develop a multivariate linear model to describe the relationships between a vector of explanatory variables and a count vector with the aforementioned internal dependency structure. To illustrate the usefulness of the model, it will be applied to the experience of formainifera species.

In Section 2, the definition of the multivariate model is included and an iterative algorithm for computing the maximum likelihood estimation of parameters is described. In addition, various adjustment for the model are included. The distributional hypothesis of model requires that the components of the response vector are ordered. This order might be known in some applications, but in general it is unknown. Section 3 offers an efficient algorithm to identify the most appropriate ordering of the response variables, and the potential problem of no real ordering in the data is also considered.

In Section 4, we provide a simulation study to check the model, the inference process and the fit of the model to the simulated data sets.

Finally, the application to real data in Paleontology (three species of formaminifera) in Section 5 confirms the usefulness of the model in experiences associated with multivariate counting data and the easiness to interpret its results by the researchers. Section 6 ends with some conclusions.

2. Multivariate log-linear conditional Poisson model

Following the known scheme of the generalized linear model, we can construct a model that allows analyzing a p-dimensional count vector ( $\mathbf{Y}$) through a d-dimensional vector of covariates ( $\mathbf{X}$) with the dependency structure and the distribution hypothesis listed below:

1. $Y_1$ depends on $\mathbf{X}$. For $k=2,\dots ,p$, $Y_k$ depends on $Y_1,\dots ,Y_{k-1}$ and $\mathbf{X}$. Further, the conditioned distributions of the components of $\mathbf{Y}$ follow the Poisson distribution. Let ${\mathbf{Y}}_{(h)}$ be the random subvector $[Y_1\dots Y_h{]}^t$,

   $$Y_k{|}_{{\mathbf{Y}}_{(k-1)},\mathbf{X}}\sim P({\lambda }_k).$$
2. The assumption structural of the model is

   $$\log {\lambda }_1={\mathbf{x}}^t{\beta }_1\mathrm{and}\log {\lambda }_j={\mathbf{x}}^t{\beta }_j+{\mathbf{y}}_{(j-1)}^t{\alpha }_j,\text{for}j=2,\dots ,p.$$

   where ${\beta }_j\in {\mathbf{R}}^d$ and ${\alpha }_j=[{\alpha }_{j1},\dots ,{\alpha }_{j(j-1)}{]}^t\in {\mathbf{R}}^{j-1}$ for $j=1,\dots ,p$

This model can be considered a multivariate extension of the log-linear Poisson model [21]. Thus the model is determined through a distributional hypothesis (I) defined on the vector of objective variables and a structural hypothesis (II) defined on the parameters of the conditioned distributions. This model can be called a multivariate log-linear conditional Poisson model (MLCP).

The dependence between $Y_j$ and the first $(j-1)$ variables is modeled through the parameter of the Poisson distribution that results from multiplying a term ( $e^{{\eta }_j}=e^{{\mathbf{x}}^t{\beta }_j}$) that depends on the covariates and the exponential function defined on the linear combination of those variables ( $e^{{\mathbf{y}}_{(j-1)}^t{\alpha }_j}$),

$${\lambda }_j=e^{{\mathbf{x}}^t{\beta }_j}{e}^{{\mathbf{y}}_{(j-1)}^t{\alpha }_j}$$ (1)

So, the parameter of the Poisson distribution of $Y_j$ is determined by an intrinsic factor of this variable and a correction factor defined on the preceding variables. The model assumes that the explanatory variables only influence on the intrinsic factor, but not the correction factor. Obviously if ${\alpha }_j=\mathbf{0}$ then the parameter coincide with the intrinsic factor, $Y_j{|}_{\mathbf{X}=\mathbf{x}}\sim P({\lambda }_j(\mathbf{x}))$ and $Y_j$ is independent of the $(j-1)$ first variables.

Given that the correction factor is an exponential function, the parameters that determine the linear combination, ${\alpha }_j$, cannot be very large in absolute values because, in the negative case, ${\lambda }_j$ is almost null and, in the positive case, they could take excessively large values.

The explanatory variables can be: metric variables, binary variables or qualitative variables. In this last case, the covariates have to be appropriately coded by a dummy vector. To include an intercept in the model, we can consider the first component of $\mathbf{X}$ equals to 1.

This multivariate model can be used to model experiences that are described by a set of correlated count variables. The correlations between the variables can be positive or negative. Therefore this model offers flexibility for the description of these types of experiences. Some applications that could be interesting to obtain conclusions about: (a) the competition for food between two species; (b) the environmental factors controlling the presence of two polymorphs in the same species; (c) speciation and phylogeny; or (d) evolutionary patterns.

2.1. Likelihood inference

In order to obtain the maximum likelihood estimates (MLE) of the parameters of the model, we consider a sample

$$\{({\mathbf{x}}_i,{\mathbf{y}}_i):{\mathbf{x}}_i=[x_{1i},\dots ,x_{di}{]}^t,{\mathbf{y}}_i=[y_{1i},\dots ,y_{pi}{]}^t,i=1,\dots ,n\}.$$

The parameters of model are ${\beta }_j\in {\mathbf{R}}^d$ and ${\alpha }_j\in {\mathbf{R}}^{j-1}$ for $j=1,\dots ,p$. Denoting $\alpha =({\alpha }_2^t\dots {\alpha }_p^t{)}^t$ and $\beta =({\beta }_1^t\dots {\beta }_p^t{)}^t$, the likelihood of the model can be expressed, except for the multiplicative constant, as follows:

$$L(\beta ,\alpha )=\prod _{i=1}^n{L}_i(\beta ,\alpha )=\prod _{i=1}^n\prod _{h=1}^p\exp (-{\lambda }_h)\exp (y_{hi}\log {\lambda }_h)$$

Therefore, the log-likelihood can be expressed by $l(\beta ,\alpha )=\sum _{i=1}^n{l}_i(\beta ,\alpha )$ where

$$\begin{array}{rl}{l}_i(\beta ,\alpha )& =[-\exp \left({\mathbf{x}}_i^t{\beta }_1\right)+y_{1i}{\mathbf{x}}_i^t{\beta }_1]\\ & +\sum _{h=2}^p[-\exp ({\mathbf{x}}_i^t{\beta }_h+{\mathbf{y}}_{i(h-1)}^t{\alpha }_h)+y_{hi}({\mathbf{x}}_i^t{\beta }_h+{\mathbf{y}}_{i(h-1)}^t{\alpha }_h)]\end{array}$$

The maximum value is obtained with the iterated Fisher scoring algorithm, that is, starting with an initial estimate $({\hat{\beta }}^{(0)},{\hat{\alpha }}^{(0)})$,

$$({\hat{\beta }}^{(k+1)},{\hat{\alpha }}^{(k+1)})=({\hat{\beta }}^{(k)},{\hat{\alpha }}^{(k)})-{[H({\hat{\beta }}^{(k)},{\hat{\alpha }}^{(k)})]}^{-1}s({\hat{\beta }}^{(k)},{\hat{\alpha }}^{(k)})\text{for }k=0,1,2\dots$$ (2)

where $s{\mathbf{(}\hat{\beta }}^{(k)},{\hat{\alpha }}^{(k)})$ is the vector of first partial derivatives and $H({\hat{\beta }}^{(k)},{\hat{\alpha }}^{(k)})$ is the Hessian matrix of second partial derivatives. As

$$\begin{array}{rl}\frac{\mathrm{\partial }{l}_i}{\mathrm{\partial }{\beta }_1}& =[y_{1i}-\exp \left({\mathbf{x}}_i^t{\beta }_1\right)]{\mathbf{x}}_i,\\ \frac{\mathrm{\partial }{l}_i}{\mathrm{\partial }{\beta }_h}& =[y_{hi}-\exp ({\mathbf{x}}_i^t{\beta }_h+{\mathbf{y}}_{i(h-1)}^t{\alpha }_h)]{\mathbf{x}}_i,\\ \frac{\mathrm{\partial }{l}_i}{\mathrm{\partial }{\alpha }_h}& =[y_{hi}-\exp ({\mathbf{x}}_i^t{\beta }_h+{\mathbf{y}}_{i(h-1)}^t{\alpha }_h)]{\mathbf{y}}_{i(h-1)},\end{array}$$

for $h=2,\dots ,p$, the vector of first partial derivatives can be expressed by $s(\beta ,\alpha )=\sum _{i=1}^n{s}_i(\beta ,\alpha )$ where $s_i$ are the $(pd+p(p-1)/2)$-dimensional vectors

$$s_i(\beta ,\alpha )={[{\left(\frac{\mathrm{\partial }{l}_i}{\mathrm{\partial }{\beta }_1}\right)}^t,{\left(\frac{\mathrm{\partial }{l}_i}{\mathrm{\partial }{\beta }_2}\right)}^t,{\left(\frac{\mathrm{\partial }{l}_i}{\mathrm{\partial }{\alpha }_2}\right)}^t,\dots ,{\left(\frac{\mathrm{\partial }{l}_i}{\mathrm{\partial }{\beta }_p}\right)}^t,{\left(\frac{\mathrm{\partial }{l}_i}{\mathrm{\partial }{\alpha }_p}\right)}^t]}^t$$

The Hessian matrix is $H(\beta ,\alpha )=diag\{\dots H_{(h)}(\beta ,\alpha )\dots \}$ with

$$H_{(1)}(\beta ,\alpha )=-\sum _{i=1}^n\exp \left({\mathbf{x}}_i^t{\beta }_1\right){\mathbf{x}}_i{\mathbf{x}}_i^t$$

and, for $h=2,\dots ,p$,

$$H_{(h)}(\beta ,\alpha )=-\sum _{i=1}^n\exp ({\mathbf{x}}_i^t{\beta }_h+{\alpha }_h^t{\mathbf{y}}_{i(h-1)})\left(\begin{array}{cc}{\mathbf{x}}_i{\mathbf{x}}_i^t& {\mathbf{x}}_i{\mathbf{y}}_{i(h-1)}^t\\ {\mathbf{y}}_{i(h-1)}{\mathbf{x}}_i^t& {\mathbf{y}}_{i(h-1)}{\mathbf{y}}_{i(h-1)}^t\end{array}\right)$$

For the selection of the initial values of the iterated procedure, the estimators of the univariate log-linear Poisson models associated to each objective variable can be considered. The stopping rule can be determined in terms of the absolute value of the gradient and the maximum number of iterations.

2.2. Goodness of fit

It can be confirmed that the distribution of the MLE of $({\beta }_1,\dots ,{\beta }_p,{\alpha }_2,\dots ,{\alpha }_p)$ is asymptotically normal with approximate mean $({\beta }_1,\dots ,{\beta }_p,{\alpha }_2,\dots ,{\alpha }_p)$ and approximate covariance matrix $[H(\hat{\beta },\hat{\alpha }){]}^{-1}$. Thus the goodness-of-fit may be checked by the deviance statistic or likelihood ratio statistic. This statistic is given by

$$D(\mathbf{y};\hat{\beta },\hat{\alpha })=-2\sum _{i=1}^n[l_i(\hat{\beta },\hat{\alpha })-l_i({\mathbf{y}}_i)]$$

where $l_i({\mathbf{y}}_i)$ is the individual log-likelihood in which ${\lambda }_j$ is replaced by $y_{ji}$ (the maximum log-likelihood achievable), that is,

$$l_i({\mathbf{y}}_i)=\sum _{k=1}^p(-y_{ki}+y_{ki}\log y_{ki})$$

So, the deviance can be equivalently expressed as $D(\mathbf{y};\hat{\beta },\hat{\alpha })=2[l(\mathbf{y})-l(\hat{\beta },\hat{\alpha })]$, twice the difference between the maximum log-likelihood achievable and the log-likelihood of the fitted model.

The formal use of this statistic becomes unsuitable in this model because the data are not grouped [21]. However, it is possible to consider a R-squared measure of goodness of fit. Cameron and Windmeijer [5] proposed the R-squared measure based on deviances for univariate Poisson regression models, instead of a measure based on residuals. Other pseudo R-squared measures have been defined for these models (see Waldhor et al. [28], Heinzl et al. [10]). In this paper, we propose a multivariate extension of this measure.

Additionally to the saturated model and the model postulated, we can consider other models. First, the model without the presence of covariates ( $\eta \in {\mathbb{R}}^p$ not dependent of $\mathbf{X}$, that is the intrinsic factors are $\exp ({\eta }_j)$, with ${\eta }_j\in \mathbb{R}$, for $j=1,\dots ,p$). Second, the intercept-only model, i.e. without the presence of the covariates and under the independence of the Poisson variables ( $\eta \in {\mathbb{R}}^p$ not dependent of $\mathbf{X}$ and ${\alpha }_j=\mathbf{0}$, for $j=2,\dots ,p$).

The without-covariates model coincides with the MLCP model with a single predictor equal to 1. Therefore, to obtain the MLE estimators, a simple adaptation of the method collected in the previous subsection can be applied. Therefore, the maximum likelihood is reached for the MLE of the parameters, that is, $\tilde{\eta }$ and ${\tilde{\alpha }}_h$, for $h=2,\dots ,p$. Thus the maximum log-likelihood of this model is $l(\stackrel{\mathbf{~}}{\eta },\stackrel{\mathbf{~}}{\alpha })=\sum _{i=1}^n{l}_i(\stackrel{\mathbf{~}}{\eta },\stackrel{\mathbf{~}}{\alpha })$ being

$$l_i(\eta ,\alpha )=[-\exp \left({\eta }_1\right)+y_{1i}{\eta }_1]+\sum _{j=2}^p[-\exp ({\eta }_j+{\mathbf{y}}_{i(j-1)}^t{\alpha }_j)+y_{ji}({\eta }_j+{\mathbf{y}}_{i(j-1)}^t{\alpha }_j)]$$

So, its deviance statistic is $D(\mathbf{y};\tilde{\eta },\tilde{\alpha })=2[l(\mathbf{y})-l(\tilde{\eta },\tilde{\alpha })]$

Considering the intercept-only model, the maximum log-likelihood for this model is

$$l(\overline{\mathbf{y}})=\sum _{i=1}^n\sum _{k=1}^p(-{\overline{y}}_{k\bullet }+y_{ki}\log {\overline{y}}_{k\bullet }),$$

where ${\overline{y}}_{k\bullet }$ is the sample average of $Y_k$. The deviance statistic of this model can be expressed as

$$D(\mathbf{y};\overline{\mathbf{y}})=2[l(\mathbf{y})-l(\overline{\mathbf{y}})]=2\sum _{i=1}^n\sum _{k=1}^p{y}_{ki}\log \left(y_{ki}/{\overline{y}}_{k\bullet }\right)$$

Obviously,

$$l(\overline{\mathbf{y}})\le l(\tilde{\eta },\tilde{\alpha })\le l(\hat{\beta },\hat{\alpha })\le l(\mathbf{y})$$ (3)

Based on (3), we can define two coefficients of determination or R-squared measures. These coefficients are based on the deviance for MLCP model and the reference interval. First, if we consider the interval $[l(\overline{\mathbf{y}}),l(\mathbf{y})]$, it can be defined as the determination coefficient the following ratio

$$R_O^2=1-\frac{D(\mathbf{y};\hat{\beta },\hat{\alpha })}{D(\mathbf{y};\overline{\mathbf{y}})}=\frac{l(\hat{\beta },\hat{\alpha })-l(\overline{\mathbf{y}})}{l(\mathbf{y})-l\left(\overline{\mathbf{y}}\right)}.$$

This ratio can be denominated as the overall R-squared measure or overall determination coefficient. This coefficient lies between 0 and 1 because the fitted log-likelihood increases as regressors are added and the maximum value is $l(\mathbf{y})$. Furthermore, it can be interpreted as the ratio between the variation of the explained log-likelihood and the variation of the achievable log-likelihood. Also, $R_O^2$ can be interpreted as the relative reduction in deviance due to the complete model, that is, the model under distributional assumption and structural assumption. Hence, values close to zero of this measure indicate a lack of global adjustment in the model. This lack of fit can be caused by a faulty distributional hypothesis or a lack of capacity of explanation in the covariates. In addition, if we define the variation ratio of the log-likelihood explained by the distributional assumption (interdependence between objective variables) by

$$VR{L}_Y=\frac{l\mathbf{(}\stackrel{\mathbf{~}}{\eta },\stackrel{\mathbf{~}}{\alpha })-l(\overline{\mathbf{y}})}{l(\mathbf{y})-l\left(\overline{\mathbf{y}}\right)}$$

and the variation ratio of the log-likelihood explained by the structural assumption (dependence of covariates) by

$$VR{L}_X=\frac{l(\hat{\beta },\hat{\alpha })-l(\stackrel{\mathbf{~}}{\eta },\stackrel{\mathbf{~}}{\alpha })}{l(\mathbf{y})-l\left(\overline{\mathbf{y}}\right)}$$

then the overall R-squared measure can be expressed as the sum of these variation ratios, that is,

$$R_O^2=VR{L}_Y+VR{L}_X$$

Second, for the reference interval $[l(\tilde{\eta },\tilde{\alpha }),l(\mathbf{y})]$, the following ratio can be considered as a determinant coefficient

$$R_r^2=\frac{l(\hat{\beta },\hat{\alpha })-l(\stackrel{\mathbf{~}}{\eta },\stackrel{\mathbf{~}}{\alpha })}{l(\mathbf{y})-l(\tilde{\eta },\tilde{\alpha })}=\frac{VR{L}_X}{1-VR{L}_Y}$$

It can be denominated as relative R-squared measure or relative determination coefficient. Obviously, this coefficient lies between 0 and 1 and can be interpreted as a measure of adjustment of the data to the structural assumption under the distributional hypothesis. So, the values close to zero of this measurement will indicate a lack of capacity of explanation in the covariates.

2.3. Selection of the model

In the MLCP model, the order of the dependent variables is essential. This order can be known in many applications. For example, in a model for studying the number of individuals of various species in a type of ecosystem, a part of the order can be known by the natural relations between the species. In a model for studying the number of tourists (T) and the number of visits (V) to a tourist point of interest (amusement park, museum, historical building, etc.), the order should be $(T,V)$. However, this information on order is not known in the experience on foraminifera species that has been presented in the introduction to this work.

If there is not complete information beforehand about this order, one has to select the most adequate model between the possible models. In this section, we present a procedure when there is not any information about the order. That is, a procedure to choose the most adequate model between the total $p!$ models. Obviously, the procedure can be based on the selection of a model with smaller deviance statistic. So the following procedure can be considered:

1. Selection of the first variable. Study of the p univariate models:

   $$Y_{h|\mathbf{X}=\mathbf{x}}\sim P({\lambda }_h^{(1)}):\log {\lambda }_h^{(1)}={\mathbf{x}}^t{\beta }_h^{(1)}h=1,\dots ,p$$

   that are denoted by $M^{(1)}[h]$. Let $D(M^{(1)}[h])$ be deviance statistic of $M^{(1)}[h]$, for $h=1,\dots ,p$. Thus, select $Y_j$ such that

   $$D(M^{(1)}[j])=\underset{h=1,\dots ,p}{min}D(M^{(1)}[h])$$

   The selected variable is noted by $Z_1^{(1)}$ and the rest by $Z_2^{(1)},\dots ,Z_p^{(1)}$.

2. Selection of the second variable.

   1. Study of the $(p-1)$ bivariate models

      $$\begin{array}{ll}{Z}_{1|\mathbf{X}=\mathbf{x}}^{(1)}\sim P({\lambda }_1^{(2)})& :\log {\lambda }_1^{(2)}={\mathbf{x}}^t{\beta }_1^{(2)}\\ Z_{h|\mathbf{X}=\mathbf{x},Z_1^{(1)}=z_1}^{(1)}\sim P({\lambda }_h^{(2)})& :\log {\lambda }_h^{(2)}={\mathbf{x}}^t{\beta }_h^{(2)}+{\alpha }^{(2)}{z}_1\\ h=2,\dots ,p& \end{array}$$

      that are noted by $M^{(2)}[1,h]$. Let $D(M^{(2)}[1,h])$ be deviance statistics, for $h=2,\dots ,p$. Thus select $Z_j^{(1)}$ such that

      $$D(M^{(2)}[1,j])=\underset{h=2,\dots ,p}{min}D(M^{(2)}[1,h])$$
   2. Study of bivariate model $M^{(2)}[j,1]$.
      • If $D(M^{(2)}[1,j])\le D(M^{(2)}[j,1])$then retaining $M^{(2)}[1,j]$
      • In other case, retaining $M^{(2)}[j,1].$
      The selected variables, in the indicated order, are noted by $Z_1^{(2)}$, $Z_2^{(2)}$ and the rest by $Z_3^{(2)},\dots ,Z_p^{(2)}$.

3. Selection of the third variable

   1. Fit the $(p-2)$ models $M^{(3)}[1,2,h]$, with the associated statistics $D(M^{(3)}[1,2,h])$. Thus select $Z_j^{(2)}$ such that

      $$D(M^{(3)}[1,2,j])=\underset{h=3,\dots ,p}{min}D(M^{(3)}[1,2,h])$$
   2. Studying the model $M^{(3)}[1,j,2]$.

      • If $D(M^{(3)}[1,2,j])\le D(M^{(3)}[1,j,2])$ then retain the model $M^{(3)}[1,2,j]$.
      • If $D(M^{(3)}[1,2,j])>D(M^{(3)}[1,j,2])$ then study the model $M^{(3)}[j,1,2]$.

        • If $D(M^{(3)}[1,j,2])>D(M^{(3)}[j,1,2])$ then retain the model $M^{(3)}[1,j,2]$
        • In other case, retain $M^{(3)}[j,1,2]$.
      The selected variables, in the indicated order, are noted by $Z_1^{(3)}$, $Z_2^{(3)}$, $Z_3^{(3)}$ and the rest by $Z_4^{(3)},\dots ,Z_p^{(3)}$.
4. Step k ( $3<k\le p$) Selection of the $k^{th}$ variable.

   1. Study the $(p-k+1)$ models $M^{(k)}[1,\dots ,k-1,h]$ with the associated statistics

      $$D(M^{(k)}[1,\dots ,k-1,h]).$$

      Thus select $Z_j^{(k-1)}$ such that

      $$D(M^{(k)}[1,\dots ,k-1,j])=\underset{h=k,\dots ,p}{min}D(M^{(k)}[1,\dots ,k-1,h])$$
   2. Study the model $M^{(k)}[1,\dots ,j,k-1]$.

      • If

        $$D(M^{(k)}[1,\dots ,k-1,j])\le D(M^{(k)}[1,\dots ,j,k-1])$$

        then retain the model $M^{(k)}[1,\dots ,k-1,j]$.
      • In other case, study the model $M^{(k)}[1,\dots ,j,k-2,k-1]$ and proceed in a similar way.

For this procedure, the maximum number of models that must be analyzed is $p^2$. For p>3 this maximum is smaller than the number of possible sorts ( $p!$). Besides, the models included in the procedure have dimensions smaller than the dimension of the objective vector, except in the last step. Therefore this procedure diminishes the computational complexity.

This procedure is based on the idea that once the order of dependency between the two variables is determined then this order should not be modified with the presence of a new variable. Possibly in many biological studies focusing on the study of the number of individuals of several species, this premise can be applied.

Our proposed procedure is successfully applied on several artificial data sets in the next section, detecting the correct order as it is defined in the population model. However, real data sets could arise from a generation process where no established order is present, but the proposed procedure would nonetheless arrive to a final ordering. To avoid this problem, the following idea is suggested.

If the CPU time required to fit a model with a certain ordering is not excessive, fit all the $p!$ models. Otherwise the proposed procedure is applied until the next-to-last step. Last step is modified in such a way that only a fraction f of the $p!-p$ remaining orderings are considered.

The deviance or the AIC for the generated models is finally analyzed. We must doubt about the appropriateness of the model if the criterion for selected ordering it not clearly the best. For example, very similar values of the criterion or the existence of other orderings with a better value can reveal that no ordering is clearly defined for the data set.

3. Simulations: artificial data sets

We run a small simulation study to analyze the performance of the estimators obtained in Section 2. First, we generate an artificial data set. We have used the following procedure of generation. We consider n = 120, p = 3, number of objective variables ( $Y_1,Y_2,Y_3$) and d = 2, one explanatory variable (X) and the intercept. For $i=1,\dots ,n$, the following values are generated

• $x_i$: value of a distribution $N(5,1)$.

• $y_{1,i}$: value of a distribution $P(e^{{\eta }_1})$, with ${\eta }_1=5.0-0.5x_i.$

• $y_{2,i}$: value of a distribution $P(e^{{\eta }_2})$, with ${\eta }_2=1.0+0.9x_i-0.3y_{1i}$

• $y_{3,i}$: value of a distribution $P(e^{{\eta }_3})$, with ${\eta }_3=1.0-0.5x_i+0.1y_{1i}-0.01y_{2i}.$

The results of the selection model algorithm are presented in Table 1. The application of this procedure provides the precise ordering, $Y_1,Y_2,Y_3$.

Table 1.

Artificial data set: model selection algorithm.

Step 1		$D(M^{(1)}[h])$
	h = 1	108.7242	Select
	h = 2	206.8058
	h = 3	161.8564
Step 2	Step 2.1	$D(M^{(2)}[1,h])$
	h = 2	222.2701
	h = 3	218.7594	Select
	Step 2.2	$D(M^{(2)}[h,t])$
	h = 1, t = 3	218.7594	Select
	h = 3, t = 1	267.1031
Step 3	Step 3.2	$D(M^{(2)}[h,t,b])$
	h = 1, t = 3, b = 2	2356.432
	h = 1, t = 2, b = 3	330.6761	Select
	h = 2, t = 1, b = 3	2265.350

Two aspects of interest of the implementation of this algorithm are: first, the initial values of the estimators in each step are obtained by applying the regression model of the Poisson univariate model for each variable objective; second, the stopping rule of the iteration process is determined in terms of the absolute value of the gradient (AV G) and the maximum number of iterations (NI). The stopping rule is: $AVG<{10}^{-3}$ or $NI\ge 50$. We have written a script in R [26] to fit our model.

In Table 1, we can see that the selection model algorithm detects the order determined in the process of the generation of data $(Y_1,Y_2,Y_3)$, even though in the second step the variable selected is $Y_3$. Furthermore, the estimation procedure provides accurate estimations (see Table 2). This table presents the real values of the parameters used in the process of the data set, the estimated values and the asymptotic confidence interval.

Table 2.

Artificial data set: estimated parameters.

Parameter	Real value	Estimated value	Confidence interval ( $\mathbf{95}\mathrm{\%}$)
${\beta }_{11}$	5.00	4.91250	4.69609	5.12891
${\beta }_{12}$	−0.50	−0.47867	−0.52670	−0.43063
${\beta }_{21}$	1.00	0.83382	0.40777	1.25986
${\beta }_{22}$	0.90	0.91601	0.85716	0.97485
${\alpha }_{21}$	−0.30	−0.29130	−0.30506	−0.27755
${\beta }_{31}$	1.00	0.71988	−1.02076	2.46053
${\beta }_{32}$	−0.50	−0.45440	−0.75899	−0.14981
${\alpha }_{31}$	0.10	0.09985	0.07350	0.12620
${\alpha }_{32}$	−0.01	−0.00521	−0.01534	0.00491

Finally, the study of adjustment of model gives the following statistics:

• $R_0^2=0.9810$. The global adjustment must be considered excellent.

• $VR{L}_Y=0.9029$ and $VR{L}_X=0.0781$. The global fit is provide by the log-likelihood explained by the distributional assumption.

• $R_r^2=0.8045$. If we consider this coefficient to measure the capacity of explanation of the explanatory variables under the distributional hypothesis then we can conclude that the variable X provides relevant information about the objective variables.

The values of these measures show that the estimated model give quite good fit to the simulated data set.

Second, we generated three artificial data sets with multivariate count data of dimensions 4, 5 and 6, respectively. The parameters of the generation process are included in Table 3, with the explanatory variable X generated according to a distribution $N(0,1)$. Given the exponential dependence between the count variables, the model parameters must be small. Some parameters have been excluded in order to simulate models in which one count variable does not influence the following variables. In all three data sets, applying the procedure to choose the most appropriate model leads to the ordering used in the generation process. Table 4 presents the estimated values of real parameters, with a clear agreement between real and estimated parameters.

Table 3.

Artificial data sets: coefficients.

Variable	Intercept	X	$Y_1$	$Y_2$	$Y_3$	$Y_4$	$Y_5$
Dimension p = 4, sample size n = 200
$Y_1$	1.0	0.5	–	–	–	–	–
$Y_2$	2.0	−0.5	0.1	–	–	–	–
$Y_3$	1.0	0.5	−0.1	0.1	–	–	–
$Y_4$	2.0	−0.5	0.1	−0.1	0.1	–	–
Dimension p = 5, sample size n = 200
$Y_1$	3.0	−0.5	–	–	–	–	–
$Y_2$	3.0	0.5	−0.1	–	–	–	–
$Y_3$	1.0	−0.5	0	0.1	–	–	–
$Y_4$	3.0	−0.5	−0.1	0	0.1	–	–
$Y_5$	3.0	−0.5	0	0	0.1	−0.1	–
Dimension p = 6, sample size n = 200
$Y_1$	3.5	1.0	–	–	–	–	–
$Y_2$	3.0	1.5	−0.01	–	–	–	–
$Y_3$	3.5	1.5	−0.10	0.10	–	–	–
$Y_4$	3.0	0	−0.05	0	0.05	–	–
$Y_5$	3.0	0	−0.01	0	−0.01	0.01	–
$Y_6$	3.0	1.0	0	0.01	0	0	−0.05

Table 4.

Artificial data sets: estimated parameters.

Variable	Intercept	X	$Y_1$	$Y_2$	$Y_3$	$Y_4$	$Y_5$
Dimension p = 4
$Y_1$	0.96 ${}^{\ast }$	0.52 ${}^{\ast }$	–	–	–	–	–
$Y_2$	2.04 ${}^{\ast }$	−0.47 ${}^{\ast }$	0.08 ${}^{\ast }$	–	–	–	–
$Y_3$	0.90 ${}^{\ast }$	0.54 ${}^{\ast }$	−0.10 ${}^{\ast }$	0.11 ${}^{\ast }$	–	–	–
$Y_4$	2.05 ${}^{\ast }$	−0.38 ${}^{\ast }$	0.07 ${}^{\ast }$	-0.08 ${}^{\ast }$	0.08 ${}^{\ast }$	–	–
Dimension p = 5
$Y_1$	2.96 ${}^{\ast }$	−0.51 ${}^{\ast }$	–	–	–	–	–
$Y_2$	2.98 ${}^{\ast }$	0.49 ${}^{\ast }$	−0.10 ${}^{\ast }$	–	–	–	–
$Y_3$	1.00 ${}^{\ast }$	−0.51 ${}^{\ast }$	0.00	0.10 ${}^{\ast }$	–	–	–
$Y_4$	2.78 ${}^{\ast }$	−0.58 ${}^{\ast }$	−0.09 ${}^{\ast }$	0.01	0.10 ${}^{\ast }$	–	–
$Y_5$	3.02 ${}^{\ast }$	−0.50 ${}^{\ast }$	0.00	0.00	0.10 ${}^{\ast }$	−0.10 ${}^{\ast }$	–
Dimension p = 6
$Y_1$	3.49 ${}^{\ast }$	1.00 ${}^{\ast }$	–	–	–	–	–
$Y_2$	2.93 ${}^{\ast }$	1.47 ${}^{\ast }$	−0.01 ${}^{\ast }$	–	–	–	–
$Y_3$	3.71 ${}^{\ast }$	1.59 ${}^{\ast }$	−0.10 ${}^{\ast }$	0.09 ${}^{\ast }$	–	–	–
$Y_4$	3.47 ${}^{\ast }$	0.26 ${}^{\circ }$	−0.06 ${}^{\ast }$	−0.01	0.05 ${}^{\ast }$	–	–
$Y_5$	3.09 ${}^{\ast }$	0.09	−0.02 ${}^{\ast }$	0.01 ${}^{\circ }$	−0.03 ${}^{\ast }$	0.02 ${}^{\ast }$	–
$Y_6$	3.08 ${}^{\ast }$	0.99 ${}^{\ast }$	0.00	0.01 ${}^{\ast }$	0.00	0.00	−0.06 ${}^{\ast }$

$(\ast ):p-value<0.001$ ; ${(}^{\circ }):p-value<0.05$

Finally, we have obtained the adjustment measures of the models associated to all possible ordernatios ( $p!$ factorial for the data set with p count variables). The results are included in Table 5, with the statistics: deviation, Akaike information criterion (AIC), square root of mean square error (RMSE) and the overall R-squared ( $R_0^2$). Table 5 includes:

• These measures for the model associated with the order used in the data generation process.

• A summary of the measures for all models: mean, minimum and maximum.

Table 5.

Artificial data sets: measures of model fit.

	Deviance	AIC	RMSE	${\mathbf{R}}_{\mathbf{0}}^{\mathbf{2}}$
Dimension p = 4
Order
$(1,2,3,4)$	819.096	3613.377	2.519	0.7573
Minimum	${}^{\ast }$819.096	${}^{\ast }$3613.377	${}^{\ast }$2.519	0.7244
Mean	884.212	3678.493	2.606	0.7511
Maximum	942.855	3737.136	2.660	${}^{†}$0.7803
${}^{\ast }$Value obtained for the order (1,2,3,4)
${}^{†}$Value obtained for the order (2,3,1,4)
Dimension p = 5
Order
$(1,2,3,4,5)$	936.654	4660.821	3.503	0.974
Minimum	${}^{\ast }$936.654	${}^{\ast }$4660.821	${}^{\ast }$3.503	0.714
Mean	3782.414	7507.021	19.776	0.888
Maximum	8786.090	12510.256	44.131	${}^{\ast }$0.974
${}^{\ast }$Value obtained for the order (1,2,3,4,5)
Dimension p = 6
Order
$(1,2,3,4,5,6)$	1109.437	5818.742	4.144	0.977
Minimum	${}^{\ast }$1109.437	${}^{\ast }$5818.742	${}^{\ast }$4.144	0.939
Mean	1897.034	6606.098	5.492	0.960
Maximum	2847.234	7557.387	6.621	${}^{†}$0.978
${}^{\ast }$Value obtained for the order (1,2,3,4,5,6)
${}^{†}$Value obtained for the order (1,2,3,5,4,6)

In conclusion, in the three cases (p = 4, 5, 6), the ordering determined by the selection procedure coincides with the ordering used in the data generation process and with the optimal ordering according to the Deviation, AIC and RMSE. Only a small discrepancy is observed in the criterion based on the coefficient $R_0^2$

4. Application: environmental control of foraminifera distribution

The foraminifera are a large group of amoeboid protists and are regarded as the most important groups of marine microfossils. Particularly, the scientific interest on these microfossils focuses on their great utility in paleontology and biostratigraphy.

Ruíz et al. [25] analyze the foraminifera collected in three estuaries (Guadiana, Piedras, Tinto-Odiel) of southwestern Spain. Fifty three observations were collected in the different sedimentary environments (subtidal channels, intertidal channels, channel borders, low salt marshes and high salt marshes) of the three estuaries studied. In this experience, the following variables were taking into account: water salinity during high tide (S), tidal height about the lowest astronomical tide (TH) and number of individuals of three species of foraminifera: Elphidium crispum (EC), Jadammina macrescens (JM), and Trochammina inflata (TI). TH is positive in subaerial environments and increases from the channel border to the high salt marshes, whereas it is negative in the channels and indicates the water depth in the two first environments mentioned above.

Table 6 presents descriptive statistics of the counts and the observed variables. For each specie, single log-linear Poisson models are obtained in the paper [24]. However, the single models do not allow to extract joint information because they do not model the correlation between the populations. In order to study the relation between the abundance of three species with the saline content and the tidal height, we consider the MLCP model constituted by count variables EC, JM and TI, and by the explanatory variables S and TH. The application of the selecting model algorithm provides the order: JM, TI and EC.

Table 6.

Foraminifera data set: descriptive statistics.

	Mean	St.Dev.	Min	Max
Elphidium crispum	3.56604	12.40736	0.0	64.0
Jadammina macrescens	7.84906	15.28875	0.0	71.0
Trochammina inflata	15.05660	42.84651	0.0	287.0
Water salinity	32.76415	7.12256	6.0	36.3
Tidal height	−0.37886	2.55246	−6.0	3.1

The results of the algorithm are presented in the Table 7. Thus, the analyzed MLCP model relates the count variables $\mathbf{Y}=(JM,TI,EC{)}^t$ to the explanatory variables $\mathbf{X}=(1,TH,S{)}^t$.

Table 7.

Foraminifera data set: model selection algorithm.

Step 1		$D(M^{(1)}[h])$
	$EC,h=1$	533.9910
	$JM,h=2$	527.6821	Select
	$TI,h=3$	1722.5150
Step 2	Step 2.1	$D(M^{(2)}[2,h])$
	$EC,h=1$	912.760	Select
	$TI,h=3$	2232.855
	Step 2.2	$D(M^{(2)}[h,t])$
	$(EC,JM),(h=1,t=2)$	1025.142
	$(JM,EC),(h=2,t=1)$	912.760	Select
Step 3	Step 3.2	$D(M^{(3)}[h,t,b])$
	$(JM,EC,TI),(h=2,t=1,b=3)$	2697.231
	$(JM,TI,EC),(h=2,t=3,b=1)$	2612.888	Select
	$(TI,JM,EC),(h=3,t=2,b=1)$	2629.255

The parameters of model are $\{({\beta }_1,{\beta }_2,{\beta }_3,{\alpha }_2,{\alpha }_3):{\beta }_j\in {\mathbf{R}}^3,{\alpha }_h\in {\mathbf{R}}^{h-1}\}$ with ${\beta }_j=({\beta }_{j1},{\beta }_{j2},{\beta }_{j3}{)}^t$ for j = 1, 2, 3, ${\alpha }_2={\alpha }_{21}$ and ${\alpha }_3=({\alpha }_{31},{\alpha }_{32}{)}^t$. So, the expected values of the Poisson distributions are ${\lambda }_1$, ${\lambda }_2$ and ${\lambda }_3$ such that

$$\begin{array}{rl}\log ({\lambda }_1)& ={\beta }_{11}+{\beta }_{12}TH+{\beta }_{13}S\\ \log ({\lambda }_2)& ={\beta }_{21}+{\beta }_{22}TH+{\beta }_{23}S+{\alpha }_{21}JM\\ \log ({\lambda }_3)& ={\beta }_{31}+{\beta }_{32}TH+{\beta }_{33}S+{\alpha }_{31}JM+{\alpha }_{32}TI\end{array}$$

The results of the analysis are summarized in Table 8. The capacity of global explanation of the model should be considered significant, particularly the distributional hypothesis. The results obtained from univariate models are:

• for EC, $R_0^2(EC)=0.3757964$

• for JM, $R_0^2(JM)=0.5250740$

• for TI, $R_0^2(TI)=0.3585918$

Table 8.

Foraminifera data set: estimated parameters.

Parameter		Estimated value	Confidence interval ( $\mathbf{95}\mathrm{\%}$)
Component: JM
Intercept	${\beta }_{11}$	0.03392	−0.68576	0.75359
Tidal height	${\beta }_{12}$	0.75441	0.66776	0.84106
Salinity	${\beta }_{13}$	0.03267	0.01257	0.05277
Component: TI
Intercept	${\beta }_{21}$	2.60108	2.26086	2.94131
Tidal height	${\beta }_{22}$	0.57358	0.51001	0.63716
Salinity	${\beta }_{23}$	−0.01916	−0.02888	−0.00944
JM	${\alpha }_{21}$	0.00845	0.00457	0.01233
Component: EC
Intercept	${\beta }_{31}$	−80.67440	−100.46653	−60.88226
Tidal height	${\beta }_{32}$	0.29280	0.21467	0.37093
Salinity	${\beta }_{33}$	2.31655	1.76659	2.86650
JM	${\alpha }_{31}$	−0.29630	−0.51093	−0.08166
TI	${\alpha }_{32}$	−0.17067	−0.34921	0.00786
Goodness of fit		$R_0^2=0.5691$	$VR{L}_Y=0.3288$	RMSE = 23.3
		$R_r^2=0.3579$	$VR{L}_X=0.2402$	AIC = 2861.0

That is, the multivariate model provides a better fit to the data than the univariate models. Therefore, the model allows the interpretation of the relationships between the three count variables and the two explanatory variables.

From Table 8, tidal height and JM show a high relation ( ${\beta }_{12}=0.75$), because the highest abundances of this species are found in the high salt marsh deposits, located over the mean high tide level. TI is the dominant species in the low salt marsh deposits and consequently presents a positive but lower relation ( ${\beta }_{22}=0.57$) with the tidal height. In addition, the positive relation ( ${\beta }_{32}=0.29$) of this variable with EC is explained by the presence of frequent individuals of this species in the channel border. Consequently, a height gradient is found when this multivariate method is applied to the sedimentary distribution of these three species, coinciding with the arrangement (JM−TI−EC) proportioned by the model selection algorithm and very similar to that observed in British estuaries [12].

The very high relation ( ${\beta }_{33}=2.316$) between salinity and EC can be explained by the marine character of this species (Villanueva [27]), which is clearly concentrated near the river mouths with salinity ranges comprised between ${33}^{0/00}$ and ${36}^{0/00}$. Nevertheless, both JM and TI show slight relations positive ( ${\beta }_{13}=0.032$) or negative ( ${\beta }_{23}=-0.019$), respectively. These species are little sensitive to changes in this parameter, although this positive relation between salinity and JM has been also detected in Canadian estuaries  [9,14].

5. Conclusion

Count data has been the subject of an increasing number of scientific works, but models taking into account multivariate count data are still scarcely applied. For single counts, the Poisson distribution is commonly used. For multiple counts, the literature reports some works using several multivariate Poisson distribution models. The disadvantage of some of these models is that the correlation between count variables is restricted to being positive.

In this work, we propose a new regression model for multivariate count data that is based on the Poisson distribution. Our proposed model does not only extend the bivariate model proposed by Berkhout and Plug[2] but also allows to model the dependence structure between count variables and the relationship with a set of explanatory variables.

The model may have certain limitations: complexity in the correlation structure and difficulty in the selection of the order of the components. However, the flexibility of the correlation structure should be regarded as an advantage of this model. Another advantage is that model estimators require little computational power and are relatively easy to obtain. To complete the study of the model we analyze two issues: analysis of adjustment measures and selection of the model.

We propose various adjustment statistics which are based on the likelihood of the model. First, an overall adjustment measure ( $R_O^2$) that can be expressed as the sum of two components: the component explained by the distributional hypothesis and the component explained by the structural hypothesis. Second, a relative adjustment measure ( $R_r^2$) that can be interpreted as a measure of adjustment to the structural assumptions under the distributional hypothesis. These measures provide relevant and useful information about the adjustment of the model and have an additional advantage: they can be applied to any parametric multivariate model whose estimators are based on the maximum likelihood method.

The distributional hypothesis of the model depends on the order of the count variables. The order of the structure of dependency between count variables can be known in many experimental studies. However, there are situations in which the order is not known. In this case, we propose an algorithm to choose an adequate order. In practice, any available information on the dependency between count variables must be incorporated into the study, which should appropriately modify the proposed procedure.

In order to illustrate this multivariate technique, a real application is included: the relation between the abundance of three benthic foraminifera and several environmental variables. The challenge to make a joint analysis of the aforementioned populations has led to develop a multivariate model. This experience allows illustrating the usefulness of this multivariate model for the analysis of species counting experiences (scientific areas such as cell biology, botany, zoology, virology, and so on).

Other methods have been recently proposed for multivariate regression count data tasks, such as mixture Poisson models and copula based Poisson models. Both approaches are very flexible, allowing the description and modeling of a wide set of structures of correlations which can appear in real data sets. However, these models are not easily interpretable, and they can drive to conclusions which can be distant from the actual process generating the data set. Our proposed model can offer a more reliable depiction of the random process behind the data in many real problems. When the hypothesis about the ordering of the variables is questionable, the other models must also be considered, selecting the most appropriate one with the aid of some goodness of fit criterion.

Finally, it is clear that other aspects of the models need to be explored. For example, a detailed study of the properties of the maximum likelihood estimators, the estimation of the marginal moments of the count vector components, design of algorithms of selection of the explanatory variables and the development of diagnostic techniques.

The proposed model assumes no overdispersion or zero-inflation in the conditional distributions. Extending the distributional hypothesis to models containing these features (overdispersion and zero-inflation) can be very interesting from a practical viewpoint, but it requires a broad development of its theoretical and computational issues. The study of these aspects should be the subject of further works.

Disclosure statement

No potential conflict of interest was reported by the author(s).