A new multivariate t distribution with variant tail weights and its application in robust regression analysis

ABSTRACT

In this paper, we propose a new kind of multivariate t distribution by allowing different degrees of freedom for each univariate component. Compared with the classical multivariate t distribution, it is more flexible in the model specification that can be used to deal with the variant amounts of tail weights on marginals in multivariate data modeling. In particular, it could include components following the multivariate normal distribution, and it contains the product of independent t-distributions as a special case. Subsequently, it is extended to the regression model as the joint distribution of the error terms. Important distributional properties are explored and useful statistical methods are developed. The flexibility of the specified structure in better capturing the characteristic of data is exemplified by both simulation studies and real data analyses.

1. Introduction

As a natural generalization of the univariate Student's t distribution, the multivariate t (MVT) distribution is a robust alternative to the multivariate normal distribution in the analysis of multivariate continuous data with heavy tails or outliers. The first derivation of the MVT distribution was accomplished by independent researchers, see [4,6], and the latter just concentrated on the bivariate case. Some characterizations on the MVT distribution were presented, see [5,16]. The work of considering an efficient computational procedure for the bivariate case was included in [1]. The maximum likelihood estimation methods for the MVT distribution with missing data were also discussed in [17,19–21]. The regression models with MVT error terms have been widely investigated by [8,18,29]. A comprehensive review on the topic was summarized in [14], as the first monograph on the MVT distribution. The corresponding mathematical properties such as stochastic representation (SR), consistency property, density expansion, moments conditional distributions can also be found in [23], and the associated estimation methods can be found in [24]. Recently, a sampling method for the MVT distribution with R package was provided, see [11].

The most common way of constructing the MVT distribution is through the SR of a normal random vector and an independent chi-squared random variable. Assume that the random vector $\mathbf{z}=(Z_1,\dots ,Z_d{)}^{\mathrm{\top }}\sim N_d(\mathbf{0}\mathbf{0},\mathbf{\Sigma })$, the random variable $V\sim {\chi }^2\left(\nu \right)$, and they are independent (denoted by $\mathbf{z}\mathrm{\perp }\mathrm{\perp }V$). Define

$$(X_1,\dots ,X_d{)}^{\mathrm{\top }}=\mathbf{x}=\mathit{\mu }+\frac{\mathbf{z}}{\sqrt{V/\nu }},$$ (1)

then $\mathbf{x}$ is said to follow a multivariate t distribution, denoted by $\mathbf{x}\sim t_d(\mathit{\mu },\mathbf{\Sigma },\nu )$, where $\mathit{\mu }=({\mu }_1,\dots ,{\mu }_d{)}^{\mathrm{\top }}$ is the location parameter vector, $\mathbf{\Sigma }=({\sigma }_{ij}{)}_{d\times d}$ is a positive-definite scale matrix and ν is the degrees of freedom. When $\nu =1$, the distribution reduces to the multivariate Cauchy distribution. As $\nu \to \mathrm{\infty }$, the distribution approaches the multivariate normal distribution. Hence, the parameter ν may be viewed as a robustness tuning parameter (see [22], pp.5332). It can be fixed in advance or inferred from the observed data. An equivalent expression of (1) is

$$\mathbf{x}=\mathit{\mu }+U^{-\frac{1}{2}}\mathbf{z},$$ (2)

where $U\sim \text{Gamma}(\nu /2,\nu /2)$ (the density of $Y\sim \text{Gamma}(\alpha ,\mathrm{\text{β}})$ is denoted by $\text{Gamma}(y\mid \alpha ,\mathrm{\text{β}})$ $={\mathrm{\text{β}}}^{\alpha }{y}^{\alpha -1}\exp (-\mathrm{\text{β}}y)/\mathrm{\Gamma }\left(\alpha \right)$ with $\alpha >0$ and $\mathrm{\text{β}}>0$), $\mathbf{z}\sim N_d(\mathbf{0}\mathbf{0},\mathbf{\Sigma })$ and $U\mathrm{\perp }\mathrm{\perp }\mathbf{z}$. We call (1) or (2) the SR of the classical MVT or Type I MVT random vector.

From the SR (1) or (2), we have observed that the Type I MVT distribution has the following particular disadvantages:

1. All components follow univariate t distributions with the same degrees of freedom ν and hence the same amount of tailweight (see [13], pp. 163).

2. The Type I MVT random vector $\mathbf{x}$ with a finite ν includes neither a component $X_i$ following the univariate normal distribution nor a sub-vector $(X_{i_1},\dots ,X_{i_r}{)}^{\mathrm{\top }}$ following the r-dimensional normal distribution, where $1\le i_1<\cdots <i_r\le d$ and $1\le r<d$.

3. The Type I MVT random vector $\mathbf{x}$ can never contain statistically independent components since all components $\{X_i{\}}_{i=1}^d$ share a common random variable V or U, even when $\mathbf{\Sigma }$ is diagonal.

These drawbacks definitely limit its application to a certain extent. To overcome the above first drawback, within the framework of copula, a class of meta-elliptical distributions including a special member of so-called asymmetric multivariate t (AMVT) distribution was proposed, see [7], whose marginals being univariate t distributions with different degrees of freedom. They further pointed out that the AMVT distribution and Type I MVT distribution have the same copulas (i.e. the same correlation structure). Hence, the AMVT distribution still cannot overcome the above second and third drawbacks. Moreover, they did not develop any statistical inference methods on this particular distribution. Then, a new bivariate t distribution with marginals having different degrees of freedom was proposed, see [13], by defining

$$X_1=\frac{Z_1}{\sqrt{V_1/{\nu }_1}}\text{and}{X}_2=\frac{Z_2}{\sqrt{(V_1+V_2)/{\nu }_2}},$$ (3)

where $Z_1,Z_2,V_1,V_2$ are assumed to be mutually independent, $Z_1,Z_2\stackrel{\mathrm{i}\mathrm{i}\mathrm{d}}{\sim }N(0,1)$, $V_1\sim {\chi }^2\left({\nu }_1\right)$ and $V_2\sim {\chi }^2({\nu }_2-{\nu }_1)$ with ${\nu }_2\ge {\nu }_1$. It also mentioned two possible multivariate extensions of (3) (see [13]) as:

$$X_1^{\prime }=\frac{Z_1}{\sqrt{V_1/{\nu }_1}},X_2^{\prime }=\frac{Z_2}{\sqrt{(V_1+V_2)/{\nu }_2}},\dots ,X_d^{\prime }=\frac{Z_d}{\sqrt{(V_1+\cdots +V_d)/{\nu }_d}},$$ (4)

where $Z_1,\dots ,Z_d,V_1,\dots ,V_d$ are assumed to be mutually independent, $Z_1,\dots ,Z_d\stackrel{\mathrm{i}\mathrm{i}\mathrm{d}}{\sim }N(0,1)$, $V_1\sim {\chi }^2\left({\nu }_1\right)$ and $V_i\sim {\chi }^2({\nu }_i-{\nu }_{i-1})$ for $i=2,\dots ,d$ with $0<{\nu }_1\le {\nu }_2\le \cdots \le {\nu }_d$; and

$$X_1^{″}=\frac{Z_1}{\sqrt{V_1/{\nu }_1}},X_2^{″}=\frac{Z_2}{\sqrt{(V_1+W_2)/{\nu }_2}},\dots ,X_d^{″}=\frac{Z_d}{\sqrt{(V_1+W_d)/{\nu }_d}},$$ (5)

where the $\{Z_i{\}}_{i=1}^d$ and $V_1$ are the same as defined above and $W_i\sim {\chi }^2({\nu }_i-{\nu }_1)$ with $0<{\nu }_1\le {\nu }_i$ for $i=2,\dots ,d$. We have three comments on (3)–(5). First, all components in (3)–(5) follow univariate t distributions with possible different degrees of freedom, i.e. the first drawback of Type I MVT distribution can be overcome; however, the second and third drawbacks cannot yet be overcome. Second, only the distributional properties for the bivariate case were studied in [13], and did not provide any statistical inference methods. Third, for multivariate cases, it remarked that ‘It appears, however, that only in the bivariate case are the joint density function and hence conditional distributions fully tractable. It is this that prompted the publication of the current special case, along with any independent interest, this special case may possess’, see [13]. An alternative t-distribution by replacing the common gamma divisor in (2) with p i.i.d. gamma divisors was then proposed, see [9], and they only mentioned the proposal of allowing these independent gamma divisors with possible different degrees of freedom, but without any detailed discussion. By inserting multidimensional weights into the Gaussian scale mixture, it is generalized to a multivariate t-distribution, see [10]. However, the resulting marginal is a linear combination of univariate t-distributions, thus has no intuitive interpretation on the degrees of freedom for marginal distributions. For the two data sets presented in Section 6 below, we may encounter that there is a dependency structure among components, while each component does not have the same tailweight. The classical MVT distribution is no longer appropriate, thus a new tool is expected to break through the limitations in existing models and have a broader application.

In this paper, we will propose a new kind of MVT distribution by allowing different degrees of freedom for each univariate component, called Type II  MVT distribution. The proposed distribution has several remarkable features including (a) all components follow univariate t-distributions with possible different degrees of freedom; (b) it could include components following the multivariate normal distributions when the corresponding ${\nu }_i\to \mathrm{\infty }$; (c) it could contain statistically independent components such that the product of independent t distributions is its special case. Compared with the classical MVT distribution, this new structure can better capture the characteristic of the data. We first derive important distributional properties and then develop useful statistical inference methods.

The rest of the paper is organized as follows. In Section 2, the Type II  MVT distribution is outlined and some distributional properties are explored. In Section 3, the maximum likelihood estimation of the parameters via the Monte Carlo expectation/conditional maximization (ECM) algorithm and test of independence are developed, and then an extended regression model is introduced and investigated. Bayesian methods are presented in Section 4. In Section 5, some simulation studies are performed to evaluate the proposed methods. Two real data sets are used to compare the proposed distribution with the classical MVT distribution in Section 6. Finally, a discussion is given in Section 7.

2. Type II  multivariate $\mathit{t}$ distribution

We define a new MVT distribution through the SR instead of the joint probability density function (pdf). A random vector $\mathbf{x}=(X_1,\dots ,X_d{)}^{\mathrm{\top }}$ is said to follow the Type II  MVT distribution, denoted by $\mathbf{x}\sim t_d^{\left(\mathrm{I}\mathrm{I}\right)}(\mathit{\mu },\mathbf{\Sigma },\mathit{\nu })$, if $\mathbf{x}$ can be stochastically represented as

$$\mathbf{x}=\mathit{\mu }+{\mathbf{U}}^{-1/2}\mathbf{z},$$ (6)

where $\mathit{\mu }=({\mu }_1,\dots ,{\mu }_d{)}^{\mathrm{\top }}$, ${\mathbf{U}}^{-1/2}=\text{diag}\left({\mathbf{u}}^{-1/2}\right)$, $\mathbf{u}=(U_1,\dots ,U_d{)}^{\mathrm{\top }}$, $U_i\stackrel{\mathrm{i}\mathrm{n}\mathrm{d}}{\sim }\text{Gamma}({\nu }_i/2,{\nu }_i/2)$ for $i=1,\dots ,d$, $\mathbf{z}=(Z_1,\dots ,Z_d{)}^{\mathrm{\top }}\sim N_d(\mathbf{0}\mathbf{0},\mathbf{\Sigma })$, $\mathit{\nu }=({\nu }_1,\dots ,{\nu }_d{)}^{\mathrm{\top }}$, and $\mathbf{u}\mathrm{\perp }\mathrm{\perp }\mathbf{z}$. The vector $\mathit{\nu }$ could be fixed in advance or it can be estimated from the observed data. Note that if ${\nu }_1=\cdots ={\nu }_d=\nu$, the distribution of $\mathbf{x}$ in (6) becomes the alternative multivariate t distribution proposed by [9].

By comparing with the Type I MVT distribution, the Type II  MVT distribution possesses the following three significant characteristics:

1. From (6), it is easy to obtain $X_i={\mu }_i+U_i^{-1/2}{Z}_i$, indicating that all components follow univariate t-distributions with possible different degrees of freedom; i.e. $X_i\sim t({\mu }_i,{\sigma }_{ii},{\nu }_i)$ for $i=1,\dots ,d$.

2. The Type II  MVT random vector $\mathbf{x}$ could include components following a multivariate normal distribution if the corresponding ${\nu }_i\to \mathrm{\infty }$.

3. The Type II  MVT random vector $\mathbf{x}$ could contain statistically independent components. From (7) and (8) below, in Type II  MVT distribution, any two components $X_i,X_j$ ( $i\ne j$) are independent provided that $Z_i,Z_j$ are independent. Whereas in Type I MVT distribution, any two components are always dependent since they share a common factor U. Particularly, when $\mathbf{\Sigma }$ is diagonal, the Type II  MVT density becomes the product of d independent univariate t densities.

Remark 2.1

The dependency structure of the Type II  MVT distribution induced by the SR (6) is totally determined by the variance–covariance matrix $\mathbf{\Sigma }$ of $\mathbf{z}$. Note that the Gamma random variables $\{U_i{\}}_{i=1}^d$ are independent, thus all $\{{\nu }_i{\}}_{i=1}^d$ are free parameters and the estimation procedure on each ${\nu }_i$ will be independent and straightforward. However, the dependency structure of the two distributions respectively induced by the SRs (4) and (5) is only own to the common random variable $V_1$ since the correlation from the multivariate normal distribution is not incorporated. Under the construction of (4) or (5), the estimations of ${\nu }_i$'s will be more complicated since an ascending order constraint ${\nu }_1\le {\nu }_2\le \cdots \le {\nu }_d$ or ${\nu }_1\le {\nu }_i$ for $i=2,\dots ,d$ is involved.

2.1. Density function of Type II  MVT distribution

To derive the joint pdf of Type II  MVT distribution, we first introduce the multivariate truncated normal distribution. A d-dimensional random vector $\mathbf{w}=(W_1,\dots ,W_d{)}^{\mathrm{\top }}$ is said to follow the multivariate normal distribution truncated in the box $[\mathit{a},\mathit{b}]$, denoted by $\mathbf{w}\sim {\text{TN}}_d(\mathit{\mu },\mathbf{\Sigma };[\mathit{a},\mathit{b}\left]\right)$ with $\mathit{a}=(a_1,\dots ,a_d{)}^{\mathrm{\top }}\in {\mathbb{R}}^d$ and $\mathit{b}=(b_1,\dots ,b_d{)}^{\mathrm{\top }}\in {\mathbb{R}}^d$, if its joint pdf is (see [12], pp. 210)

$${\text{TN}}_d(\mathit{w}|\mathit{\mu },\mathbf{\Sigma };[\mathit{a},\mathit{b}\left]\right)=\frac{\exp [-0.5(\mathit{w}-\mathit{\mu }{)}^{\mathrm{\top }}{\mathbf{\Sigma }}^{-1}(\mathit{w}-\mathit{\mu })]}{{\int }_{[\mathit{a},\mathit{b}]}\exp [-0.5(\mathit{w}-\mathit{\mu }{)}^{\mathrm{\top }}{\mathbf{\Sigma }}^{-1}(\mathit{w}-\mathit{\mu })]\text{d}\mathit{w}},\mathit{a}\le \mathit{w}\le \mathit{b},$$

where $\mathit{w}=(w_1,\dots ,w_d{)}^{\mathrm{\top }}$ is the realization of $\mathbf{w}=(W_1,\dots ,W_d{)}^{\mathrm{\top }}$.

Now, we can rewrite (6) as a mixture of

$$\mathbf{x}|(\mathbf{u}=\mathit{u})\sim N_d(\mathit{\mu },{\mathit{U}}^{-1/2}\mathbf{\Sigma }{\mathit{U}}^{-1/2})\text{and}\left\{U_i{\}}_{i=1}^d\stackrel{\mathrm{i}\mathrm{n}\mathrm{d}}{\sim }\text{Gamma}\right({\nu }_i/2,{\nu }_i/2),$$ (7)

where $\mathit{u}=(u_1,\dots ,u_d{)}^{\mathrm{\top }}$ is the realization of $\mathbf{u}=(U_1,\dots ,U_d{)}^{\mathrm{\top }}$ and ${\mathit{U}}^{-1/2}=\text{diag}\left({\mathit{u}}^{-1/2}\right)$ is the realization of ${\mathbf{U}}^{-1/2}=\text{diag}\left({\mathbf{u}}^{-1/2}\right)$. The joint pdf of $\mathbf{x}$ is obtained as (for detailed derivation, see Appendix):

$$f_{\mathbf{x}}\left(\mathit{x}\right)={\left(\frac{\pi }{2}\right)}^{-\frac{d}{2}}|\mathbf{\Sigma }{|}^{-\frac{1}{2}}\left[\prod _{i=1}^d\frac{(\frac{{\nu }_i}{2}{)}^{\frac{{\nu }_i}{2}}}{\mathrm{\Gamma }\left(\frac{{\nu }_i}{2}\right)}\right]\cdot {\mu }_{\mathit{\nu }}\left(\mathbf{w}\right)\cdot C,$$ (8)

where

$$\begin{array}{rl}{\mu }_{\mathit{\nu }}\left(\mathbf{w}\right)& ={\int }_{{\mathbb{R}}_{+}^d}\left(\prod _{i=1}^d{w}_i^{{\nu }_i}\right)\cdot C^{-1}\exp \left(-0.5{\mathit{w}}^{\mathrm{\top }}{\mathbf{\Sigma }}^{\ast -1}\mathit{w}\right)\text{d}\mathit{w},\\ C& ={\int }_{{\mathbb{R}}_{+}^d}\exp \left(-0.5{\mathit{w}}^{\mathrm{\top }}{\mathbf{\Sigma }}^{\ast -1}\mathit{w}\right)\text{d}\mathit{w},\end{array}$$ (9)

and ${\mathbf{\Sigma }}^{\ast }=({\mathit{X}}^{\ast }{\mathbf{\Sigma }}^{-1}{\mathit{X}}^{\ast }+{\mathbf{\Sigma }}_0{)}^{-1}$ with

$${\mathit{X}}^{\ast }=\text{diag}(\mathit{x}-\mathit{\mu })\text{and}{\mathbf{\Sigma }}_0=\text{diag}\left(\mathit{\nu }\right).$$ (10)

To have an insight into the function ${\mu }_{\mathit{\nu }}\left(\mathbf{w}\right)$, we first define the mixed moments of a random vector. Let a random vector $\mathbf{x}=(X_1,\dots ,X_d{)}^{\mathrm{\top }}$ have the joint pdf $f_{\mathbf{x}}\left(\mathit{x}\right)$. Given $\mathit{r}=(r_1,\dots ,r_d{)}^{\mathrm{\top }}$, where $r_1,\dots ,r_d$ are non-negative constants, then

$$E(X_1^{r_1}\cdots X_d^{r_d})=\int x_1^{r_1}\cdots x_d^{r_d}\cdot f_{\mathbf{x}}\left(\mathit{x}\right)\text{d}\mathit{x}$$

is called the mixed moments of $\mathbf{x}$ in power of $\mathit{r}$ with respect to $\mathbf{x}\sim f_{\mathbf{x}}\left(\mathit{x}\right)$. Note that C is the normalizing constant of ${\text{TN}}_d(\mathbf{0}\mathbf{0},{\mathbf{\Sigma }}^{\ast };{\mathbb{R}}_{+}^d)$, then ${\mu }_{\mathit{\nu }}\left(\mathbf{w}\right)$ given by (9) is the mixed moments of $\mathbf{w}$ in power of $\mathit{\nu }$ with respect to $\mathbf{w}\sim {\text{TN}}_d(\mathbf{0},{\mathbf{\Sigma }}^{\ast };$ ${\mathbb{R}}_{+}^d)$. It is clear from (8) that this distribution is not of the elliptical form (see [7]).

Let $\mathbf{x}=({\mathbf{x}}_1^{\mathrm{\top }},{\mathbf{x}}_2^{\mathrm{\top }}{)}^{\mathrm{\top }}$, where ${\mathbf{x}}_1$ consists of the first q components and ${\mathbf{x}}_2$ consists of the last $(d-q)$ components. To consider the conditional distribution of ${\mathbf{x}}_1$ given ${\mathbf{x}}_2={\mathit{x}}_2$, the derivation of $f_{{\mathbf{x}}_1|{\mathbf{x}}_2}\left({\mathit{x}}_1|{\mathit{x}}_2\right)=f_{\mathbf{x}}\left(\mathit{x}\right)/f_{{\mathbf{x}}_2}\left({\mathit{x}}_2\right)$ is not easy since the marginal density of ${\mathbf{x}}_2$ depends on a multiple integral over hypercubes. For visualization, we present the conditional distribution curves for the bivariate case. The parameters are set as

1. Case 1: $\mathit{\mu }=(0,0{)}^{\mathrm{\top }}$, $\mathbf{\Sigma }=[1,0.8;0.8,1]$, $\mathit{\nu }=(3,5{)}^{\mathrm{\top }}$. The plots of $f\left(x_1|x_2\right)$ are curved for $x_2=-2,-1,0,1,2$, respectively.

2. Case 2: $\mathit{\mu }=(1,3{)}^{\mathrm{\top }}$, $\mathbf{\Sigma }=[1,-0.5;-0.5,1]$, $\mathit{\nu }=(3,2{)}^{\mathrm{\top }}$. The plots of $f\left(x_1|x_2\right)$ are curved for $x_2=-1,1,3,5,7$, respectively.

From Figure 1, the curves of $f\left(x_1|x_2\right)$ are in similar shapes as the value of $x_2$ moves under the same parameter configuration.

Figure 1.

The conditional distribution curves for $f\left(x_1|x_2\right)$ in Type II MVT distribution. (a) $\mathit{\mu }=(0,0{)}^{\mathrm{\top }}$, $\mathbf{\Sigma }=[1,0.8;0.8,1]$, $\mathit{\nu }=(3,5{)}^{\mathrm{\top }}$; (b) $\mathit{\mu }=(1,3{)}^{\mathrm{\top }}$, $\mathbf{\Sigma }=[1,-0.5;-0.5,1]$, $\mathit{\nu }=(3,2{)}^{\mathrm{\top }}$.

2.2. Moments and correlation

The mean vector and variance–covariance matrix of $\mathbf{x}$ are, respectively, given by

$$E\left(\mathbf{x}\right)=\mathit{\mu },\text{for }{\nu }_i>1,i=1,\dots ,d\text{and}\text{Var}\left(\mathbf{x}\right)={\mathbf{\Sigma }}^{\prime }=\left({\sigma }_{ij}^{\prime }\right),$$

where

$${\sigma }_{ii}^{\prime }=\frac{{\nu }_i}{{\nu }_i-2}{\sigma }_{ii},{\sigma }_{ij}^{\prime }=\frac{\left({\nu }_i/2{)}^{\frac{1}{2}}\mathrm{\Gamma }\right(\frac{{\nu }_i-1}{2}\left)\right({\nu }_j/2{)}^{\frac{1}{2}}\mathrm{\Gamma }\left(\frac{{\nu }_j-1}{2}\right)}{\mathrm{\Gamma }\left(\frac{{\nu }_i}{2}\right)\mathrm{\Gamma }\left(\frac{{\nu }_j}{2}\right)}{\sigma }_{ij},i\ne j,$$

for ${\nu }_i,{\nu }_j>2$. Thus, the correlation coefficient between $X_i$ and $X_j$ is given by

$$\text{Corr}(X_i,X_j)=\frac{\sqrt{({\nu }_i-2)({\nu }_j-2)}\mathrm{\Gamma }\left(\frac{{\nu }_i-1}{2}\right)\mathrm{\Gamma }\left(\frac{{\nu }_j-1}{2}\right){\sigma }_{ij}}{2\mathrm{\Gamma }\left(\frac{{\nu }_i}{2}\right)\mathrm{\Gamma }\left(\frac{{\nu }_j}{2}\right)\sqrt{{\sigma }_{ii}{\sigma }_{jj}}},{\nu }_i,{\nu }_j>2.$$ (11)

From (11), we can see that the sign of $\text{Corr}(X_i,X_j)$ is determined by ${\sigma }_{ij}$ and can be arbitrary (positive, zero or negative).

2.3. Comparison of the densities

We compare the pdfs of Type I MVT distribution $t_d(\mathit{\mu },\mathbf{\Sigma },\nu )$ and Type II  MVT distribution $t_d^{\left(\mathrm{I}\mathrm{I}\right)}(\mathit{\mu },\mathbf{\Sigma },\mathit{\nu })$. For the same $\mathit{\mu }$ and $\mathbf{\Sigma }$, if one sets $\mathit{\nu }=\nu \mathbf{1}{\mathbf{1}}_d$ where $\mathbf{1}{\mathbf{1}}_d=(1,\dots ,1{)}^{\mathrm{\top }}$, from (2) and (6) we can see that the marginal density of each $X_i$ from the two distributions are identical, following $t({\mu }_i,{\sigma }_{ii},\nu )$. However, the shapes of the two density functions may appear totally different. Moreover, Type II  MVT distribution is more flexible since it could contain the case of $\mathit{\nu }\ne \nu \mathbf{1}{\mathbf{1}}_d$.

For more comparisons, we consider d = 2 and set parameter configurations as $\mathit{\mu }=({\mu }_1,{\mu }_2{)}^{\mathrm{\top }}$ and $\mathbf{\Sigma }=[{\sigma }_{11},{\sigma }_{12};{\sigma }_{21},{\sigma }_{22}]$ for both distributions, ν being the degrees of freedom in Type I MVT, $\mathit{\nu }=({\nu }_1,{\nu }_2{)}^{\mathrm{\top }}$ being degrees of freedom in Type II  MVT, and $\nu ={\nu }_1={\nu }_2$. To illustrate the differences, we averagely take 30 values of $X_1$ within $[{\mu }_1-3\sqrt{{\sigma }_{11}},{\mu }_1+3\sqrt{{\sigma }_{11}}]$ and 30 values of $X_2$ within $[{\mu }_2-3\sqrt{{\sigma }_{22}},{\mu }_2+3\sqrt{{\sigma }_{22}}]$. The maximum and minimum of Type I and Type II  MVT densities over these $(X_1,X_2)$ points are computed and displayed in Table 1. The correlation coefficients between two components denoted by ρ in the two distributions are also compared. Besides, the shapes of corresponding densities are curved in both contour graphs and three-dimensional perspectives by Figures 2 and 3. It is observed that when the two densities have the same amount of marginal tail weights, the Type II  density curve is a bit flatter than the Type I curve, and the correlation coefficient between components in Type II  MVT is also weaker than that in Type I MVT, which is not surprising as the dependence in Type II  MVT only relies on the dependence from the multivariate normal vector.

Table 1.

Maximum and minimum values of Type I and II MVT densities for $\nu ={\nu }_1={\nu }_2$.

	$\mathit{\mu }=(0,0{)}^{\mathrm{\top }},\mathbf{\Sigma }=\text{diag}(\mathbf{1}{\mathbf{1}}_2)$		$\mathit{\mu }=(1,-1{)}^{\mathrm{\top }},\mathbf{\Sigma }=[2,2;2,3]$
	$\nu =3$	$({\nu }_1,{\nu }_2)=(3,3)$	$\nu =10$	$({\nu }_1,{\nu }_2)=(10,10)$
	$f^{\left(\mathrm{I}\right)}$	$f^{\left(\mathrm{I}\mathrm{I}\right)}$	$f^{\left(\mathrm{I}\right)}$	$f^{\left(\mathrm{I}\mathrm{I}\right)}$
Maximum	0.156351	0.133184	0.111747	0.106234
Minimum	0.001228	0.000528	$7.0562\times {10}^{-8}$	$4.4166\times {10}^{-10}$
ρ	0	0	0.816497	0.767150

Note: $f^{\left(\mathrm{I}\right)}$: Density function of Type I MVT distribution; $f^{\left(\mathrm{I}\mathrm{I}\right)}$: Density function of Type II  MVT distribution, c.f. (8).

Figure 2.

The contour plots and 3-D perspectives of Type I and Type II  MVT density curves given $\mathit{\mu }=(0,0{)}^{\mathrm{\top }}$, $\mathbf{\Sigma }=\text{diag}\left(\mathbf{1}{\mathbf{1}}_2\right)$. (a1)–(a2) Type I MVT with $\nu =3$; (b1)–(b2) Type II  MVT with ${\nu }_1={\nu }_2=3$.

Figure 3.

The contour plots and 3-D perspectives of Type I and Type II  MVT density curves given $\mathit{\mu }=(1,-1{)}^{\mathrm{\top }}$, $\mathbf{\Sigma }=[2,2;2,3]$. (a1)–(a2) Type I MVT with $\nu =10$; (b1)–(b2) Type II  MVT with ${\nu }_1={\nu }_2=10$.

Moreover, we could have different ${\nu }_i$'s in Type II  MVT, thus we choose several combinations of parameters with ${\nu }_1\ne {\nu }_2$. Similarly, the maximum and minimum of densities are presented in Table 2 and their various shapes are displayed in Figure 4.

Table 2.

Maximum and minimum values of Type II  MVT density function for ${\nu }_1\ne {\nu }_2$.

	$\mathit{\mu }=(0,0{)}^{\mathrm{\top }},\mathbf{\Sigma }=\text{diag}(\mathbf{1}{\mathbf{1}}_2)$	$\mathit{\mu }=(1,-1{)}^{\mathrm{\top }},\mathbf{\Sigma }=[2,2;2,3]$
	$({\nu }_1,{\nu }_2)=(3,5)$	$({\nu }_1,{\nu }_2)=(4,20)$
	$f^{\left(\mathrm{I}\mathrm{I}\right)}$	$f^{\left(\mathrm{I}\mathrm{I}\right)}$
Maximum	0.137650	0.103573
Minimum	0.000397	$1.2944\times {10}^{-9}$
ρ	0	0.713626

Figure 4.

The contour plots and 3-D perspectives of Type II  MVT density curves. (a1)–(a2) $\mathit{\mu }=(0,0{)}^{\mathrm{\top }}$, $\mathbf{\Sigma }=\text{diag}\left(\mathbf{1}{\mathbf{1}}_2\right)$, $({\nu }_1,{\nu }_2)=(3,5)$; (b1)–(b2) $\mathit{\mu }=(1,-1{)}^{\mathrm{\top }}$, $\mathbf{\Sigma }=[2,2;2,3]$, $({\nu }_1,{\nu }_2)=(4,20)$.

3. Estimation of parameters and test of independence

3.1. MLEs of $(\mathit{\mu },\mathbf{\Sigma },\mathit{\nu })$ via the Monte Carlo ECM algorithm

Usually the value of $\mathit{\nu }$ cannot be known in advance in practice, we need to estimate $\mathit{\nu }$ together with $(\mathit{\mu },\mathbf{\Sigma })$. Let ${\mathbf{x}}_1,\dots ,{\mathbf{x}}_n\stackrel{\mathrm{i}\mathrm{i}\mathrm{d}}{\sim }{t}_d^{\left(\mathrm{I}\mathrm{I}\right)}(\mathit{\mu },\mathbf{\Sigma },\mathit{\nu })$ and the observed data be $Y_{\mathrm{o}\mathrm{b}\mathrm{s}}=\{{\mathit{x}}_j{\}}_{j=1}^n$, where ${\mathit{x}}_j=(x_{1j},\dots ,x_{dj}{)}^{\mathrm{\top }}$ denotes the realization of ${\mathbf{x}}_j=(X_{1j},\dots ,X_{dj}{)}^{\mathrm{\top }}$ for $j=1,\dots ,n$. The mixture expression of Type II  MVT random vector specified by (7) evokes us to employ the ECM algorithm to obtain the MLEs of the parameters. For each ${\mathbf{x}}_j$ ( $j=1,\dots ,n$), based on (7), we introduce the corresponding latent vector ${\mathbf{u}}_j=(U_{1j},\dots ,U_{dj}{)}^{\mathrm{\top }}$, where $U_{ij}\stackrel{\mathrm{i}\mathrm{n}\mathrm{d}}{\sim }\text{Gamma}({\nu }_i/2,{\nu }_i/2)$ for $i=1,\dots ,d$. The missing data are denoted by $Y_{\mathrm{m}\mathrm{i}\mathrm{s}}=\{{\mathit{u}}_j{\}}_{j=1}^n$, where ${\mathit{u}}_j=(u_{1j},\dots ,u_{dj}{)}^{\mathrm{\top }}$ is the realization of ${\mathbf{u}}_j$, and the complete data are $Y_{\mathrm{c}\mathrm{o}\mathrm{m}}=\{Y_{\mathrm{o}\mathrm{b}\mathrm{s}},Y_{\mathrm{m}\mathrm{i}\mathrm{s}}\}=\{{\mathit{x}}_j,{\mathit{u}}_j{\}}_{j=1}^n$. The complete-data likelihood function is given by

$$\begin{array}{rl}L(\mathit{\mu },\mathbf{\Sigma },\mathit{\nu }|Y_{\mathrm{c}\mathrm{o}\mathrm{m}})& \propto \left(\prod _{j=1}^n\prod _{i=1}^d{u}_{ij}^{\frac{1}{2}}\right)|\mathbf{\Sigma }{|}^{-\frac{n}{2}}\exp \left[-\frac{1}{2}\sum _{j=1}^n({\mathit{x}}_j-\mathit{\mu }{)}^{\mathrm{\top }}({\mathit{U}}_j^{\frac{1}{2}}{\mathbf{\Sigma }}^{-1}{\mathit{U}}_j^{\frac{1}{2}}\left)\right({\mathit{x}}_j-\mathit{\mu })\right]\\ & \times \prod _{j=1}^n\prod _{i=1}^d\frac{{\left(\frac{{\nu }_i}{2}\right)}^{{\nu }_i/2}}{\mathrm{\Gamma }\left(\frac{{\nu }_i}{2}\right)}{u}_{ij}^{\frac{{\nu }_i}{2}-1}\exp \left(-\frac{{\nu }_i}{2}{u}_{ij}\right)\\ \propto & |\mathbf{\Sigma }{|}^{-\frac{n}{2}}\exp \left[-\frac{1}{2}\sum _{j=1}^n({\mathit{x}}_j-\mathit{\mu }{)}^{\mathrm{\top }}({\mathit{U}}_j^{\frac{1}{2}}{\mathbf{\Sigma }}^{-1}{\mathit{U}}_j^{\frac{1}{2}}\left)\right({\mathit{x}}_j-\mathit{\mu })\right]\\ & \times \prod _{j=1}^n\prod _{i=1}^d\frac{{\left(\frac{{\nu }_i}{2}\right)}^{{\nu }_i/2}}{\mathrm{\Gamma }\left(\frac{{\nu }_i}{2}\right)}{u}_{ij}^{\frac{{\nu }_i-1}{2}}\exp \left(-\frac{{\nu }_i}{2}{u}_{ij}\right),\end{array}$$

where ${\mathit{U}}_j^{1/2}=\text{diag}\left({\mathit{u}}_j^{\frac{1}{2}}\right)$. Let ${\mathbf{W}}_j=\text{diag}\left({\mathbf{w}}_j\right)=\text{diag}\left({\mathbf{u}}_j^{1/2}\right)={\mathbf{U}}_j^{1/2}$ and ${\mathit{W}}_j=\text{diag}\left({\mathit{w}}_j\right)={\mathit{U}}_j^{1/2}$ is its realization with ${\mathbf{w}}_j=(W_{1j},\dots ,W_{dj}{)}^{\mathrm{\top }}$ and ${\mathit{w}}_j=(w_{1j},\dots ,w_{dj}{)}^{\mathrm{\top }}$. Then the log-likelihood function becomes

$$\begin{array}{rl}& \ell (\mathit{\mu },\mathbf{\Sigma },\mathit{\nu }|Y_{\mathrm{c}\mathrm{o}\mathrm{m}})\\ & \propto -\frac{n}{2}\log |\mathbf{\Sigma }|-\frac{1}{2}\mathrm{t}\mathrm{r}\left\{{\mathbf{\Sigma }}^{-1}\sum _{j=1}^n\left[{\mathit{W}}_j({\mathit{x}}_j-\mathit{\mu })\right]{\left[{\mathit{W}}_j({\mathit{x}}_j-\mathit{\mu })\right]}^{\mathrm{\top }}\right\}\\ & +\sum _{i=1}^d\left[\frac{n{\nu }_i}{2}\log \left(\frac{{\nu }_i}{2}\right)-n\log \mathrm{\Gamma }\left(\frac{{\nu }_i}{2}\right)+({\nu }_i-1)\sum _{j=1}^n\log \left(w_{ij}\right)-\frac{{\nu }_i}{2}\sum _{j=1}^n{w}_{ij}^2\right].\end{array}$$

The CM-step is to calculate the complete-data conditional MLEs of $(\mathit{\mu },\mathbf{\Sigma })$ as given by

$$\left\{\begin{array}{l}\stackrel{\mathbf{ˆ}}{\mathit{\mu }}={\left(\sum _{j=1}^n{\mathit{W}}_j{\mathbf{\Sigma }}^{-1}{\mathit{W}}_j\right)}^{-1}\left(\sum _{j=1}^n{\mathit{W}}_j{\mathbf{\Sigma }}^{-1}{\mathit{W}}_j{\mathit{x}}_j\right),\\ \stackrel{\mathbf{ˆ}}{\mathbf{\Sigma }}=\frac{1}{n}\sum _{j=1}^n{\mathit{W}}_j({\mathit{x}}_j-\mathit{\mu })({\mathit{x}}_j-\mathit{\mu }{)}^{\mathrm{\top }}{\mathit{W}}_j.\end{array}\right.$$ (12)

And the complete-data MLE of ${\nu }_i$ is the solution to the equation

$$\log \left(\frac{{\nu }_i}{2}\right)+1-\psi \left(\frac{{\nu }_i}{2}\right)+\frac{1}{n}\sum _{j=1}^n\left[2\log \left(w_{ij}\right)-w_{ij}^2\right]=0,i=1,\dots ,d,$$ (13)

where $\psi (\cdot )$ is the digamma function. The E-step is to replace those terms involved ${\mathit{W}}_j$'s in (12)–(13), i.e. $w_{ij}{w}_{kj}$ and $\log \left(w_{ij}\right)$ for $i,k=1,\dots ,d$ and $j=1,\dots ,n$, by their conditional expectations. To this end, we first need to derive the conditional distribution of ${\mathbf{w}}_j|{\mathbf{x}}_j$, which is given by

$$f_{{\mathbf{w}}_j|{\mathbf{x}}_j}\left({\mathit{w}}_j|{\mathit{x}}_j\right)\propto w_{1j}^{{\nu }_1}\cdots w_{dj}^{{\nu }_d}\exp \left(-\frac{1}{2}{\mathit{w}}_j^{\mathrm{\top }}{\mathbf{\Sigma }}_j^{\ast -1}{\mathit{w}}_j\right),{\mathit{w}}_j\ge \mathbf{0}\mathbf{0},$$ (14)

where ${\mathbf{\Sigma }}_j^{\ast }=({\mathit{X}}_j^{\ast }{\mathbf{\Sigma }}^{-1}{\mathit{X}}_j^{\ast }+{\mathbf{\Sigma }}_0{)}^{-1}$, ${\mathit{X}}_j^{\ast }=\text{diag}({\mathit{x}}_j-\mathit{\mu })$ and ${\mathbf{\Sigma }}_0$ is defined by (10). Since the normalizing constant in (14) is not available in closed form, we will perform the Monte Carlo implementation in the E-step (see [2,15,28]).

Given (14), the Monte Carlo method for calculating the conditional expectations is summarized as follows.

• Step 1:
  Generate ${\mathbf{w}}_j^{\left(1\right)},\dots ,{\mathbf{w}}_j^{\left(G\right)}\stackrel{\mathrm{i}\mathrm{i}\mathrm{d}}{\sim }{\text{TN}}_d(\mathbf{0}\mathbf{0},{\mathbf{\Sigma }}_j^{\ast };{\mathbb{R}}_{+}^d)$, and approximately calculate the conditional pdf (14) as

  $$f_{{\mathbf{w}}_j|{\mathbf{x}}_j}\left({\mathit{w}}_j|{\mathit{x}}_j\right)\approx \frac{1}{c_j}{w}_{1j}^{{\nu }_1}\cdots w_{dj}^{{\nu }_d}\cdot {\text{TN}}_d\left({\mathit{w}}_j|\mathbf{0}\mathbf{0},{\mathbf{\Sigma }}_j^{\ast };{\mathbb{R}}_{+}^d\right),$$

  where

  $$c_j=\frac{1}{G}\sum _{g=1}^G{w}_{1j}^{\left(g\right){\nu }_1}\cdots w_{dj}^{\left(g\right){\nu }_d}.$$ (15)
• Step 2:
  Approximately calculate the conditional expectations as

  $$\begin{array}{rl}E(W_{ij}{W}_{kj}|{\mathit{x}}_j,\mathit{\mu },\mathbf{\Sigma },\mathit{\nu })& \approx \frac{1}{G{c}_j}\sum _{g=1}^G\left[w_{ij}^{\left(g\right)}{w}_{kj}^{\left(g\right)}\prod _{l=1}^d{w}_{lj}^{\left(g\right){\nu }_l}\right],\end{array}$$ (16)

  $$\begin{array}{rl}E(\log W_{ij}|{\mathit{x}}_j,\mathit{\mu },\mathbf{\Sigma },\mathit{\nu })& \approx \frac{1}{G{c}_j}\sum _{g=1}^G\left[\log \left(w_{ij}^{\left(g\right)}\right)\prod _{l=1}^d{w}_{lj}^{\left(g\right){\nu }_l}\right]\end{array}$$ (17)

  for $i,k=1,\dots ,d$.

By combining (12)–(13) with (16)–(17), we could obtain the MLEs of $(\mathit{\mu },\mathbf{\Sigma },\mathit{\nu })$.

Regarding the convergence of the Monte Carlo ECM algorithm, the specification of G is very important. Large value of G could move the approximation closer to the true maximizer, but it is time-consuming. Therefore, it is inefficient to start with a large value of G when the current approximation to the maximizer is far from the true value. Instead, it is recommended to monitor the convergence of the algorithm till the process has been almost stabilized, then to terminate the algorithm and to choose the stabilized point as the new initial value to continue with a large value of G. This will further decrease the system variability and obtain a more closed maximizer to the real value. Besides, let ${\mathit{\theta }}^{\left(t\right)}$ be the t-th approximation of $\stackrel{\mathbf{ˆ}}{\mathit{\theta }}$, then the stopping rule of the Monte Carlo ECM algorithm is set as $|{\mathit{\theta }}^{(t+1)}-{\mathit{\theta }}^{\left(t\right)}|\le \delta$, where δ is a predetermined precision.

3.2. Testing hypothesis of independence

Suppose that we want to test the following hypotheses

$$H_0\text{: }\mathbf{\Sigma }\text{ is diagonal}\text{against}{H}_1\text{: }\mathbf{\Sigma }\text{ is not diagonal}.$$

Under $H_0$, the likelihood ratio test (LRT) statistic is given by

$$T=-2\left\{\ell ({\stackrel{\mathbf{ˆ}}{\mathit{\mu }}}_0,{\stackrel{\mathbf{ˆ}}{\mathbf{\Sigma }}}_0,{\stackrel{\mathbf{ˆ}}{\mathit{\nu }}}_0|Y_{\mathrm{o}\mathrm{b}\mathrm{s}})-\ell (\stackrel{\mathbf{ˆ}}{\mathit{\mu }},\stackrel{\mathbf{ˆ}}{\mathbf{\Sigma }},\stackrel{\mathbf{ˆ}}{\mathit{\nu }}|Y_{\mathrm{o}\mathrm{b}\mathrm{s}})\right\}\stackrel{.}{\sim }{\chi }^2\left(\frac{d(d-1)}{2}\right),$$ (18)

where $({\stackrel{\mathbf{ˆ}}{\mathit{\mu }}}_0,{\stackrel{\mathbf{ˆ}}{\mathbf{\Sigma }}}_0,{\stackrel{\mathbf{ˆ}}{\mathit{\nu }}}_0)$ are the constrained MLEs of $(\mathit{\mu },\mathbf{\Sigma },\mathit{\nu })$ under $H_0$ while $(\stackrel{\mathbf{ˆ}}{\mathit{\mu }},\stackrel{\mathbf{ˆ}}{\mathbf{\Sigma }},\stackrel{\mathbf{ˆ}}{\mathit{\nu }})$ are the unconstrained MLEs of $(\mathit{\mu },\mathbf{\Sigma },\mathit{\nu })$. Under $H_0$, Type II  MVT distribution reduces to the product of d independent univariate t distributions, thus the constrained MLEs $({\stackrel{\mathbf{ˆ}}{\mathit{\mu }}}_0,{\stackrel{\mathbf{ˆ}}{\mathbf{\Sigma }}}_0,{\stackrel{\mathbf{ˆ}}{\mathit{\nu }}}_0)$ are easily obtained. The test statistic T asymptotically follows a chi-squared distribution with $d(d-1)/2$ degrees of freedom under $H_0$, and the corresponding $p\text{-value}$ is given by

$$p\text{-value}=\left\{\begin{array}{ll}Pr(T>t|H_0),& \text{if }d\le 2,\\ 2min\{Pr(T>t|H_0),Pr(T\le t|H_0\left)\right\},& \text{if }d\ge 3.\end{array}\right.$$ (19)

For a given significance level α, the null hypothesis should be rejected if $p\text{-value}\le \alpha$.

3.3. Type II  multivariate t regression model

In existing multivariate linear regression models, the error terms are often assumed to follow a multivariate normal or t distribution, where the latter can improve the robustness of the normal model for data with outliers. Since the proposed Type II  MVT distribution is much more flexible than Type I MVT distribution, we adopt it as the joint distribution of the error terms. Then Type II  MVT regression model is formulated as

$$\left\{\begin{array}{l}{\mathbf{y}}_j={\mathit{\mu }}_j+{\mathit{ϵ}}_j,j=1,\dots ,n,\\ {\mu }_{ij}={\mathit{x}}_j^{\mathrm{\top }}{\mathrm{\text{β}}}_i,i=1,\dots ,m,\end{array}\right.$$ (20)

where ${\mathbf{y}}_j=(Y_{1j},\dots ,Y_{mj}{)}^{\mathrm{\top }}$ is an $m\times 1$ vector of response, ${\mathit{\mu }}_j=({\mu }_{1j},\dots ,{\mu }_{mj}{)}^{\mathrm{\top }}$, $\left\{{\mathit{ϵ}}_j{\}}_{j=1}^n\stackrel{\mathrm{i}\mathrm{i}\mathrm{d}}{\sim }{t}_m^{\left(\mathrm{I}\mathrm{I}\right)}\right(\mathbf{0}\mathbf{0},\mathbf{\Sigma },\mathit{\nu })$, $\mathbf{\Sigma }$ is a positive-definite matrix, $\mathit{\nu }=({\nu }_1,\dots ,{\nu }_m{)}^{\mathrm{\top }}$ is the vector of degrees of freedom, ${\mathit{x}}_j=(1,x_{1j},\dots ,x_{pj}{)}^{\mathrm{\top }}$ is the known covariate vector for the subject j and ${\mathrm{\text{β}}}_i=({\mathrm{\text{β}}}_{i0},{\mathrm{\text{β}}}_{i1},\dots ,{\mathrm{\text{β}}}_{ip}{)}^{\mathrm{\top }}$ is the vector of regression coefficients. Note that ${\mathit{\mu }}_j$ can be expressed in matrix form as

$${\mathit{\mu }}_j={\left(\begin{array}{cccc}{\mathit{x}}_j^{\mathrm{\top }}& \mathbf{0}& \cdots & \mathbf{0}\\ \mathbf{0}& {\mathit{x}}_j^{\mathrm{\top }}& \cdots & \mathbf{0}\\ ⋮& ⋮& \ddots & ⋮\\ \mathbf{0}& \mathbf{0}& \cdots & {\mathit{x}}_j^{\mathrm{\top }}\end{array}\right)}_{m\times m(p+1)}\left(\begin{array}{c}{\mathrm{\text{β}}}_1\\ {\mathrm{\text{β}}}_2\\ ⋮\\ {\mathrm{\text{β}}}_m\end{array}\right)\widehat{=}{\mathit{X}}_j\mathrm{\text{β}},$$

where $\mathrm{\text{β}}=({\mathrm{\text{β}}}_1^{\mathrm{\top }},\dots ,{\mathrm{\text{β}}}_m^{\mathrm{\top }}{)}^{\mathrm{\top }}$. Now we rewrite (20) as

$${\mathbf{y}}_j={\mathit{X}}_j\mathrm{\text{β}}+{\mathit{ϵ}}_j,j=1,\dots ,n,$$ (21)

and the objective is to estimate $\mathrm{\text{β}}$, $\mathbf{\Sigma }$ and $\mathit{\nu }$.

Let ${\mathit{y}}_j=(y_{1j},\dots ,y_{mj}{)}^{\mathrm{\top }}$ be the realization of ${\mathbf{y}}_j$, and the observed data be denoted by $Y_{\mathrm{o}\mathrm{b}\mathrm{s}}=\{{\mathit{y}}_j,{\mathit{x}}_j{\}}_{j=1}^n$. Similar with that in Section 3.1, we introduce latent variables $W_{ij}^2\stackrel{\mathrm{i}\mathrm{n}\mathrm{d}}{\sim }\text{Gamma}({\nu }_i/2,{\nu }_i/2)$ for $i=1,\dots ,m$ and $j=1,\dots ,n$ such that

$${\mathbf{y}}_j|({\mathbf{w}}_j={\mathit{w}}_j)\sim N_m({\mathit{X}}_j\mathrm{\text{β}},{\mathit{W}}_j^{-1}\mathbf{\Sigma }{\mathit{W}}_j^{-1}),j=1,\dots ,n,$$

where the notations ${\mathbf{w}}_j$, ${\mathit{w}}_j$ and ${\mathit{W}}_j$ are the same as those denoted in Section 3.1.

The Monte Carlo ECM algorithm is also employed to give the MLEs of parameters. The first two CM-steps are to calculate the conditional MLEs

$$\left\{\begin{array}{l}\stackrel{\mathbf{ˆ}}{\mathrm{\text{β}}}={\left({\sum }_{j=1}^n{\mathit{X}}_j^{\mathrm{\top }}{\mathit{W}}_j{\mathbf{\Sigma }}^{-1}{\mathit{W}}_j{\mathit{X}}_j\right)}^{-1}\left({\sum }_{j=1}^n{\mathit{X}}_j^{\mathrm{\top }}{\mathit{W}}_j{\mathbf{\Sigma }}^{-1}{\mathit{W}}_j{\mathit{y}}_j\right),\\ \stackrel{\mathbf{ˆ}}{\mathbf{\Sigma }}=\frac{1}{n}{\sum }_{j=1}^n{\mathit{W}}_j({\mathit{y}}_j-{\mathit{X}}_j\mathrm{\text{β}})({\mathit{y}}_j-{\mathit{X}}_j\mathrm{\text{β}}{)}^{\mathrm{\top }}{\mathit{W}}_j,\end{array}\right.$$ (22)

and the third CM-step is to calculate ${\hat{\nu }}_i$, which is the same solution to the equation (14) for $i=1,\dots ,m$. The E-step is to replace $w_{ij}{w}_{kj}$ and $\log \left(w_{ij}\right)$ for $i,k=1,\dots ,m$ by their conditional expectations, which can be calculated similarly as (15)–(17) based on the following conditional distribution

$$f_{{\mathbf{w}}_j|{\mathbf{y}}_j}\left({\mathit{w}}_j|{\mathit{y}}_j\right)\propto w_{1j}^{{\nu }_1}\cdots w_{mj}^{{\nu }_m}\exp \left[-\frac{1}{2}{\mathit{w}}_j^{\mathrm{\top }}\left({\mathit{Y}}_j^{\ast }{\mathbf{\Sigma }}^{-1}{\mathit{Y}}_j^{\ast }+{\mathbf{\Sigma }}_0\right){\mathit{w}}_j\right],{\mathit{w}}_j\ge \mathbf{0}\mathbf{0},$$ (23)

where ${\mathit{Y}}_j^{\ast }=\text{diag}({\mathit{y}}_j-{\mathit{X}}_j\mathrm{\text{β}})$ and ${\mathbf{\Sigma }}_0=\text{diag}\left(\mathit{\nu }\right)$.

It is quite difficult to obtain the standard deviations of MLEs and Wald confidence intervals of parameters due to the complexity of the observed-data likelihood function and the observed information matrix. Fortunately, the posterior samples of these parameters are relatively easy to generate by using the Gibbs sampling as shown in the next section. Based on the posterior samples, we can provide the posterior means, the posterior standard deviations and Bayesian credible intervals.

4. Bayesian methods

In this section, we discuss the Bayesian methods for Type II  MVT distribution. The prior distribution of the parameters $(\mathit{\mu },\mathbf{\Sigma },\nu )$ for the classical MVT distribution has been discussed, see [17], in which $\mathit{\mu }$, $\mathbf{\Sigma }$ and ν are assumed independent. And the locally uniform prior is assigned to $\mathit{\mu }$, the inverse Wishart prior is assigned to $\mathbf{\Sigma }$ as

$$p\left(\mathit{\mu }\right)\propto \text{constant}\text{and}p\left(\mathbf{\Sigma }\right)\propto |\mathbf{\Sigma }{|}^{-\frac{m+1}{2}}\exp \left[-\frac{1}{2}\mathrm{t}\mathrm{r}\left(\mathit{A}{\mathbf{\Sigma }}^{-1}\right)\right],$$ (24)

where m is a scalar and $\mathit{A}$ is a $d\times d$ non-negative definite matrix. If m = d and $\mathit{A}=\mathit{O}$ (i.e. zero matrix), $p\left(\mathbf{\Sigma }\right)$ is the non-informative prior; if m = −1 and $\mathit{A}=\mathit{O}$, $p\left(\mathbf{\Sigma }\right)$ is the flat prior. The prior distribution for the hyperparameter, the degrees of freedom ν, is advised by some literature (see [17,25]). One assumption of the prior of ν is the flat prior distribution for ${\nu }^{-1}$, i.e.

$$p\left(\nu \right)\propto {\nu }^{-2},\nu \ge 1.$$ (25)

For the proposed Type II  MVT distribution $t_d^{\left(\mathrm{I}\mathrm{I}\right)}(\mathit{\mu },\mathbf{\Sigma },\mathit{\nu })$, similarly we can apply (24) as the priors for $(\mathit{\mu },\mathbf{\Sigma })$, and extend the prior for the univariate hyperparameter in (25) to the multivariate case as

$$p\left(\mathit{\nu }\right)=\prod _{i=1}^d{\nu }_i^{-2},{\nu }_i\ge 1\text{ for }i=1,\dots ,d.$$ (26)

The posterior distribution of the complete-data is

$$\begin{array}{rl}& p(\mathit{\mu },\mathbf{\Sigma },\mathit{\nu }|Y_{\mathrm{c}\mathrm{o}\mathrm{m}})\propto L(\mathit{\mu },\mathbf{\Sigma },\mathit{\nu }|Y_{\mathrm{c}\mathrm{o}\mathrm{m}})\cdot p(\mathit{\mu },\mathbf{\Sigma })\cdot p\left(\mathit{\nu }\right)\\ & \propto |\mathbf{\Sigma }{|}^{-\frac{n+m+1}{2}}\exp \left[-\frac{1}{2}\mathrm{t}\mathrm{r}\left({\mathit{C}}_1{\mathbf{\Sigma }}^{-1}+\mathit{A}{\mathbf{\Sigma }}^{-1}\right)\right]\\ & \times \prod _{i=1}^d\left[\prod _{j=1}^n\frac{{\left(\frac{{\nu }_i}{2}\right)}^{{\nu }_i/2}}{\mathrm{\Gamma }\left(\frac{{\nu }_i}{2}\right)}{u}_{ij}^{\frac{{\nu }_i-1}{2}}\exp \left(-\frac{{\nu }_i}{2}{u}_{ij}\right)\right]{\nu }_i^{-2},\end{array}$$

where

$${\mathit{C}}_1=\sum _{j=1}^n{\mathit{U}}_j^{\frac{1}{2}}({\mathit{x}}_j-\mathit{\mu })({\mathit{x}}_j-\mathit{\mu }{)}^{\mathrm{\top }}{\mathit{U}}_j^{\frac{1}{2}}.$$

Firstly, the conditional predictive distribution of missing data is derived as

$$f(Y_{\mathrm{m}\mathrm{i}\mathrm{s}}|Y_{\mathrm{o}\mathrm{b}\mathrm{s}},\mathit{\mu },\mathbf{\Sigma },\mathit{\nu })=\prod _{j=1}^n{f}_{{\mathbf{u}}_j|{\mathbf{x}}_j}({\mathit{u}}_j|{\mathit{x}}_j,\mathit{\mu },\mathbf{\Sigma },\mathit{\nu }),$$

where

$$\begin{array}{rl}& f_{{\mathbf{u}}_j|{\mathbf{x}}_j}({\mathit{u}}_j|{\mathit{x}}_j,\mathit{\mu },\mathbf{\Sigma },\mathit{\nu })\\ & \propto (\prod _{i=1}^d{u}_{ij}^{\frac{{\nu }_i-1}{2}})\exp [-\frac{1}{2}({\mathit{x}}_j-\mathit{\mu }{)}^{\mathrm{\top }}({\mathit{U}}_j^{\frac{1}{2}}{\mathbf{\Sigma }}^{-1}{\mathit{U}}_j^{\frac{1}{2}}\left)\right({\mathit{x}}_j-\mathit{\mu })-\frac{1}{2}\sum _{i=1}^d{\nu }_i{u}_{ij}].\end{array}$$ (27)

To generate samples of $(\mathit{\mu },\mathbf{\Sigma },\mathit{\nu })$, we combine the Gibbs sampling method with the acceptance–rejection (AR) algorithm (see [27]). We propose that the envelope density and the envelope constant are given by

$$\begin{array}{rl}{g}_j\left({\mathit{u}}_j\right)& =\prod _{i\in \mathbb{I}}\text{Gamma}\left(u_{ij}|\frac{{\nu }_i+1}{2},\frac{{\nu }_i}{2}\right)\cdot \prod _{i\notin \mathbb{I}}U\left(u_{ij}|0,{\left(\frac{\text{e}{\nu }_i}{{\nu }_i-1}\right)}^{\frac{{\nu }_i-1}{2}}\right)\text{and}\\ c& =\prod _{i\in \mathbb{I}}\frac{\mathrm{\Gamma }\left(\frac{{\nu }_i+1}{2}\right)}{({\nu }_i/2{)}^{\frac{{\nu }_i+1}{2}}},\end{array}$$ (28)

respectively, where

$$\mathbb{I}=\{i|0<{\nu }_i\le 1.837,i=1,\dots ,d\}$$

and $U(\cdot |a,b)$ denotes the density of the uniform distribution $U(a,b)$. Note that $\mathbf{\Sigma }$ and ${\mathbf{\Sigma }}^{-1}$ are positive definite, for any non-zero vector $\mathit{z}$, we always have ${\mathit{z}}^{\mathrm{\top }}{\mathbf{\Sigma }}^{-1}\mathit{z}\ge 0$. Thus, from (27), it follows that

$$\begin{array}{rl}{f}_{{\mathbf{u}}_j|{\mathbf{x}}_j}({\mathit{u}}_j|{\mathit{x}}_j,\mathit{\mu },\mathbf{\Sigma })& \le \prod _{i=1}^d\left[u_{ij}^{\frac{{\nu }_i-1}{2}}\exp \left(-\frac{{\nu }_i{u}_{ij}}{2}\right)\right]\\ & \le \left[\prod _{i\in \mathbb{I}}{u}_{ij}^{\frac{{\nu }_i-1}{2}}\exp \left(-\frac{{\nu }_i{u}_{ij}}{2}\right)\right]\cdot \prod _{i\notin \mathbb{I}}{\left(\frac{\text{e}{\nu }_i}{{\nu }_i-1}\right)}^{-\frac{{\nu }_i-1}{2}}=c{g}_j\left({\mathit{u}}_j\right),\end{array}$$

since the function $h\left(u\right)=u^{\frac{\nu -1}{2}}\exp (-\nu u/2)$ arrives its maximum $\left[\text{e}\nu /\right(\nu -1){]}^{-\frac{\nu -1}{2}}$ at $u=1-1/\nu$ when $\nu >1.837$. For $0<\nu \le 1.837$, we have $\mathrm{\Gamma }\left(\frac{\nu +1}{2}\right)/(\nu /2{)}^{\frac{\nu +1}{2}}\ge 1$. Overall, the envelope constant c specified by (28) is always greater than or equal to 1, indicating that the function in (27) is minorized by $c{g}_j\left({\mathit{u}}_j\right)$.

Thus, the procedures to generate posterior samples by AR algorithm are given as:

• Step 1:
  For each $j\in \{1,\dots ,n\}$, draw ${\mathbf{u}}_j={\mathit{u}}_j$ from $f_{{\mathit{u}}_j|{\mathit{x}}_j}({\mathit{u}}_j|{\mathit{x}}_j,\mathit{\mu },\mathbf{\Sigma },\mathit{\nu })$ through the AR algorithm with the following two steps:

  1. Draw $V_j=v_j\sim U(0,1)$, and independently draw $Y_{ij}=y_{ij}\stackrel{\mathrm{i}\mathrm{n}\mathrm{d}}{\sim }\text{Gamma}(\frac{{\nu }_i+1}{2},\frac{{\nu }_i}{2})$ for $i\in \mathbb{I}$ and $Y_{ij}=y_{ij}\stackrel{\mathrm{i}\mathrm{n}\mathrm{d}}{\sim }U(0,[\text{e}{\nu }_i/({\nu }_i-1){]}^{\frac{{\nu }_i-1}{2}})$ for $i\notin \mathbb{I}$. Set ${\mathit{y}}_j=(y_{1j},\dots ,y_{dj}{)}^{\mathrm{\top }}$.
  2. If $v_j\le f_{{\mathbf{u}}_j|{\mathbf{x}}_j}({\mathit{y}}_j|{\mathit{x}}_j,\mathit{\mu },\mathbf{\Sigma },\mathit{\nu })/c{g}_j\left({\mathit{y}}_j\right)$ (c and $g_j(\cdot )$ are given by (28)), set ${\mathbf{u}}_j={\mathit{u}}_j={\mathit{y}}_j$; otherwise, go back to (a),
• Step 2:
  Draw $(\mathit{\mu },\mathbf{\Sigma },\mathit{\nu })$ from $p(\mathit{\mu },\mathbf{\Sigma },\mathit{\nu }|Y_{\mathrm{c}\mathrm{o}\mathrm{m}})$ via the Gibbs sampling method with the following loop:

  $$\begin{array}{rl}\mathit{\mu }|(\mathbf{\Sigma },\mathit{\nu },Y_{\mathrm{o}\mathrm{b}\mathrm{s}},Y_{\mathrm{m}\mathrm{i}\mathrm{s}})& \sim N_d({\mathit{A}}_1^{-1}{\mathit{B}}_1,{\mathit{A}}_1^{-1}),\\ \mathbf{\Sigma }|(\mathit{\mu },\mathit{\nu },Y_{\mathrm{o}\mathrm{b}\mathrm{s}},Y_{\mathrm{m}\mathrm{i}\mathrm{s}})& \sim {\text{IWishart}}_d({\mathit{C}}_1+\mathit{A},n+m-d),\end{array}$$

  where

  $${\mathit{A}}_1=\sum _{j=1}^n{\mathit{U}}_j^{\frac{1}{2}}{\mathbf{\Sigma }}^{-1}{\mathit{U}}_j^{\frac{1}{2}}\text{and}{\mathit{B}}_1=\sum _{j=1}^n{\mathit{U}}_j^{\frac{1}{2}}{\mathbf{\Sigma }}^{-1}{\mathit{U}}_j^{\frac{1}{2}}{\mathit{x}}_j,$$

  and generate $\mathit{\nu }$ from the posterior distribution of $\mathit{\nu }$ given $(\mathit{\mu },\mathbf{\Sigma },Y_{\mathrm{o}\mathrm{b}\mathrm{s}},Y_{\mathrm{m}\mathrm{i}\mathrm{s}})$ via the grid method, with the density for each ${\nu }_i$ given by

  $$f({\nu }_i|\mathit{\mu },\mathbf{\Sigma },Y_{\mathrm{o}\mathrm{b}\mathrm{s}},Y_{\mathrm{m}\mathrm{i}\mathrm{s}})\propto \left[\prod _{j=1}^n\frac{{\left({\nu }_i/2\right)}^{{\nu }_i/2}}{\mathrm{\Gamma }\left({\nu }_i/2\right)}{u}_{ij}^{\frac{{\nu }_i-1}{2}}\exp \left(-\frac{{\nu }_i}{2}{u}_{ij}\right)\right]{\nu }_i^{-2}.$$

The above methods could be extended to the regression model specified in (20) or (21) by assigning priors for $(\mathrm{\text{β}},\mathbf{\Sigma },\mathit{\nu })$.

5. Simulation studies

The components in Type I MVT distribution must be dependent and share the same degrees of freedom. While the proposed Type II  MVT distribution can accommodate different marginal amounts of tail weights on univariate t distributions and even approximate normal margins. Thus, we conduct some simulations to compare the fitting performances of the two distributions when data are generated from one of them and highlight the characteristic of Type II  MVT distribution.

We consider bivariate and trivariate cases, i.e. d = 2, 3. The sample size is set as n = 100, the mean vector $\mathit{\mu }$ is always chosen to be zero, the scale matrix $\mathbf{\Sigma }=({\sigma }_{ij}{)}_{d\times d}$ is set as the diagonal elements ${\sigma }_{ii}=1$ for all i and the off-diagonal elements ${\sigma }_{ij}$ for $i\ne j$ will be specified in each case. Under different configurations of parameters $(\mathit{\mu },\mathbf{\Sigma },\nu ,\mathit{\nu })$, samples will be generated from either $t_d(\mathit{\mu },\mathbf{\Sigma },\nu )$ or $t_d^{\left(\mathrm{I}\mathrm{I}\right)}(\mathit{\mu },\mathbf{\Sigma },\mathit{\nu })$ as indicated by the first line of each experiment in the following two tables, where the results are obtained by fitting samples by both of the two distributions. In the case of d = 2, we have Experiments 1–4, the comparisons of estimation results including the average MLE, the corresponding mean squared error (MSE) of the estimates based on L = 1000 replications are presented in Table 3. The Akaike information criterion (AIC) and Bayesian information criterion (BIC) are used to reflect their fitting performances by averaging the AIC values and BIC values for each fitted model.

Table 3.

Comparisons of estimate performances with d = 2.

	Experiment 1: $t_2(\mathit{\mu },\mathbf{\Sigma },\nu )$, ${\sigma }_{12}=0.5$, $\nu =3$
Type I	${\mu }_1$	${\mu }_2$	${\sigma }_{11}$	${\sigma }_{12}$	${\sigma }_{22}$	ν		AIC	BIC
MLE	0.0009	0.0024	0.9794	0.4919	0.9955	2.9793		682.7873	698.4183
MSE	0.0142	0.0127	0.0401	0.0194	0.0394	0.3975		–	–
Type II	${\mu }_1$	${\mu }_2$	${\sigma }_{11}$	${\sigma }_{12}$	${\sigma }_{22}$	${\nu }_1$	${\nu }_2$	AIC	BIC
MLE	0.0011	0.0026	1.0712	0.6064	1.0978	3.4333	3.4489	694.9977	713.2339
MSE	0.0158	0.0145	0.0681	0.0480	0.0785	–	–	–	–
	Experiment 2: $t_2^{\left(\mathrm{I}\mathrm{I}\right)}(\mathit{\mu },\mathbf{\Sigma },\mathit{\nu })$, ${\sigma }_{12}=0.5$, $({\nu }_1,{\nu }_2)=(3,3)$
Type I	${\mu }_1$	${\mu }_2$	${\sigma }_{11}$	${\sigma }_{12}$	${\sigma }_{22}$	ν		AIC	BIC
MLE	−0.0024	−0.0035	1.1145	0.4475	1.1175	3.2953		705.6417	721.2727
MSE	0.0140	0.0148	0.0734	0.0168	0.0802	–		–	–
Type II	${\mu }_1$	${\mu }_2$	${\sigma }_{11}$	${\sigma }_{12}$	${\sigma }_{22}$	${\nu }_1$	${\nu }_2$	AIC	BIC
MLE	−0.0006	−0.0014	0.9967	0.4965	1.0077	3.1612	3.2248	697.3014	715.5376
MSE	0.0127	0.0134	0.0553	0.0165	0.0567	0.8421	0.8900	–	–
	Experiment 3: $t_2^{\left(\mathrm{I}\mathrm{I}\right)}(\mathit{\mu },\mathbf{\Sigma },\mathit{\nu })$, ${\sigma }_{12}=0.5$, $({\nu }_1,{\nu }_2)=(3,10)$
Type I	${\mu }_1$	${\mu }_2$	${\sigma }_{11}$	${\sigma }_{12}$	${\sigma }_{22}$	ν		AIC	BIC
MLE	−0.0025	−0.0050	1.2948	0.4552	0.8495	4.8398		653.4693	669.1004
MSE	0.0166	0.0128	0.1672	0.0164	0.0475	–		–	–
Type II	${\mu }_1$	${\mu }_2$	${\sigma }_{11}$	${\sigma }_{12}$	${\sigma }_{22}$	${\nu }_1$	${\nu }_2$	AIC	BIC
MLE	−0.0024	−0.0047	0.9957	0.5007	0.9884	3.0641	10.0156	646.4751	664.7113
MSE	0.0152	0.0121	0.0551	0.0159	0.0266	0.7348	5.0330	–	–
	Experiment 4: $t_2^{\left(\mathrm{I}\mathrm{I}\right)}(\mathit{\mu },\mathbf{\Sigma },\mathit{\nu })$, ${\sigma }_{12}=0$, $({\nu }_1,{\nu }_2)=(5,5)$
Type I	${\mu }_1$	${\mu }_2$	${\sigma }_{11}$	${\sigma }_{12}$	${\sigma }_{22}$	ν		AIC	BIC
MLE	−0.0030	0.0005	1.0831	0.0013	1.0807	5.5423		666.9350	682.5660
MSE	0.0127	0.0133	0.0470	0.0097	0.0533	–		–	–
Type II	${\mu }_1$	${\mu }_2$	${\sigma }_{11}$	${\sigma }_{12}$	${\sigma }_{22}$	${\nu }_1$	${\nu }_2$	AIC	BIC
MLE	−0.0029	0.0005	0.9560	0.0006	0.9533	4.6581	4.6488	663.7192	681.9554
MSE	0.0123	0.0126	0.0372	0.0096	0.0417	1.4100	1.4080	–	–

Note: MLE is the average of 1000 point estimates with precision $\delta ={10}^{-2}$ via the Monte Carlo ECM algorithm; Mean squared error (MSE) of the estimate is equal to the sum of the variance and the squared bias of the estimator.

In the case of d = 3, we conduct Experiments 5–6, estimation accuracies on parameters are compared by MLE and MSE as summarized in Table 4. In a more direct-viewing way, box plots are adopted to present the estimation results for each experiment in Figures 5–7.

Figure 6.

(a1–a3) Experiment 3: Box plots for MLEs of the parameters by bivariate Type I and Type II  MVT distributions; (b1–b3) Experiment 4: Box plots for MLEs of the parameters by bivariate Type I and Type II  MVT distributions.

Table 4.

Comparisons of estimate performances with d = 3.

	Experiment 5: $t_3(\mathit{\mu },\mathbf{\Sigma },\nu )$, ${\sigma }_{ij}=(-1{)}^{|i-j|}\times 0.5$, $\nu =3$
Type I	${\mu }_1$	${\mu }_2$	${\mu }_3$	${\sigma }_{11}$	${\sigma }_{12}$	${\sigma }_{13}$
MLE	0.0233	0.0020	0.0250	0.9997	−0.4970	0.5032
MSE	0.0102	0.0118	0.0101	0.0230	0.0118	0.0122
Type I	${\sigma }_{22}$	${\sigma }_{23}$	${\sigma }_{33}$	ν
MLE	0.9996	−0.4974	1.0042	3.0911
MSE	0.0261	0.0118	0.0280	0.3512
Type II	${\mu }_1$	${\mu }_2$	${\mu }_3$	${\sigma }_{11}$	${\sigma }_{12}$	${\sigma }_{13}$
MLE	0.0213	0.0016	0.0218	0.9147	−0.5093	0.5146
MSE	0.0070	0.0080	0.0077	0.0398	0.0191	0.0191
Type II	${\sigma }_{22}$	${\sigma }_{23}$	${\sigma }_{33}$	${\nu }_1$	${\nu }_2$	${\nu }_3$
MLE	0.9116	−0.5081	0.9157	2.9983	2.9775	2.9868
MSE	0.0434	0.0182	0.0446	–	–	–
	Experiment 6: $t_3^{\left(\mathrm{I}\mathrm{I}\right)}(\mathit{\mu },\mathbf{\Sigma },\mathit{\nu })$, ${\sigma }_{ij}=(-1{)}^{|i-j|}\times 0.5$, $({\nu }_1,{\nu }_2,{\nu }_3)=(3,3,3)$
Type I	${\mu }_1$	${\mu }_2$	${\mu }_3$	${\sigma }_{11}$	${\sigma }_{12}$	${\sigma }_{13}$
MLE	0.0356	0.0075	0.0330	1.2886	−0.5332	0.5327
MSE	0.0155	0.0177	0.0157	0.1558	0.0181	0.0175
Type I	${\sigma }_{22}$	${\sigma }_{23}$	${\sigma }_{33}$	ν
MLE	1.2996	−0.5349	1.2854	4.0222
MSE	0.1648	0.0185	0.1547	–
Type II	${\mu }_1$	${\mu }_2$	${\mu }_3$	${\sigma }_{11}$	${\sigma }_{12}$	${\sigma }_{13}$
MLE	0.0230	0.0045	0.0206	0.9020	−0.4621	0.4629
MSE	0.0074	0.0089	0.0082	0.0405	0.0132	0.0125
Type II	${\sigma }_{22}$	${\sigma }_{23}$	${\sigma }_{33}$	${\nu }_1$	${\nu }_2$	${\nu }_3$
MLE	0.9039	−0.4646	0.8966	2.9699	2.9454	2.9617
MSE	0.0433	0.0139	0.0445	0.3139	0.3137	0.3377

Note: MLE is the average of 1000 point estimates with precision $\delta ={10}^{-2}$ via the Monte Carlo ECM algorithm; Mean squared error (MSE) of the estimate is equal to the sum of the variance and the squared bias of the estimator.

Figure 5.

(a1–a3) Experiment 1: Box plots for MLEs of the parameters by bivariate Type I and Type II  MVT distributions; (b1–b3) Experiment 2: Box plots for MLEs of the parameters by bivariate Type I and Type II  MVT distributions.

Figure 7.

(a1–a3) Experiment 5: Box plots for MLEs of the parameters by trivariate Type I and Type II  MVT distributions; (b1–b3) Experiment 6: Box plots for MLEs of the parameters by trivariate Type I and Type II  MVT distributions.

From Tables 3 and 4, it is observed that the performance of the Monte Carlo ECM algorithm in parameter estimation of Type II  MVT distribution is quite satisfactory in the sense that all the average MLEs are close to their true values. When data are indeed generated from Type I MVT distribution as in Experiments 1 and 5, the point estimates in Type I MVT are closer to their true values and are more stable with smaller MSEs for most of the parameters, and Type I MVT overwhelms Type II  MVT in model fitting in terms of AIC and BIC values. While the estimation by Type II  MVT still gives an acceptable result where the estimates of $(\mathit{\mu },\mathbf{\Sigma })$ are not too far away from the true values and the estimates on ${\nu }_i$'s are close to each other. Moreover, when samples are indeed generated from Type II  MVT distribution with identical ${\nu }_i$'s as in Experiments 2, 4, 6, the estimation performance by Type II  MVT distribution is better, as the estimates are more accurate and have smaller MSEs; while in Type I MVT, from Figures 5–7, we can see that the estimations on parameters especially $\mathbf{\Sigma }$ and ν are more likely to produce outliers. If the true values of ${\nu }_1$ and ${\nu }_2$ are distinct as in Experiment 3, the Type II  MVT distribution apparently outperforms Type I MVT distribution which has only one degrees of freedom parameter, and Type II  MVT distribution has a better depiction on different amounts of tail weights for marginal components.

6. Applications

To ease the numerical work and graphical presentation, we focus on the bivariate case to provide the numerical illustration of the presented methodologies.

6.1. Sport data

We make use of a data set, collected by the Australian Institute of Sport and reported, see [3], containing several variables measured on n = 202 Australian athletes. Specifically, we consider the pair of variables BMI (body mass index) and LBM (lean body mass). Table 5 summaries the parameter estimates from Type I bivariate t distribution $t_2(\mathit{\mu },\mathbf{\Sigma },\nu )$ with unknown ν and Type II  bivariate t distribution $t_2^{\left(\mathrm{I}\mathrm{I}\right)}(\mathit{\mu },\mathbf{\Sigma },\mathit{\nu })$ with unknown $\mathit{\nu }$.

Table 5.

MLEs of parameters for fitting (BMI, LBM).

	MLEs of parameters					Criterion
Dist.	$\mathit{\mu }$	$\mathbf{\Sigma }$	ν	$\mathit{\nu }$	log-L	AIC	BIC
Type I	$(\begin{array}{c}22.7479\\ 64.3418\end{array})$	$(\begin{array}{cc}6.2010& 21.3709\\ 21.3709& 147.3899\end{array})$	11.0467	–	−1228.471	2468.942	2488.792
Type II	$(\begin{array}{c}22.6509\\ 64.1985\end{array})$	$(\begin{array}{cc}5.4081& 22.3384\\ 22.3384& 167.2201\end{array})$	–	$(\begin{array}{c}6.2217\\ 74.1270\end{array})$	−1223.653	2461.307	2484.464

To test the independence between BMI and LBM, from (18) and (19), the value of the test statistic T is given as $t=144.9212\gg {\chi }_{0.05}^2\left(1\right)=3.84$ and the corresponding p-value $\approx 0\ll 0.05$, indicating that the null hypothesis should be rejected at 0.05 significance level.

Figure 8 displays the scatter plot and the superimposed contours of the fitted Type II  bivariate t distribution for the (BMI, LBM) pair. When comparing the two models, the proposed Type II  distribution is selected by both AIC and BIC, also with a larger value of log-likelihood. Besides, the estimated correlation coefficient from (11) is $r=0.6981$, which is very close to the sample index $\rho =0.7139$. The Kendall's tau and Spearman's rho for measuring the rank correlation between BMI and LBM are $r_k=0.5176$ and $r_s=0.7026$ respectively. All of these statistics reveal that there is an obvious positive correlation between the two variables, this feature is well captured by the proposed Type II  MVT distribution.

Figure 8.

Scatter plot (BMI, LBM) and fitted Type II  bivariate t distribution.

6.2. Tuberculosis vaccine data

The data consist of 13 trials on the efficacy of Bacillus Calmette–Guéin (BCG) vaccine against tuberculosis for a vaccinated group and a non-vaccinated control group, as presented in Table 6. Let VD, VND denote the number of disease cases and non-disease cases in the vaccinated group, NVD, NVND denote the number of disease cases and non-disease cases in the non-vaccinated group, respectively. Some covariates are available: geographic latitude of the place where the study was done; year of publication. Here, we will carry out a bivariate analysis on the log-odds of tuberculosis in the vaccinated and not-vaccinated control arm. Let $\mathbf{y}=(Y_1,Y_2{)}^{\mathrm{\top }}$ where $Y_1=\log \left(\mathrm{V}\mathrm{D}/\mathrm{V}\mathrm{N}\mathrm{D}\right)$ and $Y_2=\log \left(\mathrm{N}\mathrm{V}\mathrm{D}/\mathrm{N}\mathrm{V}\mathrm{N}\mathrm{D}\right)$, and the covariates are chosen to be $\mathit{x}=(1,x_1,x_2{)}^{\mathrm{\top }}$, where $x_1=\mathrm{L}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{t}\mathrm{u}\mathrm{d}\mathrm{e}-33$, $x_2=\mathrm{Y}\mathrm{e}\mathrm{a}\mathrm{r}-66$. The Type II  bivariate t regression model is applied to analyze this data set.

Table 6.

Data from clinical trials on the efficacy of BCG vaccine in the prevention of tuberculosis (see [26]).

	Vaccinated		Not Vaccinated
Trial	Disease	No disease	Disease	No disease	Latitude	Year
1	4	119	11	128	44	48
2	6	300	29	274	55	49
3	3	228	11	209	42	60
4	62	13536	248	12619	52	77
5	33	5036	47	5761	13	73
6	180	1361	372	1079	44	53
7	8	2537	10	619	19	73
8	505	87886	499	87892	13	80
9	29	7470	45	7232	27	68
10	17	1699	65	1600	42	61
11	186	50448	141	27197	18	74
12	5	2493	3	2338	33	69
13	27	16886	29	17825	33	76

By adopting the bivariate t regression model specified by (21) and setting the initial values of $(\mathrm{\text{β}},\mathbf{\Sigma },\mathit{\nu })$ as

$${\mathrm{\text{β}}}^{\left(0\right)}=0.1\times \mathbf{1}{\mathbf{1}}_6,{\mathbf{\Sigma }}^{\left(0\right)}=\text{Var}\left(\mathbf{y}\right)=\left(\begin{array}{cc}1.4994& 1.8882\\ 1.8882& 2.7850\end{array}\right),{\mathit{\nu }}^{\left(0\right)}=\mathbf{1}{\mathbf{1}}_2,$$

we calculate the MLEs of these parameters by using the Monte Carlo ECM algorithm (22), (13) and (23), which converged to $(\stackrel{\mathbf{ˆ}}{\mathrm{\text{β}}},\stackrel{\mathbf{ˆ}}{\mathbf{\Sigma }},\stackrel{\mathbf{ˆ}}{\mathit{\nu }})$ in 51 iterations with precision $\delta ={10}^{-2}$. These results are summarized in Table 7.

Table 7.

MLEs and posterior estimates of parameters for the tuberculosis vaccine data.

			Posterior	Posterior	95% Bayesian
Parameter	Coefficients	MLE	Mean	Std	Credible Interval
	Constant	−5.006	−5.017	0.190	[−5.408, −4.606]
${\mathrm{\text{β}}}_1$	$x_1$	0.009	0.005	0.021	[−0.031,0.046]
	$x_2$	−0.080	−0.085	0.028	[−0.128, −0.026]
	Constant	−4.137	−4.139	0.243	[−4.652, −3.685]
${\mathrm{\text{β}}}_2$	$x_1$	0.043	0.035	0.032	[−0.023, 0.116]
	$x_2$	−0.078	−0.090	0.041	[−0.168, −0.003]
	${\sigma }_{11}$	0.445	0.494	0.192	[0.235, 1.005]
$\mathbf{\Sigma }$	${\sigma }_{12}$	0.510	0.519	0.159	[0.245, 0.812]
	${\sigma }_{22}$	0.649	0.690	0.220	[0.305, 1.085]
$\mathit{\nu }$	${\nu }_1$	13.834	12.870	0.581	[12.027, 13.923]
	${\nu }_2$	3.062	3.234	0.141	[3.018, 3.479]

To illustrate the proposed Bayesian methods in Section 4, we assign the non-informative prior of (24) to $(\mathrm{\text{β}},\mathbf{\Sigma })$ (i.e. $p(\mathrm{\text{β}},\mathbf{\Sigma })\propto |\mathbf{\Sigma }{|}^{-\frac{d+1}{2}}$) and (26) to $\mathit{\nu }$, we generate 1000 posterior samples of $(\mathrm{\text{β}},\mathbf{\Sigma },\mathit{\nu })$ by using the Gibbs sampling embedded with the AR algorithm. By discarding the first half of these samples, we can calculate the posterior means, the posterior standard deviations and the 95% Bayesian credible intervals as shown in the last three columns of Table 7.

Now suppose that we want to test the independence between $Y_1$ and $Y_2$, i.e. to test whether ${\sigma }_{12}$ is zero or not. Under the null hypothesis, we have $Y_{ij}\sim t({\mathit{x}}_j^{\mathrm{\top }}{\mathrm{\text{β}}}_i,{\sigma }_{ii},{\nu }_i)$ for $j=1,\dots ,n$ and i = 1, 2. That is, we fit each component by the univariate t regression model. The parameters for each univariate distribution are estimated by

$$\begin{array}{rl}{\stackrel{\mathbf{ˆ}}{\mathrm{\text{β}}}}_1& =(-4.9466,-0.0038,-0.0809{)}^{\mathrm{\top }},{\hat{\sigma }}_{11}=0.2683,{\hat{\nu }}_1=3.6620,\\ {\stackrel{\mathbf{ˆ}}{\mathrm{\text{β}}}}_2& =(-3.8122,0.0275,-0.0582{)}^{\mathrm{\top }},{\hat{\sigma }}_{22}=0.0012,{\hat{\nu }}_2=\mathrm{0.3679.}\end{array}$$

From (18)–(19), the LRT statistic is given by t = 9.6778 and the corresponding $p\text{-value}=0.0019<0.05$. Thus, the null hypothesis should be rejected.

Moreover, from (11), the estimated correlation coefficient between $Y_1$ and $Y_2$ is $r=0.7487$, which is acceptable comparing with the sample correlation coefficient given by 0.924, indicating a positive correlation between $Y_1$ and $Y_2$. The statistics of Kendall's tau and Spearman's rho for rank correlation coefficient between $Y_1$ and $Y_2$ are $r_k=0.7692$ and $r_s=0.8956$, concurring with the above result. The difference of estimates on ${\nu }_1$ and ${\nu }_2$ from the marginal analysis indicates the distinct amounts of tail weights for $Y_1$ and $Y_2$, thus the Type II  bivariate t regression model provides a more reasonable fit to this data set than Type I model.

From the regression results, we know that the covariate $x_1$ is insignificant while $x_2$ is significant. To get a final model, we still need some extra work. We use this data set just for demonstrating that the proposed model is applicable to a regression analysis. Furthermore, incorporating covariates into the scale structure could be a potential future work.

7. Discussions

To overcome three obvious disadvantages associated with the classical MVT distribution, in this paper, we introduced a Type II  MVT distribution as a new robust alternative to the multivariate normal distribution. This new distribution has three noteworthy characteristics: (1) All components follow univariate t-distributions with not necessarily identical degrees of freedom; (2) it is applicable to the multivariate data that some components are marginally t distributed while some are approximately normal distributed, depending on small or large value of the corresponding ${\nu }_i$; and (3) it could contain some dependent components and some statistically independent components. Therefore, the proposed distribution is more flexible in model specification. Confirmed by the two real data sets presented in Section 6, the variables are moderately or strongly correlated but with very different tail weights seen from the marginal perspective. In such cases, both the dependency structure and various amounts on tails should be considered. The classical MVT distribution has a very conspicuous drawback that limits the same degrees of freedom for all components. Instead, the proposed Type II  MVT distribution covers the two aspects thus it is superior to existing models and has a better performance in our two real data fitting.

In general, although the joint density of the Type II  MVT random vector does not have a closed-form expression, the SR (6) is very useful in the derivation of the marginal distributions, mixed moments, and Monte Carlo ECM algorithm. From the characteristic (1) in Section 2, we have shown that each component follows a univariate t-distribution. In fact, any sub-vectors of $\mathbf{x}$, $(X_{i_1},X_{i_2},\dots ,X_{i_r}{)}^{\mathrm{\top }}$ say, follows $t_r^{\left(\mathrm{I}\mathrm{I}\right)}({\mathit{\mu }}^{\ast },{\mathbf{\Sigma }}^{\ast },{\mathit{\nu }}^{\ast })$, where ${\mathit{\mu }}^{\ast }=({\mu }_{i_1},{\mu }_{i_2},\dots ,{\mu }_{i_r}{)}^{\mathrm{\top }}$, ${\mathbf{\Sigma }}^{\ast }$ is a sub-matrix consisting of the $i_1,i_2,\dots ,i_r$ rows and $i_1,i_2,\dots ,i_r$ columns of $\mathbf{\Sigma }$, and ${\mathit{\nu }}^{\ast }=({\nu }_{i_1},{\nu }_{i_2},\dots ,{\nu }_{i_r}{)}^{\mathrm{\top }}$. We also applied the Type II  MVT distribution to the linear regression model by adopting it as the joint distribution of the error terms, taking advantage of its flexibility to better capture the characteristic of the data when outliers exist.

Further extension of (6) can be considered as follows:

$$X_i={\mu }_i+\frac{Z_i}{\sqrt{(V+V_i)/(\nu +{\nu }_i)}},i=1,\dots ,d,$$ (29)

where $\mathbf{z}=(Z_1,\dots ,Z_d{)}^{\mathrm{\top }}\sim N_d(\mathbf{0}\mathbf{0},\mathbf{\Sigma })$, $V\sim {\chi }^2\left(\nu \right)$, $V_i\sim {\chi }^2\left({\nu }_i\right)$ for $i=1,\dots ,d$, and $(\mathbf{z},V,V_1,\dots ,V_d)$ are mutually independent. In particular, in (29) when ${\nu }_1=\cdots ={\nu }_d=0$, it reduces to Type I MVT distribution; and when $\nu =0$, it reduces to Type II  MVT distribution. Note that the Type I MVT distribution (2) and the proposed Type II  MVT distribution (6) are non-nested. In (29), the dependency of the random vector $\mathbf{x}=(X_1,\dots ,X_d{)}^{\mathrm{\top }}$ can possibly come from both the multivariate normal vector $\mathbf{z}$ and the common Gamma random variable V. Besides, compared with the two models in (4)–(5), although similar in expressions, the model (29) will bring convenience in parameter estimations without embedding the constraint as stated in Remark 2.1. When analyzing real data, we can start with this general model (29), and turn to the reduced model based on the estimation results. The model (29) includes the Type I and Type II  MVT distributions as special cases, thus it is much more convenient and useful for model selection.

As a comparison with the model proposed in [10], similarly we decompose the scale matrix into $\mathbf{\Sigma }=\mathit{D}\mathit{A}{\mathit{D}}^{\mathrm{\top }}$, where $\mathit{D}$ is the matrix of eigenvectors of $\mathbf{\Sigma }$ and $\mathit{A}$ is a diagonal matrix whose entries are the eigenvalues of $\mathbf{\Sigma }$, then the vector $\mathbf{x}$ in (6) can be reformulated as

$$\mathbf{x}=\mathit{\mu }+{\mathbf{U}}^{-1/2}\mathit{D}{\mathit{A}}^{1/2}{\mathbf{z}}_0,$$ (30)

where ${\mathbf{z}}_0=(Z_{10},\dots ,Z_{d0}{)}^{\mathrm{\top }}$ is a d-dimensional Gaussian random vector with mean zero vector and covariance matrix equal to the identity matrix. Note that the model in (30) is different from the formulation in [10] since in general it is not commutative for matrix product, and only when $\mathit{D}$ is a diagonal matrix the two are equivalent.

Appendix. Derivation of joint pdf $f_{\mathbf{x}}\left(\mathit{x}\right)$.

Based on the mixture expressions in (7), given the conditional distribution of $\mathbf{x}\mid (\mathbf{u}=\mathit{u})$ and the marginal distributions of $U_i$'s, the joint pdf of $\mathbf{x}$ is given by

$$\begin{array}{rl}{f}_{\mathbf{x}}\left(\mathit{x}\right)& ={\int }_{{\mathbb{R}}_{+}^d}{f}_{\mathbf{x}|\mathbf{u}}\left(\mathit{x}|\mathit{u}\right)f_{\mathbf{u}}\left(\mathit{u}\right)\text{d}\mathit{u}\\ & ={\int }_{{\mathbb{R}}_{+}^d}{\left(2\pi \right)}^{-\frac{d}{2}}|{\mathit{U}}^{-1/2}\mathbf{\Sigma }{\mathit{U}}^{-1/2}{|}^{-\frac{1}{2}}\exp [-\frac{1}{2}(\mathit{x}-\mathit{\mu }{)}^{\mathrm{\top }}({\mathit{U}}^{-1/2}\mathbf{\Sigma }{\mathit{U}}^{-1/2}{)}^{-1}(\mathit{x}-\mathit{\mu })]\\ & \times \prod _{i=1}^d\frac{(\frac{{\nu }_i}{2}{)}^{\frac{{\nu }_i}{2}}}{\mathrm{\Gamma }\left(\frac{{\nu }_i}{2}\right)}{u}_i^{\frac{{\nu }_i}{2}-1}{\text{e}}^{-\frac{{\nu }_i{u}_i}{2}}\text{d}\mathit{u}\\ & ={\left(2\pi \right)}^{-\frac{d}{2}}|\mathbf{\Sigma }{|}^{-\frac{1}{2}}[\prod _{i=1}^d\frac{(\frac{{\nu }_i}{2}{)}^{\frac{{\nu }_i}{2}}}{\mathrm{\Gamma }\left(\frac{{\nu }_i}{2}\right)}]\\ & \times {\int }_{{\mathbb{R}}_{+}^d}(\prod _{i=1}^d{u}_i^{\frac{{\nu }_i-1}{2}})\exp \{-\frac{1}{2}[{\left({\mathit{u}}^{1/2}\right)}^{\mathrm{\top }}{\mathit{X}}^{\ast }{\mathbf{\Sigma }}^{-1}{\mathit{X}}^{\ast }{\mathit{u}}^{1/2}+\sum _{i=1}^d{\nu }_i{u}_i]\}\text{d}\mathit{u}.\end{array}$$

By employing the transformation of $w_i=u_i^{1/2}$ for $i=1,\dots ,d$ and then $\text{d}{u}_i=2w_i\text{d}{w}_i$, it follows that

$$\begin{array}{rl}{f}_{\mathbf{x}}\left(\mathit{x}\right)& ={\left(2\pi \right)}^{-\frac{d}{2}}|\mathbf{\Sigma }{|}^{-\frac{1}{2}}[\prod _{i=1}^d\frac{(\frac{{\nu }_i}{2}{)}^{\frac{{\nu }_i}{2}}}{\mathrm{\Gamma }\left(\frac{{\nu }_i}{2}\right)}]\\ & \times {\int }_{{\mathbb{R}}_{+}^d}(\prod _{i=1}^d{w}_i^{{\nu }_i-1})\exp \{-\frac{1}{2}[{\mathit{w}}^{\mathrm{\top }}{\mathit{X}}^{\ast }{\mathbf{\Sigma }}^{-1}{\mathit{X}}^{\ast }\mathit{w}+\sum _{i=1}^d{\nu }_i{w}_i^2]\}(\prod _{i=1}^{d}2w_i)\text{d}\mathit{w}\\ & ={\left(\frac{\pi }{2}\right)}^{-\frac{d}{2}}|\mathbf{\Sigma }{|}^{-\frac{1}{2}}[\prod _{i=1}^d\frac{(\frac{{\nu }_i}{2}{)}^{\frac{{\nu }_i}{2}}}{\mathrm{\Gamma }\left(\frac{{\nu }_i}{2}\right)}]{\int }_{{\mathbb{R}}_{+}^d}(\prod _{i=1}^d{w}_i^{{\nu }_i})\exp \left[-\frac{1}{2}{\mathit{w}}^{\mathrm{\top }}\left({\mathit{X}}^{\ast }{\mathbf{\Sigma }}^{-1}{\mathit{X}}^{\ast }+{\mathbf{\Sigma }}_0\right)\mathit{w}\right]\text{d}\mathit{w}\\ & ={\left(\frac{\pi }{2}\right)}^{-\frac{d}{2}}|\mathbf{\Sigma }{|}^{-\frac{1}{2}}[\prod _{i=1}^d\frac{(\frac{{\nu }_i}{2}{)}^{\frac{{\nu }_i}{2}}}{\mathrm{\Gamma }\left(\frac{{\nu }_i}{2}\right)}]{\int }_{{\mathbb{R}}_{+}^d}(\prod _{i=1}^d{w}_i^{{\nu }_i})\exp \left[-\frac{1}{2}{\mathit{w}}^{\mathrm{\top }}{\mathbf{\Sigma }}^{\ast -1}\mathit{w}\right]\text{d}\mathit{w}\\ & ={\left(\frac{\pi }{2}\right)}^{-\frac{d}{2}}|\mathbf{\Sigma }{|}^{-\frac{1}{2}}[\prod _{i=1}^d\frac{(\frac{{\nu }_i}{2}{)}^{\frac{{\nu }_i}{2}}}{\mathrm{\Gamma }\left(\frac{{\nu }_i}{2}\right)}]\cdot C{\int }_{{\mathbb{R}}_{+}^d}(\prod _{i=1}^d{w}_i^{{\nu }_i})\cdot \frac{1}{C}\exp \left[-\frac{1}{2}{\mathit{w}}^{\mathrm{\top }}{\mathbf{\Sigma }}^{\ast -1}\mathit{w}\right]\text{d}\mathit{w}\\ & ={\left(\frac{\pi }{2}\right)}^{-\frac{d}{2}}|\mathbf{\Sigma }{|}^{-\frac{1}{2}}[\prod _{i=1}^d\frac{(\frac{{\nu }_i}{2}{)}^{\frac{{\nu }_i}{2}}}{\mathrm{\Gamma }\left(\frac{{\nu }_i}{2}\right)}]\cdot {\mu }_{\mathit{\nu }}\left(\mathbf{w}\right)\cdot C,\end{array}$$

where ${\mathit{X}}^{\ast }$ and ${\mathbf{\Sigma }}_0$ are defined in (10).

Funding Statement

Chi Zhang's research was supported by National Natural Science Foundation of China (Grant No. 11801380). Guo-Liang Tian's research was fully supported by National Natural Science Foundation of China (Grant No. 11771199). Kam Chuen Yuen's research was supported by a Seed Fund for Basic Research of the University of Hong Kong, and a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (Project No. HKU17306220). The work of Man-Lai Tang was partially supported through grants from the Research Grant Council of the Hong Kong Special Administrative Region (UGC/FDS14/P06/17, UGC/FDS14/P02/18, and the Research Matching Grant Scheme (RMGS)) and a grant from the National Natural Science Foundation of China (11871124). The computing facilities/software were supported by SAS Viya and the Big Data Intelligence Centre at Hang Seng University of Hong Kong.

Disclosure statement

No potential conflict of interest was reported by the author(s).