A Cholesky-based estimation for large-dimensional covariance matrices

ABSTRACT

This paper develops a new method to estimate a large-dimensional covariance matrix when the variables have no natural ordering among themselves. The modified Cholesky decomposition technique is used to provide a set of estimates of the covariance matrix under multiple orderings of variables. The proposed estimator is in the form of a linear combination of these available estimates and the identity matrix. It is positive definite and applicable in large dimensions. The merits of the proposed estimator are demonstrated through the numerical study and a real data example by comparison with several existing methods.

1. Introduction

Estimation of a covariance matrix is among the most fundamental problems in statistics, since the covariance matrix plays a key role in a broad area of data analysis and statistical applications, as in principle component analysis, portfolio optimization, linear discriminant analysis (LDA) and so forth. However, the estimation is difficult due to two aspects. One is the positive definite property of a covariance matrix. The other is the fact that the number of unknown parameters is quadratically increasing as the dimensionality increases. Although the sample covariance matrix is an unbiased estimator and performs well in the traditional situation where the sample size n is far more than the number of variables p, it performs poorly in large-dimensional settings when n is close to p or n<p. In particular, the eigenvalues of the sample covariance matrix are over dispersed and the eigenvectors are inconsistent in large dimensions [15]. Additionally, we know that the classification by the LDA breaks down and reduces to random guessing as $p/n\to \mathrm{\infty }$ [1]. Therefore, it calls for the alternative estimation for a large covariance matrix.

The framework of covariance matrix estimation with large p can be loosely classified into two situations. One situation is that there exists a natural ordering among the variables. This is commonly the case in time series, longitudinal data, spatial data, etc. The estimation is routinely based on the assumption that the variables are ordered and those far apart in the ordering are only weakly correlated [10,14,26,30,33]. The other situation is that the variables do not have a natural ordering among themselves. It often happens in economic data, genetic data, industrial data, etc. In this case, one strategy is to establish an ordering by some data-driven schemes first, then estimate the covariance matrix based on it [7,24,28]. However, the established ordering may make little sense for practical data, for example, the stock market data or gene expression data. The other prevailing strategy, instead of establishing an ordering, aims at estimating the covariance matrix directly. Ledoit and Wolf [18] introduced a linear combination of the sample covariance matrix and a diagonal matrix. Bickel and Levina [2] imposed the hard thresholding technique on the small entries of the sample covariance matrix to form the estimator. At the same time, a banded structure estimate for high-dimensional covariance matrix was also proposed by Bickel and Levina [3]. However, such methods, which are based on thresholding or banding, cannot guarantee the positive definiteness of the estimated covariance matrix. Later, Bien and Tibshirani [4] developed an estimator by adding the $L_1$ penalty to the log-likelihood function. The computation, however, is much intensive due to the non-convexity of the objective function. More recent and related work of covariance matrix estimation can be found in [6,8,21,29,31], among others.

The modified Cholesky decomposition (MCD) [7,22,34] of a covariance matrix is another important technique that provides an unconstrained and statistically interpretable parameterization for a covariance matrix. Zheng et al. [34] used the MCD to obtain a set of covariance matrix estimates under multiple orderings of variables and introduced an averaging model to estimate the covariance matrix. In this paper, we adopt the idea of Zheng et al. [34] to propose an ensemble estimate of a large covariance matrix for the case where an ordering of the variables does not exist. The proposed method obtains multiple covariance matrix estimates by considering different orderings of variables in the MCD. The proposed estimator is a linear combination of the average of these available estimates and the identity matrix by employing the loss function as in Ledoit and Wolf [18]. The estimator of Ledoit and Wolf [18] was developed based on the sample covariance matrix $\mathit{S}$, while our proposed estimator is built based on the average of available estimates, which is more stable than $\mathit{S}$ in high dimensions. Hence, the proposed estimator is supposed to perform better than that of Ledoit and Wolf [18]. This is can be seen in the simulation study.

The rest of this paper is organized as follows. Section 2 briefly reviews the MCD approach to estimate the covariance matrix, and introduces the proposed estimator in detail. The numerical simulation results and a real data example are reported in Sections 3 and 4. We conclude our paper in Section 5.

2. Estimation method

2.1. The modified Cholesky decomposition

Let $\mathit{X}=(X_1,\dots ,X_p{)}^{\prime }$ be a p-dimensional vector of random variables with an unknown covariance matrix $\mathbf{\Sigma }$. Without loss of generality, we assume that $E\mathit{X}=0$. The key idea of the modified Cholesky decomposition (MCD) is that the covariance matrix $\mathbf{\Sigma }$ can be diagonalized by a lower triangular matrix resulting from the regression coefficients when $X_j$ is regressed on its predecessors $X_1,\dots ,X_{j-1}$. Specifically, for $j=2,\dots ,p$, define

$$X_j=\sum _{k=1}^{j-1}{a}_{jk}{X}_k+{ϵ}_j={\mathit{B}}_j^{\mathrm{\prime }}{\mathit{a}}_j+{ϵ}_j,$$ (1)

where ${\mathit{B}}_j=(X_1,\dots ,X_{j-1}{)}^{\mathrm{\prime }}$, and ${\mathit{a}}_j=(a_{j1},\dots ,a_{j,j-1}{)}^{\mathrm{\prime }}$ represents the vector of regression coefficients. The error terms ${ϵ}_j^{,}s$ are independent from each other with $E\left({ϵ}_j\right)=0$ and $Var\left({ϵ}_j\right)=d_j^2$, where $d_1^2$ is defined as $Var\left(X_1\right)$. Accordingly, a lower triangular matrix $\mathit{T}$ is constructed with ones on its diagonal and $-{\mathit{a}}_j$ as its jth row. Then the MCD of the covariance matrix $\mathbf{\Sigma }$ is

$$\mathbf{\Sigma }={\mathit{T}}^{-1}\mathit{D}{{\mathit{T}}^{\prime }}^{-1},$$

where $\mathit{D}=\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}(d_1^2,\dots ,d_p^2)$ is a diagonal matrix.

2.2. The proposed estimator

Note that the Cholesky-based covariance matrix estimate depends on the ordering of variables $X_1,\dots ,X_p$. This indicates that different orderings of variables will generally lead to different estimates of the covariance matrix. Hence, when the variables are not accompanied by the natural ordering, multiple estimates of $\mathbf{\Sigma }$ can be obtained via the MCD by considering a set of random permutations of the orderings. Define a permutation mapping $\pi :\{1,\dots ,p\}\to \{1,\dots ,p\}$, which gives a rearrangement of the ordering of variables, $(1,\dots ,p)\to \left(\pi \right(1),\pi (2),\dots ,\pi (p\left)\right)$. The corresponding permutation matrix ${\mathit{P}}_{\pi }$ is formed with the entries in ith column all 0 except taking 1 at position $\pi \left(i\right)$. Suppose data ${\mathit{x}}_1,\dots ,{\mathit{x}}_n$ are independently and identically distributed from a multivariate normal distribution $\mathcal{N}(\mathbf{0},\mathbf{\Sigma })$, and let $\mathbb{X}$ be the data matrix. Then, the permutated data matrix is

$${\mathbb{X}}_{\pi }=\mathbb{X}{\mathit{P}}_{\pi }=({\mathit{x}}^{\left\{\pi \right(1\left)\right\}},\dots ,{\mathit{x}}^{\left\{\pi \right(p\left)\right\}}),$$

where ${\mathit{x}}^{\left\{\pi \right(j\left)\right\}}$ is the $\pi \left(j\right)$th column of $\mathbb{X}$. Note that the normality assumption is not necessary for estimating a covariance matrix using the MCD technique. However, a number of literature studying the Cholesky-based covariance matrix estimation assumes that the data are sampled from a normal distribution for the sake of using the normal likelihood as the loss function or establishing the theoretical property of estimators [7,14,19,20,22,23,32]. Here we keep the normality assumption to derive the proposed estimator in Theorem 2.3. Some discussion on the Cholesky-Based estimation for the non-normal data is provided in Section 5.

In the situation where the sample size n is close to p or n<p, the regularization technique such as Lasso [27] can be imposed on the regressions (1) to construct the Cholesky factor matrix $\mathit{T}$. Hence, given a permutation π, we have

$${\tilde{\mathit{a}}}_{\pi \left(j\right)}=\arg \underset{{\mathit{a}}_{\pi \left(j\right)}}{min}\parallel {\mathit{x}}_{\pi }^{\left\{\pi \right(j\left)\right\}}-{\mathbb{B}}_{\pi }^{\left\{\pi \right(j\left)\right\}}{\mathit{a}}_{\pi \left(j\right)}{\parallel }_2^2+{\lambda }_{\pi \left(j\right)}\parallel {\mathit{a}}_{\pi \left(j\right)}{\parallel }_1\text{for}\pi \left(j\right)\ne 1,$$ (2)

where ${\mathbb{B}}_{\pi }^{\left\{j\right\}}$ represents the first (j-1) columns of ${\mathbb{X}}_{\pi }$, ${\lambda }_{\pi \left(j\right)}\ge 0$ is the tuning parameter, and $\parallel \cdot {\parallel }_1$ stands for the vector $l_1$ norm. The optimization problem (2) can be solved by the coordinate descent algorithm [11]. The tuning parameters are determined by the 10-fold cross validation scheme for each Lasso regression, where the mean squared error of the response variable is used as the loss measure. As a result, the corresponding estimates ${\tilde{\mathit{T}}}_{\pi }$ and ${\tilde{\mathit{D}}}_{\pi }$ can be constructed for the permutation π. Now we randomly generate M different permutations ${\pi }_k$, $k=1,\dots ,M$ and build the estimates ${\tilde{\mathit{T}}}_{{\pi }_k}$ and ${\tilde{\mathit{D}}}_{{\pi }_k}$. Consequently, ${\tilde{\mathbf{\Sigma }}}_k={\mathit{P}}_{{\pi }_k}{\tilde{\mathbf{\Sigma }}}_{{\pi }_k}{\mathit{P}}_{{\pi }_k}^{\prime }={\mathit{P}}_{{\pi }_k}{\tilde{\mathit{T}}}_{{\pi }_k}^{-1}{\tilde{\mathit{D}}}_{{\pi }_k}{\tilde{\mathit{T}}}_{{\pi }_k}^{\prime -1}{\mathit{P}}_{{\pi }_k}^{\prime }$ is an estimate of covariance matrix $\mathbf{\Sigma }$ for the permutation ${\pi }_k$.

To make it explicit that the variables and Cholesky-based estimates depend on the sample size n, we use $n=1,2,\dots$ to index a sequence of statistical models thereafter. For example, ${\mathbb{X}}^{\left(n\right)}$ is a $n\times p_n$ matrix of n i.i.d. observations for $p_n$ variables with mean zero and covariance matrix ${\mathbf{\Sigma }}^{\left(n\right)}$. Notations ${\tilde{\mathbf{\Sigma }}}_k^{\left(n\right)}$ and $({\tilde{\mathit{T}}}_k^{\left(n\right)},{\tilde{\mathit{D}}}_k^{\left(n\right)}),k=1,2,\dots ,M_n$, represent the MCD covariance matrix estimates and Cholesky factors estimates, respectively, for permutation ${\pi }_k$ associated with the sample size n. The number of variables $p_n$ can vary and even go to infinity for sample size n. Now a natural question arises as how to efficiently use these available ${\tilde{\mathbf{\Sigma }}}_k^{\left(n\right)}$ to form an ensemble covariance matrix estimate. Borrowing the idea from Ledoit and Wolf [18], we propose to consider

$${\hat{\mathbf{\Sigma }}}_{\ast }^{\left(n\right)}=\arg \underset{{\mathbf{\Sigma }}_{\ast }^{\left(n\right)}}{min}\mathit{E}||{\mathbf{\Sigma }}_{\ast }^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)}|{|}_F^2\text{s.t.}{\mathbf{\Sigma }}_{\ast }^{\left(n\right)}={\rho }_1{\mathit{I}}^{\left(n\right)}+{\rho }_2{\overline{\mathbf{\Sigma }}}^{\left(n\right)},$$ (3)

where ${\overline{\mathbf{\Sigma }}}^{\left(n\right)}=\frac{1}{M_n}\sum _{k=1}^{M_n}{\tilde{\mathbf{\Sigma }}}_k^{\left(n\right)}$ is proposed by Zheng et al. [34]. The coefficients ${\rho }_1$ and ${\rho }_2$ are constants. $||\mathit{A}|{|}_F$ is defined as $\sqrt{\mathrm{t}\mathrm{r}\left(A{A}^{\prime }\right)/p_n}$. Compared with the estimate of Ledoit and Wolf [18], we use ${\overline{\mathbf{\Sigma }}}^{\left(n\right)}$ instead of the sample covariance $\mathit{S}$ in (3). Since the estimate ${\overline{\mathbf{\Sigma }}}^{\left(n\right)}$ performs more stable than $\mathit{S}$ in large dimensions, the proposed estimate could be more accurate. On the other hand, the percentage relative improvement of the proposed estimate over the average estimate ${\overline{\mathbf{\Sigma }}}^{\left(n\right)}$ is equal to ${\beta }^{\left(n\right)}/{\delta }^{\left(n\right)}$ as discussed in [18], where ${\beta }^{\left(n\right)}$ and ${\delta }^{\left(n\right)}$ are defined in the next paragraph. Hence, our proposed method is supposed to perform better the aforementioned two approaches, which can be seen in the numerical study in Section 3.

To solve the optimization problem (3), we need to introduce some notations and assumptions. Denote the inner product as $⟨A,B⟩=\mathrm{t}\mathrm{r}\left(A{B}^{\prime }\right)/p_n$. Define ${\mu }^{\left(n\right)}=⟨{\mathbf{\Sigma }}^{\left(n\right)},{\mathit{I}}^{\left(n\right)}⟩$, ${\alpha }^{\left(n\right)}=||{\mathbf{\Sigma }}^{\left(n\right)}-{\mu }^{\left(n\right)}{\mathit{I}}^{\left(n\right)}|{|}_F^2$, ${\beta }^{\left(n\right)}=\mathit{E}||{\overline{\mathbf{\Sigma }}}^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)}|{|}_F^2$ and ${\delta }^{\left(n\right)}=\mathit{E}||{\overline{\mathbf{\Sigma }}}^{\left(n\right)}-{\mu }^{\left(n\right)}{\mathit{I}}^{\left(n\right)}|{|}_F^2$. Let ${\mathbf{\Sigma }}^{\left(n\right)}={\mathit{L}}^{\left(n\right)}{\mathit{D}}^{\left(n\right)}{\mathit{L}}^{(n{)}^{\prime }}$ denote the true covariance matrix of the observations ${\mathbb{X}}^{\left(n\right)}=(x_{ij}^{\left(n\right)}{)}_{n\times p}$, and use the set $A_k^{\left(n\right)}$ to represent the collection of nonzero entries in the lower left part of Cholesky factor matrix ${\mathit{L}}_k^{\left(n\right)}$ corresponding to the variable ordering ${\pi }_k$. Denote by $q_n$ the maximum of the cardinality of $A_k^{\left(n\right)}$ for $k=1,2,\dots ,M_n$. Define the singular values of matrix $\mathit{A}$ as $s{v}_1\left(A\right)\ge s{v}_2\left(A\right)\ge \cdots \ge s{v}_p\left(A\right)$ in a decreasing order. We assume the following conditions:

1. there exists a constant $h_n>1$ such that the singular values of ${\mathbf{\Sigma }}^{\left(n\right)}$ are bounded as $1/h_n<s{v}_p\left({\mathbf{\Sigma }}^{\left(n\right)}\right)\le s{v}_1\left({\mathbf{\Sigma }}^{\left(n\right)}\right)<h_n$;

2. the tuning parameters ${\lambda }_{\pi \left(j\right)}$ in (2) satisfy $\sum _{\pi \left(j\right)\ne 1}^{p_n}{\lambda }_{\pi \left(j\right)}=O(\log (p_n\left)/n\right)$;

3. $(p_n+q_n)\log \left(p_n\right)=o\left(n{M}_n\right)$.

The condition (C1) is also made in [12,17], which guarantees that ${\mathbf{\Sigma }}^{\left(n\right)}$ is positive definite. Now we present the following lemma and theorem.

Lemma 2.1

Suppose ${\mathit{x}}_1,\dots ,{\mathit{x}}_n$ are independently and identically distributed observations from $\mathcal{N}(\mathbf{0},{\mathbf{\Sigma }}^{\left(n\right)})$. Under (C1)–(C2), we have

$$\parallel {\tilde{\mathbf{\Sigma }}}_k^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)}{\parallel }_F=O_p\left(\sqrt{(p_n+q_n)\log \left(p_n\right)/n}\right).$$

Proof Proof of Lemma 1 —

The proof is similar to Theorem 1 in [25] and Theorem 2 in [16], thus is omitted here.

Lemma 2.2

Assume all the assumptions in Lemma 2.1 hold. Under (C3), we have $\underset{n\to \mathrm{\infty }}{lim}{\alpha }^{\left(n\right)}+{\beta }^{\left(n\right)}-{\delta }^{\left(n\right)}=0$.

Proof Proof of Lemma 2 —

We first derive the consistent property of ${\overline{\mathbf{\Sigma }}}^{\left(n\right)}$ as follows:

$$\begin{array}{rl}\parallel {\overline{\mathbf{\Sigma }}}^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)}{\parallel }_F^2& ={∥\frac{1}{M_n}\sum _{k=1}^{M_n}{\tilde{\mathbf{\Sigma }}}_k^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)}∥}_F^2\\ & =\frac{1}{M_n^2}{∥\sum _{k=1}^{M_n}{\tilde{\mathbf{\Sigma }}}_k^{\left(n\right)}-M_n{\mathbf{\Sigma }}^{\left(n\right)}∥}_F^2\\ & \le \frac{1}{M_n^2}\sum _{k=1}^{M_n}{∥{\tilde{\mathbf{\Sigma }}}_k^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)}∥}_F^2\\ & =O_p\left(\frac{(p_n+q_n)\log \left(p_n\right)}{n{M}_n}\right),\end{array}$$

where the fourth equality is provided by Lemma 1. Then we have

$$\begin{array}{rl}{\delta }^{\left(n\right)}& =\mathit{E}||{\overline{\mathbf{\Sigma }}}^{\left(n\right)}-{\mu }^{\left(n\right)}{\mathit{I}}^{\left(n\right)}|{|}_F^2\\ & =\mathit{E}||{\overline{\mathbf{\Sigma }}}^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)}+{\mathbf{\Sigma }}^{\left(n\right)}-{\mu }^{\left(n\right)}{\mathit{I}}^{\left(n\right)}|{|}_F^2\\ & =\mathit{E}||{\overline{\mathbf{\Sigma }}}^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)}|{|}_F^2+||{\mathbf{\Sigma }}^{\left(n\right)}-{\mu }^{\left(n\right)}{\mathit{I}}^{\left(n\right)}|{|}_F^2+2\mathit{E}⟨{\overline{\mathbf{\Sigma }}}^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)},{\mathbf{\Sigma }}^{\left(n\right)}-{\mu }^{\left(n\right)}{\mathit{I}}^{\left(n\right)}⟩\\ & ={\beta }^{\left(n\right)}+{\alpha }^{\left(n\right)}+2⟨\mathit{E}[{\overline{\mathbf{\Sigma }}}^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)}],{\mathbf{\Sigma }}^{\left(n\right)}-{\mu }^{\left(n\right)}{\mathit{I}}^{\left(n\right)}⟩.\end{array}$$

$\underset{n\to \mathrm{\infty }}{lim}\mathit{E}[{\overline{\mathbf{\Sigma }}}^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)}]=0$, which results from the consistent property of ${\overline{\mathbf{\Sigma }}}^{\left(n\right)}$, establishes this Lemma.

Theorem 2.3

Assume all the assumptions in Lemma 2.2 hold, then the solution to (3) is ${\hat{\mathbf{\Sigma }}}_{\ast }^{\left(n\right)}=\frac{{\beta }^{\left(n\right)}}{{\delta }^{\left(n\right)}}{\mu }^{\left(n\right)}{\mathit{I}}^{\left(n\right)}+\frac{{\alpha }^{\left(n\right)}}{{\delta }^{\left(n\right)}}{\overline{\mathbf{\Sigma }}}^{\left(n\right)}$.

Proof Proof of Theorem 1 —

The optimization problem (3) can be re-written as

$$\underset{\rho ,\upsilon }{min}\mathit{E}||{\mathbf{\Sigma }}_{\ast }^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)}|{|}_F^2\text{s.t.}{\mathbf{\Sigma }}_{\ast }^{\left(n\right)}=\rho \upsilon {\mathit{I}}^{\left(n\right)}+(1-\rho ){\overline{\mathbf{\Sigma }}}^{\left(n\right)}.$$

Therefore, the objective can be decomposed into three parts as follows:

$$\begin{array}{rl}\mathit{E}||{\mathbf{\Sigma }}_{\ast }^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)}|{|}_F^2& =\mathit{E}||\rho \upsilon {\mathit{I}}^{\left(n\right)}+(1-\rho ){\overline{\mathbf{\Sigma }}}^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)}|{|}_F^2\\ & =\mathit{E}||\rho \upsilon {\mathit{I}}^{\left(n\right)}+(1-\rho ){\overline{\mathbf{\Sigma }}}^{\left(n\right)}-\rho {\mathbf{\Sigma }}^{\left(n\right)}-(1-\rho ){\mathbf{\Sigma }}^{\left(n\right)}|{|}_F^2\\ & ={\rho }^2||{\mathbf{\Sigma }}^{\left(n\right)}-\upsilon {\mathit{I}}^{\left(n\right)}|{|}_F^2+(1-\rho {)}^2\mathit{E}||{\overline{\mathbf{\Sigma }}}^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)}|{|}_F^2\\ & +2\rho (1-\rho )⟨\mathit{E}[{\overline{\mathbf{\Sigma }}}^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)}],\upsilon {\mathit{I}}^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)}⟩\\ & ={\rho }^2||{\mathbf{\Sigma }}^{\left(n\right)}-\upsilon {\mathit{I}}^{\left(n\right)}|{|}_F^2+(1-\rho {)}^2\mathit{E}||{\overline{\mathbf{\Sigma }}}^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)}|{|}_F^2\\ & +O\left(\sqrt{(p_n+q_n)\log \left(p_n\right)/\left(n{M}_n\right)}\right).\end{array}$$

Under (C3), we have $\mathit{E}||{\mathbf{\Sigma }}_{\ast }^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)}|{|}_F^2-{\rho }^2||{\mathbf{\Sigma }}^{\left(n\right)}-\upsilon {\mathit{I}}^{\left(n\right)}|{|}_F^2-(1-\rho {)}^2\mathit{E}||{\overline{\mathbf{\Sigma }}}^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)}|{|}_F^2\to 0$ as $n\to \mathrm{\infty }$. Let ${\mathrm{\Lambda }}_1^{\left(n\right)}$ and ${\mathrm{\Lambda }}_2^{\left(n\right)}$ be the minimizers of $\mathit{E}||{\mathbf{\Sigma }}_{\ast }^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)}|{|}_F^2$ and ${\rho }^2||{\mathbf{\Sigma }}^{\left(n\right)}-\upsilon {\mathit{I}}^{\left(n\right)}|{|}_F^2+(1-\rho {)}^2\mathit{E}||{\overline{\mathbf{\Sigma }}}^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)}|{|}_F^2$, respectively. According to Lemma 2 of Hjort and Pollard [13], we have ${\mathrm{\Lambda }}_1^{\left(n\right)}-{\mathrm{\Lambda }}_2^{\left(n\right)}\to 0$. Now let us consider the minimization problem of

$${\rho }^2||{\mathbf{\Sigma }}^{\left(n\right)}-\upsilon {\mathit{I}}^{\left(n\right)}|{|}_F^2+(1-\rho {)}^2\mathit{E}||{\overline{\mathbf{\Sigma }}}^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)}|{|}_F^2.$$ (4)

It is easy to see that the optimal value of υ does not depend on ρ. Hence, the closed form for υ is developed as

$$\begin{array}{rl}\frac{\mathrm{\partial }||{\mathbf{\Sigma }}^{\left(n\right)}-\upsilon {\mathit{I}}^{\left(n\right)}|{|}_F^2}{\mathrm{\partial }\upsilon }& =-2⟨{\mathbf{\Sigma }}^{\left(n\right)},{\mathit{I}}^{\left(n\right)}⟩+2\upsilon =0\\ \upsilon & =⟨{\mathbf{\Sigma }}^{\left(n\right)},{\mathit{I}}^{\left(n\right)}⟩={\mu }^{\left(n\right)}.\end{array}$$ (5)

Replacing (5) into (4) lead to minimizing problem of ${\rho }^2{\alpha }^{\left(n\right)}+(1-\rho {)}^2{\beta }^{\left(n\right)}$. The first derivative is $2{\alpha }^{\left(n\right)}\rho -2(1-\rho ){\beta }^{\left(n\right)}$. Hence, the optimal value of ρ is $\frac{{\beta }^{\left(n\right)}}{{\alpha }^{\left(n\right)}+{\beta }^{\left(n\right)}}=\frac{{\beta }^{\left(n\right)}}{\delta \left(n\right)}$. This completes the proof.

Solving the covariance estimate ${\hat{\mathbf{\Sigma }}}_{\ast }^{\left(n\right)}$ seems challenging since the four terms ${\mu }^{\left(n\right)}$, ${\alpha }^{\left(n\right)}$, ${\beta }^{\left(n\right)}$ and ${\delta }^{\left(n\right)}$ all depend on the true covariance matrix which is apparently unobservable. [18] showed that these exist asymptotically consistent estimators for ${\mu }^{\left(n\right)}$, ${\alpha }^{\left(n\right)}$, ${\beta }^{\left(n\right)}$ and ${\delta }^{\left(n\right)}$, hence for ${\hat{\mathbf{\Sigma }}}_{\ast }^{\left(n\right)}$. Define

$$v^{\left(n\right)}=⟨{\overline{\mathbf{\Sigma }}}^{\left(n\right)},{\mathit{I}}^{\left(n\right)}⟩\text{and}{d}^{\left(n\right)}=||{\overline{\mathbf{\Sigma }}}^{\left(n\right)}-v^{\left(n\right)}{\mathit{I}}^{\left(n\right)}|{|}_F^2.$$

Then $v^{\left(n\right)}$ and $d^{\left(n\right)}$ are estimators for ${\mu }^{\left(n\right)}$ and ${\delta }^{\left(n\right)}$, respectively. Note that ${\overline{\mathbf{\Sigma }}}^{\left(n\right)}$ is the average of matrices ${\tilde{\mathbf{\Sigma }}}_k^{\left(n\right)}$, hence, we can estimate ${\beta }^{\left(n\right)}=\mathit{E}||{\overline{\mathbf{\Sigma }}}^{\left(n\right)}-{\mathbf{\Sigma }}^{\left(n\right)}|{|}_F^2$ by seeing how far each of ${\tilde{\mathbf{\Sigma }}}_k^{\left(n\right)}$ deviates from their average ${\overline{\mathbf{\Sigma }}}^{\left(n\right)}$. Define

$$\begin{array}{rl}{\ddot{b}}^{\left(n\right)}& =\frac{1}{M_n^2}\sum _{k=1}^{M_n}||{\tilde{\mathbf{\Sigma }}}_k^{\left(n\right)}-{\overline{\mathbf{\Sigma }}}^{\left(n\right)}|{|}_F^2\text{and}{b}^{\left(n\right)}=\text{min}({\ddot{b}}^{\left(n\right)},d^{\left(n\right)}),\\ a^{\left(n\right)}& =d^{\left(n\right)}-b^{\left(n\right)},\end{array}$$

where $b^{\left(n\right)}$ is introduced to ensure that $a^{\left(n\right)}$ is nonnegative. Then $a^{\left(n\right)}$ and $b^{\left(n\right)}$ are estimators for ${\alpha }^{\left(n\right)}$ and ${\beta }^{\left(n\right)}$, respectively. Now we replace the unobservable terms in the formula of ${\hat{\mathbf{\Sigma }}}_{\ast }^{\left(n\right)}$ in Theorem 2.3 with their estimators, yielding our proposed estimate of covariance matrix

$${\hat{\mathbf{\Sigma }}}^{\left(n\right)}=\frac{b^{\left(n\right)}}{d^{\left(n\right)}}{v}^{\left(n\right)}{\mathit{I}}^{\left(n\right)}+\frac{a^{\left(n\right)}}{d^{\left(n\right)}}{\overline{\mathbf{\Sigma }}}^{\left(n\right)}.$$

3. Simulation study

This section examines the finite performance of the proposed method (Prop) by comparison with several other approaches. The competing methods include the sample covariance matrix $\mathit{S}$ as a benchmark, Ledoit and Wolf's estimate (LW) [18], Bien and Tibshirani's estimate (BT) [4], minimax estimate (MX) [9] as well as the average estimate ${\overline{\mathbf{\Sigma }}}^{\left(n\right)}$ (AVE). The BT estimate minimizes the negative log-likelihood function with $L_1$ penalty

$${\hat{\mathbf{\Sigma }}}_{BT}=\arg \underset{\mathbf{\Sigma }\succ 0}{min}\{-\log |{\mathbf{\Sigma }}^{-1}|+\mathrm{t}\mathrm{r}\left({\mathbf{\Sigma }}^{-1}\mathit{S}\right)+\eta |\mathbf{\Sigma }{|}_1\},$$

where $\eta \ge 0$ is the tuning parameter selected by the cross-validation, and $\mathbf{\Sigma }\succ 0$ is a positive definite constraint for the covariance matrix imposed on the objective. This optimization problem is solved by the majorization-minimization algorithm. The MX estimate preserves the sample eigenvectors but replaces the sample eigenvalues by $\frac{n}{n+p+1-2i}{\lambda }_i^{\ast }$, where ${\lambda }_1^{\ast },{\lambda }_2^{\ast },\dots ,{\lambda }_p^{\ast }$ stand for the sample eigenvalues in decreasing order.

We consider four covariance matrix structures $\mathbf{\Sigma }=({\sigma }_{ij}{)}_{p\times p}$ as follows:

1. ${\mathbf{\Sigma }}_1^{-1}$ = diag(1, 2, …, p). The inverse covariance matrix is a diagonal matrix with diagonal elements from 1 to p.

2. ${\mathbf{\Sigma }}_2$ = AR(0.5). The conditional covariance between any two variables $X_i$ and $X_j$ is fixed to be ${0.5}^{|i-j|}$, $1\le i,j\le p$.

3. ${\mathbf{\Sigma }}_3:{\sigma }_{ij}=(1-|i-j|/50{)}_{+}$.

4. ${\mathbf{\Sigma }}_4=\left(\begin{array}{cc}\text{CS}\left(0.5\right)& \mathbf{0}\\ \mathbf{0}& \mathit{I}\end{array}\right)$, where CS(0.5) is a $20\times 20$ compound structure matrix with diagonal elements 1 and others 0.5. $\mathbf{0}$ represents zero matrix.

$\mathbf{M}\mathbf{o}\mathbf{d}\mathbf{e}\mathbf{l}$ 1 is a diagonal matrix. $\mathbf{M}\mathbf{o}\mathbf{d}\mathbf{e}\mathbf{l}$ 2 is a autoregressive structure with homogeneous variances and correlations declining. $\mathbf{M}\mathbf{o}\mathbf{d}\mathbf{e}\mathbf{l}$ 3 is similar to the matrix design used in [5,31]. $\mathbf{M}\mathbf{o}\mathbf{d}\mathbf{e}\mathbf{l}$ 4 is a block compound structure on the upper left corner. For each model, data are independently generated from $\mathcal{N}(\mathbf{0},\mathbf{\Sigma })$ with the sample size n=50 and three different settings of variable sizes $p\in \{30,50,100\}$. To measure the performance of each covariance matrix estimate $\hat{\mathbf{\Sigma }}=({\hat{\sigma }}_{ij}{)}_{p\times p}$, we consider the squared Frobenius norm (F), the mean absolute error (MAE) and quadratic loss (QL), defined as follows (up to some scale)

$$\begin{array}{rl}\text{F}& =\frac{1}{p}\sum _{i=1}^p\sum _{j=1}^p({\hat{\sigma }}_{ij}-{\sigma }_{ij}{)}^2,\\ \text{MAE}& =\frac{1}{p}\sum _{i=1}^p\sum _{j=1}^p|{\hat{\sigma }}_{ij}-{\sigma }_{ij}|,\\ \text{QL}& =\frac{1}{p}\mathrm{t}\mathrm{r}[{\hat{\mathbf{\Sigma }}}^{-1}\mathbf{\Sigma }-\mathit{I}{]}^2.\end{array}$$

We also use the entropy loss (EN) and the Kullback–Leibler loss (KL) given by

$$\begin{array}{rl}\text{EN}& =\mathrm{t}\mathrm{r}\left[{\mathbf{\Sigma }}^{-1}\hat{\mathbf{\Sigma }}\right]-\log |{\mathbf{\Sigma }}^{-1}\hat{\mathbf{\Sigma }}|-p,\\ \text{KL}& =\mathrm{t}\mathrm{r}\left[{\hat{\mathbf{\Sigma }}}^{-1}\mathbf{\Sigma }\right]-\log |{\hat{\mathbf{\Sigma }}}^{-1}\mathbf{\Sigma }|-p.\end{array}$$

Throughout this section, the number of permutations M is set to be 30 for the proposed method, as suggested by Zheng et al. [34]. The numerical results of the averaged loss measures over 100 replicates and their standard errors in parenthesis for each method are summarized in Tables 1–4. The best method with lowest averages regarding each loss measure is shown in bold for each setting of variable sizes. Dashed lines in the tables represent the corresponding values not available due to the matrix singularity.

Table 2. The averages and standard errors (in parenthesis) of estimates for $\mathbf{M}\mathbf{o}\mathbf{d}\mathbf{e}\mathbf{l}$ 2.

		F	MAE	QL	EN	KL
p=30	$\mathit{S}$	0.804 (0.005)	3.485 (0.022)	0.633 (0.006)	12.35 (0.089)	37.83 (0.075)
	BT	0.855 (0.001)	2.187 (0.003)	0.197 (0.001)	5.332 (0.032)	11.45 (0.096)
	MX	0.617 (0.003)	2.589 (0.012)	0.313 (0.003)	6.946 (0.067)	15.55 (0.316)
	AVE	0.576 (0.005)	1.917 (0.021)	0.216 (0.004)	3.307 (0.050)	4.210 (0.102)
	LW	0.593 (0.003)	2.445 (0.016)	0.475 (0.006)	5.147 (0.039)	4.860 (0.043)
	Prop	0.519 (0.004)	1.719 (0.013)	0.195 (0.003)	2.971 (0.041)	3.771 (0.073)
p=50	$\mathit{S}$	1.028 (0.005)	5.769 (0.029)	1.040 (0.006)	–	–
	BT	0.855 (0.005)	2.253 (0.011)	0.208 (0.002)	9.987 (0.177)	25.003 (1.103)
	MX	0.738 (0.002)	3.735 (0.010)	0.407 (0.001)	–	–
	AVE	0.604 (0.004)	2.229 (0.014)	0.235 (0.004)	5.977 (0.079)	7.802 (0.13)
	LW	0.668 (0.002)	3.425 (0.021)	0.632 (0.006)	10.808 (0.051)	10.231 (0.067)
	Prop	0.563 (0.003)	2.066 (0.010)	0.211 (0.002)	5.473 (0.051)	7.246 (0.095)
p=100	$\mathit{S}$	1.443 (0.004)	11.45 (0.036)	2.068 (0.010)	–	–
	BT	0.866 (0.008)	2.279 (0.015)	0.222 (0.002)	21.447 (0.312)	52.644 (1.083)
	MX	0.936 (0.001)	5.560 (0.009)	0.637 (0.001)	–	–
	AVE	0.737 (0.003)	2.957 (0.015)	0.410 (0.006)	15.085 (0.142)	20.716 (0.270)
	LW	0.757 (0.002)	5.092 (0.028)	0.875 (0.006)	27.601 (0.102)	25.466 (0.078)
	Prop	0.616 (0.002)	2.480 (0.010)	0.242 (0.002)	12.560 (0.081)	17.186 (0.153)

Table 3. The averages and standard errors (in parenthesis) of estimates for $\mathbf{M}\mathbf{o}\mathbf{d}\mathbf{e}\mathbf{l}$ 3.

		F	MAE	QL	EN	KL
p=30	$\mathit{S}$	1.043 (0.050)	5.642 (0.277)	0.627 (0.008)	12.50 (0.091)	40.03 (0.751)
	BT	1.224 (0.095)	6.219 (0.530)	1.010 (0.013)	8.732 (0.079)	7.558 (0.187)
	MX	1.671 (0.056)	8.839 (0.319)	0.363 (0.004)	7.870 (0.066)	18.53 (0.326)
	AVE	1.056 (0.066)	4.922 (0.356)	0.207 (0.006)	3.124 (0.067)	4.233 (0.122)
	LW	0.987 (0.053)	4.830 (0.290)	17.62 (1.038)	60.29 (2.138)	18.50 (0.365)
	Prop	0.945 (0.048)	4.681 (0.268)	0.180 (0.003)	2.751 (0.037)	3.617 (0.069)
p=50	$\mathit{S}$	1.286 (0.053)	8.292 (0.357)	1.049 (0.008)	–	–
	BT	3.823 (0.093)	25.70 (0.646)	2.070 (0.164)	22.25 (0.429)	28.76 (1.180)
	MX	2.501 (0.049)	16.48 (0.359)	0.469 (0.002)	–	–
	AVE	1.215 (0.077)	7.562 (0.532)	0.224 (0.004)	5.415 (0.074)	6.551 (0.140)
	LW	1.256 (0.053)	7.725 (0.366)	18.62 (0.914)	10.53 (3.258)	32.01 (0.518)
	Prop	1.196 (0.050)	7.397 (0.341)	0.194 (0.002)	4.760 (0.046)	6.363 (0.099)
p=100	$\mathit{S}$	1.751 (0.055)	14.63 (0.488)	2.045 (0.010)	–	–
	BT	5.190 (0.045)	40.21 (0.354)	108.8 (8.656)	568.9 (7.719)	277.0 (6.078)
	MX	3.562 (0.029)	28.20 (0.252)	0.695 (0.001)	–	–
	AVE	1.665 (0.068)	13.87 (0.594)	0.764 (0.006)	26.88 (0.132)	29.40 (0.310)
	LW	1.652 (0.050)	13.67 (0.452)	59.51 (2.068)	396.7 (8.192)	87.14 (0.791)
	Prop	1.610 (0.044)	13.10 (0.383)	0.688 (0.004)	24.24 (0.081)	25.93 (0.170)

Table 1. The averages and standard errors (in parenthesis) of estimates for $\mathbf{M}\mathbf{o}\mathbf{d}\mathbf{e}\mathbf{l}$ 1.

		F	MAE	QL	EN	KL
p=30	$\mathit{S}$	0.107 (0.001)	0.353 (0.002)	0.630 (0.007)	12.42 (0.082)	38.76 (0.652)
	BT	0.132 (0.001)	0.057 (0.001)	0.082 (0.001)	1.783 (0.021)	2.775 (0.041)
	MX	0.103 (0.001)	0.250 (0.001)	0.316 (0.003)	7.037 (0.059)	15.96 (0.275)
	AVE	0.049 (0.002)	0.059 (0.001)	0.071 (0.002)	1.062 (0.035)	1.281 (0.055)
	LW	0.048 (0.002)	0.074 (0.004)	0.064 (0.003)	0.945 (0.040)	1.013 (0.043)
	Prop	0.045 (0.001)	0.055 (0.001)	0.064 (0.001)	1.053 (0.024)	1.265 (0.035)
p=50	$\mathit{S}$	0.094 (0.001)	0.374 (0.002)	1.045 (0.007)	–	–
	BT	0.112 (0.001)	0.041 (0.001)	0.078 (0.001)	2.702 (0.031)	4.192 (0.063)
	MX	0.097 (0.001)	0.227 (0.001)	0.415 (0.002)	–	–
	AVE	0.041 (0.002)	0.045 (0.001)	0.073 (0.002)	1.870 (0.056)	2.308 (0.090)
	LW	0.039 (0.001)	0.072 (0.004)	0.080 (0.004)	1.899 (0.095)	2.004 (0.102)
	Prop	0.036 (0.001)	0.040 (0.001)	0.066 (0.001)	1.857 (0.036)	2.300 (0.058)
p=100	$\mathit{S}$	0.076 (0.001)	0.394 (0.001)	2.061 (0.009)	–	–
	BT	0.082 (0.001)	0.025 (0.001)	0.162 (0.001)	7.366 (0.048)	8.134 (0.066)
	MX	0.088 (0.001)	0.178 (0.001)	0.646 (0.001)	–	–
	AVE	0.029 (0.001)	0.029 (0.001)	0.078 (0.001)	4.212 (0.073)	5.504 (0.117)
	LW	0.027 (0.001)	0.067 (0.003)	0.104 (0.006)	4.666 (0.225)	4.786 (0.223)
	Prop	0.025 (0.001)	0.026 (0.001)	0.069 (0.001)	4.109 (0.058)	5.354 (0.098)

Table 4. The averages and standard errors (in parenthesis) of estimates for $\mathbf{M}\mathbf{o}\mathbf{d}\mathbf{e}\mathbf{l}$ 4.

		F	MAE	QL	EN	KL
p=30	$\mathit{S}$	0.823 (0.013)	3.611 (0.060)	0.626 (0.006)	12.46 (0.086)	39.78 (0.745)
	BT	1.795 (0.001)	6.619 (0.004)	0.162 (0.001)	4.888 (0.044)	16.11 (0.121)
	MX	0.847 (0.017)	3.736 (0.076)	0.268 (0.002)	6.372 (0.065)	14.79 (0.307)
	AVE	0.868 (0.034)	3.369 (0.135)	0.188 (0.003)	3.053 (0.036)	4.311 (0.083)
	LW	0.774 (0.013)	3.431 (0.061)	0.433 (0.006)	5.763 (0.067)	7.501 (0.132)
	Prop	0.835 (0.023)	3.228 (0.090)	0.175 (0.002)	2.863 (0.027)	3.949 (0.056)
p=50	$\mathit{S}$	1.045 (0.009)	5.864 (0.050)	1.041 (0.007)	–	–
	BT	1.415 (0.002)	4.166 (0.007)	0.184 (0.002)	8.753 (0.134)	25.83 (0.724)
	MX	0.923 (0.011)	4.595 (0.037)	0.366 (0.001)	–	–
	AVE	0.880 (0.027)	3.037 (0.080)	0.144 (0.002)	4.096 (0.059)	5.910 (0.134)
	LW	0.866 (0.009)	4.724 (0.035)	0.545 (0.011)	10.94 (0.205)	13.58 (0.355)
	Prop	0.828 (0.017)	2.858 (0.049)	0.135 (0.001)	3.816 (0.037)	5.447 (0.084)
p=100	$\mathit{S}$	1.441 (0.005)	11.44 (0.039)	2.063 (0.009)	–	–
	BT	0.891 (0.016)	2.003 (0.036)	0.252 (0.004)	21.52 (0.522)	49.61 (2.412)
	MX	1.016 (0.004)	5.837 (0.011)	0.615 (0.001)	–	–
	AVE	0.757 (0.013)	2.220 (0.026)	0.108 (0.002)	6.367 (0.101)	9.309 (0.204)
	LW	0.858 (0.003)	5.713 (0.057)	0.462 (0.012)	16.57 (0.384)	16.99 (0.394)
	Prop	0.717 (0.009)	2.094 (0.019)	0.101 (0.001)	5.903 (0.061)	8.518 (0.123)

Overall, Tables 1–4 show that the proposed estimate outperforms other methods regarding these loss measures. Although there are several settings that the proposed method is not the best, it is the second best, for example, the settings of p=100 under MAE and QL in $\mathbf{M}\mathbf{o}\mathbf{d}\mathbf{e}\mathbf{l}$ 2. The BT estimate is superior over some other approaches in a couple of settings regarding MAE and QL measures. The MX method is inferior to other approaches except $\mathit{S}$ under these loss measures, possibly due to that it cannot guarantee the positive definiteness of the estimated covariance matrix. Notably, the LW estimate performs very well with respect to KL in $\mathbf{M}\mathbf{o}\mathbf{d}\mathbf{e}\mathbf{l}$ 1, a diagonal covariance matrix. However, it is not as good as the proposed method for more general settings of matrix structures such as $\mathbf{M}\mathbf{o}\mathbf{d}\mathbf{e}\mathbf{l}$ 2, 3 and 4. In addition, the AVE method generally shows relatively good performance compared with the $\mathit{S}$, BT, MX and LW methods, but the proposed method does have some improvement over the AVE with respect to all loss measures. Furthermore, the proposed estimate substantially performs well as the number of variables p increases from 30 to 100. Thus, the proposed method is able to provide an accurate estimate for the large covariance matrices in these settings.

Notice that the MCD technique does not require the normality assumption on data to form a covariance matrix estimate. It is therefore interesting to investigate the empirical performance of the proposed method when the data are not from a normal distribution. Now we use ${\mathbf{\Sigma }}_2$ = AR(0.5) as the true covariance matrix, and consider the following two data generating processes with the sample size n=50 and variable sizes $p\in \{30,50,100\}$:

1. Independently generate data from $t_{df=4}$ distribution with mean $\mathbf{0}$, and ${\mathbf{\Sigma }}_2$ as its scale matrix.

2. Independently generate data from a mixed distribution $(1-b)\mathcal{N}(\mathbf{0},{\mathbf{\Sigma }}_2)+b\mathit{Q}$, where b is a random number from Binomial distribution $B(1,0.1)$. $\mathit{Q}$ represents $t_{df=4}$ distribution with mean $\mathbf{5}$ and the identity matrix as its scale matrix.

Table 5 summarizes the averages and corresponding standard errors for each method based on 100 replicates. The proposed method produces superior results for case I, and performs comparably with MX and LW for case II. It is worth pointing out that the proposed method becomes better for case II when the variable size increases from p=30 to p=100. Since the BT method heavily relies on the normal data and uses the normal likelihood as its objective, it gives relatively high losses in Table 5, especially regarding the measures EN and KL. We also observe that the BT method has convergence issue for the non-normal data, for example when p=100 for case II. Although the MX estimate seems to perform well for case II in the low dimensions, it is not positive definite when n<p. The LW method shows a good performance with respect to the loss functions EN and KL for case II. Overall, when the data are not normal, the proposed method appears to be superior or at least comparable to other approaches. Examining Lemma 2.1, 2.2 and Theorem 2.3, we see that the convergence rate of ${\tilde{\mathbf{\Sigma }}}_k^{\left(n\right)}$ would be different when the data are not normal. This rate may depend on the data distribution, the true covariance structure, the moments condition of variable $\mathit{X}$ and so forth. The results of Theorem 2.3 would still hold as long as such rate converges to 0 as the sample size goes to infinity.

Table 5. The averages and standard errors (in parenthesis) of estimates for non-normal data.

			F	MAE	QL	EN	KL
I	p=30	$\mathit{S}$	1.117 (0.024)	4.860 (0.110)	1.355 (0.057)	19.43 (0.344)	77.03 (2.390)
		BT	0.990 (0.033)	3.679 (0.206)	1.267 (0.060)	14.83 (0.684)	44.86 (3.905)
		MX	0.952 (0.023)	4.116 (0.100)	1.064 (0.045)	11.90 (0.255)	23.23 (0.766)
		AVE	0.713 (0.015)	2.600 (0.072)	0.598 (0.021)	5.994 (0.149)	5.980 (0.220)
		LW	0.655 (0.007)	2.634 (0.039)	0.651 (0.015)	6.242 (0.104)	5.214 (0.072)
		Prop	0.645 (0.014)	2.361 (0.065)	0.542 (0.018)	5.421 (0.133)	5.362 (0.194)
	p=50	$\mathit{S}$	1.443 (0.021)	8.146 (0.120)	2.294 (0.079)	–	–
		BT	0.868 (0.035)	3.061 (0.273)	1.199 (0.082)	21.58 (1.744)	129.7 (28.97)
		MX	1.202 (0.020)	6.741 (0.110)	1.728 (0.060)	–	–
		AVE	0.739 (0.010)	3.230 (0.062)	0.670 (0.015)	10.74 (0.174)	10.04 (0.207)
		LW	0.724 (0.007)	3.577 (0.065)	0.844 (0.014)	12.59 (0.136)	10.33 (0.088)
		Prop	0.670 (0.009)	2.923 (0.055)	0.607 (0.014)	9.712 (0.155)	8.993 (0.180)
	p=100	$\mathit{S}$	2.118 (0.067)	17.11 (0.602)	5.201 (0.489)	–	–
		BT	1.001 (0.091)	4.792 (0.880)	2.267 (0.540)	54.87 (7.447)	171.2 (35.10)
		MX	1.884 (0.065)	15.15 (0.580)	4.396 (0.443)	–	–
		AVE	0.849 (0.046)	4.847 (0.395)	0.985 (0.177)	24.06 (0.858)	23.54 (2.334)
		LW	0.813 (0.010)	5.314 (0.138)	1.057 (0.021)	29.64 (0.245)	24.34 (0.141)
		Prop	0.769 (0.042)	4.374 (0.356)	0.889 (0.159)	21.73 (0.764)	20.99 (2.019)
II	p=30	$\mathit{S}$	3.319 (0.048)	17.49 (0.268)	1.966 (0.056)	29.83 (0.568)	174.8 (5.806)
		BT	3.248 (0.057)	17.08 (0.326)	1.916 (0.057)	28.27 (0.618)	148.6 (5.208)
		MX	3.032 (0.050)	15.91 (0.277)	1.622 (0.053)	19.10 (0.461)	57.03 (2.087)
		AVE	3.326 (0.065)	17.48 (0.363)	1.806 (0.054)	18.67 (0.562)	43.60 (1.919)
		LW	3.313 (0.048)	17.42 (0.262)	1.621 (0.042)	14.95 (0.384)	35.15 (1.370)
		Prop	3.167 (0.062)	16.65 (0.346)	1.719 (0.051)	17.76 (0.536)	41.42 (1.826)
	p=50	$\mathit{S}$	4.503 (0.073)	31.04 (0.531)	3.126 (0.076)	–	–
		BT	4.296 (0.123)	29.49 (0.913)	2.981 (0.108)	68.58 (1.981)	581.7 (25.29)
		MX	4.059 (0.074)	27.90 (0.539)	2.541 (0.071)	–	–
		AVE	4.514 (0.102)	31.08 (0.735)	2.900 (0.090)	34.08 (1.094)	81.40 (3.863)
		LW	4.460 (0.081)	30.77 (0.575)	2.694 (0.077)	27.79 (0.764)	66.44 (2.889)
		Prop	4.298 (0.097)	29.59 (0.700)	2.760 (0.086)	32.42 (1.043)	77.35 (3.679)
	p=100	$\mathit{S}$	6.444 (0.091)	63.32 (0.947)	5.922 (0.145)	–	–
		BT	–	–	–	–	–
		MX	6.143 (0.095)	61.29 (0.980)	5.147 (0.137)	–	–
		AVE	6.348 (0.149)	62.31 (1.523)	5.285 (0.174)	68.50 (2.270)	161.8 (8.540)
		LW	6.413 (0.097)	63.34 (0.968)	5.076 (0.135)	56.57 (1.354)	128.0 (5.059)
		Prop	6.043 (0.142)	59.31 (1.452)	5.029 (0.166)	65.14 (2.163)	153.7 (8.125)

4. Application

In this section, we apply the proposed method in an application of portfolio optimization. The problem is formulated as

$$\begin{array}{rl}& \underset{\mathit{w}}{\text{min}}{\mathit{w}}^{\prime }\hat{\mathbf{\Sigma }}\mathit{w}\\ & \text{subject to}\sum _{i=1}^p{w}_i=1,\end{array}$$ (6)

where $\mathit{w}=(w_1,\dots ,w_p{)}^{\prime }$ is a portfolio, and $\hat{\mathbf{\Sigma }}$ is an estimate for the covariance matrix $\mathbf{\Sigma }$ of asset returns. We expect that an accurate estimate $\hat{\mathbf{\Sigma }}$ will result in a better portfolio strategy.

Now we evaluate the performance of the proposed method by analyzing real data of 97 stocks from the S$\mathrm{\&}$P100. This data set contains n=156 stock returns weekly recorded from 4 January 2010 to 12 December 2012. We use the first k observations, $k=100,101,\dots ,155$, to obtain the covariance matrix estimate ${\hat{\mathbf{\Sigma }}}_k$ from the proposed method, the sample covariance matrix, the BT method, the MX method, the AVE method and the LW method. Then the realized return ${\hat{\mathit{w}}}_k^{\prime }{\mathit{x}}_{k+1}$ at time k+1 is computed. Here the optimal portfolio ${\hat{\mathit{w}}}_k$ is calculated with $\hat{\mathbf{\Sigma }}$ replaced by ${\hat{\mathbf{\Sigma }}}_k$ in (6). We report the performance measures of the average annual realized return

$$\text{AVR}=\frac{1}{56}\sum _{k=100}^{155}52\ast {\hat{\mathit{w}}}_k^{\prime }{\mathit{x}}_{k+1},$$

their standard deviation (SD) and the information ratio AVG/SD. Note that the objective function (6) is to minimize the variance rather than maximize the realized returns. Hence, the performance of each method should be primarily evaluated by whether it could produce a small SD. A large value of AVR and AV/SD are also surely desirable as a secondary importance in evaluating the performance of each method.

Table 6 displays the results of portfolio performance for each approach. We can see that both the proposed method and the AVE produce the smallest SD, but the proposed method has a relatively large AVR. Moreover, the sample covariance matrix $\mathit{S}$, the BT method and the proposed method give the comparable AVR, but the proposed method is more reliable with a much smaller SD, hence, leading to the larger information ratio AVR/SD. In addition, the AVE and LW also show a good performance, indicated by smaller SD and larger AVR/SD than the $\mathit{S}$, the BT and the MX methods. Overall, the proposed method performs well in the portfolio selection for this set of 97 stock data. It is able to give a smaller SD and slightly larger AVR than other approaches.

Table 6. The comparison of the portfolio performance measures for 97 stock returns.

	$\mathit{S}$	BT	MX	AVE	LW	Prop
AVR	12.37	12.04	9.323	10.60	9.746	12.60
SD	19.64	15.20	11.34	8.474	9.303	8.473
AVR/SD	0.630	0.792	0.822	1.251	1.048	1.488

5. Conclusion

In this paper, we have proposed a new method for estimating the covariance matrix based on the MCD framework under the loss function of the Frobenius norm. The MCD enables us to obtain a set of estimates for the covariance matrix by permutation of variable orderings, based on which the proposed estimator is constructed following the idea of Ledoit and Wolf [18]. Simulation study indicates that the proposed estimator performs well with respect to some commonly used loss functions. It is worth noting that the proposed method does not provide a sparse estimate. In view of this, a penalty function may be imposed to the off-diagonal entries of the covariance matrix. In addition, the MCD technique does not need the normality assumption on data to provide a covariance matrix estimate. Many papers on the Cholesky-based estimation for the covariance matrix assume normal data for the reason of theoretical proof. It is an interesting topic to study our proposed estimate if the normal assumption is relaxed. We may need different assumptions instead of (C2) and (C3) to derive the convergence rate of ${\tilde{\mathbf{\Sigma }}}_k^{\left(n\right)}$ in Lemma 2.1, such as moment conditions of variable $\mathit{X}$, or proper tail condition for the distribution of data. Based on the rate of ${\tilde{\mathbf{\Sigma }}}_k^{\left(n\right)}$, it is easy to derive the convergence rate of ${\overline{\mathbf{\Sigma }}}^{\left(n\right)}$ as in Lemma 2.2. If this new derived rate is converged, Theorem 2.3 would hold, and the proposed estimate is valid. This is left as a topic for the future research.

Funding Statement

Xiaoning Kang's research was supported by the Science Education Foundation of Liaoning Province (LN2019Q21), and the National Natural Science Foundation of China (71871047). Chaoping Xie's research was supported by the National Natural Science Foundation of China (71903090). Mingqiu Wang's research was supported by the National Natural Science Foundation of China (11771250), and the Natural Science Foundation of Shandong Province (ZR2019MA002).

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

Xiaoning Kang http://orcid.org/0000-0003-0394-6240