Rate Distortion Function of Gaussian Asymptotically WSS Vector Processes

Abstract

In this paper, we obtain an integral formula for the rate distortion function (RDF) of any Gaussian asymptotically wide sense stationary (AWSS) vector process. Applying this result, we also obtain an integral formula for the RDF of Gaussian moving average (MA) vector processes and of Gaussian autoregressive MA (ARMA) AWSS vector processes.

1. Introduction

The present paper focuses on the derivation of a closed-form expression for the rate distortion function (RDF) of a wide class of vector processes. As stated in [1,2], there exist very few journal papers in the literature that present closed-form expressions for the RDF of non-stationary processes, and just one of them deals with non-stationary vector processes [3]. In the present paper, we obtain an integral formula for the RDF of any real Gaussian asymptotically wide sense stationary (AWSS) vector process. This new formula generalizes the one given in 1956 by Kolmogorov [4] for real Gaussian stationary processes and the one given in 1971 by Toms and Berger [3] for real Gaussian autoregressive (AR) AWSS vector processes of finite order. Applying this new formula, we also obtain an integral formula for the RDF of real Gaussian moving average (MA) vector processes of infinite order and for the RDF of real Gaussian ARMA AWSS vector processes of infinite order. AR, MA and ARMA vector processes are frequently used to model multivariate time series (see, e.g., [5]).

The definition of the AWSS process was first given by Gray (see [6,7]), and it is based on his concept of asymptotically equivalent sequences of matrices [8]. The integral formulas given in the present paper are obtained by using some recent results on such sequences of matrices [9,10,11,12].

The paper is organized as follows. In Section 2, we set up notation, and we review the concepts of AWSS, MA and ARMA vector processes and the Kolmogorov formula for the RDF of a real Gaussian vector. In Section 3, we obtain an integral formula for the RDF of any Gaussian AWSS vector process. In Section 4, we obtain an integral formula for the RDF of Gaussian MA vector processes and of Gaussian ARMA AWSS vector processes. We finish the paper with a numerical example where the RDF of a Gaussian AWSS vector process is computed.

2. Preliminaries

2.1. Notation

In this paper, $\mathbb{N}$, $\mathbb{Z}$, $\mathbb{R}$ and $\mathbb{C}$ denote the set of natural numbers (i.e., the set of positive integers), the set of integer numbers, the set of (finite) real numbers and the set of (finite) complex numbers, respectively. If $m,n\in \mathbb{N}$, then ${\mathbb{C}}^{m\times n}$, $0_{m\times n}$ and $I_n$ are the set of all $m\times n$ complex matrices, the $m\times n$ zero matrix and the $n\times n$ identity matrix, respectively. The symbols ⊤ and ∗ denote transpose and conjugate transpose, respectively. E stands for expectation; i is the imaginary unit; $tr$ denotes trace; $\delta$ stands for the Kronecker delta; and ${\lambda }_k\left(A\right)$, $k\in \{1,\dots ,n\}$, are the eigenvalues of an $n\times n$ Hermitian matrix A arranged in decreasing order.

Let $A_n$ and $B_n$ be $nN\times nN$ matrices for all $n\in \mathbb{N}$. We write $\left\{A_n\right\}\sim \left\{B_n\right\}$ if the sequences $\left\{A_n\right\}$ and $\left\{B_n\right\}$ are asymptotically equivalent (see ([9], p. 5673)), that is:

$$\exists M\in [0,\infty ):\parallel A_n{\parallel }_2,{\parallel B_n\parallel }_2\le M\forall n\in \mathbb{N}$$

and:

$$\underset{n\to \infty }{\lim }\frac{\parallel A_n-B_n{\parallel }_F}{\sqrt{n}}=0,$$

where ${\parallel ·\parallel }_2$ and ${\parallel ·\parallel }_F$ denote the spectral norm and the Frobenius norm, respectively. The original definition of asymptotically equivalent sequences of matrices, where $N=1$, was given by Gray (see ([6], Section 2.3) or [8]).

Let $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$ be a random N-dimensional vector process, i.e., ${\mathbf{x}}_n$ is a random (column) vector of dimension N for all $n\in \mathbb{N}$. We denote by ${\mathbf{x}}_{n:1}$ the random vector of dimension $nN$ given by:

$${\mathbf{x}}_{n:1}:=\left(\begin{array}{c}{\mathbf{x}}_n\\ {\mathbf{x}}_{n-1}\\ {\mathbf{x}}_{n-2}\\ ⋮\\ {\mathbf{x}}_1\end{array}\right),n\in \mathbb{N}.$$

Consider a matrix-valued function of a real variable $X:\mathbb{R}\to {\mathbb{C}}^{N\times N}$, which is continuous and $2\pi$-periodic. For every $n\in \mathbb{N}$, we denote by $T_n\left(X\right)$ the $n\times n$ block Toeplitz matrix with $N\times N$ blocks given by:

$$T_n\left(X\right):={\left({\mathrm{X}}_{j-k}\right)}_{j,k=1}^n=\left(\begin{array}{ccccc}{\mathrm{X}}_0& {\mathrm{X}}_{-1}& {\mathrm{X}}_{-2}& \cdots & {\mathrm{X}}_{1-n}\\ {\mathrm{X}}_1& {\mathrm{X}}_0& {\mathrm{X}}_{-1}& \cdots & {\mathrm{X}}_{2-n}\\ {\mathrm{X}}_2& {\mathrm{X}}_1& {\mathrm{X}}_0& \cdots & {\mathrm{X}}_{3-n}\\ ⋮& ⋮& ⋮& \ddots & ⋮\\ {\mathrm{X}}_{n-1}& {\mathrm{X}}_{n-2}& {\mathrm{X}}_{n-3}& \cdots & {\mathrm{X}}_0\end{array}\right),$$

where ${\left\{{\mathrm{X}}_k\right\}}_{k\in \mathbb{Z}}$ is the sequence of Fourier coefficients of X:

$${\mathrm{X}}_k=\frac{1}{2\pi }{\int }_0^{2\pi }{\mathrm{e}}^{-k\omega \mathrm{i}}X\left(\omega \right)d\omega \forall k\in \mathbb{Z}.$$

2.2. AWSS Vector Processes

We first review the well-known concept of the WSS vector process.

Definition 1.

Let $X:\mathbb{R}\to {\mathbb{C}}^{N\times N}$, and suppose that it is continuous and $2\pi$-periodic. A random N-dimensional vector process $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$ is said to be WSS (or weakly stationary) with power spectral density (PSD) X if it has constant mean (i.e., $E\left({\mathbf{x}}_{n_1}\right)=E\left({\mathbf{x}}_{n_2}\right)$ for all $n_1,n_2\in \mathbb{N})$ and $\left\{E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\ast }\right)\right\}=\left\{T_n\left(X\right)\right\}$.

We now review the definition of the AWSS vector process given in ([11], Definition 7.1).

Definition 2.

Let $X:\mathbb{R}\to {\mathbb{C}}^{N\times N}$, and suppose that it is continuous and $2\pi$-periodic. A random N-dimensional vector process $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$ is said to be AWSS with asymptotic PSD (APSD) X if it has constant mean and $\left\{E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\ast }\right)\right\}\sim \left\{T_n\left(X\right)\right\}$.

Definition 2 was first introduced by Gray for the case $N=1$ (see, e.g., ([6], p. 225)).

2.3. MA and ARMA Vector Processes

We first review the concept of real zero-mean MA vector process (of infinite order).

Definition 3.

A real zero-mean random N-dimensional vector process $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$ is said to be MA if:

$${\mathbf{x}}_n={\mathbf{w}}_n+\sum _{j=1}^{n-1}{\mathrm{G}}_{-j}{\mathbf{w}}_{n-j}\forall n\in \mathbb{N},$$ (1)

where ${\mathrm{G}}_{-j}$, $j\in \mathbb{N}$, are real $N\times N$ matrices, $\{{\mathbf{w}}_n:n\in \mathbb{N}\}$ is a real zero-mean random N-dimensional vector process and $E\left({\mathbf{w}}_{n_1}{\mathbf{w}}_{n_2}^{\top }\right)={\delta }_{n_1,n_2}\mathsf{\Lambda }$ for all $n_1,n_2\in \mathbb{N}$ with Λ being an $N\times N$ positive definite matrix.

The MA vector process $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$ in Equation (1) is of finite order if there exists $q\in \mathbb{N}$ such that ${\mathrm{G}}_{-j}=0_{N\times N}$ for all $j>q$. In this case, $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$ is called an MA$\left(q\right)$ vector process (see, e.g., ([5], Section 2.1)).

Secondly, we review the concept of a real zero-mean ARMA vector process (of infinite order).

Definition 4.

A real zero-mean random N-dimensional vector process $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$ is said to be ARMA if:

$${\mathbf{x}}_n={\mathbf{w}}_n+\sum _{j=1}^{n-1}{\mathrm{G}}_{-j}{\mathbf{w}}_{n-j}-\sum _{j=1}^{n-1}{\mathrm{F}}_{-j}{\mathbf{x}}_{n-j}\forall n\in \mathbb{N},$$ (2)

where ${\mathrm{G}}_{-j}$ and ${\mathrm{F}}_{-j}$, $j\in \mathbb{N}$, are real $N\times N$ matrices, $\{{\mathbf{w}}_n:n\in \mathbb{N}\}$ is a real zero-mean random N-dimensional vector process and $E\left({\mathbf{w}}_{n_1}{\mathbf{w}}_{n_2}^{\top }\right)={\delta }_{n_1,n_2}\mathsf{\Lambda }$ for all $n_1,n_2\in \mathbb{N}$ with Λ being an $N\times N$ positive definite matrix.

The ARMA vector process $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$ in Equation (2) is of finite order if there exist $p,q\in \mathbb{N}$ such that ${\mathrm{F}}_{-j}=0_{N\times N}$ for all $j>p$ and ${\mathrm{G}}_{-j}=0_{N\times N}$ for all $j>q$. In this case, $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$ is called an ARMA$(p,q)$ vector process (see, e.g., ([5], Section 1.2.2)).

2.4. RDF of Gaussian Vectors

Let $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$ be a real zero-mean Gaussian N-dimensional vector process satisfying that $E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)$ is positive definite for all $n\in \mathbb{N}$. If $n\in \mathbb{N}$ from [4], we know that the RDF of the real zero-mean Gaussian vector ${\mathbf{x}}_{n:1}$ is given by:

$$R_n\left(D\right)=\frac{1}{nN}\sum _{k=1}^{nN}\max \left\{0,\frac{1}{2}ln\frac{{\lambda }_k\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)}{{\theta }_n}\right\}$$ (3)

with $D\in \left(0,\frac{\mathrm{tr}\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)}{nN}\right)$ and where ${\theta }_n$ is the real number satisfying:

$$D=\frac{1}{nN}\sum _{k=1}^{nN}\min \left\{{\theta }_n,{\lambda }_k\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)\right\}.$$

The RDF of the real zero-mean Gaussian vector process $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$ is given by:

$$R\left(D\right):=\underset{n\to \infty }{\lim }{R}_n\left(D\right)$$

whenever this limit exists.

3. Integral Formula for the RDF of Gaussian AWSS Vector Processes

Theorem 1.

Let $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$ be a real zero-mean Gaussian AWSS N-dimensional vector process with APSD X. Suppose that $X\left(\omega \right)$ is positive definite for all $\omega \in \mathbb{R}$ and that $E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)$ is positive definite for all $n\in \mathbb{N}$. If $D\in \left(0,\frac{\mathrm{tr}\left({\mathrm{X}}_0\right)}{N}\right)$, then:

$$R\left(D\right)=\frac{1}{4\pi N}{\int }_0^{2\pi }\sum _{k=1}^N\max \left\{0,ln\frac{{\lambda }_k\left(X\left(\omega \right)\right)}{\theta }\right\}d\omega$$ (4)

is the operational RDF of $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$, where θ is the real number satisfying:

$$D=\frac{1}{2\pi N}{\int }_0^{2\pi }\sum _{k=1}^N\min \left\{\theta ,{\lambda }_k\left(X\left(\omega \right)\right)\right\}d\omega .$$ (5)

Proof. 

See Appendix A. ☐

Corollary 1.

Let $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$ be a real zero-mean Gaussian WSS N-dimensional vector process with PSD X. Suppose that $X\left(\omega \right)$ is positive definite for all $\omega \in \mathbb{R}$. If $D\in \left(0,\frac{\mathrm{tr}\left({\mathrm{X}}_0\right)}{N}\right)$, then:

$$R\left(D\right)=\frac{1}{4\pi N}{\int }_0^{2\pi }\sum _{k=1}^N\max \left\{0,ln\frac{{\lambda }_k\left(X\left(\omega \right)\right)}{\theta }\right\}d\omega ,$$ (6)

where θ is the real number satisfying:

$$D=\frac{1}{2\pi N}{\int }_0^{2\pi }\sum _{k=1}^N\min \left\{\theta ,{\lambda }_k\left(X\left(\omega \right)\right)\right\}d\omega .$$

Proof. 

See Appendix B. ☐

The integral formula given in Equation (6) was presented by Kafedziski in ([13], Equation (20)). However, the proof that he proposed was not complete, because although Kafedziski pointed out that ([13], Equation (20)) can be directly proven by applying the Szegö theorem for block Toeplitz matrices ([14], Theorem 3), the Szegö theorem cannot be applied since the parameter $\theta$ that appears in the expression of $R_n\left(D\right)$ in ([13], Equation (7)), depends on n, as it does in Equation (3). It should be also mentioned that the set of WSS vector processes that he considered was smaller, namely, he only considered WSS vector processes with PSD in the Wiener class. A function $X:\mathbb{R}\to {\mathbb{C}}^{N\times N}$ is said to be in the Wiener class if it is continuous and $2\pi$-periodic, and it satisfies ${\sum }_{k=-\infty }^{\infty }\left|{\left[{\mathrm{X}}_k\right]}_{r,s}\right|<\infty$ for all $r,s\in \{1,\dots ,N\}$ (see, e.g., ([11], Appendix B)).

4. Applications

4.1. Integral Formula for the RDF of Gaussian MA Vector Processes

Theorem 2.

Let $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$ be as in Definition 3. Assume that ${\left\{{\mathrm{G}}_k\right\}}_{k=-\infty }^{\infty }$, with ${\mathrm{G}}_0=I_N$ and ${\mathrm{G}}_k=0_{N\times N}$ for all $k>0$, is the sequence of Fourier coefficients of a function $G:\mathbb{R}\to {\mathbb{C}}^{N\times N}$, which is continuous and $2\pi$-periodic. Then:

• 1.  $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$ is AWSS with APSD $X\left(\omega \right)=G\left(\omega \right)\mathsf{\Lambda }{\left(G\left(\omega \right)\right)}^{\ast }$ for all $\omega \in \mathbb{R}$.
• 2. 
  If $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$ is Gaussian, $\det \left(G\right(\omega \left)\right)\ne 0$ for all $\omega \in \mathbb{R}$, and $D\in \left(0,\frac{\mathrm{tr}\left({\mathrm{X}}_0\right)}{N}\right)$ yields

  $$R\left(D\right)=\frac{1}{4\pi N}{\int }_0^{2\pi }\sum _{k=1}^N\max \left\{0,ln\frac{{\lambda }_k\left(G\left(\omega \right)\mathsf{\Lambda }{\left(G\left(\omega \right)\right)}^{\ast }\right)}{\theta }\right\}d\omega ,$$

  where θ is the real number satisfying:

  $$D=\frac{1}{2\pi N}{\int }_0^{2\pi }\sum _{k=1}^N\min \left\{\theta ,{\lambda }_k\left(G\left(\omega \right)\mathsf{\Lambda }{\left(G\left(\omega \right)\right)}^{\ast }\right)\right\}d\omega .$$

Proof. 

See Appendix C. ☐

4.2. Integral Formula for the RDF of Gaussian ARMA AWSS Vector Processes

Theorem 3.

Let $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$ be as in Definition 4. Assume that ${\left\{{\mathrm{G}}_k\right\}}_{k=-\infty }^{\infty }$, with ${\mathrm{G}}_0=I_N$ and ${\mathrm{G}}_k=0_{N\times N}$ for all $k>0$, is the sequence of Fourier coefficients of a function $G:\mathbb{R}\to {\mathbb{C}}^{N\times N}$, which is continuous and $2\pi$-periodic. Suppose that ${\left\{{\mathrm{F}}_k\right\}}_{k=-\infty }^{\infty }$, with ${\mathrm{F}}_0=I_N$ and ${\mathrm{F}}_k=0_{N\times N}$ for all $k>0$, is the sequence of Fourier coefficients of a function $F:\mathbb{R}\to {\mathbb{C}}^{N\times N}$, which is continuous and $2\pi$-periodic. Assume that $\{\parallel {\left(T_n\left(F\right)\right)}^{-1}{\parallel }_2\}$ is bounded and $\det \left(F\right(\omega \left)\right)\ne 0$ for all $\omega \in \mathbb{R}$. Then:

• 1.  $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$ is AWSS with APSD $X\left(\omega \right)={\left(F\left(\omega \right)\right)}^{-1}G\left(\omega \right)\mathsf{\Lambda }{\left({\left(F\left(\omega \right)\right)}^{-1}G\left(\omega \right)\right)}^{\ast }$ for all $\omega \in \mathbb{R}$.
• 2. 
  If $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$ is Gaussian, $\det \left(G\right(\omega \left)\right)\ne 0$ for all $\omega \in \mathbb{R}$, and $D\in \left(0,\frac{\mathrm{tr}\left({\mathrm{X}}_0\right)}{N}\right)$ yields:

  $$R\left(D\right)=\frac{1}{4\pi N}{\int }_0^{2\pi }\sum _{k=1}^N\max \left\{0,ln\frac{{\lambda }_k\left({\left(F\left(\omega \right)\right)}^{-1}G\left(\omega \right)\mathsf{\Lambda }{\left({\left(F\left(\omega \right)\right)}^{-1}G\left(\omega \right)\right)}^{\ast }\right)}{\theta }\right\}d\omega ,$$

  where θ is the real number satisfying:

  $$D=\frac{1}{2\pi N}{\int }_0^{2\pi }\sum _{k=1}^N\min \left\{\theta ,{\lambda }_k\left({\left(F\left(\omega \right)\right)}^{-1}G\left(\omega \right)\mathsf{\Lambda }{\left({\left(F\left(\omega \right)\right)}^{-1}G\left(\omega \right)\right)}^{\ast }\right)\right\}d\omega .$$

Proof. 

See Appendix D. ☐

5. Numerical Example

We finish the paper with a numerical example where the RDF of a Gaussian AWSS vector process is computed. Specifically, we compute the RDF of the MA$\left(1\right)$ vector process considered in ([5], Example 2.1), by assuming that it is Gaussian.

Let $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$ be as in Definition 3 with $N=2$,

$$G_{-1}=\left(\begin{array}{cc}-0.8& -0.7\\ 0.4& -0.6\end{array}\right),$$

${\mathrm{G}}_{-j}=0_{2\times 2}$ for all $j>1$, and:

$$\mathsf{\Lambda }=\left(\begin{array}{cc}4& 1\\ 1& 2\end{array}\right).$$

Assume that $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$ is Gaussian. Figure 1 shows $R\left(D\right)$ with $D\in (0,5.77)$ that we have computed using Theorem 2.

Figure 1.

Rate Distortion Function (RDF) of the Gaussian MA vector process considered.

Appendix A. Proof of Theorem 1

Proof. 

We divide the proof into six steps.

Step 1: We show that there exists $n_0\in \mathbb{N}$ such that ${\theta }_n$ in Equation (3) exists for all $n\ge n_0$, or equivalently, such that $D\in \left(0,\frac{\mathrm{tr}\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)}{nN}\right)$ for all $n\ge n_0$.

Since $\left\{E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right\}=\left\{E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\ast }\right)\right\}\sim \left\{T_n\left(X\right)\right\}$, applying ([11], Theorem 6.6) yields:

$$\begin{array}{cc}\hfill \underset{n\to \infty }{\lim }\frac{1}{nN}\sum _{k=1}^{nN}{\lambda }_k\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)& =\frac{1}{2\pi N}{\int }_0^{2\pi }\sum _{k=1}^N{\lambda }_k\left(X\left(\omega \right)\right)d\omega =\frac{1}{2\pi N}{\int }_0^{2\pi }\mathrm{tr}\left(X\left(\omega \right)\right)d\omega \hfill \\ \hfill & =\frac{1}{2\pi N}{\int }_0^{2\pi }\sum _{k=1}^N{\left[X\left(\omega \right)\right]}_{k,k}d\omega =\frac{1}{2\pi N}\sum _{k=1}^N{\int }_0^{2\pi }{\left[X\left(\omega \right)\right]}_{k,k}d\omega \hfill \\ \hfill & =\frac{1}{2\pi N}\sum _{k=1}^N{\left[{\int }_0^{2\pi }X\left(\omega \right)d\omega \right]}_{k,k}=\frac{1}{2\pi N}\mathrm{tr}\left({\int }_0^{2\pi }X\left(\omega \right)d\omega \right)\hfill \\ \hfill & =\frac{1}{N}\mathrm{tr}\left(\frac{1}{2\pi }{\int }_0^{2\pi }X\left(\omega \right)d\omega \right)=\frac{\mathrm{tr}\left({\mathrm{X}}_0\right)}{N}.\hfill \end{array}$$ (A1)

Consequently, as $D\in \left(0,\frac{\mathrm{tr}\left({\mathrm{X}}_0\right)}{N}\right)$, there exists $n_0\in \mathbb{N}$ such that:

$$\left|\frac{1}{nN}\sum _{k=1}^{nN}{\lambda }_k\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)-\frac{\mathrm{tr}\left({\mathrm{X}}_0\right)}{N}\right|<\frac{\mathrm{tr}\left({\mathrm{X}}_0\right)}{N}-D\forall n\ge n_0.$$

Therefore, since:

$$\frac{\mathrm{tr}\left({\mathrm{X}}_0\right)}{N}-\frac{1}{nN}\sum _{k=1}^{nN}{\lambda }_k\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)\le \left|\frac{1}{nN}\sum _{k=1}^{nN}{\lambda }_k\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)-\frac{\mathrm{tr}\left({\mathrm{X}}_0\right)}{N}\right|\forall n\ge n_0,$$

we obtain:

$$D<\frac{1}{nN}\sum _{k=1}^{nN}{\lambda }_k\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)=\frac{\mathrm{tr}\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)}{nN}\forall n\ge n_0.$$ (A2)

Step 2: We prove that the sequence of real numbers ${\{{\theta }_n\}}_{n\ge n_0}$ is bounded.

From Equation (A2), we have ${\theta }_n<{\lambda }_1\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)$ for all $n\ge n_0$. As $\left\{E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right\}\sim \left\{T_n\left(X\right)\right\}$, there exists $M\in [0,\infty )$ such that $\parallel E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right){\parallel }_2,{\parallel T_n\left(X\right)\parallel }_2\le M$ for all $n\in \mathbb{N}$. Thus,

$$\begin{array}{c}0<D=\frac{1}{nN}\sum _{k=1}^{nN}\min \left\{{\theta }_n,{\lambda }_k\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)\right\}\le \frac{1}{nN}\sum _{k=1}^{nN}{\theta }_n\hfill \\ ={\theta }_n<{\lambda }_1\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)={\parallel E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\parallel }_2\le M\forall n\ge n_0.\hfill \end{array}$$

Step 3: We show that if $\left\{{\theta }_{\sigma \left(n\right)}\right\}$ is a convergent subsequence of ${\left\{{\theta }_n\right\}}_{n\ge n_0}$, then ${\lim }_{n\to \infty }{\theta }_{\sigma \left(n\right)}=\theta$.

We denote by $\widehat{\theta }$ the limit of $\left\{{\theta }_{\sigma \left(n\right)}\right\}$. We need to prove that $\widehat{\theta }=\theta$.

Since $0<D\le {\theta }_n$ for all $n\ge n_0$, we have $0<D\le \widehat{\theta }$. Let $\{{\widehat{\theta }}_n\}$ be the sequence of real numbers such that $\{{\widehat{\theta }}_{\sigma \left(n\right)}\}=\{{\theta }_{\sigma \left(n\right)}\}$ and ${\widehat{\theta }}_n=\widehat{\theta }$ for all $n\in \mathbb{N}\backslash \sigma \left(\mathbb{N}\right)$. Obviously, ${\lim }_{n\to \infty }{\widehat{\theta }}_n=\widehat{\theta }$ and $0<{\widehat{\theta }}_n$ for all $n\in \mathbb{N}$. As ${\lim }_{n\to \infty }\frac{1}{{\widehat{\theta }}_n}=\frac{1}{\widehat{\theta }}$ and $\frac{\mathrm{rank}\left(\mathrm{E}\left({\mathbf{x}}_{\mathrm{n}:1}{\mathbf{x}}_{\mathrm{n}:1}^{\top }\right)\right)}{n}=N$ for all $n\in \mathbb{N}$, applying ([12], Lemma 1) yields $\left\{\frac{1}{{\widehat{\theta }}_n}E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right\}\sim \left\{\frac{1}{\widehat{\theta }}{T}_n\left(X\right)\right\}$. From ([11], Lemma 4.2) we obtain $\left\{\frac{1}{{\widehat{\theta }}_n}E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right\}\sim \left\{\frac{1}{\widehat{\theta }}{T}_n\left(X\right)\right\}=\left\{T_n\left(\frac{1}{\widehat{\theta }}X\right)\right\}$. Hence, applying ([11], Theorem 6.6) yields:

$$\begin{array}{cc}\hfill D& =\underset{n\to \infty }{\lim }\frac{1}{\sigma \left(n\right)N}\sum _{k=1}^{\sigma \left(n\right)N}\min \left\{{\theta }_{\sigma \left(n\right)},{\lambda }_k\left(E\left({\mathbf{x}}_{\sigma \left(n\right):1}{\mathbf{x}}_{\sigma \left(n\right):1}^{\top }\right)\right)\right\}\hfill \\ \hfill & =\underset{n\to \infty }{\lim }{\theta }_{\sigma \left(n\right)}\frac{1}{\sigma \left(n\right)N}\sum _{k=1}^{\sigma \left(n\right)N}\min \left\{1,\frac{{\lambda }_k\left(E\left({\mathbf{x}}_{\sigma \left(n\right):1}{\mathbf{x}}_{\sigma \left(n\right):1}^{\top }\right)\right)}{{\theta }_{\sigma \left(n\right)}}\right\}\hfill \\ \hfill & =\widehat{\theta }\underset{n\to \infty }{\lim }\frac{1}{\sigma \left(n\right)N}\sum _{k=1}^{\sigma \left(n\right)N}\min \left\{1,\frac{{\lambda }_k\left(E\left({\mathbf{x}}_{\sigma \left(n\right):1}{\mathbf{x}}_{\sigma \left(n\right):1}^{\top }\right)\right)}{{\widehat{\theta }}_{\sigma \left(n\right)}}\right\}\hfill \\ \hfill & =\widehat{\theta }\underset{n\to \infty }{\lim }\frac{1}{nN}\sum _{k=1}^{nN}\min \left\{1,\frac{{\lambda }_k\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)}{{\widehat{\theta }}_n}\right\}\hfill \\ \hfill & =\widehat{\theta }\underset{n\to \infty }{\lim }\frac{1}{nN}\sum _{k=1}^{nN}\min \left\{1,{\lambda }_k\left(\frac{1}{{\widehat{\theta }}_n}E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)\right\}\hfill \\ \hfill & =\widehat{\theta }\frac{1}{2\pi N}{\int }_0^{2\pi }\sum _{k=1}^N\min \left\{1,{\lambda }_k\left(\frac{1}{\widehat{\theta }}X\left(\omega \right)\right)\right\}d\omega \hfill \\ \hfill & =\widehat{\theta }\frac{1}{2\pi N}{\int }_0^{2\pi }\sum _{k=1}^N\min \left\{1,\frac{{\lambda }_k\left(X\left(\omega \right)\right)}{\widehat{\theta }}\right\}d\omega \hfill \\ \hfill & =\frac{1}{2\pi N}{\int }_0^{2\pi }\sum _{k=1}^N\min \left\{\widehat{\theta },{\lambda }_k\left(X\left(\omega \right)\right)\right\}d\omega .\hfill \end{array}$$

Thus, $\widehat{\theta }$ is a real number satisfying Equation (5). Since $D<\frac{\mathrm{tr}\left({\mathrm{X}}_0\right)}{N}=\frac{1}{2\pi N}{\int }_0^{2\pi }{\sum }_{k=1}^N{\lambda }_k\left(X\left(\omega \right)\right)d\omega$, there exists a unique real number $\theta$ satisfying Equation (5), and consequently, $\widehat{\theta }=\theta$.

Step 4: We prove that ${\lim }_{n\to \infty }{\theta }_n=\theta$. From Steps 2 and 3, we have ${\lim \inf }_{n\to \infty }{\theta }_n={\lim \sup }_{n\to \infty }{\theta }_n=\theta$. Consequently, the sequence of real numbers ${\{{\theta }_n\}}_{n\ge n_0}$ is convergent, and its limit is $\theta$ (see, e.g., ([15], p. 57)).

Step 5: We show that Equation (4) holds.

Let $\left\{{\widehat{\theta }}_n\right\}$ be the sequence of positive numbers defined in Step 3 for the case in which $\left\{\sigma \left(n\right)\right\}=\{n+n_0-1\}$, that is, ${\widehat{\theta }}_n={\theta }_n$ if $n\ge n_0$ and ${\widehat{\theta }}_n=\theta$ if $n<n_0$. From ([11], Theorem 6.6), we obtain:

$$\begin{array}{cc}\hfill R\left(D\right)& =\underset{n\to \infty }{\lim }{R}_n\left(D\right)=\underset{n\to \infty }{\lim }\frac{1}{nN}\sum _{k=1}^{nN}\max \left\{0,\frac{1}{2}ln\frac{{\lambda }_k\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)}{{\theta }_n}\right\}\hfill \\ \hfill & =\underset{n\to \infty }{\lim }\frac{1}{nN}\sum _{k=1}^{nN}\frac{1}{2}ln\max \left\{1,\frac{{\lambda }_k\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)}{{\theta }_n}\right\}=\frac{1}{2}\underset{n\to \infty }{\lim }\frac{1}{nN}\sum _{k=1}^{nN}ln\max \left\{1,\frac{{\lambda }_k\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)}{{\widehat{\theta }}_n}\right\}\hfill \\ \hfill & =\frac{1}{2}\underset{n\to \infty }{\lim }\frac{1}{nN}\sum _{k=1}^{nN}ln\max \left\{1,{\lambda }_k\left(\frac{1}{{\widehat{\theta }}_n}E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)\right\}=\frac{1}{4\pi N}{\int }_0^{2\pi }\sum _{k=1}^{N}ln\max \left\{1,{\lambda }_k\left(\frac{1}{\theta }X\left(\omega \right)\right)\right\}d\omega \hfill \\ \hfill & =\frac{1}{4\pi N}{\int }_0^{2\pi }\sum _{k=1}^{N}ln\max \left\{1,\frac{{\lambda }_k\left(X\left(\omega \right)\right)}{\theta }\right\}d\omega =\frac{1}{4\pi N}{\int }_0^{2\pi }\sum _{k=1}^N\max \left\{0,ln\frac{{\lambda }_k\left(X\left(\omega \right)\right)}{\theta }\right\}d\omega .\hfill \end{array}$$

Step 6: We prove that Equation (4) is the operational RDF of $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$. Following the same arguments that Gray used in [16] for Gaussian AR AWSS one-dimensional vector processes, to prove the negative (converse) coding theorem and the positive (achievability) coding theorem, we only need to show that the sequence $\left\{d_{\max }^{\ast }\left(n\right)\right\}$ defined in ([17], p. 490), is bounded. Hence, Equation (A1) finishes the proof. ☐

Appendix B. Proof of Corollary 1

Proof. 

Since $X\left(\omega \right)$ is positive definite for all $\omega \in \mathbb{R}$, from ([11], Theorem 4.4) and ([18], Corollary VI.1.6), we have:

$$0<\underset{\omega \in [0,2\pi ]}{\min }{\lambda }_N\left(X\left(\omega \right)\right)=\underset{\omega \in [0,2\pi ]}{\inf }{\lambda }_N\left(X\left(\omega \right)\right)\le {\lambda }_{nN}\left(T_n\left(X\right)\right)={\lambda }_{nN}\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\ast }\right)\right)={\lambda }_{nN}\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)$$

for all $n\in \mathbb{N}$, and consequently, $E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)$ is positive definite for all $n\in \mathbb{N}$. Combining ([11], Lemma 3.3) and ([11], Theorem 4.3) yields $\left\{E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\ast }\right)\right\}=\left\{T_n\left(X\right)\right\}\sim \left\{T_n\left(X\right)\right\}$. The proof finishes by applying Theorem 1. ☐

Appendix C. Proof of Theorem 2

Proof. 

(1) From Equation (1), we have:

$$\left(\begin{array}{c}{\mathbf{x}}_n\\ {\mathbf{x}}_{n-1}\\ {\mathbf{x}}_{n-2}\\ ⋮\\ {\mathbf{x}}_1\end{array}\right)=\left(\begin{array}{ccccc}{I}_N& {\mathrm{G}}_{-1}& {\mathrm{G}}_{-2}& \cdots & {\mathrm{G}}_{1-n}\\ 0_{N\times N}& I_N& {\mathrm{G}}_{-1}& \cdots & {\mathrm{G}}_{2-n}\\ 0_{N\times N}& 0_{N\times N}& I_N& \cdots & {\mathrm{G}}_{3-n}\\ ⋮& ⋮& ⋮& \ddots & ⋮\\ 0_{N\times N}& 0_{N\times N}& 0_{N\times N}& \cdots & I_N\end{array}\right)\left(\begin{array}{c}{\mathbf{w}}_n\\ {\mathbf{w}}_{n-1}\\ {\mathbf{w}}_{n-2}\\ ⋮\\ {\mathbf{w}}_1\end{array}\right),$$

or more compactly,

$${\mathbf{x}}_{n:1}=T_n\left(G\right){\mathbf{w}}_{n:1}$$

for all $n\in \mathbb{N}$. Consequently,

$${\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }=T_n\left(G\right){\mathbf{w}}_{n:1}{\mathbf{w}}_{n:1}^{\top }{\left(T_n\left(G\right)\right)}^{\top }\forall n\in \mathbb{N},$$

and applying ([11], Lemma 4.2), yields:

$$\begin{array}{cc}\hfill \left\{E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right\}& =\left\{T_n\left(G\right)E\left({\mathbf{w}}_{n:1}{\mathbf{w}}_{n:1}^{\top }\right){\left(T_n\left(G\right)\right)}^{\top }\right\}\hfill \\ \hfill & =\left\{T_n\left(G\right)T_n\left(\mathsf{\Lambda }\right){\left(T_n\left(G\right)\right)}^{\ast }\right\}=\left\{T_n\left(G\right)T_n\left(\mathsf{\Lambda }\right)T_n\left(G^{\ast }\right)\right\},\hfill \end{array}$$ (A3)

where $G^{\ast }\left(\omega \right)={\left(G\left(\omega \right)\right)}^{\ast }$, $\omega \in \mathbb{R}$. Combining ([11], Lemma 3.3) and ([11], Theorem 4.3), we obtain $\left\{T_n\left(G\right)\right\}\sim \left\{T_n\left(G\right)\right\}$. Moreover, applying ([10], Theorem 3) yields $\left\{T_n\left(\mathsf{\Lambda }\right)T_n\left(G^{\ast }\right)\right\}\sim \left\{T_n\left(\mathsf{\Lambda }{G}^{\ast }\right)\right\}$. Hence, from ([10], Lemma 2) and ([10], Theorem 3), we have:

$$\begin{array}{cc}\hfill \left\{E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\ast }\right)\right\}& =\left\{E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right\}=\left\{T_n\left(G\right)T_n\left(\mathsf{\Lambda }\right)T_n\left(G^{\ast }\right)\right\}\hfill \\ \hfill & \sim \left\{T_n\left(G\right)T_n\left(\mathsf{\Lambda }{G}^{\ast }\right)\right\}\sim \left\{T_n\left(G\mathsf{\Lambda }{G}^{\ast }\right)\right\}=\left\{T_n\left(X\right)\right\}.\hfill \end{array}$$ (A4)

Thus, as the relation ∼ is transitive (see ([11], Lemma 3.1)), $\{{\mathbf{x}}_n:n\in \mathbb{N}\}$ is AWSS with APSD X. (2) First, we prove that $X\left(\omega \right)$ is positive definite for all $\omega \in \mathbb{R}$. Fix $\omega \in \mathbb{R}$, and consider $\mathbf{y}\in {\mathbb{C}}^{N\times 1}$. Since $\mathsf{\Lambda }$ is positive definite, we have:

$${\mathbf{y}}^{\ast }X\left(\omega \right)\mathbf{y}={\mathbf{y}}^{\ast }G\left(\omega \right)\mathsf{\Lambda }{\left(G\left(\omega \right)\right)}^{\ast }\mathbf{y}={\left({\left(G\left(\omega \right)\right)}^{\ast }\mathbf{y}\right)}^{\ast }\mathsf{\Lambda }{\left(G\left(\omega \right)\right)}^{\ast }\mathbf{y}>0$$

whenever ${\left(G\left(\omega \right)\right)}^{\ast }\mathbf{y}\ne 0_{N\times 1}$. As $\det \left(G\right(\omega \left)\right)\ne 0$, ${\left(G\left(\omega \right)\right)}^{\ast }\mathbf{y}=0_{N\times 1}$ if and only if $\mathbf{y}={\left({\left(G\left(\omega \right)\right)}^{-1}\right)}^{\ast }{0}_{N\times 1}=0_{N\times 1}$, and consequently, $X\left(\omega \right)$ is positive definite.

Secondly, we prove that $E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)$ is positive definite for all $n\in \mathbb{N}$. To do that, we only need to show that $\det \left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)\ne 0$ for all $n\in \mathbb{N}$, because as $E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)$ is a correlation matrix, it is positive semidefinite. We have:

$$\begin{array}{cc}\hfill \det \left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)& =\det \left(T_n\left(G\right)T_n\left(\mathsf{\Lambda }\right)T_n\left(G^{\ast }\right)\right)=\det \left(T_n\left(G\right)\right)\det \left(T_n\left(\mathsf{\Lambda }\right)\right)\det \left({\left(T_n\left(G\right)\right)}^{\ast }\right)\hfill \\ \hfill & =|\det \left(T_n\left(G\right)\right){|}^2{\left(\det \left(\mathsf{\Lambda }\right)\right)}^n={\left|{\left(\det \left(I_N\right)\right)}^n\right|}^2{\left(\det \left(\mathsf{\Lambda }\right)\right)}^n={\left(\det \left(\mathsf{\Lambda }\right)\right)}^n\ne 0\hfill \end{array}$$ (A5)

for all $n\in \mathbb{N}$.

The result now follows from Theorem 1. ☐

Appendix D. Proof of Theorem 3

Proof. 

(1) From Equation (2), we have:

$$\sum _{j=0}^{n-1}{\mathrm{F}}_{-j}{\mathbf{x}}_{n-j}=\sum _{j=0}^{n-1}{\mathrm{G}}_{-j}{\mathbf{w}}_{n-j},$$

or more compactly,

$$T_n\left(F\right){\mathbf{x}}_{n:1}=T_n\left(G\right){\mathbf{w}}_{n:1}$$

for all $n\in \mathbb{N}$. Consequently,

$$T_n\left(F\right){\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }{\left(T_n\left(F\right)\right)}^{\top }=T_n\left(F\right){\mathbf{x}}_{n:1}{\left(T_n\left(F\right){\mathbf{x}}_{n:1}\right)}^{\top }=T_n\left(G\right){\mathbf{w}}_{n:1}{\mathbf{w}}_{n:1}^{\top }{\left(T_n\left(G\right)\right)}^{\top }\forall n\in \mathbb{N},$$

and applying Equation (A3) yields:

$$\begin{array}{cc}\hfill \left\{T_n\left(F\right)E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\ast }\right){\left(T_n\left(F\right)\right)}^{\ast }\right\}& =\left\{T_n\left(F\right)E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right){\left(T_n\left(F\right)\right)}^{\top }\right\}\hfill \\ \hfill & =\left\{T_n\left(G\right)E\left({\mathbf{w}}_{n:1}{\mathbf{w}}_{n:1}^{\top }\right){\left(T_n\left(G\right)\right)}^{\top }\right\}=\left\{T_n\left(G\right)T_n\left(\mathsf{\Lambda }\right)T_n\left(G^{\ast }\right)\right\}.\hfill \end{array}$$

Since $\det \left(T_n\left(F\right)\right)={\left(\det \left(I_N\right)\right)}^n=1\ne 0$ for all $n\in \mathbb{N}$, we obtain:

$$\begin{array}{cc}\hfill \left\{E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\ast }\right)\right\}& =\left\{{\left(T_n\left(F\right)\right)}^{-1}{T}_n\left(G\right)T_n\left(\mathsf{\Lambda }\right)T_n\left(G^{\ast }\right){\left({\left(T_n\left(F\right)\right)}^{\ast }\right)}^{-1}\right\}\hfill \\ \hfill & =\left\{{\left({\left(T_n\left(F\right)\right)}^{\ast }{T}_n\left(F\right)\right)}^{-1}{\left(T_n\left(F\right)\right)}^{\ast }{T}_n\left(G\right)T_n\left(\mathsf{\Lambda }\right)T_n\left(G^{\ast }\right)T_n\left(F\right){\left({\left(T_n\left(F\right)\right)}^{\ast }{T}_n\left(F\right)\right)}^{-1}\right\}.\hfill \end{array}$$

From Equation (A4) and the fact that the relation ∼ is transitive (see ([11], Lemma 3.1)), we have $\left\{T_n\left(G\right)T_n\left(\mathsf{\Lambda }\right)T_n\left(G^{\ast }\right)\right\}\sim \left\{T_n\left(G\mathsf{\Lambda }{G}^{\ast }\right)\right\}$. Combining ([11], Lemma 3.3) and ([11], Theorem 4.3) yields $\left\{T_n\left(F\right)\right\}\sim \left\{T_n\left(F\right)\right\}$. Therefore, applying ([10], Lemma 2) and ([10], Theorem 3), we obtain:

$$\left\{T_n\left(G\right)T_n\left(\mathsf{\Lambda }\right)T_n\left(G^{\ast }\right)T_n\left(F\right)\right\}\sim \left\{T_n\left(G\mathsf{\Lambda }{G}^{\ast }\right)T_n\left(F\right)\right\}\sim \left\{T_n\left(G\mathsf{\Lambda }{G}^{\ast }F\right)\right\}.$$

Using ([11], Lemma 3.1) yields $\left\{{\left(T_n\left(F\right)\right)}^{\ast }\right\}\sim \left\{{\left(T_n\left(F\right)\right)}^{\ast }\right\}$, and applying ([10], Lemma 2), ([11], Lemma 4.2), and ([10], Theorem 3), we have:

$$\begin{array}{cc}\hfill \left\{{\left(T_n\left(F\right)\right)}^{\ast }{T}_n\left(G\right)T_n\left(\mathsf{\Lambda }\right)T_n\left(G^{\ast }\right)T_n\left(F\right)\right\}& \sim \left\{{\left(T_n\left(F\right)\right)}^{\ast }{T}_n\left(G\mathsf{\Lambda }{G}^{\ast }F\right)\right\}\hfill \\ \hfill & =\left\{T_n\left(F^{\ast }\right)T_n\left(G\mathsf{\Lambda }{G}^{\ast }F\right)\right\}\sim \left\{T_n\left(F^{\ast }G\mathsf{\Lambda }{G}^{\ast }F\right)\right\}.\hfill \end{array}$$ (A6)

If $\omega \in \mathbb{R}$ and $\mathbf{y}\in {\mathbb{C}}^{N\times 1}$, then,

$${\mathbf{y}}^{\ast }{\left(F\left(\omega \right)\right)}^{\ast }F\left(\omega \right)\mathbf{y}={\left(F\left(\omega \right)\mathbf{y}\right)}^{\ast }F\left(\omega \right)\mathbf{y}={\parallel F\left(\omega \right)\mathbf{y}\parallel }_2>0$$

whenever $F\left(\omega \right)\mathbf{y}\ne 0_{N\times 1}$. As $\det \left(F\right(\omega \left)\right)\ne 0$, $F\left(\omega \right)\mathbf{y}=0_{N\times 1}$ if and only if $\mathbf{y}={\left(F\left(\omega \right)\right)}^{-1}{0}_{N\times 1}=0_{N\times 1}$, hence ${\left(F\left(\omega \right)\right)}^{\ast }F\left(\omega \right)$ is positive definite for all $\omega \in \mathbb{R}$, and applying ([11], Theorem 4.4) and ([18], Corollary VI.1.6), yields:

$$0<\underset{\omega \in [0,2\pi ]}{\min }{\lambda }_N\left({\left(F\left(\omega \right)\right)}^{\ast }F\left(\omega \right)\right)=\underset{\omega \in [0,2\pi ]}{\inf }{\lambda }_N\left({\left(F\left(\omega \right)\right)}^{\ast }F\left(\omega \right)\right)\le {\lambda }_{nN}\left(T_n\left(F^{\ast }F\right)\right)\forall n\in \mathbb{N}.$$

Thus,

$$\begin{array}{cc}\hfill \parallel {\left(T_n\left(F^{\ast }F\right)\right)}^{-1}{\parallel }_2& =\underset{k\in \{1,\cdots ,nN\}}{\max }\left|{\lambda }_k\left({\left(T_n\left(F^{\ast }F\right)\right)}^{-1}\right)\right|=\underset{k\in \{1,\cdots ,nN\}}{\max }\left|\frac{1}{{\lambda }_k\left(T_n\left(F^{\ast }F\right)\right)}\right|\hfill \\ \hfill & =\underset{k\in \{1,\cdots ,nN\}}{\max }\frac{1}{{\lambda }_k\left(T_n\left(F^{\ast }F\right)\right)}=\frac{1}{{\lambda }_{nN}\left(T_n\left(F^{\ast }F\right)\right)}\le \frac{1}{{\min }_{\omega \in [0,2\pi ]}{\lambda }_N\left({\left(F\left(\omega \right)\right)}^{\ast }F\left(\omega \right)\right)}\hfill \end{array}$$

for all $n\in \mathbb{N}$. Observe that $\left\{\parallel {\left({\left(T_n\left(F\right)\right)}^{\ast }{T}_n\left(F\right)\right)}^{-1}{\parallel }_2\right\}$ is also bounded, because:

$$\begin{array}{cc}\hfill \parallel {\left({\left(T_n\left(F\right)\right)}^{\ast }{T}_n\left(F\right)\right)}^{-1}{\parallel }_2& =\parallel {\left(T_n\left(F\right)\right)}^{-1}{\left({\left(T_n\left(F\right)\right)}^{\ast }\right)}^{-1}{\parallel }_2={\parallel {\left(T_n\left(F\right)\right)}^{-1}{\left({\left(T_n\left(F\right)\right)}^{-1}\right)}^{\ast }\parallel }_2\hfill \\ \hfill & \le \parallel {\left(T_n\left(F\right)\right)}^{-1}{\parallel }_2\parallel {\left({\left(T_n\left(F\right)\right)}^{-1}\right)}^{\ast }{\parallel }_2={\parallel {\left(T_n\left(F\right)\right)}^{-1}\parallel }_2^2\forall n\in \mathbb{N}.\hfill \end{array}$$

Moreover, from ([10], Theorem 3), we obtain $\left\{{\left(T_n\left(F\right)\right)}^{\ast }{T}_n\left(F\right)\right\}=\left\{T_n\left(F^{\ast }\right)T_n\left(F\right)\right\}\sim \left\{T_n\left(F^{\ast }F\right)\right\}$. Consequently, applying Lemma A1 (see Appendix E) and ([11], Theorem 6.4), yields:

$$\left\{{\left({\left(T_n\left(F\right)\right)}^{\ast }{T}_n\left(F\right)\right)}^{-1}\right\}\sim \left\{{\left(T_n\left(F^{\ast }F\right)\right)}^{-1}\right\}\sim \left\{T_n\left({\left(F^{\ast }F\right)}^{-1}\right)\right\}=\left\{T_n\left(F^{-1}{\left(F^{\ast }\right)}^{-1}\right)\right\}.$$ (A7)

Therefore, from Equation (A6), ([10], Lemma 2), and ([10], Theorem 3), we have:

$$\begin{array}{cc}\hfill \left\{{\left({\left(T_n\left(F\right)\right)}^{\ast }{T}_n\left(F\right)\right)}^{-1}{\left(T_n\left(F\right)\right)}^{\ast }{T}_n\left(G\right)T_n\left(\mathsf{\Lambda }\right)T_n\left(G^{\ast }\right)T_n\left(F\right)\right\}& \sim \left\{T_n\left(F^{-1}{\left(F^{\ast }\right)}^{-1}\right)T_n\left(F^{\ast }G\mathsf{\Lambda }{G}^{\ast }F\right))\right\}\hfill \\ \hfill & \sim \left\{T_n\left(F^{-1}G\mathsf{\Lambda }{G}^{\ast }F\right))\right\}.\hfill \end{array}$$

Hence, applying Equation (A7), ([10], Lemma 2), and ([10], Theorem 3), we deduce that:

$$\begin{array}{cc}\hfill \left\{E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\ast }\right)\right\}& \sim \left\{T_n\left(F^{-1}G\mathsf{\Lambda }{G}^{\ast }F\right))T_n\left(F^{-1}{\left(F^{\ast }\right)}^{-1}\right)\right\}\hfill \\ \hfill & \sim \left\{T_n\left(F^{-1}G\mathsf{\Lambda }{G}^{\ast }{\left(F^{\ast }\right)}^{-1}\right)\right\}=\left\{T_n\left(F^{-1}G\mathsf{\Lambda }{G}^{\ast }{\left(F^{-1}\right)}^{\ast }\right)\right\}=\left\{T_n\left(X\right)\right\}.\hfill \end{array}$$

(2) First, we prove that $X\left(\omega \right)$ is positive definite for all $\omega \in \mathbb{R}$. Fix $\omega \in \mathbb{R}$, and consider $\mathbf{y}\in {\mathbb{C}}^{N\times 1}$. Since $G\left(\omega \right)\mathsf{\Lambda }{\left(G\left(\omega \right)\right)}^{\ast }$ is positive definite (see the proof of Theorem 2), we have:

$$\begin{array}{cc}\hfill {\mathbf{y}}^{\ast }X\left(\omega \right)\mathbf{y}& ={\mathbf{y}}^{\ast }{(F\left(\omega \right))}^{-1}G\left(\omega \right)\mathsf{\Lambda }{\left(G\left(\omega \right)\right)}^{\ast }{({\left(F\left(\omega \right)\right)}^{-1})}^{\ast }\mathbf{y}\hfill \\ \hfill & ={({({\left(F\left(\omega \right)\right)}^{-1})}^{\ast }\mathbf{y})}^{\ast }G\left(\omega \right)\mathsf{\Lambda }{\left(G\left(\omega \right)\right)}^{\ast }{({\left(F\left(\omega \right)\right)}^{-1})}^{\ast }\mathbf{y}>0\hfill \end{array}$$

whenever ${\left({\left(F\left(\omega \right)\right)}^{\ast }\right)}^{-1}\mathbf{y}={\left({\left(F\left(\omega \right)\right)}^{-1}\right)}^{\ast }\mathbf{y}\ne 0_{N\times 1}$. As ${\left({\left(F\left(\omega \right)\right)}^{\ast }\right)}^{-1}\mathbf{y}=0_{N\times 1}$ if and only if $\mathbf{y}={\left(F\left(\omega \right)\right)}^{\ast }{0}_{N\times 1}=0_{N\times 1}$, $X\left(\omega \right)$ is positive definite.

Secondly, we prove that $E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)$ is positive definite for all $n\in \mathbb{N}$, or equivalently, $\det \left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)\ne 0$ for all $n\in \mathbb{N}$. Applying Equation (A5) yields:

$$\begin{array}{cc}\hfill \det \left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)& =\det \left({\left(T_n\left(F\right)\right)}^{-1}{T}_n\left(G\right)T_n\left(\mathsf{\Lambda }\right)T_n\left(G^{\ast }\right){\left({\left(T_n\left(F\right)\right)}^{\ast }\right)}^{-1}\right)\hfill \\ \hfill & =\frac{\det \left(T_n\left(G\right)T_n\left(\mathsf{\Lambda }\right)T_n\left(G^{\ast }\right)\right)}{\det \left(T_n\left(F\right)\right)\det \left({\left(T_n\left(F\right)\right)}^{\ast }\right)}=\frac{{\left(\det \left(\mathsf{\Lambda }\right)\right)}^n}{|\det \left(T_n\left(F\right)\right){|}^2}={\left(\det \left(\mathsf{\Lambda }\right)\right)}^n\ne 0\forall n\in \mathbb{N}.\hfill \end{array}$$

The result now follows from Theorem 1. ☐

Appendix E. A Property of Asymptotically Equivalent Sequences of Invertible Matrices

Lemma A1.

Let $A_n$ and $B_n$ be $nN\times nN$ invertible matrices for all $n\in \mathbb{N}$. Suppose that $\left\{A_n\right\}\sim \left\{B_n\right\}$ and $\{\parallel A_n^{-1}{\parallel }_2\}$ and $\{\parallel B_n^{-1}{\parallel }_2\}$ are bounded. Then, $\left\{A_n^{-1}\right\}\sim \left\{B_n^{-1}\right\}$.

Proof. 

If $M\in [0,\infty )$ such that $\parallel A_n^{-1}{\parallel }_2,{\parallel B_n^{-1}\parallel }_2\le M$ for all $n\in \mathbb{N}$, then:

$$\begin{array}{cc}\hfill 0& \le \frac{{∥A_n^{-1}-B_n^{-1}∥}_F}{\sqrt{n}}=\frac{{∥B_n^{-1}-A_n^{-1}∥}_F}{\sqrt{n}}=\frac{{∥B_n^{-1}{A}_n{A}_n^{-1}-B_n^{-1}{B}_n{A}_n^{-1}∥}_F}{\sqrt{n}}=\frac{{∥B_n^{-1}\left(A_n-B_n\right)A_n^{-1}∥}_F}{\sqrt{n}}\hfill \\ \hfill & \le {∥B_n^{-1}∥}_2\frac{{∥\left(A_n-B_n\right)A_n^{-1}∥}_F}{\sqrt{n}}\le {∥B_n^{-1}∥}_2\frac{{∥A_n-B_n∥}_F}{\sqrt{n}}{∥A_n^{-1}∥}_2\le M^2\frac{{∥A_n-B_n∥}_F}{\sqrt{n}}\to 0.\hfill \end{array}$$

☐

This result was presented in ([6], Theorem 1) for the case $N=1$.

Author Contributions

Authors are listed in order of their degree of involvement in the work, with the most active contributors listed first. J.G.-G. conceived the research question. All authors proved the main results. J.G.-G. and X.I. performed the simulations. All authors wrote the paper. All authors have read and approved the final manuscript.

Funding

This work was supported in part by the Spanish Ministry of Economy and Competitiveness through the CARMEN project (TEC2016-75067-C4-3-R).

Conflicts of Interest

The authors declare no conflict of interest.