Dynamic principal component analysis with missing values

Abstract

Dynamic principal component analysis (DPCA), also known as frequency domain principal component analysis, has been developed by Brillinger [Time Series: Data Analysis and Theory, Vol. 36, SIAM, 1981] to decompose multivariate time-series data into a few principal component series. A primary advantage of DPCA is its capability of extracting essential components from the data by reflecting the serial dependence of them. It is also used to estimate the common component in a dynamic factor model, which is frequently used in econometrics. However, its beneficial property cannot be utilized when missing values are present, which should not be simply ignored when estimating the spectral density matrix in the DPCA procedure. Based on a novel combination of conventional DPCA and self-consistency concept, we propose a DPCA method when missing values are present. We demonstrate the advantage of the proposed method over some existing imputation methods through the Monte Carlo experiments and real data analysis.

1. Introduction

Dynamic principal component analysis (DPCA) was originally proposed by Brillinger [4,5], which generalizes the static PCA by considering serial dependence of data and reduces the dimension of time series in the frequency domain while preserving as much information as possible. One of its most common uses is the estimation of factors in the dynamic factor model (DFM). DFM assumes that multivariate time series is generated from a linear combination of the common factors in a lower dimension and these common factors, unlike the ones in the classical static factor model, have their own dynamics. Sargent and Sims [25] and Geweke [15] first extended the classical factor model to dynamic models, and several researchers developed various versions of the DFM [9,27,28].

Although DFMs and DPCAs differ when the number of variables is small, their differences diminish as the number of variables increases [1,2,29]. For example, Forni et al. [14] developed a non-parametric DFM procedure in the frequency domain, based on dynamic principal components, and Bai  [3] provided a comprehensive review of large-sample results for high-dimensional factor models estimated via dynamic PCA. More recently, Doz et al. [8] proposed a two-step estimation method that is based on a DFM with the factor loadings set equal to the eigenvectors which are associated with a set of principal components, and Eichler et al. [12] estimated the common components of DFM by the time-varying principal components approach in the frequency domain.

Not only has DPCA been widely used to estimate factors in DFMs, but also is useful by itself. DPCA had been developed to replace the static PCA when analyzing dynamic data. One major shortcoming of static PCA is the lack of focus on time dependence, and several problems can arise when applying static PCA to dynamic data directly [7]. DPCA attempts to address the dynamic problem by modeling the auto-correlation structure present in the data [19]. Therefore, it can recover the dynamics of a group of time series efficiently and has become popular in various fields including macroeconomics and chemometrics [6,13]. Numerous methods have been derived from DPCA. Stoffer [31] proposed a spectral envelope using DPCA to detect common signals of multiple time series, and Hörman et al. [16] suggested a functional version of DPCA to analyze functional time-series data such as particulate matter with an aerodynamic diameter of less than 10 µm (PM10). More recently, Peña and Yohai [24] discussed a new perspective and generalization for principal component series in DPCA and further developed a robust version of it.

DPCA is successful in analyzing time series; however, it cannot be efficient in the presence of missing values. For the conventional PCA, there have been some studies on the missing problem [10,17,26], but there are few studies on DPCA for missing data [18,30]. In time-series data related to macroeconomics, missing data are common because some variables have a late start of observation or might have different observing interval than the others.

Since the dynamic principal component is obtained from the eigenanalysis of the spectral density matrix, we may apply the imputation methods for the spectral density estimation [22,32], which enables DPCA in the presence of missing values. More recently, Lee and Zhu [21] proposed a spectrum estimation method based on self-consistency concept. The method imputes the missing data until the estimate of the spectrum is stabilized. In this paper, we modify the spectrum estimation method of [21] and propose a new algorithm which enables DPCA for the incompletely observed data.

We organize the rest of this paper as follows. Section 2 reviews DPCA, and in Section 3, we propose an imputation algorithm for DPCA, which is the main contribution of this study. Section 4 compares the proposed method to other existing imputation methods such as linear interpolation and Kalman smoothing of Durbin and Koopman [11] through the Monte Carlo experiment. In Section 5, we apply the new method to two real data. Finally, concluding remarks are given in Section 6.

2. Dynamic principal component analysis

First of all, we briefly review the traditional static PCA that is performed in the time domain. Let $\mathbf{X}$ be a p-dimensional random vector with mean $\mathit{\mu }$ and covariance matrix $\mathbf{\Sigma }=E[(\mathbf{X}-\mathit{\mu }){\overline{(\mathbf{X}-\mathit{\mu })}}^{\prime }]$. Static PCA aims at reducing the dimension of given data while preserving as much information as possible. So, the static PCA finds a projection of $\mathbf{X}-\mathit{\mu }$ to q-dimensional subspace, which can be reconstructed to the original data most closely. The projected and reconstructed data are expressed as $\mathbf{B}(\mathbf{X}-\mathit{\mu })$ and $\mathbf{C}\mathbf{B}(\mathbf{X}-\mathit{\mu })$, respectively, where $\mathbf{B}$ is a $q\times p$ matrix and $\mathbf{C}$ is a $p\times q$ matrix. Then, the reconstruction criterion is defined as

$$E[(\overline{\mathbf{X}-\mathit{\mu }-\mathbf{C}\mathbf{B}(\mathbf{X}-\mathit{\mu })}{)}^{\prime }(\mathbf{X}-\mathit{\mu }-\mathbf{C}\mathbf{B}(\mathbf{X}-\mathit{\mu }))],$$ (1)

and the principal components are obtained by minimizing it. The solutions are given by

$$\mathbf{C}=[{\mathbf{V}}_1,\dots ,{\mathbf{V}}_q]={\overline{\mathbf{B}}}^{\prime },$$

where ${\mathbf{V}}_j$ is an eigenvector of $\mathbf{\Sigma }$ which corresponds to the j-th largest eigenvalue ${\lambda }_j$ ($j=1,\dots ,q$). Hence, we obtain the j-th principal component of $\mathbf{X}$ as ${\zeta }_j={\overline{{\mathbf{V}}_j}}^{\prime }\mathbf{X}$, $j=1,\dots ,q$.

However, the complex non-linear relations among observations cannot be approximated by the static PCA when the data have a time-dependent structure. Therefore, DPCA has been developed to analyze the data that have a dynamic structure. For a review of DPCA, we summarize Chapter 9 of Brillinger [5] from the aspect of dimension reduction and reconstruction of time series, which is closely related to the static PCA criterion in (1).

Suppose that we have a p vector-valued second-order stationary time series $\mathit{X}(t)=(X_1(t),X_2(t),\dots ,X_p(t){)}^{\prime }$ with zero mean, absolutely summable autocovariance function $\mathit{c}(u)$ and the corresponding spectral density matrix $\mathbf{f}(\omega )$, $-\mathrm{\infty }<\omega <\mathrm{\infty }$. Similarly, as in the time domain, DPCA reduces the dimension of given time-series data that can be reconstructed to the original one as close as possible. Hence, we consider minimizing the least squares criterion of reconstruction error defined as

$$E[\{\overline{\mathbf{X}(t)-\sum _u\mathbf{c}(t-u)\mathit{\zeta }(u)}{\}}^{\prime }\{\mathbf{X}(t)-\sum _u\mathbf{c}(t-u)\mathit{\zeta }(u)\}],$$ (2)

where $\mathbf{c}(t)$ is a $p\times q$ filter and $\mathit{\zeta }(t)=\sum _u\mathbf{b}(t-u)\mathbf{X}(u)$ is a principal component series with $q\times p$ filter $\mathbf{b}(t)$. Then, the solution is given by

$$\mathbf{b}(u)=\frac{1}{2\pi }{\int }_0^{2\pi }\mathbf{B}(\omega )e^{iu\omega }\mathrm{d}\omega ,\mathrm{and}\mathbf{c}(u)=\frac{1}{2\pi }{\int }_0^{2\pi }\mathbf{C}(\omega )e^{iu\omega }\mathrm{d}\omega ,$$

where $\mathbf{C}(\omega )=[{\mathit{V}}_1(\omega ),\dots ,{\mathit{V}}_q(\omega )]={\overline{\mathbf{B}(\omega )}}^{\prime }$ and ${\mathit{V}}_j(\omega )$ is an eigenvector of $\mathbf{f}(\omega )$ which corresponds to the j-th largest eigenvalue ${\lambda }_j(\omega )$. Furthermore, the PC series ${\mathit{\zeta }}_j(t)$ has ${\lambda }_j(\omega )$ as its spectrum, and it has zero coherence with ${\mathit{\zeta }}_k(t)$ for $k\ne j$. Detailed derivation can be found in Brillinger [5].

DPCA is also closely related to the DFM [14]. Under the assumptions in Forni et al. [14], the DFM can be written as

$$\mathbf{X}(t)=\sum _u\mathbf{c}(u)\mathbf{U}(t-u)+\mathit{\xi }(t),$$

where $\mathbf{U}(t)=(U_1(t),U_2(t),\dots ,U_q(t){)}^{\prime }$ is a q-dimensional common shock (or a common factor in statistical term), $\mathbf{c}(t)$ is a $p\times q$ filter, and $\mathit{\xi }(t)$ is a zero-mean stationary process called idiosyncratic component. Our main interest lies in estimating the common component $\mathcal{X}(t)=\mathbf{X}(t)-\mathit{\xi }(t)$ and the estimator is computed as

$$\hat{\mathcal{X}}(t)=\sum _u\mathbf{c}(t-u)\mathit{\zeta }(u),$$

which is the orthogonal projection of $\mathbf{X}(t)$ on $\mathcal{U}:=\overline{span}({\mathit{\zeta }}_j(t),j=1,\dots ,q)$, the minimal closed subspace of Hilbert space $L_2(\mathrm{\Omega },\mathcal{F},P)$ containing the first q PC series. It converges to $\mathcal{X}(t)$ in mean square as p tends to infinity [14].

3. Dynamic principal component analysis with missing values

Here we present a new approach for DPCA when data contain missing values. As stated in the previous section, it is necessary to estimate the spectral density matrix $\mathbf{f}(\omega )$ to perform DPCA. Hence, in the presence of missing values, the estimation of $\mathbf{f}(\omega )$ might be distorted. To cope with this problem, we adopt an imputation method for estimating the spectral density matrix $\mathbf{f}(\omega )$ based on self-consistency [21] and then combine it with the DPCA.

3.1. Review: Monte Carlo self-consistent imputation algorithm

It is important to start with the concept of self-consistency. When we approximate a random variable ${\mathbf{X}}_1$ with another random variable ${\mathbf{X}}_2$, the $L_2$ approximation criterion $E\Vert {\mathbf{X}}_1-g({\mathbf{X}}_2){\Vert }^2$ is minimized when $g({\mathbf{X}}_2)=E[{\mathbf{X}}_1|{\mathbf{X}}_2]$. Here, $||\cdot ||$ is $L_2$ norm. Therefore, the following inequality holds

$$E\Vert {\mathbf{X}}_1-E[{\mathbf{X}}_1|{\mathbf{X}}_2]{\Vert }^2\le E\Vert {\mathbf{X}}_1-{\mathbf{X}}_2{\Vert }^2,$$

and we say that ${\mathbf{X}}_2$ is self-consistent, if ${\mathbf{X}}_2=E[{\mathbf{X}}_1|{\mathbf{X}}_2]$ almost surely. This concept has been applied to find efficient estimators in various studies such as wavelet image denoising with missing data [20] and EM-algorithm [23].

Recently, Lee and Zhu [21] developed a self-consistent estimation method of the spectral density function for missing data, termed Monte Carlo Self-Consistent (MCSC) algorithm. MCSC algorithm is a non-parametric spectrum estimation method using an iterative method. During the iteration, missing values are imputed with the random numbers generated from the conditional normal distribution of the missing data $X^{mis}$ given the observed data $X^{obs}$. Then the spectrum is estimated based on the imputed values. The iteration is continued until the convergence of $\hat{f}$ in order to satisfy the self-consistency, that is

$$\hat{f}=E\{{\hat{f}}^{com}|X^{obs},f=\hat{f}\},$$

where $\hat{f}$ and ${\hat{f}}^{com}$ are spectrum estimates obtained from the imputed and complete data, respectively.

3.2. Proposed DPCA

For a new DPCA method to deal with missing values, we modify the MCSC algorithm because the uncertainty arises due to the random numbers which impute the unobserved ones. This inherent randomness of the MCSC algorithm prevents the estimated spectrum from converging. Therefore, we use the conditional expectation in our algorithm and apply the DPCA to the converged estimate to obtain dynamic principal components.

Here are the detailed implementing steps for the proposed DPCA method for missing data, and we name the whole process as Iterative Self-Consistent (ISC) algorithm. Let ${\mathbf{X}}^{com}=({\mathbf{X}}_1,{\mathbf{X}}_2,\dots ,{\mathbf{X}}_p)$ be the complete data, and ${\mathbf{X}}_j^{obs}$ and ${\mathbf{X}}_j^{mis}$ be observed and missing portions of ${\mathbf{X}}_j$, respectively, for $j=1,2,\dots ,p$. We assume that ${\mathbf{X}}_j$ has $M_j$ missing values, where ${\mathbf{X}}_{j,m}^{mis}$ denotes the mth missing value, $m=1,\dots M_j$.

1. To obtain the converged spectral density estimate ${\hat{f}}_j^{\ast }$ of ${\mathbf{X}}_j$, repeat the following steps for $j=1,2,\dots ,p.$

   1. Impute the missing data ${\mathbf{X}}_j^{mis}$ by linear interpolation and denote the imputed complete data by ${\mathbf{X}}_j^{(0)}$. Then, obtain the initial spectral density estimate ${\hat{f}}_j^{(0)}$ from ${\mathbf{X}}_j^{(0)}$.
   2. Iterate the following steps for $\ell =1,2,\dots$, until

      $$n^{-1}\sum _{k=1}^n({\hat{f}}_j^{(\ell -1)}({\omega }_k)-{\hat{f}}_j^{(\ell )}({\omega }_k){)}^2<ϵ,$$

      for a pre-determined $ϵ<0$.

      1. For $m=1,2,\dots ,M_j,$

         1. update the imputation of $X_{j,m}^{mis}$ with the conditional mean

            $$X_{j,m}^{(\ell )}:=E[X_{j,m}^{mis}|{\mathbf{X}}_j^{obs},\gamma ={\hat{\gamma }}^{(\ell -1)}],$$

            where γ is an autocovariance function whose estimate is obtained by

            $${\hat{\gamma }}^{(\ell )}(h)=\frac{1}{2\pi }{\int }_{-\pi }^{\pi }{\hat{f}}_j^{(\ell )}(\omega )e^{i\omega h}\mathrm{d}\omega ,$$
         2. obtain the m-th periodogram $I_m^{(\ell )}$ from

            $$\{{\mathbf{X}}_j^{obs}\}\cup \{X_{j,k}^{(\ell -1)}:k=1,\dots ,m-1,m+1,\dots ,M_j\}\cup \{X_{j,m}^{(\ell )}\},\mathrm{and}$$
         3. apply the kernel smoothing on $I_m^{(\ell )}$ and obtain ${\hat{f}}_{j,m}^{(\ell )}$.
      2. Compute the ℓth spectral density estimate by ${\hat{f}}_j^{(\ell )}=\frac{1}{M_j}\sum _{m=1}^{M_j}{\hat{f}}_{j,m}^{(\ell )}$

2. Perform DPCA with the spectral density matrix ${\hat{\mathbf{f}}}^{\ast }$ obtained from the converged imputed complete data.

For an illustration purpose of the above algorithm, Figure 1 shows the flow of estimating ${\hat{f}}_j^{\ast }$ in Step 1.

Figure 1.

Estimation procedure of ${\hat{f}}_j^{\ast }$ from $X_j$

4. Monte Carlo experiment

We use DFM as a data generating process in the Monte Carlo experiment of this study. For a p-dimensional DFM $\mathbf{X}(t)=\mathcal{X}(t)+\mathit{\xi }(t),1\le t\le T$, we estimate the common component $\mathcal{X}(t)$ by the orthogonal projection of $\mathbf{X}(t)$ on $\mathcal{U}:=\overline{span}({\mathit{\zeta }}_j(t),j=1,\dots ,q)$ as described in Section 2. We assume that the dimension of the PC series is half of that of $\mathcal{X}(t)$, i.e. $q=p/2$.

Simulation data with missing values are generated under various scenarios. A weighted sum of sinusoids or vector MA(1) process is considered as a common component. Regarding the proportion of the missing data, we consider four scenarios that $5\mathrm{\%}$, $10\mathrm{\%}$, $15\mathrm{\%}$ or $20\mathrm{\%}$ of the given data is missing completely at random. Then, the missing values are imputed by three different imputation methods: linear interpolation, Kalman smoothing of Durbin and Koopman [11], and the proposed ISC algorithm. Kalman smoothing assumes that the incompletely observed process is a linearly transformed version of the unobservable state process with noise,

$$\mathbf{X}(t)=\mathbf{A}\mathbf{U}(t)+\mathit{\xi }(t),1\le t\le T,$$

where $\mathbf{U}(t)$ is a q-dimensional autoregressive state process and $\mathit{\xi }(t)\sim N(0,\mathbf{Q})$. For imputation, $\mathbf{U}(t)$ is first estimated by $E[\mathbf{U}(t)|{\mathbf{X}}_{obs}]$ and then it is post-multiplied by the observation matrix $\mathbf{A}$.

Using the imputed data, we perform DPCA and estimate the common components. As a measure that evaluates the empirical performance of the proposed method, we use the relative deviation criterion defined as

$$D(\mathcal{X},\hat{\mathcal{X}})=\frac{\sum _{i,t}({\mathcal{X}}_i(t)-{\hat{\mathcal{X}}}_i(t){)}^2}{\sum _{i,t}{\mathcal{X}}_i(t{)}^2},$$

where $\hat{\mathcal{X}}$ denotes the estimate of the common components by q PC series.

For the simulation study, we consider two different dimensions of observations $p=20,40$, and three different lengths of observations T = 1000, 2000, 4000. For each combination of p, T and missing proportion, the average values and their standard deviations of $D(\mathcal{X},\hat{\mathcal{X}})$ over 100 simulations are computed.

4.1. Common component: sum of sinusoids

We consider a weighted sum of sinusoids as a common component and MA(1) process as an idiosyncratic component. The simulation data, $\mathbf{X}(t)=(X_1(t),X_2(t),\dots ,X_p(t){)}^{\prime }$ is defined as

$$\mathbf{X}(t)=2\mathbf{A}\mathbf{U}(t)+\mathit{\xi }(t)-0.5\mathit{\xi }(t),1\le t\le T,$$

where $\mathbf{A}=(A_{i,j}{)}_{1\le i\le p,1\le j\le q}$ with $A_{i,j}\stackrel{iid}{\sim }N(0,1)$, and $\mathit{\xi }(t)=({\xi }_1(t),{\xi }_2(t),\dots ,{\xi }_p(t){)}^{\prime }$ with ${\xi }_j(t)\stackrel{iid}{\sim }N(0,1).$ The common factor $\mathbf{U}(t)=(U_1(t),U_2(t),\dots ,U_q(t){)}^{\prime }$ has a sinusoidal form of

$$U_j(t)=\cos ({\omega }_j(2\pi t-{\tau }_j)),$$

where ${\omega }_j={\nu }_j/200$, with ${\nu }_j$ as the j-th prime number, and ${\tau }_j\sim Unif(0,2\pi )$,  $j=1,\dots ,q$.

Results are listed in Table 1. For all cases, the proposed method based on the ISC algorithm outperforms the other two methods. To determine whether one method is better than the other significantly, we can construct an approximated 95% confidence interval of the difference between two performance measures. For example, when T = 1000, p = 20, and missing data rate is $20\mathrm{\%}$, the approximate the confidence interval between the average values of Kalman smoothing and ISC algorithm is

$$(0.062-0.057)\pm z_{0.025}\frac{\sqrt{{0.015}^2+{0.014}^2}}{\sqrt{100}},$$

which implies that the ISC algorithm performs better than Kalman smoothing since this confidence interval does not contain zero. Furthermore, we observe that the superiority of the proposed method becomes significant as p and missing data rate increase, and T decreases.

Table 1. Average and standard deviation in parentheses of $D(\mathcal{X},\hat{\mathcal{X}})$ over 100 simulations.

			Missing proportion
		Method	5%	10%	15%	20%
T = 1000	p = 20	Linear interpolation	0.057 (0.014)	0.063 (0.019)	0.065 (0.016)	0.066 (0.016)
		Kalman smoothing	0.057 (0.016)	0.060 (0.018)	0.061 (0.015)	0.062 (0.015)
		ISC algorithm	0.054 (0.012)	0.055 (0.013)	0.056 (0.015)	0.057 (0.014)
	p = 40	Linear interpolation	0.083 (0.012)	0.113 (0.015)	0.142 (0.016)	0.170 (0.016)
		Kalman smoothing	0.066 (0.012)	0.081 (0.013)	0.093 (0.014)	0.110 (0.013)
		ISC algorithm	0.059 (0.011)	0.066 (0.012)	0.075 (0.013)	0.085 (0.013)
T = 2000	p = 20	Linear interpolation	0.059 (0.013)	0.062 (0.013)	0.064 (0.015)	0.067 (0.015)
		Kalman smoothing	0.057 (0.012)	0.059 (0.012)	0.062 (0.015)	0.063 (0.016)
		ISC algorithm	0.057 (0.013)	0.056 (0.011)	0.057 (0.014)	0.060 (0.016)
	p = 40	Linear interpolation	0.076 (0.012)	0.103 (0.015)	0.127 (0.015)	0.153 (0.015)
		Kalman smoothing	0.061 (0.012)	0.073 (0.014)	0.083 (0.013)	0.097 (0.015)
		ISC algorithm	0.058 (0.014)	0.063 (0.013)	0.070 (0.014)	0.079 (0.015)
T = 4000	p = 20	Linear interpolation	0.062 (0.013)	0.062 (0.014)	0.063 (0.013)	0.064 (0.014)
		Kalman smoothing	0.061 (0.013)	0.062 (0.015)	0.061 (0.014)	0.063 (0.015)
		ISC algorithm	0.059 (0.012)	0.060 (0.012)	0.059 (0.013)	0.060 (0.012)
	p = 40	Linear interpolation	0.075 (0.013)	0.095 (0.013)	0.116 (0.015)	0.136 (0.015)
		Kalman smoothing	0.063 (0.011)	0.071 (0.012)	0.080 (0.013)	0.089 (0.013)
		ISC algorithm	0.059 (0.012)	0.064 (0.013)	0.069 (0.012)	0.073 (0.012)

Note: Note that a sum of sinusoids as a common component.

4.2. Common component: MA(1)

Here we consider MA(1) process as a common component [14],

$$\mathbf{X}(t)={\mathbf{A}}_0\mathbf{U}(t)+{\mathbf{A}}_1\mathbf{U}(t-1)+2\mathit{\xi }(t),1\le t\le T,$$

where $\mathbf{U}(t)=(U_1(t),U_2(t),\dots ,U_q(t){)}^{\prime }$ with $U_j(t)\stackrel{iid}{\sim }N(0,1)$, and other variables are generated in the same way as in the previous case of Section 4.1.

From Table 2, we observe that the performance of the proposed method is similar to that of Kalman smoothing when the missing proportion is equal to or less than 10%. Kalman smoothing works slightly better than the proposed method as the missing data rate increases and T decreases, but most of the differences are not significant.

Table 2. Average and standard deviation in parentheses of $D(\mathcal{X},\hat{\mathcal{X}})$ over100 simulations.

			Missing proportion
		Method	5%	10%	15%	20%
T = 1000	p = 20	Linear interpolation	0.171 (0.009)	0.225 (0.012)	0.282 (0.013)	0.339 (0.015)
		Kalman smoothing	0.148 (0.009)	0.179 (0.009)	0.211 (0.009)	0.245 (0.008)
		ISC algorithm	0.149 (0.009)	0.180 (0.009)	0.214 (0.009)	0.255 (0.009)
	p = 40	Linear interpolation	0.137 (0.005)	0.194 (0.006)	0.253 (0.007)	0.316 (0.007)
		Kalman smoothing	0.114 (0.006)	0.148 (0.006)	0.184 (0.005)	0.223 (0.005)
		ISC algorithm	0.114 (0.005)	0.149 (0.006)	0.188 (0.005)	0.232 (0.005)
T = 2000	p = 20	Linear interpolation	0.167 (0.011)	0.217 (0.013)	0.270 (0.016)	0.327 (0.018)
		Kalman smoothing	0.145 (0.010)	0.173 (0.010)	0.204 (0.010)	0.236 (0.010)
		ISC algorithm	0.145 (0.010)	0.174 (0.009)	0.205 (0.010)	0.242 (0.010)
	p = 40	Linear interpolation	0.133 (0.006)	0.187 (0.006)	0.243 (0.007)	0.301 (0.009)
		Kalman smoothing	0.112 (0.005)	0.144 (0.005)	0.178 (0.005)	0.213 (0.004)
		ISC algorithm	0.112 (0.005)	0.145 (0.005)	0.180 (0.005)	0.218 (0.005)
T = 4000	p = 20	Linear interpolation	0.161 (0.009)	0.211 (0.011)	0.263 (0.013)	0.317 (0.015)
		Kalman smoothing	0.140 (0.008)	0.168 (0.009)	0.197 (0.008)	0.228 (0.009)
		ISC algorithm	0.140 (0.008)	0.168 (0.008)	0.197 (0.008)	0.230 (0.009)
	p = 40	Linear interpolation	0.129 (0.005)	0.179 (0.005)	0.232 (0.006)	0.287 (0.007)
		Kalman smoothing	0.109 (0.005)	0.139 (0.005)	0.171 (0.005)	0.204 (0.005)
		ISC algorithm	0.109 (0.005)	0.139 (0.005)	0.172 (0.005)	0.207 (0.005)

Note: Note that MA(1) as a common component.

5. Empirical application

5.1. Macroeconomic data

We consider four yearly macroeconomic indicators: gross domestic product (GDP), gross fixed capital formation (GFCF), consumer price index (CPI), and industrial production (IP). GDP and GFCF are measured in US dollars, and the base year of CPI and IP is 2015. We use the data for the period of years 1971–2017 from 15 countries in the Organization for Economic Co-operation and Development (OECD), which are Austria, Belgium, Canada, Finland, France, Germany, Greece, Italy, Luxembourg, Netherlands, Norway, Portugal, Spain, Sweden, and the United Kingdom. The dataset is available in the OECD website https://data.oecd.org.

To apply the proposed method, we artificially generate missing values completely at random with a rate of 10% or 20% and estimate the coincident indicator defined by Forni et al. [14]. We compare this indicator with the one from the missing-free data. The procedure is described as follows:

1. Construct a panel pooling four yearly macroeconomic indicators for 15 countries of the OECD from 1971 to 2017. Take a difference of logarithm of each indicator for each country and normalize it by subtracting the mean and dividing with the standard deviation. Then, we have 46 observations for each of 60 variables.

2. Apply the DPCA with the proposed ISC algorithm. Here, the dimension of the common factor is set as q = 2 by following the suggestion of Forni et al. [14].

3. Estimate the common component of GDP, ${\mathcal{X}}_i(t)$ and compute the coincident indicator defined as

   $$\mathrm{\Delta }C(t)=\frac{\sum W_i{\mathcal{X}}_i(t)}{\sum W_i},$$

   where $W_i$ is the average GDP level of the i-th country for the period of 1971–2017.

The left plot in Figure 2 shows the estimated coincident indicators obtained from missing-free, 10% missing and 20% missing data, and the right plot is a deviation of coincident indicator obtained from missing data from that of missing-free data. We observe that the indicators obtained from missing data do not deviate significantly from the missing-free result.

Figure 2.

(Left) Coincident indicators for the 15 OECD countries. (Right) Deviation of coincident indicators due to the missing observations.

5.2. Particulate matter concentration in South Korea

We analyze daily particulate matter ($P{M}_{10}$) concentration observed at 23 cities in South Korea for the period of years 2006–2015. Recorded $P{M}_{10}$ values often have missing values and, sometimes, observation failure lasts for a long time due to facility maintenance.

Though we cannot compare the performance of our method with the other ones since the data had missing values in the first place, we can make interesting and reasonable interpretations by comparing the estimated common component with the imputed complete data. We assume that the data are generated from the DFM with a two-dimensional common factor, and perform DPCA with the proposed ISC algorithm. Figure 3 shows the result of common component estimation for four cities (Seoul, Suwon, Mokpo, and Taean) in South Korea for January–June in 2009. Dots denote the imputed values, the solid line is the resultant imputed full data, and the dashed line is the estimated common component for each city. While the observations in Seoul show very similar behavior with the common component, Mokpo has higher $P{M}_{10}$ concentrations in general than the common component. On the other hand, at some time points in Suwon or Taean, $P{M}_{10}$ concentrations are distinguishably lower or higher than the common component. These indicate some short-term affecting events in each city and are explained by the idiosyncratic component in the DFM.

Figure 3.

$P{M}_{10}$ concentration of four cities (Seoul, Suwon, Mokpo and Taean) in South Korea between January and June in 2009. Dots denote the imputed values, and solid line is the resultant imputed full data. Dashed line is the estimated common component for each city.

6. Concluding remark

6.1. Application to non-stationary data

Recently, Eichler et al. [12] proposed a non-stationary DFM based on time-varying principal components. Since the stationarity is assumed for both DPCA and ISC algorithms, it is inappropriate to simply apply our approach to non-stationary multivariate time-series data. However, we expect that the proposed method may further be generalized to a non-stationary process. Here we illustrate a toy simulation to show the applicability of the proposed method under the non-stationary process.

The simulation is conducted by referring to the study of Eichler et al. [12], which consider a sum of a time-varying AR(2) and an ordinary AR(2) process as a common component. The data generating process has its specific form as

$$\mathbf{X}(t)={\mathbf{U}}^{(1)}(t)+{\mathbf{U}}^{(2)}(t)+\mathit{\xi }(t),1\le t\le 1000,$$

where

$$\begin{array}{rl}{U}_i^{(j)}(t)& =\sum _{k=1}^2{a}_{ij,k}\left(\frac{t}{T}\right)U_i^{(j)}(t-k)+\mathit{\eta }(t),\\ a_{ij,1}(t)& =\frac{2}{\alpha }\cos ({\varphi }_{ij}-\cos ({\nu }_{ij}t)),a_{ij,2}(t)=-\frac{1}{{\alpha }^2},\\ \mathit{\xi }(t)& =({\xi }_1(t),{\xi }_2(t),\dots ,{\xi }_{20}(t){)}^{\prime },\\ \mathit{\eta }(t)& =({\eta }_1(t),{\eta }_2(t),\dots ,{\eta }_{20}(t){)}^{\prime },\end{array}$$

$\alpha =-10/9$, ${\nu }_{i1}=0.4\pi i$, ${\nu }_{i2}=0.2\pi i$, ${\varphi }_{i1}=0.5i$, ${\varphi }_{i2}=-1.2i$, and ${\xi }_i(t),{\eta }_i(t)\sim i.i.dN(0,1)$ for $i=1,2,\dots ,20$ and j = 1, 2.

We have set 20% of the data as missing completely at random and impute these with three different imputation methods. Then, we estimate the common components using ten dynamic principal components. Average values and their standard deviation of $D(\mathcal{X},\hat{\mathcal{X}})$ are computed over 100 simulations, which are listed in Table 3. Even if the proposed method is valid only under the stationarity, the proposed DPCA with ISC algorithm provides the closest estimate to the true common components. We expect that the results may be improved if we generalize the proposed method for non-stationary data as future research.

Table 3. Non-stationary common component – averages and their standard deviation of $D(\mathcal{X},\hat{\mathcal{X}})$ over 100 simulations.

Method	Average	Standard deviation
Linear interpolation	0.355	0.026
Kalman smoothing	0.148	0.016
ISC algorithm	0.133	0.010

6.2. Summary

DPCA is a popular statistical method to reveal the dynamics of the underlying data structure. However, we cannot estimate the spectral density matrix, which is essential in DPCA, when there exist missing values in the data. In this paper, we propose a new algorithm to impute the missing data based on a self-consistent inference on the frequency domain information.

‘Through the numerical experiments including simulations and real data analysis, we observe that the proposed DPCA with ISC algorithm works well compared to the other methods such as linear interpolation and Kalman smoothing.’ When the data is a sum of sinusoids, the proposed method works best. However, when the common components are generated from the process with less distinctive frequency such as MA(1), Kalman smoothing works best while the proposed method gives comparable results.

We expect that the proposed method can be generalized further to the non-stationary case, and various kind of real data containing missing values can be analyzed by the proposed method.

Funding Statement

This research was supported by the Seoul National University Research Grant in 2018 and the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Korea government (NRF-2018R1D1A1B07042933, NRF-2019R1A2C4069453).

Disclosure statement

No potential conflict of interest was reported by the authors.