Abstract
To handle principal component analysis (PCA)-based missing data with high correlation, we propose a novel imputation algorithm to impute missing values, called iterated score regression. The procedure is first to draw into a transformation matrix, which puts missing values and observed values into two data blocks, and then by using the data blocks, the score matrix, and PCA model to construct the related regression equations. The estimation update at the iteration is highlighted. We examine the sensitivity of the proposed algorithm, including the effects of standard deviations, correlation coefficients, missing proportions, variable numbers, and sample sizes with different intervals of the standard deviations and correlation coefficients. To compare some existing algorithms, we suggest the modifications of three popularly used algorithms that are also used to deal with missing data but are not highly correlated. In the numerical studies we conducted, the MSE values of the algorithm, to show its stability and accuracy, are always the smallest among the competitors we consider. It also shows the advantage, as the illustration, for three real missing data sets.
Keywords: Principal component analysis, Missing data, High correlation, Iterated score regression, Sensitivity
Subject terms: Operant learning; Statistical physics, thermodynamics and nonlinear dynamics
Introduction
Problems with missing data are very common in statistical analysis. To deal with the massive data sets with a small missing proportion (MP), a naive method is to delete the missing cases. However, in a data set with a large MP or small sample size, the method causes a severe loss of data information such that the total-data likelihood failed to obtain. This is often the case for the research fields where the data information is challenging to collect, for example, in cases with fast data updating and/or short timeliness of data. Thus, we have to estimate their missing values due to higher data utilization requirements. Some important types of missing data include categorical missing data, missing longitudinal data, random missing data, and small-size missing data.
To use factor models to fit categorical missing data sets, Josse et al.22 proposed a regularized iterative method by removing noises to avoid prediction instability. This method has fast speed and strong stability but may need to handle the algorithm convergence due to its iteration. Sun and Tchetgen33 used inverse probability-weighted estimate for longitudinal missing data sets to recover the missing information. This does not need to establish the distribution of complete data and improves the asymptotic efficiency. Its main issue concerns the estimation bias in small and medium data sets. To use a generalized linear model to fit random missing data sets, Zhao and Shao37 selected some of the covariates as instrumental variables to estimate the unknown parameters. This method mainly applies to generalized linear models under normal and Poisson distribution. McNeish27 established multiple imputations with a non-ridged before estimating the missing values in small-size missing data sets. Compared with existing missing data methods, its convergence is relatively slow though it often works well in practice.
This paper aims at the missing data problem in principal component analysis (PCA), a very important and popular method in multivariate statistics. It can reduce the computational complexity while retaining the primary data information by performing dimension reduction-based projections on the original data. We call the data set suitable for PCA a PCA-based data set. The PCA-based missing data problem arises in many practical applications, including chemical model building.9,10, ecological data analysis8 and medical data processing17,18. When the missing values cannot be reasonably estimated in PCA-based data sets, it will cause large errors in the results and the subsequent computation. It is noted that the existing estimation methods often have relatively strict requirements on data distributions and often suffer from discouraging accuracy and stability. For different types of PCA-based missing data, we may encounter other problems. For example, for PCA-based missing data with high missing proportion (MP), a severe over-fitting problem would occur; for mixed-type PCA-based missing data, especially those with rare categories in classification variables, the calculation accuracy could often be low; for high-dimensionally PCA-based missing data, it would often lead to high-computational cost.
For the above problems, Ilin and Raiko19 proposed the variational Bayesian PCA method to tackle PCA-based missing data with high MP. The method is used the estimated posterior covariance matrices to reconstruct the missing values and perform well in preventing the severe overfitting problem. However, the technique performs poorly in time cost.
There are some proposals available in the literature to deal with these problems. The following are some examples. Audigier et al.4 used the iterative factor analysis for mixed data to weight these variables to balance their effects for mixed-type PCA-based data with the continuous and categorical variables. This method has advantages in the estimation accuracy for the mixed-type PCA-based missing data with a linear correlation between categorical and continuous variables and with rare categories in categorical variables. It does not consider the mixed-typed PCA-based missing data on the interactions of different categorical variables. Audigier et al.5 used the Bayesian multiple imputation under a linear relationship of differenimputationss for high-dimensionally PCA-based missing data. The advantage of this method is that the confidence interval is short. But its performance does not seem very good when handling the low-rank PCA-based missing data high-dimensionally.
For the PCA-based missing data with high correlation (HC), the estimators of the missing data, the right singular matrix, and the simple covariance matrix face convergence problems; the estimation accuracies of the existing algorithms are often not high. We propose an iterated score regression (ISR) algorithm to solve the problems in this work.
We construct the difference between the ISR estimators of the centralized missing values and their true values, obtain the threshold on the centralized matrix, and establish the convergence on the ISR estimator of the missing values. It is shown that the ISR has a good convergence on the missing values.
Note that the right singular matrix of the centralized matrix is an important content of the ISR; it is necessary to research its convergence problem. We construct the difference between the estimator of the right singular matrix and the true value matrix of it and establish the convergence of the estimator of the right singular matrix. It is shown that the ISR has a good convergence on the right singular matrix.
Note that the column vector reflects the sample characteristics of a variable; it is significant to research its convergence problem of it. The sample variance matrix of a variable can be obtained simply by multiplying the centralized column vector by its transpose vector and reflecting the variance and correlation. We construct the difference of the sample covariance matrices of the true values and their estimators of them and establish the convergence on the estimator of the simple covariance matrix on the ISR. It is shown that the ISR has a good convergence on the sample covariance matrix.
With the ISR, we enhance the estimation accuracy and decentralized to construct the estimator in the missing data set. We simulate the discrete, continuous, and mixed PCA-based missing data with HC to examine the sensitivity and advantages of the ISR and apply the ISR to estimate the real missing data sets.
The rest of the paper is structured as follows. Section PCA-based missing data with high correlation introduces PCA-based missing data with HC and its missing mechanism. Section The iterated score regression estimation algorithm describes the generic estimation procedure. For comparison purposes, Section Modifications of three existing algorithms introduces the modifications of the three algorithms that have been popularly used in the literature to deal with missing data but not highly correlated. These algorithms are the regularized PCA-based algorithm (RPCA,21 &26), the Nonlinear iterative partial least squares algorithm (NIPALS,9), and the Maximum likelihood PCA-based algorithm (MLPCA,10). In Section Numerical analysis, we conduct several simulation studies to examine their performances and analyze some real data sets for illustration via several indicators, including mean squared error (MSE), mean absolute error (MAE), and relative error (RE). In Section Discussions, we discuss the proposed algorithm. Section Appendix: Technical proofs is an appendix including technical proofs of the main results.
PCA-based missing data with high correlation
An 
PCA-based data matrix 
is expressed as
 |
where the vectors 
and 
are the ith row vector and the jth column vector of the matrix 
, for 
and 
, respectively. Assume that p-dimensional vector 
is the mean vector of 
. The centralized matrix of 
is expressed as
 |
1 |
where the vectors 
and 
are the ith row vector and the jth column vector of the matrix 
, respectively. We construct an 
index matrix 
expressed as
 |
2 |
where the vectors 
and 
are the ith row vector and the jth column vector of the matrix 
, respectively. An 
PCA-based missing data matrix 
is expressed as
 |
3 |
where 
is the Hadamard product; the vectors 
and 
are the ith row vector and the jth column vector of the matrix 
, respectively. With the matrix 
in (2), if 
, the position ij corresponds to an observed value and is marked by subscript NA; otherwise, the position ij corresponds to a missing value, and is marked by subscript OB; that is, the missing values and observed values in 
is distinguished by 
and 
, respectively, see also Little and Rubin25. In addition, we denote the numbers of observed values and missing values in the vector 
as 
and 
, respectively, while the numbers of observed values and missing values in the vector 
are 
and 
, respectively. We set the number of missing values in the matrix 
as 
, the MP value is written as
 |
Further, we denote the p-dimensional column mean vector of observed values by 
, and use 
for 
to be the initial interpolation of 
for 
. We then obtain an updated matrix X written as
 |
4 |
where 
and 
are the ith row vector and jth column vector of the matrix X, respectively. The matrix X is the estimator of the matrix 
with the mean algorithm. The matrix Z is an 
column centralized matrix of the matrix X, written as
 |
5 |
where 
and 
are the ith row vector and jth column vector of the matrix Z, respectively. With singular value decomposition (SVD), the matrix Z can be expressed as
 |
where 
is an 
diagonal matrix with the singular values of 
on its diagonal, the singular values are usually denoted as 
, and the remaining entries are zero if 
. The left singular matrix B and right singular matrix A are 
and 
column orthogonal matrices, respectively.
We now construct the 
sample covariance matrix S on the matrix Z, written as
 |
6 |
where 
for 
. When we square 
, we are effectively squaring each of the singular values, and the resulting matrix 
is a 
diagonal matrix with the squared singular values on its diagonal. The diagonal matrices consist of the dominant k singular values, and the smaller 
singular values in (6) are written respectively
 |
7 |
The k-right singular matrix and the k-left singular matrix corresponding to the matrix 
are written respectively
 |
8 |
See also Abdi and Williams1. With the matrices 
and 
in (8), the data matrix 
in (3) is a PCA-based missing data matrix.
The correlation matrix R on the matrix Z is written as
 |
9 |
where 
and 
for 
. For the KMO test, the null hypothesis (H0) is that variables are uncorrelated, implying no underlying common factors. The alternative hypothesis (H1) is that variables are correlated, suggesting the presence of common factors. For the Bartlett’s Test of Sphericity (BTS), H0 assumes a correlation matrix equal to the identity matrix (no correlation among variables), while H1 assumes the correlation matrix is not the identity matrix (indicating correlations among variables), which is necessary for factor analysis. These tests determine the suitability of data for factor analysis, see also Kaiser23, Guo et al.14,15. Whether a PCA-based missing data set is highly correlated is determined by two tests of KMO and BTS, whose statistics are respectively expressed as
 |
where 
is a partial correlation coefficient of the matrix Z, expressed as
 |
where 
is the algebraic cofactor of 
. The BTS statistic follows the Chi-square distribution. In the null hypothesis, the correlation coefficient matrix R is to be an identity matrix; under the alternative hypothesis, the matrix is not an identity matrix.
PCA-based data refers to data sets suitable for PCA, where variables exhibit some level of correlation, though not necessarily high. PCA-based missing data indicates data sets with missing values that necessitate special handling to effectively apply PCA, ensuring that the analysis can proceed without issues from incomplete data. The PCA-based missing data with HC (in short, PCAMHC) emphasizes data sets that not only contain missing values but are also characterized by strong correlations between variables; handling such data requires preserving the robust correlation structure while addressing the missing values, which is critical for maintaining the effectiveness of PCA. Therefore, when a data set 
in (3) has a KMO value exceeding 0.7 and a BTS P-value below 0.1, it is defined as PCAMHC data, indicating suitability for PCA, presence of missing values requiring special handling, and the presence of high correlation among variables.
We determine the missing mechanism of the PCA-based missing data according to the conditional probability density (CPD) function of the index matrix 
in (2). Suppose the CPD function is 
, and 
. In that case, the missing mechanism is missing completely at random (MCAR), missing at random (MAR), and not missing at random (NMAR), respectively, see also Little and Rubin25. The missing mechanism of the PCA-based missing data is MCAR.
The iterated score regression estimation algorithm
In this section, we present a novel regression estimation algorithm–the ISR algorithm, using principal component scores and LS to estimate the existing PCAMHC.
To handle the missing data, we construct the 
elementary transformation matrix 
for 
to convert the missing data. With the subscripts 
and 
for 
, the positions of observed values and missing values only in the ith row of 
, the matrix 
on the ith row of 
is expressed as,
 |
10 |
where the subscripts u and v mean the order of missing values and observed values in the ith row of 
.
We fix the column number of the matrix 
as 
, and denote a 4-row vector 
, expressed as
 |
where the numbers of observed values and missing values are, 
and 
. The order of observed values and missing values are 
, and 
; the subscripts j of missing values and observed values are, 
and 
, respectively. Thus, for the subscript pairs 
and 
, the values are defined as 
. For all other pairs 
, 
.
The 
elementary transformation matrix 
is expressed as
 |
By using 
, we map the vector 
in (5), expressed as
 |
11 |
where 
represents right multiply by the matrix 
, the vectors 
and 
are 
and 
-dimensional vectors, respectively. We map the matrices Z in (5) and 
in (8) with 
, written as respectively
 |
12 |
where 
and 
are 
and 
matrices, 
and 
are 
and 
matrices, respectively. We structure the 
score matrix 
on Z, written as
 |
13 |
where 
is the ith row of 
. We construct the score regression equation as
 |
14 |
where 
is a 
coefficient matrix and 
is an 
error matrix.
By using the LS method, we obtain the regression coefficient estimator, expressed as
 |
15 |
By using the score vector in (13) and the estimated coefficient matrix in (15), we estimate the missing values and obtain the estimated vectors expressed as
 |
16 |
The iteration equation (16) includes two terms 
and 
. Here 
is a score vector, 
is an estimator. The vector 
is replaced with 
, and the estimator 
is obtained by using the inverse mapping matrix 
; that is,
 |
17 |
where 
represents right multiply by 
. The estimation result in 
is then expressed as
 |
18 |
The final estimator 
is expressed as
 |
The proposed algorithm uses the score matrix 
in (13) to construct the regression equation in (14). It obtains the estimation result through the iteration process in (16), then the algorithm is called as ISR algorithm. Figure 1 shows the flow diagram of the ISR. Algorithm 1 displays a flow diagram of the ISR for the PCAMHC.
Fig. 1.
Flow diagram of the ISR for PCA-based missing data with high correlation.
Note that the score regression (SR) algorithm was used by Arteaga and Ferrer2 to handle missing data in process control, applied by Arteaga and Ferrer3 to handle online missing data in process control, and used by Folch-Fortuny et al.9 to handle with PCA-based missing data with non-high-correlation. However, the estimation accuracy of it is not high for PCA-based missing data, the process of converting missing data is not standardized, and the estimation result on missing data with HC is not ideal.
Here the ISR algorithm adds an iteration step in (16) to improve the estimation accuracy, uses a given matrix 
in (12) to obtain the specification of the conversion process on missing data, and centralizes the updated data matrix in (4). The ISR has the following advantages compared with the SR. The demands of data information are relatively small for the SR; the estimation error is reduced for the SR; the matrix 
in (12) displays the mapping processes of the estimation on the ith row of 
while the estimator is obtained on the original structure of 
by using the inverse matrix of 
.
Algorithm 1.
The ISR for PCA-based missing data with high correlation.
With the matrix 
in (10), we map the vector 
in (1), expressed as 
for 
We now have the following convergence theorem on the ISR estimator 
.
Theorem 1
(Convergence of the ISR estimator) Assume
that
, 
, 
with the
matrix
for
. We have, for any
,
 |
We perform SVD on the matrix 
in (18) and obtain 
where the matrix 
is a 
right singular matrix on 
. The matrix 
is a 
inverse matrix of the matrix 
, the matrix 
is a 
pseudo-inverse matrix of the matrix 
. With 
and 
, we construct the matrix 
. Assume that the matrices 
and 
are the true value matrices of 
and 
, respectively. With 
and 
, the true value matrix of 
is expressed as 
Theorem 2
(Convergence of the right singular matrix estimator) Assume that
, 
and
; there is a constant
such that
, 
, 
and
. We have, for any
,
 |
Moreover, because the column vector reflects the sample characteristics of a variable, it is also necessary to consider the convergence of the column vector estimator. Here, we present the convergence of the column vector on the ISR estimator 
.
Theorem 3
(Convergence of the centralized matrix estimator) Assume that, with the
j-th column vectors
and
, for the true value
and the ISR estimator
, 
, 
; 
, 
. There exists a constant
such that
 |
Then, we have, for any
,
 |
Modifications of three existing algorithms
To make comparisons with our algorithm, we modify the three existing algorithms to handle the PCAMHC. Three following algorithms have been used to deal with the missing data. The first is the regularized PCA-based algorithm (RPCA); see the references21,24 & Loisel and Takane26.
The second is the nonlinear iterative partial least squares (NIPALS) (e.g.9). The third is maximum likelihood principal component analysis (MLPCA). The relevant references are Wentzell et al.36 and Folch-Fortuny, et al.10, among others. The key modifications are to use the elementary transformation matrix 
in (10) to modify the existing algorithms. On the one hand, these modifications select the data information and reduce the estimation error; on the other hand, these modifications further reduce computational costs. The details are described as follows.
Modified RPCA
First, we map the vector 
in (5) with the matrix 
in (5), and obtain the vectors 
and 
. We construct the matrices 
and 
in (7), and the matrices 
and 
in (8). We map 
with 
, and obtain the matrices 
and 
. The modified RPCA (MRPCA) algorithm can then be expressed as
 |
19 |
where the matrix C is a 
loading matrix; the matrices 
and 
are, the 
and 
matrix, respectively; the k-dimensional transposed score vector 
is distributed as 
; the p-dimensional transposed error vector 
is distributed as 
; 
and 
-dimensional transposed vectors 
and 
are distributed as 
and 
, respectively. Thus it is shown that 
is distributed as 
, see also Bellas, et al.6.
We construct the estimator of the matrix 
in (19), expressed as
 |
20 |
and the estimator of the score vector 
in (19) can be written as
 |
21 |
where the k-dimensional vector 
is the ith row of 
in (8). With 
in (7), the estimator of 
can be expressed as
 |
We use 
in (20) and 
in (21) to be the MLEs of 
and 
, see also Loisel and Takane26. Then, we substitute 
and 
into the equation (19) to estimate 
, expressed as
 |
With the estimator vector 
, we modify the estimation of the existing RPCA algorithm with the matrix 
and obtain the matrix 
. The absolute convergence criterion is met. Finally, the final estimator 
is obtained. With regularized strategy, the MRPCA algorithm estimates the load matrix C and the score vector 
Algorithm 2 displays the MRPCA for the PCAMHC.
Algorithm 2.
The MRPCA for PCA-based missing data with high correlation.
Modified NIPALS
We construct the vectors 
and 
with the matrix 
in (10), and denote the n-dimensional vector 
as the first score vector of the matrix Z in (5). We choose the column vector with the biggest modulus in Z as the initial value of 
; that is,
 |
The regression of the matrix Z on 
is written as
 |
where the p-dimensional vector 
is a coefficient vector, and the 
matrix 
is an error matrix of Z. The LS estimator of the transposed vector 
is expressed as
 |
The estimator vector 
is centralized as the estimator of the first right singular vector 
in (8), expressed as
 |
The regression of 
on 
is written as
 |
where the 
matrix 
is an error matrix of 
. The LS estimator of the transposed vector 
is expressed as
 |
The vector 
is assumed as the true value of 
, and an absolute convergence criterion is given by the following threshold,
 |
22 |
where 
is a constant. According to the above steps, we sequentially generate the first score-right singular vector pair 
.
We map 
with the matrix 
, expressed as
 |
The estimator of the missing value vector 
is expressed as
 |
where 
is the ith element of the estimator vector 
. We modify the estimation of the existing NIPALS algorithm with 
. The estimator vector 
is obtained by the given vectors 
and 
. With the estimator 
, we obtain the final estimator 
. Figure 2 shows the flow diagram of the modified NIPALS (MNIPALS) algorithm for the missing data. Algorithm 3 displays the MNIPALS algorithm for the missing data.
Fig. 2.
Flow diagram of the MNIPALS for PCA-based missing data with high correlation.
Note that the MNIPALS only uses the first score vector and the first right singular vector; the amount of computation is small, especially when it has large MP values in PCA-based missing data set; the MNIPALS has an advantage in time costs. The MNIPALS algorithm uses the centralization matrix Z to modify the NIPALS and deals with the HC problem.
Algorithm 3.
The MNIPALS for PCA-based missing data with high correlation.
Modified MLPCA
Again, we use the matrix 
in (10) to modify the row step of the existing MLPCA algorithm. For the row step, with the vectors 
in (5) and 
in (1), we construct 
, and map it, expressed as
 |
where 
is the error vector of 
. We map the matrix 
in (8) with 
, and obtain the matrices 
and 
. Assume that the errors are independent, and the covariance matrices of 
and 
are 
and 
, respectively. Thus, the 
covariance matrix of 
is expressed as
 |
For the row step, we construct the MLE of 
as
 |
23 |
With the inverse mapping matrix 
, the vector 
is obtained by the given vectors 
and 
. The estimation result on the row step is expressed as
 |
24 |
The summation of the weighted squared residual is written as
 |
25 |
where the estimator vector 
is defined in (23), and 
is the true value of the vector 
.
For the column step, we firstly construct an 
elementary transformation matrix 
of the jth column in the original matrix 
, as the same as (10). We use 
to modify the column step of the existing MLPCA. With the vectors 
in (24) and 
in (1), we construct 
, and map it with 
, expressed as
 |
26 |
where 
represents left multiply by 
, 
is the error vector of 
. We perform SVD on 
and obtain the k-left singular matrix 
, expressed as
 |
where 
and 
are 
and 
matrices, respectively. As the same as the row step, we further assume that the errors are independent and the covariance matrices of 
and 
are 
and 
, respectively. Thus, we obtain the 
covariance matrix of 
as
 |
At the column step, we construct the MLE of 
as
 |
27 |
With the vectors 
and 
, we use the inverse matrix 
to modify the result of the column step of the existing MLPCA, expressed as
 |
where 
represents left multiply by 
. With the column step, the estimator is expressed as
 |
The summation of the weighted squared residual is expressed as
 |
28 |
where 
is the MLE defined in (27), and 
is the true value of the vector 
.
Then, the absolute convergence criterion is written as
 |
29 |
where 
is a constant; 
and 
are defined in (25) and (28), respectively. The absolute convergence criterion is met, and the estimator of Z is expressed as 
. Finally, the final estimator 
is given. Figure 3 shows the flow diagram of the modified MLPCA (MMLPCA) for the missing data. Algorithm 4 displays the MMLPCA for the missing data. The MMLPCA is an alternating iteration PCA algorithm with a row step and column step, which combines the low-rank model and generalized regression equation to construct the MLE. The matrices 
and 
(
) modify the column step of the existing MLPCA algorithm and display the mapping inverse mapping processes in the column step, respectively. With the centralization matrix Z, the MMLPCA may deal with the missing data problem.
Fig. 3.
Flow diagram of the MMLPCA for PCA-based missing data with high correlation.
Algorithm 4.
The MMLPCA for PCA-based missing data with high correlation.
Numerical analysis
In this section, we examine the performance of the ISR for handling the PCAMHC both in simulation and in real data analysis. We have implemented the algorithm with an R package called ISR16, which is available at https://CRAN.R-project.org/package=ISR. To ensure the robustness of our findings, each scenario was replicated 5 times.
Preparation
We select three indicators to examine the performance of the ISR, including the MSE of 
the MAE of 
and the RE of 
. The computation formulas of these three indicators are expressed respectively,
 |
The smaller the absolute values of these three indicators are, the better the estimation effect is. Among them, the MSE values reflect the concentration of the estimates on the best-fit line, and the MAE and RE values reflect the deviation of the predicted values from the true values. The values of MSE, MAE, and RE are related to the values of 
and suitable for the simulation study. Note that the values of 
are unavailable in missing data sets; we use another indicator. The GCV is used to obtain the loss function on the ISR and assesses its performance in missing data sets. The GCV is expressed as
 |
where 
is the eigenvalue of the estimator 
; that is, the eigenvalues obtained from a matrix derived from 
which is used to assess the performance of the imputation algorithm. It is obvious that the smaller the GCV values are, the higher the accuracy of the ISR is; see also Josse and Husson20.
Simulations
In this subsection, we examine the influence of several parameters on the ISR and verify the performance of the ISR in a simulation example.
Sensitivities
Note that these algorithms are all based on the centralization matrix Z in (5). Thus, in the following three cases, we directly simulate the centralization matrix Z with n samples that obey p-dimensional multivariate distribution. For several missing values with the MP values, we use the MSE values to examine the accuracy of the ISR.
Case 1: (Multivariate Normal Distribution)
The samples obey p-dimensional normal distribution 
where 
, see also Staedler and Buehlmann32. To simplify the computation, we construct a positive definite correlation matrix R in (9), and select the variances 
to construct the covariance matrix 
, expressed as
 |
In Case 1, the standard deviation is the first parameter we consider to affect the accuracy of the ISR. For fixed 
, we set the correlation coefficients 
for 
. The standard deviation 
is put in four regions [0, 1], (1, 2], (2, 3], and (3, 4]. We show the accuracies of different standard deviations on the algorithm in Case 1; see Table 1.
Table 1.
MSE values of different standard deviations in Case 1.
 |
I | II | III | IV | |
|---|---|---|---|---|---|
| Algorithms | SR | 0.307 | 1.997 | 5.332 | 10.435 |
| ISR | 0.225 | 1.679 | 4.661 | 9.266 | |
| MRPCA | 0.298 | 2.273 | 6.403 | 12.732 | |
| MNIPALS | 0.252 | 1.733 | 4.797 | 9.498 | |
| MMLPCA | 0.302 | 2.302 | 6.461 | 12.877 | |
| Mean | 0.300 | 2.287 | 6.429 | 12.770 |
I: [0,1], II: (1,2], III:(2,3], IV:(3,4].
As shown in Table 1, the MSE values of the ISR are between 0.225 to 9.266, those of the SR are between 0.307 to 10.435, those of the MRPCA are between 0.298 to 12.732, those of the MNIPALS are between 0.252 to 9.498, those of the MMLPCA are between 0.302 to 12.877, and those of the mean algorithm is between 0.300 to 12.770. For 
, the MSE values of these six algorithms are the smallest; that is, the smaller the standard deviations, the better the estimation accuracies of these algorithms on the missing data. In summary, for the same range of standard deviations, the accuracy of the ISR is the best, and the smallest of the MSE values reaches 0.225.
The second parameter is the correlation coefficient. For fixed 
= 
with 
, the range of 
is divided into four regions: [0, 0.25], (0.25, 0.5], (0.5, 0.75], and (0.75, 1] for 
. The influences of different correlation coefficients are shown in Table 2. As shown in Table 2, with the increase of 
for 
, the MSE values of these algorithms are decreased rapidly; that is, the bigger the correlation coefficients are, the better the estimation accuracies of these algorithms on the missing data are. In summary, the estimation accuracy of the ISR is also the best among these algorithms.
Table 2.
MSE values of different correlation coefficients in Case 1.
 |
I | II | III | IV | |
|---|---|---|---|---|---|
| Algorithms | SR | 0.353 | 0.283 | 0.194 | 0.093 |
| ISR | 0.296 | 0.206 | 0.124 | 0.039 | |
| MRPCA | 0.300 | 0.290 | 0.287 | 0.276 | |
| MNIPALS | 0.305 | 0.215 | 0.137 | 0.055 | |
| MMLPCA | 0.303 | 0.301 | 0.299 | 0.296 | |
| Mean | 0.301 | 0.299 | 0.296 | 0.294 |
I: [0,0.25], II: (0.25,0.5], III: (0.5,0.75], IV: (0.75,1].
We now consider the influence of the MP values for the ISR for fixed 
. Taking the five values of MP as 10%, 20%, 30%, 40%, and 50%, we examine the performance of the ISR. The examination results are shown in Fig. 4.
Fig. 4.
Comparison results of different MP values in Case 1. (a) MSE. (b) time costs of three algorithms (s). (c) time costs of the MMLPCA (s).
As shown from Panel (a) in Fig. 4, the MSE values of the ISR are between 0.039 and 0.065 when the MP values range from 10% to 50%, those of the SR are between 0.093 to 0.924, those of the MRPCA are between 0.276 and 0.319, those of the MNIPALS are between 0.055 and 0.147, those of the MMLPCA are between 0.295 and 0.338, and those of the mean algorithm is between 0.294 and 0.335. The MSE values of our proposed algorithm are the smallest compared to those of the other five methods, those of the MNIPALS are followed, those of the SR are the third, those of the MRPCA are the fourth, those of the mean algorithm are the fifth, and those of the MMLPCA are the largest. When the MP value is 40% and 50%, the MSE value of the SR is the largest among these six algorithms.
The MSE values of the SR and the MNIPALS increase as the MP values increase. The values of the ISR increase when the MP values increase from 10% to 30% and 40% to 50%, while the values decrease when the MP values increase from 30% to 40%. The values of the MRPCA, the MMLPCA, and the mean algorithm increase when the MP values increase from 10% to 20% and 40% to 50%, while the values decrease when the MP values increase from 20% to 40%. Moreover, with the change of the MP values, the slope of the MSE values of the SR is lower than those of the ISR and the other four algorithms.
As shown from Panels (b) and (c) in Fig. 4, the time costs of the MNIPALS range from 0.697 to 1.008 seconds, those of the MRPCA range from 3.569 to 5.627 seconds, those of the ISR range from 6.127 to 13.905 seconds, and those of the MMLPCA range from 13.499 to 20.909 seconds. The time costs of the MNIPALS are the smallest; those of the MRPCA are followed, those of the ISR are the third, and those of the MMLPCA are the largest. The time costs of all four algorithms are significantly upward with the increase of the MP values.
The ISR has a higher computational cost compared to MRPCA and MNIPALS due to its iterative nature and complex operations. Each iteration involves recalculating the score matrix and solving regression equations, which are computationally intensive, especially with large data sets. Additionally, the ISR requires SVD for constructing the score matrix, a process that is computationally demanding. Despite this, ISR offers significant advantages in terms of accuracy and robustness, particularly in data sets with high correlation. Its iterative refinement leads to more precise estimates of missing values, making it a valuable tool for complex data analysis, even though it demands more computational resources.
We now consider the influence of the sample size n on the estimation effects of the ISR. For fixed 
, we examine the performance of the algorithm when 
, and 4000. The examination results on different sample sizes are shown in Fig. 5.
Fig. 5.
Comparison results of different sample sizes in Case 1. (a) MSE. (b) time costs of three algorithms (s). (c) time costs of the MMLPCA (s).
As shown in Panel (a) in Fig. 5, the MSE values of the ISR are the smallest, those of the MNIPALS are followed, those of the SR are the third, those of the MRPCA are the fourth, those of the mean algorithm are the fifth, and those of the MMLPCA are the largest. Moreover, the MSE values of the SR decrease when n increases from 500 to 2000, while the values increase when n increases from 2000 to 4000, while those of the other five algorithms have downward trends when n increases from 500 to 1000 while the values have upward trends when n increases from 1000 to 5000. Moreover, with the change of n, the slopes of the MSE values of the ISR, the SR, and the MNIPALS are lower than those of the MRPCA, the MMLPCA, and the mean algorithm. As shown from Panels (b) and (c) in Fig. 5, the time costs of these four algorithms are in the same order as those in Fig. 4. Moreover, the time costs of these four algorithms all have upward trends with an increase of n.
For fixed 
, we examine the performance of the ISR when taking p as 10, 20, 30, and 40, respectively. The examination results on different variable numbers are shown in Fig. 6.
Fig. 6.
Comparison results of different dimension numbers in Case 1. (a) MSE. (b) time costs of three algorithms (s). (c) time costs of the MMLPCA (s).
As shown from Panel (a) in Fig. 6, for different p, the MSE values of the MNIPALS have upward trends when p increases from 10 to 20 while having downward trends when p increases from 20 to 40, those of the SR have downward trends with the rise of p, those of the ISR, the MRPCA, the MMLPCA, and the mean algorithm all have upward trends with the increase of p. In addition, with the change of p, the slopes of the MSE values of the ISR and the MNIPALS are lower than those of the SR, the MRPCA, the MMLPCA, and the mean algorithm. As shown in Panels (b) and (c), the time costs of these algorithms are in the same order as those in Figs. 4 and 5. The MNIPALS’s computational efficiency is a significant advantage in these scenarios, particularly in applications where time is a critical factor. It makes the MNIPALS a particularly attractive option for real-time data processing or when dealing with large data sets. Moreover, the time costs of these four algorithms all have upward trends with an increase of p.
Case 2: (Multivariate Poisson Distribution)
The samples obey the Poisson distribution 
where 
is both the mean values and the variance values.
In Case 2, the first parameter that we consider to affect the accuracy of the ISR is the parameter 
. For fixed 
, putting the parameter 
in four regions [0, 7], [0, 8], [0, 9], and [0, 10], we show the accuracies of different values of 
on the algorithm in Case 2, see Table 3. As shown in Table 3, for 
, the MSE values of these six algorithms are the smallest. In summary, for the same range of 
, the accuracy of the ISR is the best.
Table 3.
MSE values of different 
in Case 2.
 |
I | II | III | IV | |
|---|---|---|---|---|---|
| Algorithms | SR | 3.642 | 4.198 | 4.716 | 5.300 |
| ISR | 3.619 | 4.171 | 4.692 | 5.272 | |
| MRPCA | 7.148 | 8.877 | 10.714 | 12.753 | |
| MNIPALS | 3.632 | 4.220 | 4.782 | 5.393 | |
| MMLPCA | 7.241 | 9.039 | 10.878 | 12.909 | |
| Mean | 7.209 | 8.949 | 10.798 | 12.849 |
I: [0,7], II: [0,8], III: [0,9], IV: [0,10].
The second parameter that we consider to affect the accuracy of the ISR is the MP value. For fixed 
, and taking the five values of MP as 10%, 20%, 30%, 40%, and 50%, we examine the perform of the ISR. The examination results of different MP are shown from Panel (a) in Fig. 7. The MSE values of the ISR when the MP value is 10%, and 20% are the smallest among these six algorithms. The values increase when the MP values increase from 10% to 20% and 30% to 50%, while the values decrease when the MP values increase from 20% to 30%. The values of five other algorithms increase when the MP values increase from 10% to 20% and 30% to 40%, while the values decrease when the MP values increase from 20% to 30% and 40% to 50%.
Fig. 7.
Comparison results of the ISR algorithm in Case 2. (a) MSE values of different MP (101). (b) MSE values of different n (101). (c) MSE values of different p (101).
We now consider the influence of the sample size n on the estimation effects of the ISR. For fixed 
, we examine the performance of the ISR when 
, and 4000. The examination results on different sample sizes are shown from Panel (b) in Fig. 7. When n is 1000, 2000, 3000, and 4000, the MSE values of the algorithm are the smallest. The MSE values of the ISR, the SR, and the MNIPALS decrease when n increases from 500 to 1000 and 2000 to 4000, while the values decrease when n increases from 1000 to 2000.
Finally, we consider the influence of the dimension number p on the estimation effects of the ISR. For fixed 
, we examine the performance of the ISR when taking p as 10, 20, 30, and 40, respectively. The examination results on different variable numbers are shown from Panel (c) in Fig. 7. When p is 10, the values of the ISR are the smallest. The values of the MNIPALS increase when p increases from 10 to 30, while the values decrease when p increases from 30 to 40. The MSE values of the ISR and the other four algorithms increase when p increases from 10 to 20 and 30 to 40, while the values decrease when p increases from 20 to 30.
Case 3: (Multivariate Mixed Distribution)
The samples of 
variables obey 
-dimensional normal distribution 
where 
, 
While the samples of the other 
variables obey the Poisson distribution 
, where 
is both the mean values and the variance values. Here, we set 
.
For Case 3, the first parameter that we consider to affect the accuracy of the ISR is 
. For fixed 
, setting the parameter 
, we show the accuracies of different values of 
on the algorithm in simulation, see Table 4. As shown in Table 4, for 
, the MSE values of these six algorithms are the smallest. In summary, for the same value of 
, the accuracy of the ISR is the best.
Table 4.
MSE values of the ISR algorithm in Case 3.
 |
8 | 6 | 4 | 2 | |
|---|---|---|---|---|---|
| Algorithms | SR | 1.093 | 1.689 | 3.001 | 4.249 |
| ISR | 1.018 | 1.587 | 2.885 | 3.634 | |
| MRPCA | 1.250 | 2.223 | 3.785 | 8.030 | |
| MNIPALS | 1.301 | 1.813 | 3.623 | 3.676 | |
| MMLPCA | 1.262 | 2.239 | 3.810 | 8.135 | |
| Mean | 1.254 | 2.232 | 3.793 | 8.068 |
The second parameter that we consider to affect the accuracy of the ISR is the MP value. For fixed 
, and taking the five values of MP as 10%, 20%, 30%, 40%, and 50%, we examine the performance of the ISR. The examination results of different MP are shown from Panel (a) in Fig. 8. The MSE value of the ISR is the smallest among these six algorithms. The values of these six algorithms increase when the MP values increase from 10% to 20% and 30% to 40%, while the values decrease when the MP values increase from 20% to 30% and 40% to 50%.
Fig. 8.
Comparison results of the ISR algorithm in Case 3. (a) MSE values of different MP (101). (b) MSE values of different n (101). (c) MSE values of different p (101).
We now consider the influence of the sample size n on the estimation effects of the ISR. For fixed 
, we examine the performance of the ISR when 
, and 4000. The examination results on different sample sizes are shown from Panel (b) in Fig. 8. For different n, the ISR’s MSE values are always the smallest. The values of the algorithm, the SR, and the MNIPALS increase when n increases from 500 to 2000, while the values decrease when n increases from 2000 to 4000. The values of the MRPCA, the MNIPALS, and the mean algorithm increase when n increases from 500 to 3000, while the values decrease when n increases from 3000 to 4000.
Finally, we consider the influence of the dimension number p on the estimation effects of the ISR. For fixed 
, we examine the performance of the ISR when taking p as 10, 20, 30, and 40, respectively. The examination results on different variable numbers are shown in Panel (c) in Fig. 8. For different p, the MSE values of the ISR are still the smallest. The values of the MNIPALS decrease as p increases, while the values of five other algorithms decrease as p increases from 10 to 30 and increase as p increases from 30 to 40.
In summary, the MSE values of the ISR in Cases 1 and 3 are the lowest among these six algorithms; the values of the ISR in Case 2 are not bad among these six algorithms. Moreover, the time costs of the ISR are similar in these three cases. Thus, we display the values in Case 1.
Simulation examples
We select a complete real data set – Travel reviews data set provides 980 travelers’ reviews of 10 different types of travel facilities in East Asia. All the variables are continuous; see Renjith et al.31.
The examined results of KMO and BTS are shown in Table 5. The KMO value is 0.610, and the P-value of the BTS is 0. The data set is then suitable for PCA and does not meet the requirements of HC. Moreover, 230 samples are selected as a data subset with HC, the KMO value is 0.704, and P-value is 
-124. The data subset meets the requirements for HC and is named as HC data subset.
Table 5.
KMO and BTS of simulation example in simulation.
| Data set | KMO | BTS | ||
|---|---|---|---|---|
| Chi-square | df | P-value | ||
| Travel reviews | 0.610 | 442.088 | 45 | 0 |
| HC data subset | 0.704 | 730.371 | 45 | 1.397e-124 |
For fixed 
in the data set, we take the MP values as 10%, 20%, 30%, 40%, and 50% in simulation, and obtain the estimates of the ISR, the SR, the MRPCA, the MNIPALS, the MMLPCA, and the mean algorithm. The examination results of the MSE, the MAE, and the RE for the ISR are shown in Fig. 9. Panels (a), (b), and (c) in Fig. 9 present the results in the HC data subset, and Panels (d), (e), and (f) in Fig. 9 give the results in the total data set.
Fig. 9.
Comparison results of the ISR algorithm both in HC data subset and in total data set. (a) MSE in HC data subset. (b) MAE in HC data subset. (c) RE in HC data subset. (d) MSE in total data set. (e) MAE in total data set. (f) RE in total data set.
As is shown from Panels (a), (b), and (c) in Fig. 9, most of the indicator values of the ISR in the HC data subset are the smallest while those of the MNIPALS are followed, those of the MRPCA are the third, those of the mean algorithm are the fourth, those of the MMLPCA is the fifth, and those of the SR is the biggest. The MSE values of the MNIPALS are the smallest when the MP value is 
, while all the indicator values of the MNIPALS are the smallest when the MP value is 
.
As is shown from Panels (d), (e), and (f) in Fig. 9, most of the indicator values on these six algorithms in the total data set are in the same order as those in the HC data subset. The MSE values of the SR are less than those of the MRPCA when the MP value is 
, those of the MMLPCA and the mean algorithm.
Compared with Panels (d), (e), and (f) in Fig. 9 in the total data set, most of the indicator values from Panels (a), (b), and (c) in the HC data subset are smaller. In contrast, the indicator values of the ISR are larger than those in the total data set when the MP value is 50%, and the difference in the indicator values on these six algorithms is more prominent.
Real data analysis
We select three real data sets with missing values to examine the ISR. The first one is chronic kidney disease (CKD) data set from the UCI repository; there are 18 variables, 13 discrete variables, and 5 continuous variables. It includes 400 instances where 250 instances of them with CKD and others without CKD, see also Polat et al.30. There are 1010 missing values in the CKD data set, accounting for 
. The second one is the Ozone data set, which has 112 daily measurements on 13 meteorological variables; see also Audigier et al.4. Of these variables are discrete variables, 8 are continuous variables. There are 115 missing values in the Ozone data set, accounting for 
. The third is the Orange data set, which describes 12 orange juices on 8 continuous sensory descriptors. There are 19 missing values in the data set, accounting for 19.79%. The values of KMO and BTS in these data sets all meet our requirements for HC and are shown in Table 6.
Table 6.
The results of KMO and BTS in real data sets.
| Data sets | KMO | BTS | ||
|---|---|---|---|---|
| Chi-square | df | P-value | ||
| CKD | 0.84 | 2730.464 | 153 | 0 |
| Ozone | 0.85 | 945.6959 | 55 | 0 |
| Orange | 0.70 | 39.2214 | 28 | 0.077 |
The GCV indicator is used to assess the accuracy of the ISR in these three missing data sets. The GCV values are shown in Fig. 10. We can see that the GCV value of the ISR in the CKD data set is 0.6341, the value in the Orange data set is 0.3927, and the value in the Ozone data set is 0.2250. The GCV values of the ISR are always the smallest among these six algorithms in these three data sets. It means that the estimates of the ISR are the most reasonable.
Fig. 10.
GCV values of the ISR algorithm in three missing data sets. (a) CKD. (b) Ozone. (c) Orange.
Discussions
Note that the score matrix for the ISR reflects the information of the reasonable sub-matrix after the mean interpolation; the score matrix we used is more reasonable than the ordinary score matrix. The similarity between the ISR and the MRPCA, the MNIPALS, and the MMLPCA is that they all use PCA knowledge, such as the score vectors and the loading vectors, to estimate the PCA-based missing values.
It is noted that in Section Simulations, with multivariate normal distribution and Poisson distribution, we simulate continuous, discrete, and mixed PCAMHC. We examine the ISR’s effect on the missing data with MCAR. For the cases of continuous and mixed PCAMHC, the MSE values of the ISR are the lowest among these six algorithms. It is worthy of application in the data sets with other distributions. For discrete PCAMHC, the MSE values of the ISR are the lowest under some settings. It is meaningful to modify the ISR further to improve the accuracy of the data sets. Note that the time costs of the ISR are similar in all these 3 cases; we only show the time costs in Case 1.
It is shown in Fig. 4 all the KMO values exceed 0.9; the higher the KMO value in the PCA-based missing data set is, the more obvious the advantage of the algorithm is. In Fig. 9, the values of the ISR in HC data subsets are always the best among five of those algorithms when the MP values are less than or equal to 30%, and 
may be an inflection point.
Due to the advantage in estimation accuracy, the ISR will further bring convenience to life and scientific research, for example, the early stage of major diseases and the recording of precision instruments. The time costs of the ISR mainly depend on the iteration number because it is an iterative algorithm. Thus in the future, we can further combine the ISR with other ideas to improve the convergence speeds and save time costs. Moreover, it is worth noting that the research object here is the PCAMHC and its strong simulation conditions. In the future, we can appropriately weaken these conditions to make the research more universal.
The ISR, initially developed for unsupervised PCA to handle missing data, can be adapted for supervised learning. In this context, the ISR aids in dimensionality reduction and feature selection by extracting principal components that capture significant data variance, thus serving as effective inputs for predictive models. Its capability to impute missing values also ensures data set completeness, potentially boosting model accuracy. Furthermore, the ISR can augment features, adding valuable information that may enhance a model’s predictive power. However, care must be taken to balance these benefits against potential information loss and increased model complexity. Integrating the ISR into a supervised pipeline requires careful consideration of how transformations might affect feature-target relationships. Combining the ISR with methods that utilize label information can help maintain the integrity of these relationships while leveraging ISR’s strengths.
Accelerated distributed EMs enhance large-scale data processing with fast convergence, see Guo, et al.13. ISR addresses PCA-based missing values but could integrate with distributed EM for better performance. The COR criterion provides an effective method for optimal subset selection in distributed settings (see11), which may offer valuable insights into optimizing the iterative process in ISR. The ORKM introduces regularization terms to better capture trends (see12), making ISR more adaptable to dynamic data environments with real-time updates. The distributed online EM for Poisson mixture models showcases strong capabilities in processing count-type data (see35), offering potential ideas for expanding ISRs application scope by incorporating sparsity and distribution concepts.
Supplementary materials
This section details the supplementary materials associated with this paper, which are housed within the accompanying supplementary archive:
Codes: The R code necessary to implement the discussed algorithms is provided. Interested readers are directed to the document titled “Code ISR.pdf”.
Function: The R functions associated with the algorithms are also made available. The functions are stored in the archive “Function.rar”.
Data sets: The data sets for real data analysis are provided in the supplementary materials archive. They can be found in the file “Data sets.rar”.
Supplementary Information
Acknowledgements
The authors thank the editors and the reviewers for their careful work and thoughtful suggestions that have helped improve this paper substantially. The research of Lixing Zhu, described herewith was supported by the NSFC grants (NSFC11671042, NSFC12471276), a grant (CI2023C063YLL) funded by the China Academy of Chinese Medical Science, and the research of Guangbao Guo, was supported by the NSSC grant (NSSC23BTJ059).
Appendix: Technical proofs
Proof of Theorem 1
We denote an n-dimensional vector 
, where the ith value is 1 and others are all 0. We construct the difference of the vectors 
and 
, expressed as
 |
With 
and 
, we obtain
 |
With 
, we construct the threshold on the vector 
, expressed as
 |
We construct the threshold on the centralized matrix Z, expressed as
 |
Finally, with 
, we construct the absolute convergence criterion with the following threshold
 |
There exists any 
such that
 |
Proof of Theorem 2
We construct the difference of 
and 
, expressed as
 |
Under the given conditions, we construct the F-norm of 
with the following threshold
 |
There exists any 
such that
 |
Proof of Theorem 3
Note that,
 |
and
 |
For 
or 
, we then have, for any 
,
 |
Note that,
 |
Note also that,
 |
We then have, for any 
,
 |
Author contributions
Guangbao Guo and Lixing Zhu wrote the main manuscript text and Haoyue Song prepared the figures and the tables. All authors reviewed the manuscript.
Data availability
The data that can be used to replicate the results presented in this paper is available by reasonable request after contacting the corresponding author.
Declarations
Competing interests
The authors declare no competing interests.
Footnotes
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Guangbao Guo, Haoyue Song and Lixing Zhu: These authors contributed equally.
Supplementary Information
The online version contains supplementary material available at 10.1038/s41598-025-93333-6.
References
- 1.Abdi, H. & Williams, L. Principal component analysis. Wiley Interdisciplinary Reviews: Computational Statistics2(4), 433–459 (2010). [Google Scholar]
- 2.Arteaga, F. & Ferrer, A. Dealing with missing data in MSPC: several methods, different interpretations, some examples. Journal of Chemometrics16(8–10), 408–418 (2002). [Google Scholar]
- 3.Arteaga, F. & Ferrer, A. Framework for regression-based missing data imputation methods in on-line MSPC. Journal of Chemometrics19(8), 439–447 (2005). [Google Scholar]
- 4.Audigier, V., Husson, F. & Josse, J. A principal components method to impute missing values for mixed data. Advances in Data Analysis and Classification10(1), 5–26 (2014). [Google Scholar]
- 5.Audigier, V., Husson, F. & Josse, J. Multiple imputations for continuous variables using a bayesian principal component analysis. Journal of Statistical Computation and Simulation86(11), 2140–2156 (2015). [Google Scholar]
- 6.Bellas, A., Bouveyron, C., Cottrell, M. & Lacaille, J. Model-based clustering of high-dimensional data streams with an online mixture of probabilistic PCA. Advances in Data Analysis and Classification7(3), 281–300 (2013). [Google Scholar]
- 7.Chronic Kidney Disease Dataset. (2015). UCI Machine Learning Repository. https://archive.ics.uci.edu/ml/datasets/Chronic+Kidney+Disease
- 8.Dray, S. & Josse, J. Principal component analysis with missing values: a comparative survey of methods. Plant Ecology216(5), 657–667 (2015). [Google Scholar]
- 9.Folch-Fortuny, A., Arteaga, F. & Ferrer, A. PCA model building with missing data: New proposals and a comparative study. Chemometrics and Intelligent Laboratory Systems146, 77–88 (2015). [Google Scholar]
- 10.Folch-Fortuny, A., Arteaga, F. & Ferrer, A. Assessment of maximum likelihood PCA missing data imputation. Journal of Chemometrics30(7), 386–393 (2016). [Google Scholar]
- 11.Guo, G., Song, H. & Zhu, L. The COR criterion for optimal subset selection in distributed estimation. Statistics and Computing34, 163–176 (2024). [Google Scholar]
- 12.Guo, G., Song, H., & Zhu, L. X. (2024b). ISR: The iterated score regression-based estimation algorithm. https://CRAN.R-project.org/package=ISR. [DOI] [PubMed]
- 13.Guo, G., Wang, Q., Allison, J. & Qian, G. Accelerated distributed expectation-maximization algorithms for the parameter estimation in multivariate Gaussian mixture models. Applied Mathematical Modelling137, 115709 (2025). [Google Scholar]
- 14.Guo, G., Wei, C. & Qian, G. Sparse online principal component for the parameter estimation in factor model. Computational Statistics38(2), 1095–1116 (2023). [Google Scholar]
- 15.Guo, G., Wei, C., & Qian, G. (2022). SOPC: The Sparse Online Principal Component Estimation Algorithm. https://CRAN.R-project.org/package=SOPC.
- 16.Guo, G., Yu, M. & Qian, G. ORKM: Online regularized K-means clustering for online multi-view data. Information Sciences680, 121133 (2024). [Google Scholar]
- 17.Hoffmann, G., Andreas, B., Ralf, L. & Frank, K. Using machine learning techniques to generate laboratory diagnostic pathways-a case study. Journal of Laboratory and Precision Medicine3, 58–67 (2018). [Google Scholar]
- 18.Howard, W., Rhemtulla, M. & Little, T. Using principal components as auxiliary variables in missing data estimation. Multivariate Behavioral Research50(3), 285–299 (2015). [DOI] [PubMed] [Google Scholar]
- 19.Ilin, A. & Raiko, T. Practical approaches to principal component analysis in the presence of missing values. Journal of Machine Learning Research11(3), 1957–2000 (2010). [Google Scholar]
- 20.Josse, J. & Husson, F. I am selecting the number of components in principal component analysis using cross-validation approximations. Computational Statistics and Data Analysis.56(7), 1869–1879 (2011). [Google Scholar]
- 21.Josse, J. & Husson, F. Handling missing values in exploratory multivariate data analysis methods. Journal de la Societe Francaise de Statistique.153(2), 1–21 (2012). [Google Scholar]
- 22.Josse, J., Chavent, M., Liquet, B. & Husson, F. Handling missing values with regularized iterative multiple correspondence analysis. Journal of Classification29(1), 91–116 (2012). [Google Scholar]
- 23.Kaiser, H. F. An index of factorial simplicity. Psychometrika39(1), 31–36 (1974). [Google Scholar]
- 24.Kao, Y. & Roy, B. Learning a factor model via regularized PCA. Machine Learning91(3), 279–303 (2013). [Google Scholar]
- 25.Little, R. & Rubin, D. Statistical Analysis with Missing Data 3rd edn. (John Wiley and Sons, 2019). [Google Scholar]
- 26.Loisel, S. & Takane, Y. Comparisons among several methods for handling missing data in principal component analysis (PCA). Advances in Data Analysis and Classification13(2), 495–518 (2019). [Google Scholar]
- 27.McNeish, D. Missing data methods for arbitrary missingness with small samples. Journal of Applied Statistics44(1), 24–39 (2016). [Google Scholar]
- 28.Orange Data set. (2024). R package ISR. https://cran.r-project.org/web/packages/ISR/index.html.
- 29.Ozone Data set. (2008). UCI Machine Learning Repository. https://archive.ics.uci.edu/ml/datasets/Ozone+Level+Detection
- 30.Polat, H., Danaei-Mehr, H. & Cetin, A. Diagnosis of chronic kidney disease based on support vector machine by feature selection methods. Journal of Medical Systems41(4), 1–11 (2017). [DOI] [PubMed] [Google Scholar]
- 31.Renjith, S., Sreekumar, A. & Jathavedan, M. Evaluation of partitioning clustering algorithms for processing social media data in tourism domain. IEEE Recent Advances in Intelligent Computational Systems (RAICS)2018, 127–131 (2018). [Google Scholar]
- 32.Staedler, N. & Buehlmann, P. Missing values sparse inverse covariance estimation and an extension to sparse regression. Statistics and Computing22(1), 219–235 (2012). [Google Scholar]
- 33.Sun, B. & Tchetgen, E. On inverse probability weighting for nonmonotone missing at random data. Journal of the American Statistical Association113(521), 369–379 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Travel Reviews Data set. (2018). UCI Machine Learning Repository. https://archive.ics.uci.edu/ml/datasets/Travel+Reviews
- 35.Wang, Q., Guo, G., Qian, G. & Jiang, X. Distributed online expectation-maximization algorithm for Poisson mixture model. Applied Mathematical Modelling124, 734–748 (2023). [Google Scholar]
- 36.Wentzell, P., Andrews, D., Hamilton, D., Faber, K. & Kowalski, B. Maximum likelihood principal component analysis. Journal of Chemometrics11(4), 339–366 (1997). [Google Scholar]
- 37.Zhao, J. & Shao, J. Semiparametric pseudo-likelihoods in generalized linear models with nonignorable missing data. Journal of the American Statistical Association110(512), 1577–1590 (2015). [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The data that can be used to replicate the results presented in this paper is available by reasonable request after contacting the corresponding author.