A novel MissForest-based missing values imputation approach with recursive feature elimination in medical applications

Abstract

Background

Missing values in datasets present significant challenges for data analysis, particularly in the medical field where data accuracy is crucial for patient diagnosis and treatment. Although MissForest (MF) has demonstrated efficacy in imputation research and recursive feature elimination (RFE) has proven effective in feature selection, the potential for enhancing MF through RFE integration remains unexplored.

Methods

This study introduces a novel imputation method, “recursive feature elimination-MissForest” (RFE-MF), designed to enhance imputation quality by reducing the impact of irrelevant features. A comparative analysis is conducted between RFE-MF and four classical imputation methods: mean/mode, k-nearest neighbors (kNN), multiple imputation by chained equations (MICE), and MF. The comparison is carried out across ten medical datasets containing both numerical and mixed data types. Different missing data rates, ranging from 10 to 50%, are evaluated under the missing completely at random (MCAR) mechanism. The performance of each method is assessed using two evaluation metrics: normalized root mean squared error (NRMSE) and predictive fidelity criterion (PFC). Additionally, paired samples t-tests are employed to analyze the statistical significance of differences among the outcomes.

Results

The findings indicate that RFE-MF demonstrates superior performance across the majority of datasets when compared to four classical imputation methods (mean/mode, kNN, MICE, and MF). Notably, RFE-MF consistently outperforms the original MF, irrespective of variable type (numerical or categorical). Mean/mode imputation exhibits consistent performance across various scenarios. Conversely, the efficacy of kNN imputation fluctuates in relation to varying missing data rates.

Conclusion

This study demonstrates that RFE-MF holds promise as an effective imputation method for medical datasets, providing a novel approach to addressing missing data challenges in medical applications.

Introduction

Missing values, also known as missing data, are defined as data points that are not recorded for a variable in a given observation of interest [1]. This pervasive issue spans across various domains [2–6], often arising from a combination of factors such as human and machine errors, data processing difficulties, privacy concerns, or situations where relevant information is either unavailable or unobserved [7–10]. In medical research, missing data presents significant challenges, potentially impairing downstream statistical analyses and predictive models [11, 12]. These challenges can have broad implications, influencing clinical decision-making processes and ultimately affecting the quality of patient care [10, 13]. The severity of these consequences highlights the critical need to address the missing data problem in medical research.

To mitigate these challenges, researchers have developed and implemented various missing value imputation (MVI) techniques, which aim to replace missing values with derived estimates, thereby preserving dataset integrity and utility [14–18]. In the medical domain, several traditional imputation methods have been widely employed, including mean/mode imputation [19], multiple imputation by chained equations (MICE) [20], and k-nearest neighbor (kNN) imputation [21]. While these conventional techniques offer valuable solutions in certain contexts, they are subject to innate limitations that may affect either the accuracy of the imputed data or the applicability of the method itself. For instance, mean/mode imputation, despite its simplicity, replaces missing values with the mean or mode of the observed data for a given variable [22, 23]. However, this approach disregards the inherent uncertainty in such imputations, often yielding biased or unrealistic outcomes [24, 25]. MICE, renowned for its flexibility, is frequently employed as a multiple imputation method [26, 27]. Nevertheless, MICE, along with other multiple imputation techniques, faces challenges in high-dimensional settings [28], particularly those involving interactive and nonlinear relationships among variables [26, 29]. In such scenarios, the complexity of specifying conditional models for each variable with missing data increases substantially, rendering the imputation process both intricate and computationally demanding, potentially compromising the accuracy and efficiency of MICE [4, 30]. Similarly, while kNN imputation is widely utilized for its robustness and effectiveness [9, 31], its computational complexity and sensitivity to parameter settings—such as the number of neighbors, choice of distance metrics, and imputation order—present notable limitations, constraining its practical applicability in real-world settings [32, 33].

In response to these challenges, tree-based imputation methods have emerged as promising alternatives [34]. Notably, MissForest (MF), an iterative imputation algorithm based on random forests (RF), distinguishes itself from traditional imputation methods by neither assuming normality nor requiring parameter specifications for modeling [35, 36]. Furthermore, its capacity to effectively handle mixed data types renders it particularly adept in heterogeneous data contexts [37, 38]. Consequently, MF has garnered increasing attention in the field of MVI research, attributed to its favorable performance relative to traditional imputation methods [35, 36]. Moreover, several studies have demonstrated MF’s promising efficacy within the medical domain [37, 39]. However, while effective at imputing missing data, MF lacks inherent feature selection, which is critical for reducing dimensionality and improving model interpretability, particularly in high-dimensional medical datasets.

Feature selection reduces model complexity by identifying relevant features and removing irrelevant or redundant ones [40, 41]. Recursive feature elimination (RFE), a wrapper method, is particularly effective among feature selection strategies [42, 43] and has gained considerable acclaim within the biomedical domain for its efficacy across numerous studies [44, 45]. It iteratively removes the least important features based on their impact on model performance, aiming to optimize the feature subset for better classification accuracy [46, 47]. Recently, numerous studies have shown that conducting feature selection on observed data to filter out unrepresentative features can significantly enhance the efficiency of the imputation process, as certain missing features deemed unrepresentative may not be essential for effective imputation [48–51].

While MF has gained widespread recognition in MVI research, demonstrating its efficacy across various applications [35–37, 39], efforts to further optimize and fully explore its potential remain limited. Concurrently, RFE is a well-established feature selection method known for reducing dimensionality and improving computational efficiency [44, 45, 48]. However, its use has primarily been limited to a preprocessing role, aimed at enhancing predictive models rather than directly improving imputation methods. A significant research gap exists in integrating RFE feature selection and MF imputation techniques to improve both tasks simultaneously.

To address this, we propose RFE-MF, a novel approach that combines MF with RFE to mitigate the influence of irrelevant features and enhance imputation quality. This study introduces RFE-MF and demonstrates its effectiveness using medical datasets. We perform a comparative analysis to evaluate the performance of our proposed RFE-MF method against four conventional imputation approaches—mean/mode imputation, MICE, kNN, MF—using ten medical datasets. Furthermore, we evaluate their performance on both numerical and mixed data types, with simulated missing rates ranging from 10 to 50%, addressing key practical challenges in the medical field.

The rest of this paper is organized as follows. Section 2 reviews the related literature, covering missing data mechanisms, imputation techniques, and the RFE feature selection method. In Sect. 3, we describe the proposed RFE-MF algorithm. Sections 4 and 5 present the experimental evaluation and the results, respectively. Finally, conclusions are drawn in Sect.  6.

Literature review

Missing data mechanisms

According to [52], there are three mechanisms for missing data: missing completely at random (MCAR), missing at random (MAR), and not missing at random (NMAR). These classifications are based on the relationship between the missingness of the data and the observed or unobserved values in the dataset [53].

MCAR occurs when the probability of missingness is independent of both observed and unobserved variables [54]. An illustrative example is a heart rate monitoring study where data points are missing due to equipment malfunction, such as battery failure. In this scenario, the missing data are unrelated to participants’ heart rates or any other measured variables, thus satisfying the MCAR criteria. MAR is characterized by missingness that is contingent upon observed variables but remains independent of unobserved data [54]. Consider a large-scale health survey where participants periodically report on their health status. If older individuals are more likely than younger participants to omit questions related to dietary habits, the missingness is associated with the observed variable (age) but not with the unobserved dietary information. Once age is controlled for, the missingness can be treated as random, thereby fulfilling the MAR condition. MNAR, the most complex mechanism, occurs when missingness is directly related to unobserved data, such as the value of the missing variable itself [55]. This is exemplified in longitudinal studies of depression, where participants experiencing more severe symptoms may be less inclined to complete follow-up assessments. In this instance, the likelihood of missing data correlates with the unobserved severity of depression, as those with the most pronounced symptoms are the ones most likely to be absent. This represents a case of MNAR, where the missing data are systematically linked to unobserved characteristics.

Imputation methods are predicated on specific missingness mechanisms, and deviations from these underlying assumptions may introduce bias into subsequent analyses [53]. When data adhere to the MCAR condition, results derived from various imputation techniques maintain their validity, and complete case analysis does not introduce systematic bias [56]. To evaluate the efficacy of our proposed imputation method relative to established classical techniques, we conducted a comparative analysis under the MCAR mechanism. The selection of MCAR as a fundamental basis for this comparative analysis is supported by its relatively straightforward assumptions, which provide a well-defined benchmark for assessing imputation performance. Furthermore, previous studies [14, 15] have revealed that MCAR is the most commonly used missingness mechanism in simulation studies, due to its simplicity and ease of implementation, making it an ideal starting point for evaluating imputation methods.

Missing value imputation

Current MVI strategies can be broadly classified into four main categories: single imputation, multiple imputation, machine/deep learning, and tree-based imputation [34, 50, 57, 58]. The first category comprises single imputation methods, including mean/mode imputation [19], regression imputation [59], and expectation–maximization [60]. Among these methods, mean/mode imputation replaces missing numerical values with the mean and categorical values with the most frequent value (mode), effectively using central tendencies to fill in the gaps for the corresponding variables [22, 23]. It is often favored for its simplicity and serves as a common reference technique [61]. However, despite its ease of use, these methods tend to underestimate the variance of estimates and overlook the correlations between variables, potentially resulting in biased or unrealistic outcomes [24, 25].

The second category, multiple imputation, was proposed by Rubin [62, 63], who developed a method for averaging outcomes across multiple imputed datasets. This approach diverges from single imputation methods by substituting each missing observation with multiple plausible values, thereby more accurately reflecting the inherent uncertainty associated with the imputation process [64, 65]. Among the various multiple imputation methods, MICE exhibits flexibility and robustness in managing mixed data structures by offering a full range of conditional distributions and regression-based methods [20, 29]. Its unique design, based on chained equations, enables the estimation of each variable using the model best suited to its specific distribution characteristics [66, 67]. However, MICE is not without limitations. While it imputes missing data through a series of conditional distributions, there is no guarantee that these align with the appropriate joint distribution, potentially compromising the validity and reliability of the imputed results [26]. This issue is particularly pronounced in massive, multivariable datasets, where the complexity of specifying appropriate models is exacerbated by nonlinear and interactive relationships between variables [26, 29].

The third category encompasses machine learning and/or deep learning methods such as kNN [21], support vector machine [68], clustering [69], and multi-layer perceptron [70], etc. The kNN imputation method has been widely studied for its efficacy in addressing missing data [9, 65]. It works by classifying the nearest neighbors of missing values and use those neighbors for imputation using a distance measure between instances [71]. Configuring kNN typically requires selecting an appropriate distance metric—such as Hamming, Euclidean, or Manhattan distance—and determining the optimal number of neighbors, k, to predict each missing value [9]. However, the efficacy of kNN imputation is not only critically dependent on several key parameters (i.e., the number of neighbors, and choice of distance metrics)—factors for which standardized determination methods are currently lacking [9, 72]—but it also faces a substantial limitation due to its computational complexity, particularly in high-dimensional datasets [32].

The fourth category, tree-based imputation, includes decision tree [73], RF [36], and MF [38]. Among these methods, MF employs the RF algorithm for missing data imputation, efficiently managing multivariate datasets that include both numerical and categorical variables [37, 38]. Additionally, MF requires no parameter tuning and imposes no assumptions about the underlying distribution of the data [35, 36], demonstrating its effectiveness in managing missing data, particularly in medical research [74, 75]. Despite its efficacy, MF exhibits limitations, particularly in handling high-dimensional datasets where the computational burden of iterative imputation presents a significant challenge [37, 76, 77].

RFE feature selection

Feature selection in machine learning is essential for reducing data dimensionality and constructing models that are both simplified and interpretable [40, 41]. This process identifies relevant features while eliminating irrelevant or redundant ones through various approaches, including filter, wrapper, or embedded methods [78, 79].

Among these, RFE, a wrapper selection method, has gained prominence for its ability to identify optimal feature subsets based on model performance and classification accuracy [42, 46]. RFE’s operational mechanism involves iteratively eliminating features, generating a ranking of features and candidate subsets, along with a list of accuracy values corresponding to each subset [47, 80]. This approach allows for a comprehensive evaluation of feature importance and their impact on model performance. Notably, RFE is frequently employed in conjunction with various classification algorithms, such as support vector machines [42] and RF [44], to construct more efficient classifiers. This synergistic combination enhances the overall model efficacy by focusing on the most informative features, thereby potentially improving both accuracy and interpretability in complex classification tasks while simultaneously reducing storage and computational costs [81, 82].

Proposed RFE-MF algorithm

This study integrates RFE into MF, aiming to leverage MF’s inherent capability to handle mixed data types while enhancing its utility in real-world clinical settings. By incorporating RF-RFE, the extended MF method performs both missing data imputation and feature selection, yielding more efficient and interpretable models. The details of the RF-RFE mechanism and how it complements MF in achieving both imputation and feature selection are as follows.

Suppose we have a data matrix $X=\left\{X_1,,,X_2,,,\cdots ,,,X_p\right\}$, where n denotes the number of observations and p the number of predictors, of size $n\times p$. For an arbitrary variable $X_s$ ($s=1,\cdots ,p$) with missing values at certain entries, the RFE-MF algorithm divides the dataset into four distinct parts [38]:

1. Observed values of $X_s$: These are the entries in $X_s$ that are not missing, denoted by $y_{\mathit{obs}}^{(s)}$.

2. Missing values of $X_s$: These are the variables other than $X_s$ for which the corresponding observations in $X_s$ are not missing, denoted by $y_{\mathit{mis}}^{(s)}$.

3. Other variables with complete observations: These are the variables other than $X_s$ with observations, denoted by $x_{\mathit{obs}}^{(s)}$.

4. Other variables with missing observations: These represent the variables other than $X_s$, corresponding to the rows has missing values. These variables with observations are denoted as $x_{\mathit{mis}}^{(s)}$.

Figure 1 presents the pseudo-code of the proposed RFE-MF algorithm, which consists of six steps: (1) initial imputation, (2) iterative imputation, (3) feature selection, (4) model fitting, (5) convergence, and (6) outputting the final imputed dataset.

Fig. 1.

The proposed RFE-MF algorithm

In Step 1, the variables in $X$ are sorted by the number of missing values in each $X_s$, starting with the variable that has the fewest missing values. Initially, all missing values in the dataset are imputed using simple strategies: the mean for numerical variables and the mode for categorical variables. In Step 2, the RFE-MF algorithm iteratively updates the imputed dataset. Specifically, given a previously imputed dataset (denoted as $X_{\mathit{old}}^{\mathit{imp}}$) and a stopping criterion $\gamma$, Step 2 generates a new imputed dataset (denoted as $X_{\mathit{new}}^{\mathit{imp}}$) until the imputed values stabilize. The stopping criterion will be discussed in later paragraphs.

In each iteration, Steps 3–5 are executed. Let k denote the vector of sorted indices of variables in $X$. For each $X_s$ in k, Step 3 begins by applying the RF-RFE procedure, RF-RFE($y_{\mathit{obs}}^{(s)}\sim x_{\mathit{obs}}^{(s)}$), to perform feature selection using the response variable $y_{\mathit{obs}}^{(s)}$ and the predictors $x_{\mathit{obs}}^{(s)}$, resulting in $x_{\mathit{obs}}^{FS(s)}$ (i.e., the important predictors selected by RF-RFE).

The Procedure RF-RFE ($y_{\mathit{obs}}^{(s)}\sim x_{\mathit{obs}}^{(s)}$) facilitates the elimination of irrelevant or redundant features through an iterative process to optimize model performance. Initially, a random forest model is trained using the full set of predictors, $x_{\mathit{obs}}^{(s)}$, to predict the response variable, $y_{\mathit{obs}}^{(s)}$. After the model is trained, the importance of each predictor is evaluated and ranked based on its contribution to the model's accuracy. In each iteration, the algorithm removes a certain number of the least important predictors and retrains the random forest model with the reduced subset. This process is repeated, with the model’s performance being evaluated and the remaining predictors ranked after each iteration. The goal is to identify the subset of predictors that results in the best model performance. Once the optimal set of predictors is determined, it is returned as the final list of important features, $x_{\mathit{obs}}^{FS(s)}$.

In Step 4, a random forest model is trained using $y_{\mathit{obs}}^{(s)}$ and $x_{\mathit{obs}}^{FS(s)}$. The missing values $y_{\mathit{mis}}^{(s)}$ are then predicted by applying the trained RF model to $x_{\mathit{mis}}^{(s)}$ in $X_s$. The imputed matrix is continually updated for all variables listed in k, ultimately yielding $X_{\mathit{new}}^{\mathit{imp}}$.

In Step 5, convergence is checked by comparing the imputed values from the current iteration (i.e., $X_{\mathit{new}}^{\mathit{imp}}$) with those from the previous iteration ($X_{\mathit{old}}^{\mathit{imp}}$). Convergence is defined as when the difference between $X_{\mathit{new}}^{\mathit{imp}}$ and $X_{\mathit{old}}^{\mathit{imp}}$ in the current iteration exceeds the difference between them in the previous iteration. Once the imputed data matrix has converged, the stopping criterion $\gamma$ is met, and the algorithm return $X_{\mathit{new}}^{\mathit{imp}}$ from the current iteration as the final result (Step 6).

Similar to MF, the proposed RFE-MF method can impute values for both numerical and categorical variables. To assess convergence in RFE-MF, the difference for the set of numerical variables $N$ is defined as:

$${\mathrm{\Delta }}_N=\frac{{{\sum }_{j\in N}\left(X_{\mathit{new}}^{\mathit{imp}},-,X_{\mathit{old}}^{\mathit{imp}}\right)}^2}{{\sum }_{jϵN}{\left(X_{\mathit{new}}^{\mathit{imp}}\right)}^2}$$ 1

Similarly, the difference for the set of categorical variables $F$ is defined as:

$${\mathrm{\Delta }}_F=\frac{{\sum }_{j\in F}{\sum }_{i=1}^{n}I\left(X_{\mathit{new}}^{\mathit{imp}},\ne ,X_{\mathit{old}}^{\mathit{imp}}\right)}{\#NA}$$ 2

where $I(X_{\mathit{new}}^{\mathit{imp}}{\ne X}_{\mathit{old}}^{\mathit{imp}})$ is an indicator function that equals 1 when the newly and previously imputed values differ, and #NA is the number of missing values in the categorical variables.

Experimental evaluation

Dataset source

We assessed RFE-MF on ten medical datasets from the UCI repository of machine learning databases [83] and Kaggle.1These datasets included numerical and mixed data types. See Table 1 for dataset descriptions.

Table 1.

Medical datasets used for experimental analysis

Data type	Dataset	Year	Instances	Features
Numerical	Parkinson Disease Detection	2020	195	22
	Mehmet Diabetes	2020	768	8
	Prostate Cancer	2018	100	8
	Lower Back Pain Symptoms	2016	310	12
	Liver Disorders	1990	345	7
Mixed	Pre-processed Stroke	2021	5109	11
	Heart Failure Prediction	2020	299	12
	Early-Stage Diabetes Risk Prediction	2020	520	16
	Indian Liver Patient Records	2017	583	10
	Contraceptive Method Choice	1997	1473	9

Experimental setup

The experimental process, illustrated in Fig. 2, begins with simulating ten complete datasets using the MCAR mechanism, across five missing rates: 10%, 20%, 30%, 40%, and 50%. For each missing rate, the simulation is repeated ten times to generate incomplete datasets. Five imputation methods, including mean/mode, kNN, MICE, MF, and RFE-MF, are then applied to impute the missing values. The imputation quality is assessed using two metrics: normalized root mean squared error (NRMSE) for numerical variables and the proportion of falsely classified entries (PFC) for categorical variables. To compare the performance of each imputation method against RFE-MF, paired samples t-tests are conducted, utilizing results from ten repetitions of the simulated tests.

Fig. 2.

The experimental process

In addition to standard mean/mode imputation, kNN imputation was applied with k = 5. MICE was used to generate five multiple imputed datasets, with a threshold of 1 to address multicollinearity. MF parameters were optimized following recommendations from [38], employing 10 iterations and 100 forests. The proposed RFE-MF used the RF-RFE algorithm with enhanced resampling over 10 cross-validation iterations. The number of forests for predicting missing values matched those in MF, though only 3 iterations were found sufficient after experimentation. All methods were implemented in R, and categorical variables were preprocessed using label encoding.

Evaluation metrics

The performance for numerical variables is evaluated using NRMSE, as proposed by [84], defined as:

$$NRMSE=\frac{mean\left[{\left(X^{\mathit{imp}},-,X^{\mathit{ture}}\right)}^2\right]}{{var\left(X^{\mathit{ture}}\right)}^2}$$ 3

where $X^{\mathit{imp}}$ is the imputed data matrix and $X^{\mathit{ture}}$ is the complete data matrix. "Mean" and "var" are shorthand notations for the empirical mean and variance, computed over the numerical missing values.

For categorical variables, PFC is used as the evaluation metric, defined as:

$$PFC=\frac{{\sum }_{i=1}^n{X}_i^{\mathit{imp}}\ne X_i^{\mathit{ture}}}{\#NA}$$ 4

where $X_i^{\mathit{ture}}$ is the true value, $X_i^{\mathit{imp}}$ is the imputed value, and #NA is the number of missing values in the categorical variables. In both cases, better performance results in values closer to 0, while poorer performance approaches a value of 1.

Experimental results

Results of numerical datasets

The NRMSE results for numerical data and the paired t-test for the differences in population means are presented in Table 2. In the Parkinson Disease Detection dataset, at a 10% missing rate, MF performs optimally with an NRMSE of 0.342. However, the difference between MF (0.342) and our proposed RFE-MF (0.343) is negligible. Notably, the discrepancy increases slightly at the 20%, 40%, and 50% missing rates. Interestingly, at a 30% missing rate, RFE-MF marginally outperforms MF (NRMSE = 0.310 vs. 0.312). In the Mehmet Diabetes dataset, RFE-MF achieves the lowest NRMSE (0.627) at a 10% missing rate, followed by MF (0.641) and kNN (0.694), a trend that persists across all missing rates up to 50%. In the Prostate Cancer dataset, mean/mode imputation performs poorly at a 10% missing rate but stabilizes as missing rates increase. Conversely, kNN’s performance deteriorates as missing rates rise, with MICE exhibiting a similar trend but with slightly better performance than kNN. MF and RFE-MF alternate as the top-performing methods. In the Lower Back Pain Symptoms dataset, RFE-MF consistently outperforms all other methods across every level of missingness. Similarly, in the Liver Disorders dataset, RFE-MF consistently delivers the best results.

Table 2.

NRMSE and paired t-test results for numerical datasets across five imputation methods

Dataset	Imputation method	Missing Rates					Mean	t	p
		10%	20%	30%	40%	50%
Parkinson Disease Detection	Me/Mo	0.419	0.390	0.387	0.392	0.394	0.396	12.725	< 0.001***
	kNN	0.411	0.391	0.402	0.409	0.411	0.405	16.392	< 0.001***
	MICE	0.506	0.469	0.485	0.471	0.498	0.485	15.961	< 0.001***
	MF	0.342	0.313	0.312	0.338	0.341	0.329	-2.234	0.0301**
	RFE-MF	0.343	0.322	0.310	0.342	0.353	0.334
Mehmet Diabetes	Me/Mo	0.810	0.841	0.819	0.814	0.815	0.850	12.184	< 0.001***
	kNN	0.694	0.732	0.768	0.787	0.796	0.755	9.466	< 0.001***
	MICE	0.910	0.897	0.928	0.969	0.989	0.939	16.745	< 0.001***
	MF	0.641	0.672	0.713	0.752	0.763	0.708	5.182	< 0.001***
	RFE-MF	0.627	0.662	0.698	0.739	0.750	0.695
Prostate Cancer	Me/Mo	0.418	0.478	0.433	0.454	0.451	0.447	15.683	< 0.001***
	kNN	0.340	0.415	0.408	0.466	0.459	0.418	16.898	< 0.001***
	MICE	0.246	0.267	0.351	0.401	0.416	0.336	4.076	< 0.001***
	MF	0.191	0.255	0.284	0.351	0.348	0.286	0.611	0.544
	RFE-MF	0.180	0.260	0.281	0.341	0.356	0.284
Lower Back Pain Symptoms	Me/Mo	0.397	0.473	0.474	0.462	0.438	0.449	27.984	< 0.001***
	kNN	0.370	0.453	0.477	0.478	0.462	0.448	33.971	< 0.001***
	MICE	0.396	0.477	0.461	0.455	0.469	0.452	18.625	< 0.001***
	MF	0.280	0.369	0.384	0.371	0.371	0.355	5.074	< 0.001***
	RFE-MF	0.275	0.365	0.381	0.369	0.365	0.351
Liver Disorders	Me/Mo	0.579	0.595	0.575	0.579	0.583	0.582	13.906	< 0.001***
	kNN	0.521	0.568	0.546	0.542	0.566	0.585	11.781	< 0.001***
	MICE	0.648	0.695	0.653	0.677	0.699	0.674	15.501	< 0.001***
	MF	0.500	0.542	0.502	0.511	0.546	0.520	4.933	< 0.001***
	RFE-MF	0.489	0.529	0.493	0.497	0.529	0.507

The optimal values across five simulated missing rates and their mean value for the different medical datasets are highlighted in bold

Me/Mo mean/mode imputation, kNN k-nearest neighbor imputation, MICE multiple imputation by chained equations, MF MissForest, RFE-MF recursive feature elimination-MissForest. Significant differences at the 99% and 99.9% levels are indicated by ** and ***, respectively

Regarding the paired t-tests results, RFE-MF consistently demonstrates the lowest means values across four datasets: Mehmet Diabetes, Prostate Cancer, Lower Back Pain Symptoms, and Liver Disorders, with all p-values < 0.001*** (except for the difference between RFE-MF and MF in the Prostate Cancer dataset, which is not statistically significant). Conversely, in the Parkinson Disease Detection dataset, MF yields superior results compared to RFE-MF, with a p-value of 0.030**, indicating a statistically significant difference in favor of MF.

Results of mixed datasets

The results of the NRMSE and PFC for both numerical and categorical variables, along with the paired t-test for the difference in population means, are presented in Table 3. In terms of the NRMSE metric, in the Heart Failure Prediction dataset, RFE-MF slightly outperforms MF, although the mean/mode imputation method exhibits the best performance. Similar trends are observed in the PFC metric, with RFE-MF closely trailing MF at a 10% missing rate. To better understand the efficacy of the mean/mode imputation method in this dataset, we analyzed the statistical characteristics of each variable under complete conditions and various simulated missing rates. The findings suggest that the dataset's characteristics favor the mean/mode imputation method, as the continuous variables closely approximate their complete-condition mean values, while the mode values for categorical variables consistently align with the complete data.

Table 3.

NRMSE, PFC results, and paired t-test results for mixed datasets across five imputation methods

Dataset	Imputation method	NRMSE								PFC
		Missing Rates					Mean	t	p	Missing Rates					Mean	t	p
		10%	20%	30%	40%	50%				10%	20%	30%	40%	50%
Heart Failure Prediction	Me/Mo	0.354	0.402	0.356	0.376	0.369	0.371	-14.085	< 0.001***	0.374	0.377	0.372	0.378	0.368	0.374	9.001	< 0.001***
	kNN	0.370	0.429	0.387	0.404	0.396	0.397	-2.097	0.0411**	0.422	0.430	0.420	0.440	0.439	0.430	-6.006	< 0.001***
	MICE	0.546	0.568	0.519	0.514	0.530	0.536	-14.085	< 0.001***	0.429	0.422	0.442	0.445	0.457	0.439	-7.129	< 0.001***
	MF	0.375	0.424	0.384	0.405	0.415	0.401	-5.071	< 0.001***	0.408	0.407	0.406	0.426	0.423	0.414	-1.504	0.139
	RFE-MF	0.366	0.406	0.378	0.400	0.403	0.391			0.415	0.404	0.395	0.418	0.415	0.409
Pre-processed Stroke	Me/Mo	0.571	0.570	0.570	0.572	0.573	0.571	12.995	< 0.001***	0.316	0.324	0.320	0.320	0.319	0.320	19.581	< 0.001***
	kNN	0.574	0.588	0.602	0.608	0.628	0.600	36.539	< 0.001***	0.302	0.310	0.316	0.321	0.330	0.316	36.158	< 0.001***
	MICE	0.691	0.709	0.709	0.718	0.721	0.710	57.120	< 0.001***	0.339	0.341	0.346	0.347	0.356	0.346	50.703	< 0.001***
	MF	0.503	0.514	0.538	0.562	0.585	0.540	8.847	< 0.001***	0.297	0.306	0.309	0.315	0.324	0.310	30.866	< 0.001***
	RFE-MF	0.496	0.508	0.528	0.547	0.558	0.527			0.279	0.287	0.293	0.296	0.303	0.291
Early-Stage Diabetes Risk Prediction	Me/Mo	1.004	1.003	1.002	1.000	1.000	1.002	13.232	< 0.001***	0.386	0.391	0.391	0.388	0.386	0.388	20.971	< 0.001***
	kNN	0.850	0.899	0.943	1.009	1.011	0.942	15.062	< 0.001***	0.172	0.210	0.248	0.284	0.306	0.244	17.691	< 0.001***
	MICE	1.229	1.197	1.215	1.211	1.256	1.222	24.680	< 0.001***	0.318	0.331	0.339	0.352	0.367	0.341	21.197	< 0.001***
	MF	0.703	0.741	0.797	0.900	0.950	0.818	0.425	0.673	0.118	0.167	0.208	0.253	0.289	0.207	5.819	< 0.001***
	RFE-MF	0.711	0.743	0.813	0.879	0.932	0.816			0.110	0.157	0.202	0.245	0.282	0.199
Indian Liver Patient Records	Me/Mo	0.773	0.862	0.860	0.863	0.865	0.845	15.627	< 0.001***	0.247	0.244	0.239	0.237	0.251	0.243	-3.083	0.0034**
	kNN	0.737	0.829	0.851	0.858	0.864	0.828	15.616	< 0.001***	0.276	0.301	0.293	0.292	0.292	0.291	6.581	< 0.001***
	MICE	1.185	0.933	0.960	0.957	0.977	1.002	8.674	< 0.001***	0.335	0.348	0.354	0.363	0.373	0.355	12.688	< 0.001***
	MF	0.619	0.676	0.718	0.726	0.824	0.713	3.497	0.001**	0.245	0.256	0.263	0.284	0.288	0.267	2.455	0.0177*
	RFE-MF	0.606	0.663	0.704	0.714	0.788	0.695			0.257	0.247	0.255	0.257	0.266	0.256
Contraceptive Method Choice	Me/Mo	0.383	0.386	0.387	0.386	0.384	0.385	13.099	< 0.001***	0.370	0.374	0.369	0.372	0.373	0.372	7.983	< 0.001***
	kNN	0.371	0.392	0.405	0.416	0.417	0.400	27.304	< 0.001***	0.365	0.378	0.379	0.395	0.403	0.384	22.883	< 0.001***
	MICE	0.426	0.434	0.453	0.470	0.482	0.453	52.507	< 0.001***	0.411	0.413	0.418	0.425	0.438	0.421	41.197	< 0.001***
	MF	0.325	0.339	0.362	0.383	0.398	0.361	13.035	< 0.001***	0.385	0.391	0.393	0.400	0.405	0.395	25.186	< 0.001***
	RFE-MF	0.314	0.324	0.343	0.358	0.370	0.342			0.338	0.344	0.349	0.365	0.376	0.354

The optimal values across five simulated missing rates and their mean value for the different medical datasets are highlighted in bold

Me/Mo mean/mode imputation, kNN k-nearest neighbor imputation, MICE multiple imputation by chained equations, MF MissForest, RFE-MF recursive feature elimination-MissForest. Significant differences at the 95%, 99%, and 99.9% levels are indicated by *, **, and ***, respectively

In the Pre-processed Stroke dataset, RFE-MF consistently performs best across both the NRMSE and PFC metrics, followed by MF, while kNN’s performance declines with increasing missing rates. In the Early-Stage Diabetes Risk Prediction dataset, MF excels in NRMSE at missing rates between 10 and 30%, whereas RFE-MF performs slightly better at missing rates between 40 and 50%. RFE-MF consistently outperforms the PFC metric. In the Indian Liver Patient Records dataset, RFE-MF consistently outperforms MF in NRMSE across all missing rates. The mean/mode imputation method shows optimal PFC performance, with RFE-MF trailing MF slightly only at a 10% missing rate. In the Contraceptive Method Choice dataset, RFE-MF exhibits the best NRMSE performance across all missing rates. RFE-MF consistently outperforms in PFC, except for being slightly surpassed by mean/mode imputation method at a 50% missing rate.

Regarding the paired t-tests for NRMSE, RFE-MF performs optimally across four datasets: Pre-processed Stroke, Early-Stage Diabetes Risk Prediction, Indian Liver Patient Records, and Contraceptive Method Choice, with statistically significant results. However, in the Early-Stage Diabetes Risk Prediction dataset, the difference between RFE-MF and MF is not statistically significant. In the Heart Failure Prediction dataset, mean/mode imputation method exhibits the best performance, with RFE-MF slightly trailing behind. Similar trends are observed in the paired t-tests for PFC, where RFE-MF performs optimally across most datasets. However, in the Heart Failure Prediction and the Indian Liver Patient Records datasets, the mean/mode imputation method outperforms RFE-MF.

In summary, as shown in Table 4, the results highlight the effectiveness of the proposed RFE-MF method in handling missing values across the selected datasets. In the experimental evaluation of 10 medical datasets, RFE-MF achieved the top rank in seven datasets, while securing the second rank in the remaining three. These findings confirm that RFE-MF outperforms the other four classical imputation methods, demonstrating its suitability for medical datasets.

Table 4.

The rankings of five imputation methods across all datasets

Data type	Dataset	Metric	Me/Mo	kNN	MICE	MF	RFE-MF
Numerical	Parkinson Disease Detection	NRMSE	3	4	5	1	2
	Mehmet Diabetes	NRMSE	4	3	5	2	1
	Prostate Cancer	NRMSE	5	4	3	2	1
	Lower Back Pain Symptoms	NRMSE	4	3	5	2	1
	Liver Disorders	NRMSE	3	4	5	2	1
Mixed	Heart Failure Prediction	NRMSE	1	3	5	4	2
		PFC	1	4	5	3	2
	Pre-processed Stroke	NRMSE	3	4	5	2	1
		PFC	2	4	5	3	1
	Early-Stage Diabetes Risk Prediction	NRMSE	4	3	5	2	1
		PFC	5	3	4	2	1
	Indian Liver Patient Records	NRMSE	4	3	5	2	1
		PFC	1	4	5	3	2
	Contraceptive Method Choice	NRMSE	3	4	5	2	1
		PFC	2	3	5	4	1
Total scores			45	53	72	36	19
Overall ranking			3	4	5	2	1

Me/Mo mean/mode imputation, kNN k-nearest neighbor imputation, MICE multiple imputation by chained equations, MF MissForest, RFE-MF recursive feature elimination-MissForest, NRMSE normalized root-mean-square error, PFC proportion of falsely classified samples

Conclusions

In this study, the proposed RFE-MF exhibits superior performance across the majority of medical datasets compared to four classical imputation methods (mean/mode imputation, kNN, MICE, and MF), underscoring its efficacy in MVI tasks. Notably, RFE-MF consistently outperforms the original MF, regardless of variable type (numerical or categorical), indicating the effectiveness of our integrated approach in improving imputation quality for datasets with mixed characteristics. Additionally, the results highlight the sensitivity of kNN imputation to varying missing rates, whereas mean/mode imputation maintains consistent performance across all missing data rates, depending on the dataset's characteristics. Our proposed RFE-MF demonstrates potential for practical applications, offering a valuable imputation technique for healthcare data analysis and the development of predictive models. Furthermore, this study emphasizes the importance of considering data type and missingness rate when selecting imputation techniques, as these factors significantly impact the performance of different methods.

This study acknowledges several limitations. Firstly, certain parameters, such as the k value in kNN imputation method and the number of iterations for MICE, were not optimized. Default hyperparameter values were employed for MF, indicating a need for further investigation into parameter tuning. Secondly, while this study focuses on RF-RFE, future research could explore the integration of other feature selection methods with MF, potentially uncovering novel synergies and enhancing imputation performance. Nevertheless, the current selection of RF-RFE is justified by the limitations of alternative methods. For instance, genetic algorithms [85] often face issues of high computational cost for fitness calculation [86], especially when dealing with high-dimensional datasets prevalent in medical research. Finally, it is crucial to note that the analyses and conclusions presented in this study are predicated on the assumption of MCAR data. Future research should explore the performance of these methods under different missing data mechanisms, including MAR and MNAR, to enhance the generalizability and applicability of the findings across various scenarios in medical research.

Authors’ contributions

All authors had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: YHH and TYL. Acquisition of data: YCL. Analysis and interpretation of data: YHH, RYW, and YCL. Drafting of the manuscript: All authors. Critical revision of the manuscript for important intellectual content: All authors. Study supervision: YHH.

Funding

The authors report no funding.

Data availability

The datasets generated and/or analyzed during the current study are available in the UCI Machine Learning Repository (https://archive.ics.uci.edu/datasets) and Kaggle (https://www.kaggle.com/).

Declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.