Feature learning augmented with sampling and heuristics (FLASH) improves model performance and biomarker identification

Abstract

Big biological datasets, such as gene expression profiles, often contain redundant features that degrade model performance and limit generalization across independent datasets with complexities like class imbalance and hidden sub-clusters. To overcome challenges, we present ‘FLASH’, a novel feature selection method combining filtration and heuristic-based systematic elimination. FLASH generates random samples and computes p-values for each feature using multiple statistical tests (t-test, ANOVA, Wilcoxon Rank-Sum, Brunner–Munzel, Mann–Whitney). Features are scored by aggregating significant p-values across samples. The coefficient from the machine learning model with the highest accuracy on the filtered features is used to rank them. Recursive elimination with cross-validation systematically removes features while monitoring accuracy. The final subset is selected based on the highest performance during elimination, to achieve effective feature selection. We show that our method preserves predictive performance on independent datasets. Our comprehensive evaluation across diverse datasets showed that FLASH outperforms the compared feature selection methods dRFE, Mutual information, MRMR, ElasticNet, NeuralNet, Permutation test and SAGA within the scope of our tested datasets and evaluation settings. Additionally, features selected by FLASH demonstrated greater biological relevance, as evidenced by higher overlap with disease-associated genes from DisGeNET in an independent dataset.

Introduction

In the era of modern biomedical technologies, rapid advancements and innovations have accumulated vast amounts of biological omics data^1–3. These datasets consist of thousands of features (p) for each sample (n), providing valuable insights into various biological phenomena. However, the number of available samples is very small due to challenges in recruiting participants. The number further narrows down in the case of omic studies due to the high cost associated with data generation^4,5. This high cost leads to fewer samples with omics data, which comes under the big data category, causing smaller n with larger p. Analyzing these omics data with “large p, small n” poses significant challenges, such as identifying features associated with specific phenotypes or diseases⁶.

The curse of dimensionality may occur when training a model with a significantly smaller number of samples using all the omic features⁷. Feature selection algorithms offer a potential solution to this challenge by identifying a subset of features significantly associated with phenotypes of interest⁸. These algorithms aim to reduce the dimensionality of the data, improving computational efficiency and enhancing the interpretability of the results. The need for more accurate and efficient feature selection tools is evident, especially in biomedical research. For instance, the PAM50 gene panel has proven effective in accurately defining major breast cancer subtypes based on the expression profiles of 50 genes⁹. Similarly, the commercial availability of the 21-gene panel OncoType DX has contributed significantly to estimating the likelihood of recurrence in early-stage female ER+ breast cancer¹⁰. There is an increasing demand for guided diagnostic and treatment approaches^11–13. These tests’ success depends on identifying disease signatures, which capture disease patterns robustly in large populations. So, developing advanced feature selection algorithms becomes more crucial for enhancing patient care and overall outcomes. Feature selection is a challenging task, as it falls under the category of problems in computational complexity theory called NP-hard problems (nondeterministic polynomial time), which requires exponential time to find the global optimal solution¹⁴. As a result, existing feature selection algorithms rely on heuristic rules to find local optimal solutions, leading to variations in their performance across different datasets. These rules can be categorized into three classes. Filter methods utilize the intrinsic properties of the data to select the optimal feature subset without relying on any learning algorithm¹⁵. Wrapper methods employ a learning algorithm to evaluate different feature subsets and evolve towards an optimal solution through multiple iterations¹⁶. Embedded methods perform feature filtration during model training and perform simultaneous optimization with feature elimination and classification¹⁷.

Several algorithms have been developed to address the feature selection problem in this context. The AltWOA enhances the conventional Whale Optimization Algorithm by introducing altruism within the whale population, resulting in more effective global optimization¹⁸. Similarly, enCFS employs ensemble techniques and sampling to enhance feature selection’s discrimination and stability¹⁹. The VNLHHO offers a metaheuristic approach that balances global exploration and local exploitation for gene feature extraction²⁰. The MBAO combines a filtering approach with an efficient wrapper to select the most informative genes²¹. The MGRFE leverages an embedded integer-coded genetic algorithm for minimal gene combinations with maximal information²². The RIFS2D algorithm challenges conventional wisdom by showcasing the predictive potential of low-ranked features, even when top-ranked features fall short²³. Numerous approaches rely on wrapper methods that evaluate and compare subsets of features according to their predictive value. Like many other feature selection methods, they tend to be computationally intensive and require substantial time and resources²⁴. These computational limitations motivate the need for techniques that improve efficiency and scalability when handling large-scale or high-dimensional data. Most methods use a filter score like a t-test, Fisher score, Wilcoxon test correlation, etc. It requires users to choose a predefined cut-off value depending on the nature of the data. The elimination step requires users to select a minimal number of features as a model parameter. In our opinion, the algorithm should choose the minimum number of features. One common drawback among these methods is the absence of sampling, which can limit their ability to handle imbalanced or high-dimensional datasets effectively. By incorporating sampling techniques, we can achieve a more balanced and representative view of the data, which helps reduce variance and improves the stability of feature selection–particularly in complex or high-dimensional settings. When these sampling strategies are embedded directly within feature selection methods, they enable the identification of features that consistently perform well across various data subsets. This consistency mitigates the model’s sensitivity to data fluctuations and reduces the risk of overfitting. As a result, it enhances generalizability, boosts computational efficiency, and yields more robust and accurate feature subsets for downstream analysis and modeling tasks.

In this paper, we propose a feature selection algorithm that employs a sampling technique to identify features that are consistently informative across subsets of the data, to enhance the classification of new samples. This algorithm aims to provide more robust results in complex datasets with class imbalance and many features. The proposed feature selection algorithm, FLASH, which stands for Feature learning augmented with sampling and heuristics, is a hybrid of filter and recursive elimination methods. First, we created the filter scores by applying statistical tests to large randomly sampled subsets from the data. Random sampling helps reduce bias toward specific patterns in the dataset and improves the generalizability of the selected features to unseen or external datasets. We employed five statistical tests to ensure robustness across different distributional assumptions one-way ANOVA²⁵, rank-sum test²⁵, Mann–Whitney U test²⁵, t-test²⁵, and the Brunner–Munzel test²⁶. Random sample selection from the population reduces bias towards the data under investigation and increases its chance for global application on other data²⁷. This global application increases the scope of our algorithm by mitigating the potential impact of outliers or specific characteristics present in a non-randomly selected sample, thereby enhancing the chance of improved accuracy when applied to a general population²⁸. Through extensive evaluation across diverse datasets, we demonstrate the effectiveness and efficiency of our proposed algorithm, FLASH, in selecting informative features associated with the phenotype of interest. To assess the biological relevance of the selected features, we benchmarked FLASH by comparing its outputs on an independent dataset against known disease-associated genes curated in the DisGeNET database. This comparison highlights FLASH’s ability to generate reliable and meaningful biomedical insights.

Results

The impact of FLASH on different ML algorithms

This study evaluates a feature subset by assessing its classification performance using multiple representative classifiers employing a repeated stratified 10-fold cross-validation approach. The metric employed, called maximum accuracy (mAcc), represents the highest classification accuracy achieved among these classifiers. A total of six popular classifiers are used to evaluate the given feature subset (Fig. 1). These classifiers include support vector machine (SVM), Random Forest, XGBoost, logistic regression (LR), decision tree (DTree), and K nearest neighbor (KNN). The result is derived from the model performance on the selected feature sets. By leveraging the diverse capabilities of these classifiers, a comprehensive assessment of the feature subset’s classification performance is achieved, providing valuable insights for feature selection in the studied context.

Fig. 1. Performance comparison over multiple ML algorithms.

A mAcc, B F1-Score.

Among the evaluated classifiers, SVM, KNN, and Logistic Regression consistently demonstrated superior performance in compared to non-linear methods such as Decision Trees, particularly regarding accuracy and F1-score. These findings suggest that the effectiveness of FLASH features may vary depending on the choice of ML algorithm, highlighting the importance of specific characteristics and requirements of the dataset when selecting an appropriate ML approach.

Ablation study: FLASH boosts performance beyond baseline

The filtering approach requires fewer computational resources than RFECV and helps identify initial features²⁹. Since applying RFECV directly to a dataset with numerous features is inefficient²⁹, a preliminary filtering step becomes essential. However, it is important to note that additional efforts are required to systematically optimize the final set of features for the classification algorithm. The evaluation of an ML model is required at every iteration after elimination to capture the best subset based on accuracy. This process allows the selection of features to contribute to the classification performance.

In some cases, the mAcc has reached its peak value of 1, indicating good classification performance (Fig. 2). The improvement becomes more significant when the F1-score is considered. For instance, in the case of the GSE755 dataset, the F1-Score initially measured below 0.4 but experienced a substantial jump after embedding RFECV, resulting in an F1-Score of 0.77. These findings highlight the effectiveness and impact of incorporating RFECV in enhancing classification performance, particularly regarding accuracy and precision. The performance has also been compared with the baseline, which contains the complete set of features. We observed that using all the features led to a lower model accuracy. Literature also supports the observation that including redundant information may hinder the learning behavior of the model^30–32.

Fig. 2. Importance of feature selection.

Performance comparison of filter score with and without RFECV over baseline (using all features). Results are shown for two metrics: A mAcc, B F1-Score.

Association between FLASH selected feature and outcome label

To assess the interpretability and coherence of FLASH-selected features. We visualized the correlation coefficient of the top-ranked genes concerning class labels across 11 datasets. Each dataset exhibits a distinct distribution of gene-phenotypic association as shown in (Fig. 3). For the remaining five datasets, refer to (Supplementary Fig. 4).

Fig. 3. Correlation profiles of FLASH-selected genes across six transcriptomics datasets.

Each plot shows horizontal barplots of correlation coefficients between top-ranked genes and class labels. The supplementary material (Supplementary Table 3) provides the Gene Name/ID list plotted along the x-axis for each dataset.

GSE37751 and GSE40419 exhibited strongly negative correlation peaks (ρ < 0.6), which could be potential tumor promoters in the disease state. GSE18864 displayed multiple transcripts with high positive correlation (ρ > 0.4). These positively correlated genes may represent candidates for tumor-suppressive roles or markers of normal physiological states.

Glioblastoma_2 presents a sparse but high magnitude correlation. It could be the presence of oncogenic drivers. These findings underscore FLASH’s strength: to adapt feature selection to the statistical structure of each dataset. While still covering biologically meaningful signals across diverse conditions.

Benchmarking FLASH with contemporary feature selection algorithms

The proposed algorithm FLASH’s performance is compared with recently and widely used feature selection algorithms like MRMR³³, Mutual information (Mut-Info)³⁴, dRFE³⁵, and SAGA³⁶. We also compared FLASH’s performance with intrinsic ML models such as a neural network and ElasticNet³⁷ to demonstrate the necessity of extrinsic feature selection in imbalanced datasets. Additionally, we used a permutation test³⁸, a non-parametric method that assesses the significance of observed group differences. The test shuffles labels to build a reference distribution and compute a p-value, without relying on data distribution assumptions. The repeated cross-validation approach and the metrics,mAcc and F1 score, were also used.

The FLASH showed better results than other algorithms, reaching over 90% mean accuracy on 8 out of 11 datasets. In the Illumina HiSeq datasets, the accuracy exceeded 95%. The value sometimes reaches nearly 100% (see Fig. 4A). For instance, in the GSE99309 dataset, the other algorithms achieved ~0.65 mAcc, while FLASH attained a good mAcc 1.

Fig. 4. Comparing FLASH with other feature selection methods.

All algorithms except NeuralNet, ElasticNet, and Permutation test have explicit feature selection methods. NeuralNet, ElasticNet, and Permutation test have intrinsic feature selection methods. Results are presented in four metrics: A mAcc, B F1-Score, C Precision, D Recall.

It is noteworthy that FLASH was performed on both simple and complex datasets. For example, in GSE18864, a less complex dataset, other feature selection algorithms³⁵ achieved an mAcc of 1, and FLASH also demonstrated an mAcc of 1. In the GSE35725 dataset, FLASH exhibits a significantly better result than other methods. The datasets GSE233242 and GSE40419, based on Illumina HiSeq, showed superior performance with the FLASH algorithm, achieving an average accuracy of 98.55%. For a detailed table, refer to (Supplementary Table 4). These findings show better performance of the FLASH compared to the other methods across multiple datasets, demonstrating its effectiveness in achieving high classification accuracy on different datasets.

Comparison of FLASH features with other algorithms

To compare the selected gene sets of FLASH with methods like dRFE, Mut_info, MRMR, and SAGA. We performed the Jaccard similarity analysis across 11 datasets, as shown in (Fig. 5). For details on the total number of features selected, please refer to (Supplementary Table 5). Jaccard index between genes selected by FLASH and those selected by different algorithms. FLASH showed low similarity with most algorithms across several datasets. Reinforcing its tendency to select a more parsimonious and informative feature set.

Fig. 5. Comparing features selected by FLASH with other feature selection methods.

Jaccard index between genes selected by FLASH and other methods.

The Jaccard matrix shows distinct overlap trends, indicating that dRFE shared a moderate Jaccard similarity with FLASH as well as with datasets like Glioblastoma_2, GSE19159, and GSE5764, from which we can suggest that recursive feature elimination methods may identify comparable biologically significant features. Mutual Information and SAGA consistently showed low overlap across the datasets. It shows a difference in their feature selection approaches. FLASH recovers stable and biologically plausible features while offering unique gene sets that differ meaningfully from traditional methods. It potentially contributes novel insights into disease-specific molecular signatures.

To examine the characteristics of selected features, we evaluated their effect size distributions using Cohen’s d. Analysis of Cohen’s d³⁹ showed variation in effect size distributions across different feature selection methods. For a detailed table, refer to (Supplementary Table 5). Mut_info and mRMR often selected features with large effect sizes (Cohen’s d > 0.8), while FLASH and SAGA usually selected features with moderate effect sizes (Cohen’s d 0.2–0.8). In Glioblastoma_2, FLASH showed an enrichment of strong-effect features (Cohen’s d > 0.8) and achieved notably high accuracy compared to other feature selection methods, reaching a maximum of 100%. In GSE99039, FLASH prioritized features with moderate effect sizes (~0.5) while reaching a maximum accuracy of 100%. FLASH did not focus only on strong univariate signals, but it also detected dispersed patterns of variation, and balanced statistical evidence with biological relevance by selecting features that represented broad distributions.

FLASH improves ML model performance on independent datasets

We conducted a case study using two breast cancer datasets, Illumina GSE52194^40,41 and microarray GSE21422^42,43 to evaluate the generalizability and, in contrast, the specificity of our selected features. This analysis also positions our method as a potential biomarker identification tool for gene expression datasets. The feature set identified from the breast cancer datasets GSE233242 and GSE5764 was applied to predict the disease categories of two independent datasets, GSE52194 and GSE21422. The performance of FLASH was evaluated on multiple metrics, in the present data, and the accuracy obtained by FLASH is comparable with other methods, (Fig. 6A). However, FLASH’s significance lies in its ability to capture features related to the system on which it is applied.

Fig. 6. Evaluating FLASH on an independent dataset.

A Performance comparison of features selected from breast cancer data on independent datasets. B Matched breast cancer-related genes (from DisGeNet) and selected genes for different methods.

To further validate the biological relevance of the selected features, we compared them with breast cancer-related genes from the DisGeNet database⁴⁴. FLASH-identified features exhibited a 70% overlap for GSE52194 and an 83% overlap for GSE223242 with disease-associated genes from the DisGeNet database, significantly higher than the overlap achieved by the other methods (Fig. 6B). This high overlap underscores the effectiveness of FLASH in identifying biologically meaningful features pertinent to diabetes, thereby demonstrating its potential as a robust tool for biomarker discovery in gene expression studies.

Discussion

Building predictive models using molecular signatures is crucial in biomedical research. There are many applications of such models, like diagnosis, prognosis, etc., for example, MammaPrint⁴⁵, Decipher⁴⁶, VeriStrat⁴⁷ etc., have been developed for predicting recurrence and survival in breast and lung cancer. Unfortunately, the number of such tests is very low and is only available for a few diseases. The limited availability is due to the dimensionality of the data, which limits the models’ training ability and makes them less robust. Feature selection provides a possible solution to this problem by finding a smaller subset from a large pool of features. Identification of such a meaningful subset of features improves model interpretability and computational efficiency.

In this study, we have introduced a feature selection algorithm, FLASH, that combines elements of both filter and recursive elimination methods. The fusion approach first filters the identified candidate genes with significant discriminatory power. First, we created thousands of data snapshots to filter the genes by randomly sampling subsets from the original data. Then, we implemented a t-test, One-way ANOVA, Wilcoxon Rank-Sum, Brunner–Munzel, and Mann–Whitney U on each randomly drawn set and obtained p-values for every feature in the respective random set. The average of statistically significant p-values (<0.01) was taken after transforming them to -log10(p-value) to obtain a score which describes the character of a feature being significant in most of the data snapshots. The sampled significant score ensures the reliability and validity of the selected features. Subsequently, we utilize recursive elimination to refine the feature set, systematically removing less informative features and evaluating the cross-validated accuracy at each iteration. We selected the One-way ANOVA test among the five statistical methods based on accuracy scores, using the Kneedle algorithm to identify the optimal cutoff. We have used twenty gene expression datasets from the literature to evaluate FLASH with other well-established and recent algorithms like dRFE, MRMR, SAGA, etc. These datasets have various degrees of complexity, such as large feature sizes, smaller samples, and an imbalanced distribution of classes/ labels. Considering the selected datasets, first, we have demonstrated the importance of feature selection by comparing the predictive performance of the feature subset over the complete feature set. The result showed a gain in accuracy in all the datasets, and in eight out of eleven datasets, this gain was more than 90%. In the evaluation datasets, Gliboblastoma_2, GSE35725 and GSE18864 have an accuracy of 100% previously reported in the literature^22,23,35. However, datasets like GSE233242, GSE5764, GSE40419, and GSE99039 have an average accuracy of approximately 90%, with limited consensus among different studies. Some of the datasets’ reported accuracy was below 70%, making it very challenging to translate them at a clinical level. In our experimentation with such challenging datasets, FLASH achieved more than 90% accuracy in 8 out of 11 datasets. Other metrics, such as precision and recall, which are relevant to understanding category-wise prediction results, are also consistent.

Our case study on a diabetic dataset underscores our approach’s practical utility for disease-associated signature identification. The other prediction algorithms have shown around 90% accuracy in the original dataset^22,23,35. However, after evaluating the diabetes ML model on a new dataset as a test set, its performance dropped to 50%, equivalent to random prediction. Nonetheless, FLASH maintained its predictive accuracy with training and testing data around 90%. We have also observed that the selected features matched well with the known breast cancer-associated genes from the DisGeNet database.

In conclusion, this work introduced FLASH, an algorithm that addresses the limitations of existing feature selection methods by incorporating a sampling technique. FLASH enhances model performance, yielding better results on independent datasets. This method offers a new feature selection strategy that can generate robust biomarkers for disease diagnosis.

Methods

Dataset description

We have used 19 gene expression datasets to evaluate the feature selection algorithm, including three Illumina HiSeq datasets: GSE233242, GSE40419, and GSE134878. These datasets have complex characteristics such as high-class imbalance, large feature vs sample ratio and poor predictions by previous feature selection methods. Six datasets are widely recognized within the research community^22,23,35 as standard benchmarks for evaluating feature selection methodologies. The description of the dataset is given in Table 1.

Table 1.

The datasets used in this study with defined characteristics

Name	Characteristics	Category	No. of features	Ref.
ALL-AML	Category imbalance	ALL: 47, AML: 25	7129	63
ALL2	Category imbalance	relapse: 65, non-relapse: 35	12625	63
ALL3	Category imbalance	non-mdr: 101, mdr: 24	12625	63
Breast	Large feature size	non-relapse: 51, relapse: 46	24481	64
CNS	Category imbalance	survivor: 39, non-survivor: 21	7129	65
Colon	Category imbalance	tumor: 40, non-tumor: 22	2000	66
Breast cancer (GSE19159)	Category imbalance	relapse: 111, non-relapse: 57	2905	67
Autistic (GSE25507)	Large feature size	autism: 82, non-autism: 64	54613	68
Lung cancer (GSE30219)	Category imbalance+	early-stage: 198, late-stage: 93	54675	69
Diabetes (GSE35725)	Large feature size	T1D: 57, Healthy: 44	54675	70
Myeloma (GSE755)	Category imbalance	lesion: 137, non-lesion: 36	12625	71
Parkinson (GSE99039)	Large feature size	non-IPD: 233, IPD: 205	54675	72
Glibolastoma_2	Category imbalance	glioblastoma: 77, non-tumor: 23	15435	73
Human Breast (GSE37751)	Large feature size	Tumor: 61, Non-tumor: 47	33297	74–76
Human Breast (GSE18864)	Category imbalance+ Large feature size	treated: 60, non-treated: 24	54675	77–79
Human Breast (GSE5764)	Large feature size	non-cancer: 20, cancer: 10	54675	80
Human Breast (GSE233242)	Large feature size	tumor: 43, normal: 43	15044	81
Lung Cancer (GSE40419)	Large feature size	tumor: 86, normal: 77	36742	82
Essential Tremor (GSE134878)	Large feature size	tremor: 33, control: 21	25931	83

Additionally, we selected independent test datasets GSE52194^40,41 and GSE21422^42,43, which share characteristics with GSE223242 and GSE5764, respectively, both related to breast cancer. This dataset was used to validate the trained diabetes model. All the experiments were conducted on a computing server with 128 GB of system memory and a 48-core Intel Xeon CPU (2.20 GHz).

The feature learning augmented with sampling and heuristics (FLASH) algorithm has been established in the context of binary classification problems. Such problems have two groups of samples: positive and negative. The predictive model uses a certain set of features and an ML classifier to predict the groups using unknown/ new feature values.

FLASH employs an embedded approach comprising two components (Fig. 7). The first component is a filtration method that selects an initial pool of features, also known as candidate features. The second component of the algorithm is the recursive feature elimination with cross-validation (RFECV), which can handle univariate and multicollinearity^48,49.

Fig. 7. Design of feature learning framework FLASH.

The dataset block represents the initial data, and the color indicates the group/class distribution. The framework has three major phases: sampling, feature scoring, and feature learning. Sampling involves drawing a random set with repetition while maintaining a similar group representation as the original data. Feature scoring calculates the filter score by performing a significance test on the random set. The final step, feature learning, includes RFECV to generate an optimal set of features from the filtered pool.

The proposed algorithm: FLASH

The two-step approach of obtaining the filter score F_score for each feature is first to randomly draw subsets from the complete dataset using pseudo code (See Algorithm 1). This subset selection strategy employs a stratified hybrid bootstrap sampling approach, ensuring that each class is proportionally represented in every resampled subset. In every iteration, we begin by selecting a portion of the samples from each class without replacement to preserve the diversity and uniqueness of the dataset. The remaining samples are then chosen with replacement, allowing us to introduce a deliberate level of redundancy in a controlled manner^50,51.

To enhance the robustness of statistical assessment scores, varied data distributions. We applied five significance tests to each dataset: one-way ANOVA, rank-sum test, Mann–Whitney U test, t-test, and Brunner–Munzel test^52,53. For each test, p-values were computed for all features across K = 1000 randomized subsets. The filter score for each feature is derived by transforming the p-values using a negative log transformation p_mod = −log₁₀(p-value)⁵⁴. The transformed score for the i^th subset is denoted as $p_{mod}^i$.

To calculate the F_score, we consider only significant p-values, i.e., only those i’s for which $p_{mod}^i>2$. These $p_{mod}^i$’s are added and then divided by the total number of random samples.

So, for K randomization, we have

$$F_{score}=\frac{{\sum }_{i=1}^K\mathbf{1}\left\{p_{mod}^i>2\right\}}{K}$$ 1

Thus, we calculated the fraction of significant features over the total number of features to assess the influence of stratified random sampling. The fraction is sorted from lowest to highest value (Fig. 8). This fraction is sorted in ascending order, and the observed increase in filtration suggests inflation of type 1 error. A considerable variation is observed in the fraction of the significant features for different random sampling, with the highest in breast cancer data (0.53) and the lowest in GSE30219(0.22) and ALL-AML (0.20).

Fig. 8. Fraction of significant features for different random sampling.

The fraction of significant features is plotted for different random samplings. Each panel shows results for a different dataset.

This large variation supports our hypothesis that substructures present in the data could influence the choice of significant features. Thus, removing features based on sampled p-values would improve the feature filtration for the downstream process.

Algorithm 1

Pseudo code to calculate p-values for sampled subsets using five statistical tests

Require: Dataset D with S samples, N features, and class labels Y; number of iterations K; sampling ratio α ∈ (0, 1)

Ensure: Matrices TTEST, WILCOX, ANOVA, BM, $MWU\in {\mathbb{R}}^{N\times K}$ storing p-values

 1: Partition D into class-wise subsets: D₀, D₁, …, D_C where C is the number of classes

 2: Initialize matrices: TTEST, WILCOX, ANOVA, BM, MWU ← 0^N×K

 3: for i = 1 to K do

 4: $D_{sub}\leftarrow \varnothing$

 5: for each class c ∈ {0, …, C} do

 6: S_c ← ∣D_c∣

 7: Draw ⌊α ⋅ S_c⌋ samples from D_cwithout replacement$\to D_{unique}^c$

 8: Draw S_c − ⌊α ⋅ S_c⌋ samples from D_cwith replacement$\to D_{rep}^c$

 9: $D_{sub}\leftarrow D_{sub}\cup \left(D_{unique}^c\cup D_{rep}^c\right)$

10: end for

11: for each feature j = 1 to N do

12: Compute t-test p-value → TTEST[j, i]

13: Compute Wilcoxon Rank-Sum p-value → WILCOX[j, i]

14: Compute One-way ANOVA p-value → ANOVA[j, i]

15: Compute Brunner–Munzel p-value → BM[j, i]

16: Compute Mann–Whitney U p-value → MWU[j, i]

17: end for

18: end for

The core principle of FLASH lies in training a supervised model using the initial screened features based on a predefined percentage cut-off on F_score. This initial step filters and selects statistically significant features. We used these features to train classification models, including Decision Tree, K-Nearest Neighbours (KNN), Support Vector Machine (SVM), Logistic Regression, Random Forest, and XGBoost. The absolute value of the model’s coefficient with maximum accuracy ranked the features. By iterating through these ranked features, we systematically eliminated lower-ranked features from the set using recursive feature elimination with cross-validation (RFECV) and continued until the last feature. During this process, the model’s accuracy was monitored at each step, and the set with the highest model performance was selected as the optimal set. RFECV inherently includes an early stopping mechanism to optimize feature selection. However, in FLASH, we want to ensure that the optimal set corresponds to maximum accuracy, so we iterate through the entire feature set. If multiple optimal sets have the same accuracy, then FLASH choose the set with fewer features. This dynamic elimination strategy ensures that the final selected features are statistically significant and optimally contribute to the model’s predictive performance. (See Algorithm 2). The pseudo-code 2 outlines the step-by-step feature selection process within the FLASH framework.

Algorithm 2

Pseudo code to perform stepwise feature elimination.

 1: Input: Samples with N features

 2: Output: An optimal set of features F_final for maximum accuracy

 3: S ← Features_initial

 4: Initialize F_master ← {}

 5: fori = Num_Features to 1 do

 6: Build predictive model on S

 7: S_rank ← Sorted features based on current model’s coefficient values from high to low

 8: f_least ← S_rank[i]

 9: S_new ← S − {f_least}

10: S_acc ← accuracy using cross-validation on S_new

11: Store (S_new, S_acc) to F_master

12: S ← S_new

13: end for

14: F_final ← element from F_master with S_acc maximum among all sets

Quantifying model effectiveness

The number of positive samples is denoted by N_p, and the negative samples by N_n. The precision (P) is defined as the proportion of predicted positives that were actually correct, while recall (R) is defined as the proportion of actual positive samples that were correctly identified⁵⁵. The formulas for P and R are $P=\frac{T_P}{T_P+F_P}$ and $R=\frac{T_P}{T_P+F_N}$, where T_P represents the number of correctly predicted positive samples, and T_N represents the number of correctly predicted negative samples. The overall prediction accuracy, denoted as Acc, is calculated as $\frac{T_P+T_N}{N_p+N_n}$, and mACC is the maximum accuracy achieved by any ML algorithm, similar to the approach used in previous studies^56–58. To estimate a more robust and balanced predictive performance, we also used the F1-Score (F1)⁵⁹, which is defined as the harmonic mean of precision and recall, i.e., $F1=\frac{2*P*R}{P+R}$. We have used 10 times repeated 10-fold cross-validation to obtain the model predictions withf these metrics^60,61.

Evaluation of statistical filters and recursive elimination strategy

To evaluate our algorithm, we assessed the performance of five statistical tests: one-way ANOVA²⁵, rank-sum test²⁵, Mann–Whitney U test²⁵, t-test²⁵, and the Brunner–Munzel test²⁶ across 8 datasets, as we can refer to the (Supplementary Table 1). We computed the classification accuracy after feature selection and calculated the proportion of datasets where the accuracy exceeded a 95% threshold. One-way ANOVA test achieved the highest score of 0.6875, exceeding 95% accuracy in the datasets. It was followed by the t-test, which scored 0.63. Mann–Whitney U, Brunner–Munzel, and Ranksum tests recorded lower scores, all falling below 0.59, as seen in the (Supplementary Table 2. Based on this, we used a One-way ANOVA for the subsequent analysis.

To determine the optimal feature pool size, we applied the kneedle algorithm⁶² across all five statistical tests to identify the “elbow point”. The algorithm identified 6% as the optimal test cutoff (Supplementary Fig. 2). We present the results using the five statistical tests at 6% (Fig. 9A) and ANOVA tests in (Fig. 9B) demonstrate a 6% cutoff yielded across the eight datasets. To evaluate the robustness of our cutoff, we calculated the mean accuracy for different p-values (<0.05, and <0.001) (Supplementary Fig. 3). The mean accuracy for features with p-values <0.001 was lower than that observed for other significance thresholds and was excluded from further analysis. We found that the mean accuracies for p-value thresholds of <0.05 and <0.01 were comparable, with both showing maximum accuracy of ~6%. We adopted a p-value cutoff of <0.01 for subsequent accuracy calculations to ensure the selection of robust features while maintaining statistical relevance.

Fig. 9. Parameter calibration in selected datasets.

A Comparison of different tests at 6% obtained using mAcc. B Comparison of different cut-off percentages obtained from F_score using mAcc. C Comparison of different elimination steps for RFECV using mAcc.

We experimented with feature elimination step sizes ranging from 1 to 5 in the RFECV process. A step size of 1 showed the most stable and optimal performance across all eight datasets (Fig. 9C). These findings lead to our final parameter selection for all subsequent experiments.

Author contributions

S.K.: Conceptualization, Data curation, Formal analysis, Methodology, Visualization, Writing original draft. A.A.: Formal analysis, Methodology, Visualization, Data curation, Writing original draft. S.C.: Conceptualization, Methodology, Writing review and editing, Supervision, funding.

Data availability

The datasets analyzed in this study are publicly available and have associated GEO identifiers and references. These can be accessed through the GEO database. We have provided access to datasets not associated with a GEO identifier in the GitHub repository https://github.com/samrat-lab/FLASH/.

Code availability

The scripts used in this work are available at https://github.com/samrat-lab/FLASH/.

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at 10.1038/s41540-025-00614-x.