Table 1.

Overview of available PAS tools included in this benchmarking.

Name	Date	Platform	Description / Exclusion reason	Inclusion	Reference
PLAGE	2005	R*	Singular value decomposition	True	[31]
z-score	2008	R*	Combined z-score	True	[30]
ssGSEA	2009	R*	Kolmogorov-Smirnov-like rank statistic based on gene expression of single sample	True	[16]
GSVA	2013	R	Kolmogorov-Smirnov-like rank statistic based on kernel estimation of the cumulative density	True	[15]
Pagoda2	2017	R	First principal component of gene sets	True	[19]
AUCell	2017	R	Area under the ranked gene expression curve	True	[26]
Vision	2019	R	Summarizing the normalized expression of genes in the gene sets	True	[21]
ROMA	2016	R/Python/Matlab	Running time is too slow (costs 2.8 h on Test Data* with 4 cores)	False	[55]
f-scLVM	2017	R	Running time is too slow (costs 4.3 h on Test Data*)	False	[56]
PROGENY	2018	R	Non-extensible (This method only inferred pathway activity scores for predefined 14 signaling pathways)	False	[28]
Single Cell Signature Explore	2019	GO	not implemented in R/Python	False	[57]

Note: R*: original article did not have implemented it, cooperated in R package GSVA; Test Data*: 33,694 genes × 10000 cells, combining with KEGG database.