Machine Learning of Single-Cell Transcriptome Highly Identifies mRNA Signature by Comparing F-Score Selection with DGE Analysis

Abstract

Human preimplantation development is a complex process involving dramatic changes in transcriptional architecture. For a better understanding of their time-spatial development, it is indispensable to identify key genes. Although the single-cell RNA sequencing (RNA-seq) techniques could provide detailed clustering signatures, the identification of decisive factors remains difficult. Additionally, it requires high experimental cost and a long experimental period. Thus, it is highly desired to develop computational methods for identifying effective genes of development signature. In this study, we first developed a predictor called EmPredictor to identify developmental stages of human preimplantation embryogenesis. First, we compared the F-score of feature selection algorithms with differential gene expression (DGE) analysis to find specific signatures of the development stage. In addition, by training the support vector machine (SVM), four types of signature subsets were comprehensively discussed. The prediction results showed that a feature subset with 1,881 genes from the F-score algorithm obtained the best predictive performance, which achieved the highest accuracy of 93.3% on the cross-validation set. Further function enrichment demonstrated that the gene set selected by the feature selection method was involved in more development-related pathways and cell fate determination biomarkers. This indicates that the F-score algorithm should be preferentially proposed for detecting key genes of multi-period data in mammalian early development.

Introduction

Human preimplantation embryo development refers to the first 7 days of fertilization, which proceeds through stages of the two-cell stage, four-cell stage, eight-cell stage, morula, blastocyst, and late hatched blastocyst.^1,2 The first process of zygote development is zygotic genome activation (ZGA) when the embryo gradually stops depending on maternally inherited transcripts and proteins and initiates zygotic genome transcription.^3,4 After a small transcriptional activation wave from oocytes to the four-cell stage, major ZGA genes are upregulated between the four-cell and eight-cell stage and start to regulate the biological development of the embryo.^5,6 Then, the differences between embryonic cells begin to appear, and three different blastocyst cell lineages are formed.⁷ The formation of the trophectoderm (TE) reflects the first lineage segregation, followed by the next lineage segregation when the inner cell mass (ICM) is divided into primitive endoderm (PE) and epiblast (EPI) cells.⁸ In fact, there have been numerous studies of mRNA identification for embryo development. For example, it has been found that pioneering factors (ARGFX, CPHX1, LEUTX, and DUX4) activate the ZGA program by an overexpression experiment and transcriptional analysis.^9,10 In mice, CDX2 represses OCT4 expression in the outer cells, leading to TE-ICM lineage segregation,¹¹ but in human CDX2-OCT4 antagonism may not be necessary.¹² DPPA2 and DPPA4 regulate expression of Dux and LINE-1 in mouse embryonic stem cells, suggesting that they are an upstream factor of ZGA.^13,14 Moreover, Yan et al.¹⁵ have identified 2,733 potential novel long non-coding RNAs (lncRNAs) that were involved in preimplantation. However, potential molecular events of embryo development are not fully understood.

Recently, the single-cell RNA sequencing (RNA-seq) techniques are the main method for detecting developmental trajectories and cellular heterogeneity in early preimplantation embryos;16, 17, 18, 19 however, such techniques could only provide detailed clustering signatures, and the identification of decisive factors remains difficult and requires high experimental cost and a long experimental period. As good complements to experimental techniques, computational methods play high potential roles for cancer diagnosis and sequence classification.20, 21, 22, 23, 24, 25, 26, 27 For example, Capper et al.²⁸ proposed random forest (RF) to classify approximately 100 known tumor types of the central nervous system based on DNA methylation data. Based on a single-cell transcriptome, single-cell variational inference (scVI) aggregates information across similar cells and genes by stochastic optimization and deep neural networks,²⁹ and Scialdone et al.³⁰ constructed a predictor for identifying cell-cycle stage. In addition, feature selection methods are independent of prior knowledge of biological dependencies, having been applied in bioinformatics, including protein prediction and biomarker discovery.^31,32 The QSPred-FL tool is based on the fact that quorum-sensing peptides in large-scale proteomic data can be detected by feature representation learning and machine learning algorithms,³¹ and the GF-ICF (gene frequency-inverse cell frequency) pipeline can provide an effective and simple workflow for feature selection and subsequent analyses. However, many studies have advanced new computational methods to interpret single-cell RNA-seq data,^29,30,33,34 but most existing methods cannot build predictive models of development. To the best of our knowledge, so far there is no computational tool available for identifying signature genes of development.

Here, we develop the EmPredicitor, a novel machine learning-based tool for predicting stages of human embryonic development. In this predictor, we compared three traditional differential gene expression (DGE) analyses with a feature selection method based on a single-cell RNA-seq dataset of preimplantation embryos. Figure 1 shows a schematic diagram of the model establishment workflow. The dataset was first integrated and removed genes with no expression in all cells. Then, we applied three DGE methods (edgeR, limma, and DESeq) and a feature selection method (F-score algorithm) to obtain signature genes. By comparing these method performances based on support vector machine (SVM) and functional enrichment analysis, the F-score algorithm had the highest performance and obtained an area under the receiver operating characteristic (ROC) curve (AUC) of 0.95. Our results also suggested that DGE analysis relied on pairwise comparison and overlap, inducing the loss of some key genes that were highly expressed at multiple stages, and the F-score algorithm considered gene expression at all stages and ignored low expression of transcripts.

Figure 1.

The Workflow of Construction and Validation of the Embryo Development Signatures

The best signature from the F-score method consisted of 1,881 genes related to embryo development that was constructed and validated using gene expression from publicly available datasets.

Results

Global Expression Profiles of Human Embryos

Global transcriptome profiles were first analyzed based on a dataset of early human embryos (Figure 2A, reads per kilobase transcript per million mapped reads [RPKM] > 0). The gene expression level of E3 cells was higher than that for other stages, indicating that the zygote genome was activated and began to identify the genetic program that may control this process.¹⁵ Also, the gene expression level of the cells began to decrease during the E3–E6 stage but increased during the E7 stage, suggesting that E7 embryos may initiate new transcriptional activation to begin embryo implantation. To determine whether embryos at the same stage showed a high correlation, we analyzed RNA-seq data of the E3–E7 embryos using the SC3 package (Figure 2C).³⁵ Most of the cells from the same stage were clustered into one cluster. E3 cells and E4 cells have a high correlation. However, E5 cells have two clusters, because after ZGA, differences of embryos begin to emerge and E5 cells appear to segregate ICM and TE.¹² Interestingly, the embryos at the E6 and E7 stages were clustered together and divided into three clusters, suggesting that preimplantation of the early embryo resolved in the formation of three distinct cell lineages of blastocysts.³⁶

Figure 2.

Global Transcriptome Profiles of Human Embryos

(A) Expression-level boxplots for the expressed gene in E3–E7 embryos normalized to the Z score. (B) Heatmap of E3–E7 cells and the top 1,881 genes based on the F-score using unsupervised clustering. (C) Two-dimensional t-SNE representation of 1,529 single-cell preimplantation transcriptomes using the top 1,881 genes according to the F-score feature selection. (D) Heatmap of E3–E7 cells and 4,876 differentially expressed genes based on DESeq.

In order to investigate whether these gene expression profiles were related to developmental stages, we conducted t-distributed stochastic neighbor embedding (t-SNE) on all individual embryos (Figure 2B)^37,38 and found that embryos at the same developmental stage were clustered together, and the primary segregating factor was developmental time. With the development of embryos, the heterogeneity of embryos increased gradually. In addition, we used differentially expressed genes (DEGs) to plot a heatmap by DESeq, which reflected that embryo cells segregated into five groups, that E6 cells were less different from cells in adjacent stages, and that E3 cells have more DEGs than do other stages (Figure 2D).

Identification of the Developmental Signature by Comparing with F-Score and Differential Expression Analysis

To identify the best signature genes related to embryonic development, we obtained 24,444 gene expression profiles of 1,529 individual cells (81 E3, 190 E4, 377 E5, 415 E6, and 466 E7 cells) from a public database. As DGE analysis usually applied sequence count data, count data were analyzed by comparing three DGE analyses and the F-score algorithm. By using the same parameter (fold change > 2, p < 0.05), limma, DESeq, and edgeR identified 3,754, 4,876 and 6,231 DEGs, respectively, and the number of overlapping genes based on these methods was 2,976 (Figure 3A; Figure S1A). E3 cells had the highest number of DEGs compared to other stages (Figure S1A), and edgeR had more differential genes than did other methods (Figure 3A). The F-score algorithm calculated and ranked each gene score, but we still did not know how many genes should finally be selected. To optimize signature gene selection, we tried the number of signature genes (ranging from 10 to 24,444 genes) for training the support vector machine model and calculated their prediction performance, which applied incremental feature selection (IFS) (Figure 3B).^39,40 More genes may bring the best performance but lead to more hardware and time loss, so we chose 1,881 of the top genes by considering gene number and their performance, the accuracy of which was 0.93, and this cost low memory consumption and took only 43 min to obtain the best model. Then, we plotted a heatmap of 1,881 genes based on F-score algorithms (FSGs) and using the SC3 package (Figure S1B). Interestingly, E5 cells were separated into two clusters, suggesting that embryo lineage separation showed the formation of TE and ICM.^41,42

Figure 3.

Identification of Development Signature by Comparing the F-Score with Differential Expression Analysis

(A) Venn diagram of DEGs from limma, edgeR, and DESeq. The number of overlapping genes from the three DGE methods is 2,976. (B) The IFS curve with the number of genes and the performance of classifiers. The x axis is the number of genes used for SVM classifier construction and the y axis is the accuracy of the SVM classifier evaluated with 5-fold cross-validation. (C) Venn diagram showing a comparison of DEGs and FSGs. Representative gene ontology (GO) terms are listed. (D) Boxplots of POU5F1 relative expression level at the E3–E7 stages.

We also compared DEGs and FSGs by gene function enrichment (Figure 3C; Table S1). Unique DEGs were enriched for regulation of the cellular process, positive regulation of the multicellular organismal process, positive regulation of the metabolic process, and multicellular organism development. Unique FSGs were enriched for embryo development, regulation of cell shape, cell-cell adhesion, and the cell cycle, which are most relevant to embryonic development. The genes overlapped by DEGs and FSGs were enriched for embryo implantation, regulation of transcription, DNA-templated, multicellular organism development, and positive regulation of cell proliferation, which related to transcription and development. Briefly, unique FSGs related more to embryonic development compared to unique DEGs.

As shown in Table 1, some key genes were selected from FSGs and DEGs. DPPA5, ZSCAN4, and SOX2 were selected by using these methods (rank 9, 22, and 1,520, respectively). DPPA5 stabilizes NANOG and supports human pluripotent stem cell (hPSC) self-renewal and cell reprogramming in feeder-free conditions.⁴³ ZSCAN4 is a unique gene highly expressed at the zygotic genome activation stage.^44,45 The POU5F1 gene is vital for PSC maintenance in the mammalian embryo.^46,47 Interestingly, POU5F1 (rank 1,470) was not selected by the DGE method but was obtained by F-score. Therefore, we analyzed the expression of POU5F1 among E3–E7 stages, and POU5F1 was highly expressed in E4 and E5 cells (Figure 3D). Therefore, DGE analysis mainly relies on pairwise comparison and overlap, so if a gene is highly expressed at two or more stages, differential expression analyses may lose the gene, suggesting that DGE analysis only considered DEGs highly expressed at a stage. The F-score algorithm showed the importance of a gene in all stages. If the expression level of a gene was too low, the F-score algorithm would give a low score for this gene, especially similar to ERVFRD-1 (Figure S1C; Table S2). Although transcripts of low expression may be important, most of these are outliers, suggesting that transcripts with low expression levels were preprocessed, in line with previous studies.⁴⁸

Table 1.

Comparison between Key Genes of FSGs and DEGs

Genes	Gene Ontology Terms	FSGs (1,881)	F-Score	limma	edgeR	DESeq
GDF9	positive regulation of cell proliferation	●	1.97	∗∗∗	∗∗∗	∗∗∗
DPPA5	multicellular organism development	●	1.51	∗	∗	∗
RNF168	zinc ion binding	●	1.42	∗∗∗	∗∗∗	∗∗∗
KLF17	transcription factor activity, sequence-specific DNA binding	●	1.39	∗∗∗	∗∗∗	∗∗∗
ZSCAN4	transcription, DNA templated	●	1.20	∗	∗	∗
TLE6	repressing transcription factor binding	●	1.14	∗∗∗	∗∗∗	∗∗∗
CCKBR		●	0.65	∗∗∗	∗∗∗	∗∗∗
PTN		●	0.49	∗∗∗	∗∗∗	∗∗∗
CD24		●	0.48
OSR2	embryo development	●	0.34
DLX3	transcription from RNA polymerase II promoter	●	0.31	∗∗∗
SOX2	transcription factor endodermal cell fate specification	●	0.22	∗	∗	∗
POU5F1	somatic stem cell population maintenance	●	0.22
STC2	embryo implantation	●	0.19
ERVFRD-1		○	0.09	∗∗∗	∗∗∗	∗∗∗

If a gene belongs to the dataset, replace it with ●; otherwise, replace it with ○. F-score shows the importance of features selected by the F-score algorithm. ∗p < 0.05, ∗∗∗p < 0.001.

In addition to known markers, several less described markers were identified, such as RNF168, CCKBR, PTN, CD24 and STC2 (rank 12, 151, 332, 349, and 1,851). CCKBR, a cholecystokinin B receptor, has been found in a diverse range of cancers.⁴⁹ We found that in the late blastocyst, E6 and E7 cells expressed high levels of CCKBR, indicating that CCKBR may be involved in the ICM segregation of EPI and PE cells. PTN-encoded protein has significant roles in cell growth, migration, and tumorigenesis,^50,51 and it was expressed in the late blastocyst, suggesting that PTN may be involved in embryonic cell migration. Then, we found that most of the 500 top-ranked genes were high relative expression genes of E3 stages (Table S2), similar to what has been previously reported.¹⁵

Predictor of Human Preimplantation Development and the Web Server

To develop a predictor to identify developmental stages of human preimplantation embryogenesis, we applied the support vector machine classifier to train models based on three DGE analyses and F-score algorithm in 5-fold cross-validation, and we obtained the performance of the four methods (Table 2). The models of the four method showed high performances; however, FSGs achieved precision, recall, accuracy, and F1 measure values of 0.933, 0.929, 0.930, and 0.930, respectively, and the number of FSGs had the fewest (Table 2). In addition, the classifier using the F-score algorithm also showed high performance, with an AUC greater than 95% (Figure 4).

Table 2.

Performance of Stage Predicting Models with 5-Fold Cross-Validation

Method	Gene No.	Precision (%)	Recall (%)	Accuracy (%)	F1 Measure (%)
DESeq	4,876	90.23	89.81	89.85	89.73
limma	3,754	91.5	91.23	91.24	91.24
edgeR	6,231	90.82	90.36	90.42	90.31
F-score	1,881	93.3	92.91	93.01	93.2

Underlined text represents the maximum value of every performance evaluation criterion.

Figure 4.

The Predictor of Human Preimplantation Development Based on Machine Learning

ROC curve and AUC showing that the F-score method obtained high performance.

Based on our proposed model, a user-friendly and publicly accessible web server for EmPredictor was established (available at http://bioinfor.imu.edu.cn/empredictor), where users can upload or paste a dataset of the eight key genes to predict the stage of their samples. The home page of EmPredictor is shown in Figure 5. We also considered that users may want to know the relative expression trend of a gene, so the server provides the function of searching for a gene on a single-cell dataset from E-MTAB-3929. The user guide is available on the web page.

Figure 5.

A Semi-screenshot Showing the Home Page of the EmPredictor Web Server

Discussion

Herein, we have proposed the first EmPredicitor, a novel machine learning-based tool for predicting stages of human embryonic development. Based on three DGE analyses (limma, edgeR, DESeq) and F-score algorithm, the single-cell transcriptomes data obtain 3,754, 4,876, 6,231, and 1,881 signature genes, respectively. Then, supervised machine learning is used to estimate the contribution of embryonic development to these signature genes. Toward the application of 5-fold cross-validation on a benchmark dataset, the F-score algorithm can achieve the highest accuracy of 0.93 and AUC of 0.95. Furthermore, functional enrichment analysis showed that the F-score algorithm can obtain key signaling pathways related to embryo development. Based on prior biological knowledge, some key genes were used to estimate the assessment of F-score and DGE analyses. DGE analyses rely on pairwise comparison and overlap to obtain differentially expressed genes. F-score detected key genes of multi-period data that contributed to identifying early embryo stages. In addition, we constructed a user-friendly and publicly accessible web server where users can upload or paste a dataset of the eight key genes to predict the stage of their samples.

There are still some disadvantages of this work. Here, we investigated only predicting embryonic days. However, embryonic development is a complex process involving lineage specification and X chromosome dosage compensation.^7,12,46 Integrating genetic and epigenetic data with gene expression may provide a more comprehensive view of embryonic development. In addition, feature selection methods have irreplaceable advantages in processing single-cell transcriptome data and are independent of prior knowledge of biological dependencies, which extend the development analysis pipeline. In the future, we will use advanced feature selection methods to study embryonic development based on more accurate molecular events and multi-omics data.

Materials and Methods

Data and Preprocessing

We downloaded a single-cell transcriptome dataset of human preimplantation embryos from ArrayExpress under accession E-MTAB-3929,¹² including 1,529 samples. The dataset has five different cell stages, which are embryonic day (E)3, E4, E5, E6, and E7. The E3 stage has 81 cells, the E4 stage has 190 cells, the E5 stage has 377 cells, the E6 stage has 415 cells, and the E7 stage has 466 cells.

The data were processed using TrueSeq dual-index sequencing primers (Illumina) according to the manufacturer’s recommendations on an Illumina HiSeq 2000.¹² The data quality was checked and reads were mapped to the human genome (hg19) using STAR with default settings.⁵² RPKM were calculated using rpkmforgenes⁵³ by the uniquely mapped read counts. Genes were filtered, keeping 24,444 out of 26,178 genes that were expressed in at least 1 out of 1,529 cells (count > 0).

Feature Selection

Linear Model

During the past decade, the limma package⁵⁴ has been a popular choice for gene discovery through differential expression analyses of microarrays. Recently, limma has also provided differential expression and differential splicing analyses of RNA-seq data. limma uses the voom function by converting mean variance to precision weights and using a linear model,

$$E\left(y_{gi}\right)={\mu }_{gi}={\text{X}}_i^T{\beta }_g,$$ (Equation 1)

where X_i is a vector of covariates and β_g is a vector of unknown coefficients representing log₂ fold changes between experimental conditions. In matrix terms

$$E\left(y_g\right)=\frac{\text{X}}{{\beta }_g},$$ (Equation 2)

where y_g is the vector of log cpm values for gene g, and X is the design matrix with the X_i as rows. The limma package is available at https://bioconductor.org/packages/release/bioc/html/limma.html.

Negative Binomial Distribution

edgeR⁵⁵ is designed for the analysis of replicated count-based expression data. Data are modeled as negative binomial (NB) distributed

$$Y_{gi}\sim NB\left(M_i{p}_{gj},{\phi }_g\right)$$ (Equation 3)

for gene g and sample i. Here, M_i is the library size (total number of reads), φ_g is the dispersion, and p_gi is the relative abundance of gene g in experimental group j to which sample i belongs. The edgeR package is available at https://bioconductor.org/packages/release/bioc/html/edgeR.html.

DESeq⁵⁶ provides methods to test for differential expression by use of the negative binomial distribution and a shrinkage estimator for the distribution’s variance,

$$K_{ij}\sim NB\left({\mu }_{ij},{\sigma }_{ij}^2\right),$$ (Equation 4)

which has two parameters, the mean μ_ij and the variance ${\sigma }_{ij}^2$. The read counts K_ij are non-negative integers. The DESeq package is available at https://bioconductor.org/packages/release/bioc/html/DESeq.html.

F-Score Algorithm

F-score is a simple and basic but effective algorithm for evaluating the importance of each feature in the dataset. F-score is a computed each feature values and

$$F\left(i\right)=\frac{{\left({\bar{x}}_i^{(+)}-{\bar{x}}_i\right)}^2+{\left({\bar{x}}_i^{(-)}-{\bar{x}}_i\right)}^2}{\frac{1}{n_{+}-1}{\sum }_{k=1}^{n_{+}}{\left(x_{k,i}^{(+)}-{\bar{x}}_i^{(+)}\right)}^2+\frac{1}{n-1}{\sum }_{k=1}^{n_{-}}{\left(x_{k,i}^{(-)}-{\bar{x}}_i^{(-)}\right)}^2},$$ (Equation 5)

where ${\bar{x}}_i,{\bar{x}}_i^{(+)},{\bar{x}}_i^{(-)}$ are the average of the ith feature of the whole, positive, and negative datasets respectively; $x_{k,i}^{(+)}$is the ith feature of the kth positive instance; and $x_{k,i}^{(-)}$is the ith feature of the kth negative instance. A Python program fselect.py can compute each feature value and rank the feature downloaded from https://www.csie.ntu.edu.tw/∼cjlin/libsvmtools/.⁵⁷

Machine Learning Model Implementation

The support vector machine (SVM) was proposed by Vapnik et al.⁵⁸ SVM shows many advantages in solving small sample, nonlinear, and high-dimensional pattern recognition. The idea of SVM is based on transforming the input vector into a high-dimensional Hilbert space and finding a separating hyperplane in this space. Gaussian radial basis function (RBF) kernel function⁵⁹ is a widely used kernel function because of its high performance in non-line classification:

$$K_{Gaussion}\left(x_i,x_j\right)=e^{\frac{{\left|\left|x_i-x_j^{}\right|\right|}^2}{2\sigma }}.$$ (Equation 6)

We applied LIBSVM as an SVM model with a one-against-one strategy⁶⁰ and RBF kernel. A grid search strategy with a cross-validation test is always utilized to obtain the best values of the regularization parameter C and kernel parameter g. We used the gird.py file (https://www.csie.ntu.edu.tw/∼cjlin/libsvmtools/) in LIBSVM to search for the best C value and g value (the range of the C parameter is between 2⁻⁵ and 2¹⁰, and the range of the g parameter is between 2⁻¹⁵ and 2³).⁶⁰ Classifier performance was evaluated by 5-fold cross-validation analysis,^28,59 where each training dataset was randomly partitioned into four equal parts with one part being used for model training and the remaining part used for testing. We used the cross-validation method to limit overfitting of the classifier. To have a complete measurement of the prediction performance, four statistics, i.e., accuracy, recall, precision, and F1 measure,^30,59 were calculated as follows:

$$Accuracy=\frac{TP+TN}{TP+FN+TN+FP}$$ (Equation 7)

$$Recall=\frac{TP}{TP+FN}$$ (Equation 8)

$$Precision=\frac{TP}{TP+FP}$$ (Equation 9)

$$F1measure=\frac{2\ast \left(precision\ast recall\right)}{\left(precision+recall\right)}=\frac{2\ast TP}{2\ast TP+FN+FP},$$ (Equation 10)

where TP is the true positive correct result, FP is the false unexpected result, FN is the false missing result, and TN is the true correct absence of result.

Code Available

The code for the implementation of the EmPredictor is available on GitHub: https://github.com/liameihao/EmPredictor.

Author Contributions

Y.Z. designed this work. P.L. and W.Y. performed the whole bioinformatics analysis and wrote the manuscript. X.C. and C.L. performed the experiments and helped edit the manuscript. L.Z. and H.L. assisted the experiments.

Conflicts of Interest

The authors declare no competing interests.