Towards Mechanism Classifiers: Expression-anchored Gene Ontology Signature Predicts Clinical Outcome in Lung Adenocarcinoma Patients

Abstract

We aim to provide clinically applicable, reproducible, mechanistic interpretations of gene expression changes that lack in gene overlap among predictive gene-signatures. Using a method we recently developed, Functional Analysis of Individual Microarray Expression (FAIME), we provide evidence that Gene Ontology-anchored signatures (GO-signatures) show reliable prognosis in lung cancer. In order to demonstrate the biological congruence and reproducibility of FAIME-derived mechanism classifiers, we chose a disease where gene expression classifiers signatures alone had failed to significantly stratify a larger collection of samples and that exhibited poor or no genetic overlap. For each patient in the two lung adenocarcinoma studies, personalized FAIME-profiles of GO biological processes are generated from genome-wide expression profiles. For both training studies, GO-signatures significantly associated to patient mortality were identified (Prediction Analysis for Microarrays; three-fold cross-validation). These two GO-signatures could effectively stratify patients from an independent validation cohort into sub-groups that show significant differences in disease-free survival (log-rank test P=0.019; P=0.001). Importantly, significant mechanism overlaps assessed by information-theory similarity were detected between the two GO-signatures (Fischer Exact Test p=0.001). Hence, together with machine learning technologies, FAIME could be utilized to develop an ontology-driven and expression-anchored prognostic signature that is personalized for an individual patient.

1. INTRODUCTION

Gene-signatures derived from genome-wide expression profiling have been extensively used in oncology to predict poor prognosis [1], recurrence [2], invasiveness [3], and metastasis [4–6]. Despite the fact that gene signatures derived from different cohorts of patients are highly comparable in predicting clinical outcome for a specific cancer, gene-signatures have the following deficiencies as noted by Khatris et al [7]: (i) the underlying mechanisms or clinical relevance is not readily attainable from gene lists [8], (ii) gene lists derived from distinct cohorts or studies exhibit minimal overlap [8], and (iii) methods of pathway (e.g. Gene Ontology (GO)) enrichment over a gene signature of the entire sample set are incapable of generating individual GO profiles for personalized prognosis (or quantitative trait). In clinical settings, these limitations could be further compounded by conflicting interpretations from distinct gene signatures conducted over the same patient biopsy. Further, the biological function of a significant number of genes in a signature has not been characterized in the cancer of interest, making the determination of true diagnostic or prognostic genes unattainable. Nonetheless, development of bioinformatics methods for generating diagnostic or prognostic signatures from expression profiling has remained focused on gene signatures.

Biological mechanism-driven strategies are promising to discover prognosis signatures that will provide clinicians with mechanistic interpretations of biological changes associated with the deduced gene signatures, enabling decisions about the most appropriate therapeutic options with molecular guidance [9]. In particular, the Gene Set Enrichment Analysis (GSEA) method can identify mechanistically related genes in a geneset exhibiting coordinated expression patterns, and are now recognized as a valuable approach for pattern recognition of biological mechanisms [10, 11]. However, GSEA is not designed for generating a personalized mechanistic profile for each patient. Further, Acharya, et al. developed a mechanism-anchored breast cancer prognostic signature using eleven genesets and a personalized score. Bayesian binary regression was assigned to each sample of each of the eleven genesets [12]. However, this method is not scalable, as such scores are not normalized for geneset size and it is neither designed for high dimensionality (large number of genesets), nor for mechanism profiling (e.g. the method provides a capability for discovering a single mechanism and is not designed for identifying multiple mechanisms)

We and others hypothesized that because different genes may regulate the same biological function, mechanism-anchored signatures by directly assessing gene functions will exhibit significantly more overlap, as we have previously shown using a novel ontology-transformation approach, the Functional Analysis of Individual Microarray Expression (FAIME) method [13, 14]. Further, by providing mechanistic interpretations of biological changes, such signatures are more clinically applicable. Indeed, other complex physiological signal readouts are one routinely computationally simplified for improved human interpretation (e.g. just-in-time textual machine-readouts of electrocardiograms, automated electroencephalogram, automated analysis in fetal monitoring, etc.). Specifically, we have explored that (i) using an ontology-transformation approach derived from the FAIME method exclusively utilizing GO biological processes, such ontology transformation of individual expression arrays can produce individualized GO-wide scores for biological mechanisms; (ii) reapplying ontology transformation methods to analyze gene expression datasets that are traditionally challenging to analyze due to their small sample size is not only feasible but also robust; and (iii) these personalized quantitative profiles of FAIME-derived GO biological processes (GO-signatures) for each patient are more reproducible among different studies than gene signatures [14].

The goal of identifying GO terms as signatures of genome-wide expression datasets is a departure from the common approaches of identifying a geneset, and is supported by Blois’s scalar theory of biomedical information in clinical genomics, in that intermediate phenotypes of genes’ expression, such as biological processes, may serve as emergent properties for clinical phenotypes [15]. Here, we demonstrate the feasibility of learning and validating potentially paradigm-shifting personalized prognostic signatures ab initio using FAIME [14] over three lung cancer datasets. To demonstrate the biological congruence and reproducibility of FAIME-derived mechanism classifiers as compared to expression-based ones, this study differs from our previous methodological development work in that we specifically targeted a challenging disease where gene expression classifiers signatures alone had failed to significantly stratify a larger collection of samples [16] and that exhibited poor or no genetic overlap [16]. Importantly, the identified GO-signatures are anchored in both biological mechanisms and gene-expression and are reproducible in different datasets where gene-based classifiers fail to overlap. After demonstrating the robustness and reproducibility of the FAIME method [14], we further illustrate that FAIME can reproducibly identify a prognostic signature from a small sample size of lung cancer patients that would otherwise have been prohibitive to analyze with gene-level classifiers due to the lack of statistical power of smaller samples.

Additional Background. Lung adenocarcinoma is the predominant subtype of non-small cell lung cancers and accounts for 40% of all lung carcinomas. Prognostication of lung adenocarcinoma (LA) is important because about 17% of patients will survive more than 5 years after diagnosis, and current clinical and histological scores are insufficient for comprehensive prognostication. Gene signatures derived from genome-wide expression profiling have thus been extensively used for LA prognostication, outcome prediction, and adjuvant targeted therapy selection [16–19]. However, studies showed that gene signatures alone failed to significantly stratify a larger collection of LA including 442 samples [16]. In addition, gene expression signatures derived from oligonucleotide microarray [1, 2] and cDNA microarray [21] studies showed minimal genetic overlap [11]. In this study, we first employed FAIME to analyze these three datasets and showed the significant overlap between the results of FAIME and those from GSEA in Table 1. Subsequently, GO-signatures derived from two datasets were evaluated for their prognosis in the third dataset.

Table 1.

MSigDB mechanism-signature of FAIME were compared to those of the GSEA for discriminating between “good” vs “poor” prognosis as defined by the authors. Note that the FAIME predictions occur with FDR≤10% in all datasets, while the GSEA method requires a FDR=25% (thus less sensitive) and fails to predict mechanisms in the Stanford dataset (below, red arrow). Every predicted mechanisms of this table are reported in Supplement Table S1 with their respective method and dataset.

Lung Adeno-carcinoma Array Datasets	Number of predicted MSigDB mechanisms (their FDR threshold)			Enrichment betweenFAIME and GSEA; Fischer Exact Test P-value
	using FAIME-Scores (FDR threshold)	using GSEA as published in PNAS [11]	Overlap between FAIME & GSEA
- Boston [20]	4 (1%)*	8 (25%)	4	<2×10–8
- Michigan [1]	16 (10%)	11 (25%)	4	<2×10–4
- Stanford [21]	5 (10%)	➜0 (25%)	NA	NA

Legend: GSEA results from annotations and ontologies were compared to the FAIME results using the same proxy gold standard genesets (“Functional category” of MSigDB published in 2005) used by the GSEA authors.

^*:unadjusted p-values were used to select the top genesets overlapping with GSEA (thus a lower FDR).

2. METHODS

Figure 1 summarizes the overall design for developing personalized ontology-transformed patient expression profiles using FAIME [14], in which each step was described as follows.

Figure 1.

Overall methods for the discovery of Gene Ontology signatures based on ontology transform of microarray profiles. Overall methods for the discovery of Gene Ontology signatures based on ontology transform of microarray profiles. An analogy is the Fourier transform that effectively translates complex frequencies of electrical signal into simple linear equations. Similarly the FAIME pathway transform can translate gene patterns expression on a single sample into molecular mechanisms corresponding to a score calculates from genesets assigned to this mechanism (Steps 1–2). In other words, a biological mechanism or pathway profile of a single sample can be aligned with that of other patient samples and create the pathway expression map shown in Step 2. Further, for each pathway, this score is available across patients and can be utilized to calculate a regression against a continuous phenotype of each samples such as the survival (months) as shown in step 5.

Data (Figure 1, Step 1). For FAIME profiling of patient samples, GO annotations of human genes were downloaded from the NCBI (ftp://ftp.ncbi.nlm.nih.gov/gene/DATA/gene2go.gz) on December 11, 2009. We employed the 1,730 Biological Process (BP) GO terms each annotated with 5 to 200 genes, covering approximately 10k genes. We selected this range of annotation of GO BP for the following reasons: (i) a single gene expression artifact could potetnially bias a FAIME score of small genesets – 5 and more were selected, (ii) we have thoroughly analyze the effect of robustness of signals in GO and KEGG genesets in a previous study and found that large ones >350 did not carry any significant patterns (figure 4 of [14]).

To perform an equivalent comparison to the results from the GSEA method, a collection of 522 mechanisms genesets (the “Functional” category in the Molecular Signatures Database (MSigDB), a collection of canonical pathways and literature genesets, downloaded from the BROAD Institute [11]) was used for validating our FAIME method and for three lung cancer study (presented in Table 1). The FAIME method was applied to all datasets to compare to the GSEA method (results shown in Table 1).

For learning and validating GO-signatures, we utilized three gene expression datasets from lung adenocarcinoma studies (clinical and pathological features of patients and signature statistics summarized in Table 2). Additional patient survival tables for the three lung adenocarcinoma studies were downloaded from: (i) the NIH caArray (https://caarraydb.nci.nih.gov/caarray/) [1], (ii) BROAD Institute (http://broad.harvard.edu/publications/broad902) [20], and (iii) (http://www.pnas.org/content/98/24/13784/suppl/DC1) [21]; the lung adenocarcinoma subset of this “Stanford” lung cancer dataset was used in the evaluation of the GO-signatures.

Table 2.

Clinical and pathological features of lung tumor samples

The three datasets were downloaded from http://www.broadinstitute.org/gsea/datasets.jsp

	Michigan [1]	Boston [20]	Stanford [21]
Microarray	oligo	oligo	cDNA
Sample size	86	62	24
Age (mean, s.d.)	64 (10)	63 (10.5)	Not available
Gender (% male)	41	40	Not available
Stage I	67	50	7
Stage II	0	12	14
Stage III	19	0	3
Follow-up months (median)	30	50	17.5
Number of death	24	31	14

2.1. FAIME generates the biological mechanism-anchored GO profiles of an individual patient sample (Figure 1, Step 2–3)

To identify GO terms significantly associated with disease, a novel profile score measuring the gene expression effect of a GO term in a single biological sample has been described by our group [14]. We also previously showed that the predictive performance of GO profiles in head and neck cancer patient samples are comparable to that of gene-profiles[14]. The R function OrderedList module offers an insight into the mathematical-comparison of two biological-comparisons of clinical states [22, 23]. The algorithm is in principle a weighted sum of the number of overlapping genes in the top ranks, with more weight put on the top ranks. Although each single study of the biological-comparison might not necessarily reveal significant changes, we observe considerable overlap in the top ranking genes, using weights to decay exponentially with rank [24]. Extending this idea to single array measures rather than comparisons of lists, the FAIME score [14] (Equation 1) calculates the distance between the centroid of weighted expression [22, 23] of genes (CWEG) annotated by a GO term (GO_i,). The CWEG of all the remaining genes in GO that were not annotated to this GO term (also referred to as the complement set of genes, noted as A_i). In order to calculate the FAIME score, the gene expression levels of each tumor sample (j) were normalized (into s_x,j) by their ranks (r_x,j, integer from 1 to the number of expressed genes) of original expression values. In Equations 1–3, i is the identifier of the GO terms, and j is the identifier of the patient samples. CGO_i,j is the centroid of weighted expression of genes in GO_i; C_Ai,j is the centroid of weighted expression of genes in A.; s_x,j is the normalized value assigned to the expression rank (r_x,j) of a gene x in sample j. G is the total number of distinct GO terms. The personalized profile of every FAIME score of a patient sample is noted Pj={OT_1,j, OT_2,j,…, OT_G,j}. Moreover, the centroids were further normalized for the cardinality (count of genes) of their respective GO terms (e.g. cardinality of GO_i is |GO_i|).

$${\mathit{OT}}_{i,j}=\mathit{OT}({\mathit{GO}}_{i,j})=C_{{\mathit{GO}}_{i,j}}-C_{A_{i,j}}$$ (Equation 1)

$$C_{{\mathit{GO}}_{i,j}}=\frac{1}{\left|{\mathit{GO}}_i\right|}\sum _{x\in {\mathit{GO}}_i}(s_{x,j})\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\mathit{and}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{C}_{A_{i,j}}=\frac{1}{\left|A_i\right|}\sum _{x\in A_i}(s_{x,j})$$ (Equation 2)

$$S_{x,j}=(r_{x,j})\cdot (e^{\frac{r_{x,j}}{G}})$$ (Equation 3)

2.2. Learning robust mechanism-anchored signature classifiers from differential expressed FAIME scores

To identify the robust FAIME scores derived from GO Biological Processes (GO-signature) that are significantly associated to clinical/pathological parameters of interest, we utilized supervised machine learning methods (classification) on Michigan and Boston datasets, in which patients’ mortality status were used as class labels. (good vs bad as reported by the authors). Generating significant differentially expressed mechanisms from FAIME scores is addressed in detail in the following paragraphs and these become features for the classifier – here we focus on the machine learning approach to identify robust features within a dataset. These mortality statuses were provided to the R version of the GSEA program (v1.0). Training was conducted on the Michigan and Boston datasets separately using 100 repeated cross-validations (Figure 3; Suppl. Tables S2 & S3), and the selected features from the two datasets were tested in an independent testing (Stanford dataset; Figure 2). Specifically, to avoid over-training in conducting each of the 100 trials, a conservative two-step validation procedure was followed to select candidate GO-signatures (Figure 1, Step 3):

1. The samples from the Michigan or Boston datasets were separately randomly split into training and testing sets with a respective proportion of 2:1 for three-fold cross-validation and feature selection;

2. An optimal training model was selected in each training set via the PAM machine learning algorithm using the optimal threshold, the nearest shrunken centroid classifier, and default parameters [25];

3. The independent validation Stanford dataset was used to evaluate more conservatively the features of the fully specified 24-GO term and 25-GO term classifiers learned from the Boston and Michigan datasets, respectively (Suppl. Tables S2–S3). These operations resulted in identifying a subset of GO terms (candidates) stratified according to optimal thresholds. Additionally, a probability for features being selected is assigned to each candidate GO term. Robust signatures were set to comprise the top 1% to 2% of the 1,730 GO terms that were most frequently selected as candidates in the 100 trials and were above 50% probability of being selected in all trials. The probability distribution of a GO term to be part of the GO-signature on the frequency plot was used to identify the optimal number of distinct GO terms per GO-signature.

Figure 3.

Information theoretic similarity and ontological hierarchy concordance of the GO biological processes predictive of lung adenocarcinoma prognosis as discovered by FAIME in three independent analyses of distinct datasets. In the network plot, the robust GO biological processes are plotted as grey nodes, while their ancestors are transparent circles. The datasets from where these GO terms were identified are shaped differently (Michigan, square; Boston, triangle). Additionally, the GO-signatures that are similar to each other based on a threshold of ITSS ≥ 0.5 are linked. A solid line means a cross-study similarity, and a dash line refers to a within-study similarity. The GO terms related by similarity between the Michigan and the Boston GO-signatures are more numerous than expected by chance (Empirical permutation resampling of sample’s group assignment p=0.001).

Figure 2.

Robust GO-signatures predict survival of patients using Kaplan-Meier plot for two sub-classes of samples per dataset. The x-axis is the survival time (months), and the y-axis is the cumulative hazard. The left plot shows that the sample classification using the 24 GO terms of the GO-signature derived from and trained over the Boston dataset predicts patient survival in the Stanford dataset (log-rank p=0.001). Similar results are observed for the GO-signature (25 GO terms, right plot) derived from the Michigan dataset when predicting patient survival in the Stanford dataset (Log-rank p=0.019). Detailed mechanisms of these robust signatures are provided in Suppl. Tables S2–S3.

2.3. Preliminary evaluation of MSigDB-FAIME scores as compared to GSEA (Table 1)

Prediction of MSigDB mechanism-signatures using GSEA and FAIME from the same gene expression gold standard (522 genesets of the “Functional category” of MSigDB) in five datasets (Table 1). As FAIME scores can be derived from any form of organized genesets (e.g. GO terms, KEGG pathways, MSigDB), the accuracy of FAIME scores for predicting mechanism-signatures between two groups/subclasses of samples was evaluated using the same class labels of lung cancer patients’ mortality and geneset annotations that have been previously analyzed by GSEA (good vs bad prognosis as annotated by the authors). Datasets from published GSEA results were analyzed in addition to the lung adenocarcinoma datasets in this study. A collection of 522 common genesets from the MSigDB (“Functional category”) was considered as the proxy gold standard for comparing GSEA vs. FAIME methods. The FAIME scores of each MSigDB geneset of the gold standard were calculated via Equation 1–3 (using MSigDB genesets instead of GO terms) to identify significant differentially expressed MSigDB mechanism-signatures using the R function twilight.pval (default parameters, t statistic, reporting empirical FDR) for each datasets. Significant MSigDB mechanism associated to the patient mortality in the three lung adenocarcinoma datasets computed by the FAIME scores were compared the associated MSigDB mechanism calculated by the GSEA algorithm. Multiple thresholds of adjusted and unadjusted p-values were scanned for significant enrichment between the mechanism-signatures discovered by FAIME scores and those identified by GSEA. An initial FDR of 5% or 10% was tested. Enrichment p-values of overlap between MSigDB mechanism-signatures were calculated by the Fischer Exact Test (FET). The Boston and Michigan dataset are independent in term of patients and published by different authors. Thus their gene expression classifier signatures were independently developed. We applied FAIME independently on each sample and prioritized the FAIME-prioritized pathways and biological mechanisms on each dataset independently.

2.4. GO-FAIME analysis of Lung adenocarcinoma studies (Figure 1, Steps 4–5; Tables 2–3)

Table 3.

Comparison of computed prognostic GO-signatures to traditional prognostic gene profiles

	Michigan [1]	Boston [20]
GO-profiling based results:
# robust GO terms	25	24
P-value of stratified samples in the same dataset §	0.008	0.0001
P-value of stratified samples in the independent dataset	0.019	0.001
Traditional Gene-profiling based results in the original publication:
# genes reported by the authors	50	70
P-value reported by the authors	0.024	0.005*

^*:Bhattacharjee et al. reported one of six subclasses of adenocarcinomas, the C2 sub-group, derived from gene-expression profiling of 70 transcripts, which were associated with poor outcome and resulted in this P-value, using 126 samples that include the samples we used (provided by BROAD Institute for GSEA analysis) [20].

^§:Cox regression P-value reported for internal validation

Predicting survival outcome in lung adenocarcinoma datasets. A more convincing validation of the prognostic power of the GO-signatures is to predict the outcome of new patients. For conducting this evaluation, “robust” GO-signatures were independently generated via FAIME scores for each of two distinct datasets of lung adenocarcinoma through cross-validation by leaving out samples (Table 2, Michigan and Boston datasets). These two GO-signatures were irrespectively applied to predict survival outcome of the lung adenocarcinoma subset of the Stanford lung cancer datasets (Methods; Table 2, Figure 2). Specifically, an unsupervised classifier, the k-means method (k=2, R Bioconductor kmean function, defaults parameters), was carried out on the FAIME-derived ontology-anchored data consisting of only the GO-signatures prioritized independently in both the Boston and Michigan datasets (Robust GO-signatures) for the Stanford dataset. The prediction of survival and the resultant two classes were analyzed for prognosis by Kaplan-Meier survival probabilities, using the Bioconductor package survival (Table 3).

Mechanism overlap and biological relevance of two validated prognostic GO-signatures of lung adenocarcinoma. To determine whether the two Boston and Michigan prognostic GO-signatures share mechanism genomic spaces, the enrichment of identical or similar GO terms between the two robust GO-signatures was performed using the FET and Information Theory-based Semantic Similarity (ITSS) [26, 27], respectively. The latter determines GO terms that can be repeatedly found either by exact identification or by similarity in the structure of GO, or within the network structure of GO terms. A system view of the relationships between the mechanisms of two GO-signatures from Michigan and Boston datasets and their structural relationship was generated using information theoretic similarity algorithms that we previously published [27] and the hierarchical topology information of the GO parent-child directed acyclic graph (Figure 3). To this end, the pairwise semantic similarity among these GO-signatures were calculated [26], as well as an enrichment study of observed pairs of GO terms related by similarity as compared to putative pairs that we have previously derived for studying molecular interaction networks [26, 27]. Then, the nearest common ancestor GO terms were first found for similar GO terms, and thereafter for individual GO terms that bear no similarities to other terms. The former have a pairwise ITSS larger than or equal to 0.5. Subsequently, a network visualization was generated by Cytoscape software [28].

3. RESULTS

In the current study, ontology transformation of gene expression was employed to analyze three clinically characterized public microarray datasets. Specifically, FAIME scoring of traditional gene expression datasets was applied to lung adenocarcinoma samples of two studies (Table 2, Michigan, Boston). Prognostic GO-signatures derived from FAIME analyses were validated using the lung adenocarcinoma subset of an independent lung cancer dataset that was not used in the training of these signatures (Table 2, Stanford).

First, we performed a preliminary validation of the accuracy of FAIME scores on MSigDB mechanisms genesets collected by the BROAD Institute and analyzed by the GSEA method (Methods 2.2). Comparisons were made with the GSEA method for identification of MSigDB mechanisms significantly associated with clinical outcome in the lung adenocarcinoma datasets. Significant overlap was obtained between the genes identified by FAIME scores and by the GSEA, resulted a p-value from 0.00002 to 2e–8 (Table 1), indicating the capability of the FAIME scores in recapitulating results generated by traditional geneset-based methods. Furthermore, FAIME is shown more sensitive as it identifies results at lower FDRs than GSEA.

3.1. Generation of MSigDB- and GO- mechanisms signatures predictive of lung adenocarcinoma survival

The primary goals of utilizing GO FAIME scoring on multiple independent lung adenocarcinoma datasets are: 1) to demonstrate that FAIME-derived mechanism signatures can outperform published methods in predicting lung adenocarcinoma prognosis (Table 1), 2) to validate these predictions in an independent dataset (Figure 2), and 3) to demonstrate a higher concordance of FAIME-derived mechanisms-level classifiers/signatures between distinct studies than those conducted at the gene expression-level (Figure 3). A total of 1,730 GO terms with a gene-size range from 5 to 200 were unbiasedly examined in this study.

Samples. Three independent, clinically characterized microarray datasets of lung adenocarcinomas were downloaded from the MIT BROAD institute [11]. Conventional gene signatures of the three studies all classified the patients into clinical subtypes that exhibited statistically separable clinical outcomes of disease survival. Clinicopathological parameters for the three lung cancer datasets are shown in Table 2. Interestingly, Subramanian et al. found no significant association between clinical outcome and differentially expressed genes at significance levels of 5% after correcting for multiple testing [11]. Though they found evidence for shared common biological pathways, the GSEA method could only identify significant clinical outcome associated MSigDB mechanisms in only two of the three datasets at a high false discovery rate of 25% (Table 1). GSEA was not used by Subramanina et al. for survival analysis.

3.2. Independent validation of the GO-signatures prognostication and qualitative review (Fig.2, Suppl. Tables S2–S3)

The number of robust GO biological processes is provided in Table 3 for Michigan and Boston datasets. The prognostic power of the Michigan GO-signature was comparable to that of the original prognostic gene signature which stratified Stage I tumors into high-risk and low-risk groups (data not shown). Both Boston and Michigan GO-signatures accurately classified all high-risk Stage I tumors identified by gene signatures in the Boston and Michigan datasets. Similarly, as shown in Figure 2 (detailed mechanisms in Suppl. Tables S2–S3), the 24-term GO-signature derived from the Boston dataset also effectively stratified independent adenocarcinomas into two prognostic groups that differ significantly in survival. Further, the Michigan GO-signature was also capable of providing prognostic stratification on survival of tumors of all subclasses in the Stanford dataset. In contrast, the traditional published gene signature of the Boston dataset was only capable of associating one subclass of its own dataset (C2/neuroendocrine) with poor clinical outcome. It is noteworthy that all poor-outcome C2 tumors used in the gene signature analysis by Bhattacharjee et al. were recovered in the GO-signature in the predicted poor survival outcome group (not shown). Further, in comparison with prognostic stratification via the gene signature of the Boston study, the GO-signature provided a clean classification of two prognostic groups among adenocarcinoma patients, while the clinical outcome association among the three sub-groups classified by the traditional gene signature was weak and could not be clearly related to any clinical/pathological parameters. Collectively, these observations demonstrate that GO-signatures derived via our approach are robust and possess validated prognostic power comparable or superior to the original published three distinct gene signatures. Further, the GO-signature we derived can provide personalized predictions, a feature that is unattainable by gene signatures used in our comparison.

3.3. Prognostic GO-signatures derived from distinct gene profiling expression datasets exhibit significant mechanism overlaps (Figure 3, Table 4)

Table 4.

Selected GO-signatures and their annotated genes from the “positive regulation of cytokine production” network module.

GO ID	GO term (Biological Process)	Prognosis	Annotated genes
GO:0032755	positive regulation of interleukin-6 production	poor	ADORA2B, AKIRIN2, CARD9, HSPD1, IL1B, IL6, IL6R, LBP, NOD1, P2RX7, RIPK2, TICAM1, TLR2, TLR3, TLR4, TLR7, TLR9, TNF,UCN.
GO:0032722	positive regulation of chemokine production	poor	ADAM17, ADORA2B, HIF1A, IL6, IL6R, RIPK2, TLR2, TLR3, TLR4, TLR7, TLR9, TNF.
GO:0032757	positive regulation of interleukin-8 production	poor	AFAP1L2, CALCA, LBP, TLR2, TLR3, TLR4, TLR5, TLR7, TLR9.
GO:0045409	negative regul. of interleukin-6 biosynthetic process	good	FOXJ1, GHRL, GHSR, INPP5D, NLRP12.

We subsequently analyzed the significant overlap of GO terms between the two prognostic GO-signatures we derived. Indeed, we observed a significant enrichment of exact overlap between the GO terms derived from the GO-signature associated to the Boston data and for those derived from the Michigan data (FET p=0.046). This significant enrichment was also supported by a measurement of structural and mechanism similarity (Empirical permutation resampling of sample’s annotation p=0.001) using the semantic similarity of Lin [26]. In summary, we observe 32,420 pairs of similar GO terms with a Lin score larger than or equal to 0.5 among all 1,730×1,729 possible pairs (∼3 million), 14 similarity pairs (solid lines, Fig. 3) were shared among the 25×24 (300) potential pairs of GO terms between the Michigan and Boston GO-signatures. This is in drastic contrast with the reported lack of significant overlap at the individual gene level among genes in the gene signatures.

To understand the hierarchical relationship between the prognostic GO-terms identified from independent data sources, a network was generated based on the GO ontology structure between the Michigan and Boston GO-signatures. Albeit many GO-terms are only associated to their original dataset, we observed significant direct or indirect links connecting the GO-terms in the two prognostic GO-signatures (Figure 3, Suppl. Tables S2–S3). Two overlapping GO terms were found in both Michigan and Boston GO-signature – “neutrophil activation” (GO:0042119) and “negative regulation of growth” (GO:0045926). Increased numbers of tumor-infiltrating neutrophils are reported to confer poorer clinical outcome in patients with adenocarcinoma of the bronchioloalveolar carcinoma subtype of lung cancer [29]. The microarray profiling in the Michigan and Boston studies were conducted over total RNAs extracted from whole tumors. Therefore gene expression changes in the lung cancer cells and in normal cells present in tumor stroma would both contribute to the observed genome wide gene expression profile from which the gene signatures and prognostic GO-signatures were developed. It appears that our GO-signatures contain significant input from the tumor stroma, as “neutrophil activation” (GO:0042119) was present in both GO-signatures. Further, cytokines produced by lung cancer cells may have various effects on immune responses and tumor growth [30]. A shared GO sub-network module of regulation of cytokine production was found in both GO-signatures, and we present the details of annotated genes and association of GO terms in the module to poor survival outcome of lung adenocarcinoma patients in Table 4. A parental node, “positive regulation of cytokine production” (GO:0001819), was found in the Michigan [1] GO-signature. The four descendants of this node were clustered together and provided a mechanistic link between the two GO-signatures. In particular, we identify the “positive regulation of interleukin-8 production” and note that increased IL-8 production is associated with poor prognosis of lung cancer [31]. Similarly, we identify “positive regulation of interleukin-6 production” and note that elevated expression of IL-6 is detected in multiple epithelial tumors [32] including lung cancer [33]. Examination of the differentially expressed genes voted for the GO enrichment of altered cytokine functions (Table 4) show that cytokine and chemokine-mediated innate immunity is significantly altered. Most notably, various toll-like receptors (TLR) highly expressed by leukocytes, cytokines produced by macrophages (IL1B, IL6) and cytokine receptors regulating innate immunity (IL6R, P2RX7, NOD1) were altered (Table 4). However, cross examination of the top 100 gene list of the Michigan gene signature failed to recover these genes representing potentially important stromal gene changes. The majority of the similar GO terms, as well as those specific to one signature, can be expected in cellular pathophysiology, and some stand out as strongly associated with lung adenocarcinoma or stromal response to cancer. For example, the neurological markers have been well described in the previously discussed neuroendocrine subtype of lung adenocarcinoma strongly associated to poor prognosis [34] and in Fig. 3 (middle network).

These findings suggest that our approach may be more sensitive in identifying tumor stromal changes that are usually masked by more profound gene expression changes in cancer cells in gene signature analysis. Biological validation is required to confirm the validity of the GO-signatures in assessing tumor stromal changes as a prognostic role of the cytokine production in lung adenocarcinoma.

Collectively, these observations provide mechanistic explanations of the occupation of identical prognostic space by non-overlapping distinct gene signatures. They also suggest that mechanism-anchored signatures, such as the two personalized GO-signatures identified by FAIME scores, share both the mechanism genomic space as well as prognostic space. Thus, they are poised for future development of personalized diagnosis and prognosis.

4. DISCUSSION

Comparison to alternative methods

Compared with the post-signature enrichment analysis which tests the over-representation of genesets (for example GOstats[35], DAVID [36], GENMAPP [37]), GSEA [11], SigPathway [13] and FAIME [14] are threshold-free methods. Such tolerance toward within-set changes is important to robustly depict cancer cells. On the other hand, unlike GSEA methods, both SigPathway and FAIME assign a statistical score to every GO term for each sample on the microarray. In other words, it generates a mechanism’s geneset “expression” profile (e.g. GO process expression) that can be analyzed using any modern method developed for prognostication in high dimensional datasets. As the proposed approach is conducted ab initio on the expression data, by design, it is more applicable for prognostication with a continuous variable (e.g. cancer survival time) than GSEA approaches that require mutually exclusive grouping.

Future studies and limitations

Future studies will focus on the prospective clinical of the utility of mechanism-anchored signature. To that effect, we have extracted the RNA of 70 patients in several clinical trials and are awaiting the three year prognosis to test the classifiers. Further, gene expression arrays do not represent the complexity of measurements and biological deregulation and a single biological scale method such as FAIME lacks intrinsically access to splicing variants, RNA-seq, microRNA regulation, protein expression, loss of heterogeneity, polymorphisms, etc. We are developing of comprehensive multiscale/multianalyte mechanism scoring using both curated mechanisms (e.g. GO, KEGG) and unbiased derived ones (e.g. network modeling of conditional/co-expression and protein interactions). We also noted in Figure S1 our original FAIME paper that employing ontology transformation without cross-sample normalization, as would be needed for a true single patient classifier, did not result in a significant change of overlap of the mechanistic signature [14]. There are limitations to the current study. For example, deriving FAIME scores requires geneset annotations, while conventional gene expression signatures is simpler as it requires less transformation of the data. This is the limitation for all methods designed for a signature of the predefined genesets. An alternate approach is to utilize unbiased created genesets such as those arising from co-expression groups, however the mechanistic meaning of their co-regulation remain more difficult to interpret.

Conclusion

In summary, this study presents the first independent validation in lung adenocarcinoma patients of the machine learning of the recently reported GO profiling analysis method using FAIME scores, and the feasibility of developing individualized and mechanism-based prognostic signatures using ontology. Specifically, these observations have several implications: First, they provide a mechanistic explanation for the shared prognostic space of non-overlapping distinct gene signatures. Second, they suggest that mechanism-anchored signatures, such as the two personalized GO-signatures that we identified, deriving FAIME scores is dependent deriving FAIME scores is dependent are more computationally reproducible and biologically congruent than gene-level classifiers in the prognostic space. Third, GO-signatures are capable of analyzing “challenging” patient populations that are small in size and thus limiting to the generation of traditional classification schemes built on gene signatures. Fourth, such mechanism-anchored signatures that provide mechanistic interpretations of the biological changes are poised to be further developed for personalized cancer diagnosis and prognosis as they can be computed on a single sample without the requirement of a reference cohort, in other words a clinical laboratory test can be optimized over the transcriptome. Further, this approach is potentially paradigm shifting because biologists and physicians can recognize and interpret key features of the classifier while gene-based classifiers remain generally opaque as few or none of their constituent genes have been mechanistically confirmed as relevant to the clinical context.