Qualitative transcriptional signature for predicting pathological response of colorectal cancer to FOLFOX therapy

Abstract

FOLFOX (5‐fluorouracil, leucovorin and oxaliplatin) is one of the main chemotherapy regimens for colorectal cancer (CRC), but only half of CRC patients respond to this regimen. Using gene expression profiles of 96 metastatic CRC patients treated with FOLFOX, we first selected gene pairs whose within‐sample relative expression orderings (REO) were significantly associated with the response to FOLFOX using the exact binomial test. Then, from these gene pairs, we applied an optimization procedure to obtain a subset that achieved the largest F‐score in predicting pathological response of CRC to FOLFOX. The REO‐based qualitative transcriptional signature, consisting of five gene pairs, was developed in the training dataset consisting of 96 samples with an F‐score of 0.90. In an independent test dataset consisting of 25 samples with the response information, an F‐score of 0.82 was obtained. In three other independent survival datasets, the predicted responders showed significantly better progression‐free survival than the predicted non‐responders. In addition, the signature showed a better predictive performance than two published FOLFOX signatures across different datasets and is more suitable for CRC patients treated with FOLFOX than 5‐fluorouracil‐based signatures. In conclusion, the REO‐based qualitative transcriptional signature can accurately identify metastatic CRC patients who may benefit from the FOLFOX regimen.

FOLFOX is one of the main chemotherapy regimens for colorectal cancer (CRC), but only half of CRC patients respond to this regimen. The REO‐based qualitative transcriptional signature can accurately identify CRC patients who may benefit from the FOLFOX regimen.

Abbreviations

5‐FU

5‐fluorouracil

AUC

area under the receiver operating characteristic curve

CR

complete response

CRC

colorectal cancer

DEG

differentially expressed genes

FN

false negative

FOLFOX

5‐FU, leucovorin and oxaliplatin

FP

false positive

FPKM

number of Fragments Per Kilobase exons per Million mapped reads

GEO

Gene Expression Omnibus

GPS

gene pair signature

NSCLC

non‐small cell lung cancer

PD

progressive disease

PFS

progression‐free survival

PPI

protein‐protein interaction

PR

partial response

RD

rank difference

REO

relative expression orderings

RMA

Robust Multi‐array Average

SD

stable disease

TCGA

The Cancer Genome Atlas

TN

true negative

TP

true positive

1. INTRODUCTION

Colorectal cancer is one of the most common malignant tumors. Approximately 20% of CRC patients have synchronous distant metastases at the initial diagnosis,1 and 50% of patients initially without metastases develop distant metastases within 3 years of diagnosis.2 Although the use of targeted drugs, such as cetuximab, erlotinib and bevacizumab has been improving patient prognosis significantly, 5‐fluorouracil (5‐FU)‐based chemotherapy is still the main treatment for metastatic CRC, as a result of the cost and the limited applicable population of the targeted drugs. According to the NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines) for CRC, FOLFOX is one of the first‐line treatments for metastatic CRC.3, 4 However, only half of CRC patients respond to FOLFOX and the others undergo resistance to this chemotherapy together with detrimental, life‐threatening side‐effects.5, 6 Thus, stratification of patients for FOLFOX therapy response is indispensable for better therapeutic results.

To tackle this problem, transcriptional signatures have been developed to predict the response of metastatic CRC to FOLFOX chemotherapy.7, 8, 9 However, these transcriptional signatures are based on quantitative gene expression values and often subject to batch effects introduced by the differences in laboratory conditions and personnel.10, 11, 12 Therefore, data normalization is necessary, which requires that a single sample should be normalized together with a set of samples. This makes the signatures hardly applicable to common clinical settings. In contrast, qualitative REO of gene pairs within a sample have been reported as robust against experimental batch effects.13, 14 More importantly, our previous studies have shown that qualitative REO are less sensitive to partial degradation of RNA,15 amplification bias of low‐input RNA samples16 and variation of tumor cell percentage from different sampling locations17 than quantitative gene expression measurements. Because of these unique advantages, we have developed many qualitative REO‐based signatures for predicting the prognosis and drug response of breast cancer,18 colorectal cancer,19 gastric cancer,20 hepatocellular carcinoma21 and lung cancer.22 Also, our lab also had developed some qualitative signatures to predict response for 5‐FU‐based therapies in CRC, such as signatures for stage II‐III CRC patients19, 23 and locally advanced rectal cancers,24 but we found these signatures could not be carried out very well in metastatic CRC treated with FOLFOX. Thus, it is worthwhile to develop a qualitative signature for predicting the response of metastatic CRC treated with FOLFOX.

Currently, RECIST25 is widely used in evaluating the response of patients to anticancer treatments after chemotherapy.7, 8, 26, 27, 28, 29 According to RECIST, a patient’s response to a chemotherapy regimen is classified into four groups, including CR, PR, SD and PD, based on change in lesion size derived from imaging or clinical examination. In CRC studies, researchers usually classify FOLFOX‐treated metastatic CRC patients who achieved CR or PR as responders but SD or PD patients as non‐responders.7, 8, 27 However, studies for patients with other types of advanced cancers such as NSCLC showed the response classification of SD patients is controversial with regards to the prognosis.30, 31

In the present study, we first analyzed the relationship between survival benefits and RECIST response status in metastatic CRC patients treated with FOLFOX. Then, using the response information, we developed a robust qualitative transcriptional signature, based on the within‐sample REO of gene pairs, for predicting responders with improved PFS after FOLFOX treatment. Finally, we compared our signature with other published signatures.

2. MATERIALS AND METHODS

2.1. Data sources and preprocessing

A total of 243 samples from CRC patients undergoing FOLFOX therapy were used in this study. Clinical and pathological characteristics of these patients are summarized in Table 1 and the detailed characteristics of these patients can be seen in Table S1. Tumor response was evaluated according to RECIST recommendations for the evaluation of cancer treatment in solid tumors.32, 33 PFS was defined as the time from the beginning of first‐line treatment until disease progression or death. Alive patients without progression were censored at the date of the last follow up. Primary tumors originating in the splenic flexure, descending colon or sigmoid colon were classified as left‐sided colon, and in the appendix, cecum, ascending colon, hepatic flexure or transverse colon as right‐sided colon.34 Sixty‐three unresectable patients in http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE28702 had only response information available and were used in the training group (part of CRC96). The 55 patients who underwent surgery in http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE104645 had both response and survival information, of which 33 patients (including 1 CR, 27 PR and 5 PD) with response information were also used in the training group (part of CRC96) and all 55 patients with survival information were used in the validation group (CRC55). The 32 patients who underwent surgery in http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE72970 also had both response and survival information, 25 patients (including 20 PR and 5 PD) with response information were used as the validation dataset of CRC25 and all the 32 patients with survival information were used as the validation dataset (CRC32). All CRC samples from the three datasets (http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE28702, http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE104645 and http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE72970) were metastatic CRC (including synchronous and metachronous CRC and for the metachronous patients the distant metastases occurred beyond several months of the primary diagnosis of CRC). Samples who underwent surgery from http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE39582 (20 patients) and TCGA‐CRC (73 patients) had survival information only and part of these samples did not have clear information about metastasis, and these samples were used in the validation stage only (CRC93). More information on training and validation is shown in Table 2. By comparing the clinical and pathological characteristics between the responders and non‐responders, only primary lesion location was found to be significantly associated with response to chemotherapy in CRC25 (Table S2), which is consistent with the known fact that left colon cancer gains more benefit from FOLFOX chemotherapy than right colon cancer.35

Table 1.

Baseline clinical and pathological characteristics of patients in the present study

Characteristic	http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE28702	http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE104645	http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE72970	http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE39582	TCGA‐CRC
	n = 63	n = 55	n = 32a	n = 20	n = 73
Age (y)
Mean ± SD	62.86 ± 11.57	62.71 ± 10.05	63.63 ± 11.77	57.7 ± 9.56	60.01 ± 12.59
Median	65	63	66	56.5	63.21
Gender
Male	40	35	19	13	37
Female	23	20	13	7	36
Stage
III	0	17	2	9	60
IV	63	38	28	11	13
PFS (mo)
Mean ± SD	—	12.33 ± 8.35	17.42 ± 18.07	12.4 ± 7.53	22.50 ± 11.97
Median	—	10.07	13.14	13	20.27
PFS event
0 (censored)	—	4	5	11	56
1	—	51	27	9	17
Response
CR	2	1	1	—	—
PR	40	27	19	—	—
SD	b	17	7	—	—
PD	21	5	5	—	—
Unknown	0	5	0	—	—
Location
Right colon	—	11	7	16	22
Left colon	—	15	15	4	22
Rectum	—	29	10	0	22
Unknown	—	0	0	0	7

Abbreviations: CR, complete response; PD, progressive disease; PFS, progression‐free survival; PR, partial response; SD, stable disease; TCGA, The Cancer Genome Atlas.

—, no such data.

^aIncluding two stage‐I metachronous metastases colorectal cancer (CRC) samples.

^bSamples of 20 SD patients were not used in this study.

Table 2.

Training group and validation group data used in the present study

Group	Dataset	Accession	Responder		Non‐responder	Total	Platform
			CR	PR	PD
Training	With response data
	CRC96	http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE104645	1	27	5	33	GPL6480 (Agilent)
		http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE28702	2	40	21	63	GPL570 (Affymetrix)
Validation	With response data
	CRC25	http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE72970	0	20	5	25	GPL570 (Affymetrix)
	With survival data
	CRC55	http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE104645	/	/	/	55	GPL6480 (Agilent)
	CRC32	http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE72970	/	/	/	32	GPL570 (Affymetrix)
	CRC93a	http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE39582	/	/	/	20	GPL570 (Affymetrix)
		TCGA‐CRC	/	/	/	73	RNA‐seq (Illumina)

Abbreviations: CR, complete response; PD, progressive disease; PR, partial response.

^aProgression‐free survival (PFS) of 11 colorectal cancer (CRC) patients in http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE39582 and 56 in TCGA‐CRC were censored and they were combined as one validation dataset.

All gene expression profiles were downloaded from GEO (http://www.ncbi.nlm.nih.gov/geo/) or TCGA (https://portal.gdc.cancer.gov/). For TCGA data, FPKM data derived from Illumina’s RNA‐Sequencing technique were extracted as the gene expression measurements. For the data measured by the Affymetrix platform, raw mRNA expression data (.CEL files) was downloaded and the RMA algorithm was used for preprocessing.36 For the data measured by the Agilent platform, we directly downloaded the processed data. All gene expression measurements were log2 transformed. The probe sets were mapped to Entrez gene ID using the corresponding platform files. For each sample, the expression measurements of all probe sets corresponding to the same Gene ID were averaged to obtain the final value. Probe sets that did not match any Gene ID or matched multiple Gene IDs were discarded.

2.2. Differential expression analyses

The RankProd algorithm37 was used to identify DEG between two groups.

2.3. Calculating the rank difference of gene pairs

In a sample, all the genes are ranked according to their expression levels in an ascending order. Given a gene pair, gene i and gene j, in sample t, the RD is calculated as:

$${\text{RD}}_{\mathit{tij}}=R_{\mathit{ti}}-R_{\mathit{tj}},$$

where R_ti is the rank of gene i in sample t, R_tj is the rank of gene j in sample t.

The average RD of a gene pair in the samples of the two groups is calculated as the geometric mean of two arithmetic means of the RD in the responder group and in non‐responder group, respectively, as follows:

$${\text{RD}}_{\mathit{ij}}=\sqrt{\left[\frac{1}{n}\sum _{t=1}^n{\text{RD}}_{\mathit{tij}}^r\right]\times \left[\frac{1}{m}\sum _{t=1}^m{\text{RD}}_{\mathit{tji}}^{\mathit{nr}}\right]}.$$

If the two arithmetic means have opposite signs, the geometric mean will be a complex number and the pair is dropped. We only keep the gene pairs with a real geometric mean value.

2.4. Accuracy and F‐score

When developing the signature, accuracy and F‐score were used to evaluate the predictive performance of gene pairs and signatures. For each gene pair (gene i and gene j) in sample t, the REO pattern R_ti > R_tj voted for response and the other REO pattern R_ti < R_tj voted for non‐response. For a signature with a set of gene pairs, a sample was predicted as a responder if half or more than half of gene pairs in the signature voted for FOLFOX response, otherwise as a non‐responder. Samples labeled as responders (CR and PR) were defined as positive samples whereas negative samples included the samples labeled as non‐responders (PD). TP represent the samples labeled as responders that were correctly predicted as responders. Similarly, TN, FP and FN denote the number of true negatives, false positives and false negatives, respectively. Sensitivity, specificity, accuracy and F‐score are calculated as follows:

$$\text{Sensitivity}=\frac{\text{TP}}{\text{TP}+\text{FN}}$$

$$\text{Specificity}=\frac{\text{TN}}{\text{TN}+\text{FP}}$$

$$\text{Accuracy}=\frac{\text{TP}+\text{TN}}{\text{TP}+\text{FN}+\text{TN}+\text{FP}}$$

$$F-\text{score}=\frac{2\times \text{sensitivity}\times \text{specificity}}{\text{sensitivity}+\text{specificity}}$$

In order to determine if the accuracy for a gene pair was higher than a random guess (50%), the one‐sided exact binomial test was done at the 1% significance level.

2.5. Developing the predictive gene pair signature for FOLFOX therapy

First, DEG were identified between the responder group and the non‐responder group using the RankProd algorithm. Then the gene pairs consisting of at least one DEG were selected with real RD_tj as the rank‐selected gene pairs. For each rank‐selected gene pair, accuracy and F‐score were calculated. Some of these gene pairs with a significantly high accuracy to predict patients’ response were defined as response‐associated gene pairs using the exact binomial test. All the response‐associated gene pairs were used as candidates to develop a predictive GPS. A forward selection procedure was applied to the candidate gene pairs to obtain the largest F‐score for predicting patient response. In brief, the gene pairs were sorted in a descending order according to their F‐scores, and each of the gene pairs among the top 20 with the largest F‐scores were chosen as a seed. For each seed gene pair, another candidate gene pair was added to the signature set to increase the F‐score until the F‐score could not further increase. Among the results derived from the 20 seeds, the set of gene pairs with the largest F‐score were chosen as the predictive signature for FOLFOX.

2.6. Area under the curve and survival analyses

The AUC was used to evaluate the predictive performance of signatures. It was calculated in R using package “pROC.”38Multivariate Cox proportional‐hazards regression model was used to evaluate the independent association between the signature and the survival of patients after adjusting for clinical factors such as stage, age, gender and tumor location. Survival curves were estimated using the Kaplan‐Meier method.39

2.7. Human protein‐protein interaction data

Protein‐protein interaction data were downloaded from KEGG,40 HPRD,41 IntAct,42 MIPS,43 MINT,44 DIP,45 BIND46 and neighboring reactions.47 We compiled an integrated interaction network of 101 729 distinct interactions involving 12 372 human proteins.

In the network, 21 genes involved in 5‐FU and 20 genes involved in oxaliplatin transport, metabolism and other downstream effects (denoted as 5‐FU‐related genes and oxaliplatin‐related genes) were collected from DRUGBANK (https://www.drugbank.ca/).48

3. RESULTS

3.1. Controversial classification of stable disease patients

In the present study, we found that the responder group (CR + PR) and the non‐responder group (SD + PD) did not show significant difference in PFS in the 50 CRC patients with both response and survival information from the http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE104645 dataset (multivariate Cox, HR = 1.25, 95% CI = 0.67‐2.30, P = .48; Figure 1A) and 32 CRC patients with both response and survival information from the http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE72970 dataset (multivariate Cox, HR = 2.52, 95% CI = 0.97‐6.58, P = .06; Figure 1B). The lack of significant difference might be due to uncertainty in the classification of SD patients, as they showed similar survival benefits to PR patients (Figure 1C,D). Similar results have been reported on platinum‐based chemotherapy for NSCLC.30, 31 Because of the controversial relationship between the response classification and the prognosis for SD patients, we classified patients who achieved CR or PR as responders and PD as non‐responders, excluding the SD patients, in the signature development.

Figure 1.

Kaplan‐Meier curves of progression‐free survival (PFS) among complete response (CR), partial response (PR), stable disease (SD) and progressive disease (PD) of colorectal cancer patients with 5‐fluorouracil, leucovorin and oxaliplatin (FOLFOX) treatment. A and B, CR and PR vs SD and PD in http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE104645 and http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE72970; C and D, PR vs SD in http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE104645 and http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE72970

3.2. Development of the predictive signature for FOLFOX

The discovery workflow is shown in Figure 2. First, using CRC96 as the training data (including 33 patients from http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE104645 and 63 patients from http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE28702), based on the RankProd algorithm,37 we identified 1227 DEG between the 28 responders and five non‐responders in http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE104645 and 3151 DEG between the 42 responders and 21 non‐responders in http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE28702 with P < .05. The two lists of DEG were combined to obtain a total of 3997 DEG. Second, from all the gene pairs involving at least one of the DEG, 66 529 gene pairs with real RD_ij values between 70 responders and 26 non‐responders in the CRC96 dataset were selected as rank‐selected gene pairs. From these rank‐selected gene pairs, we extracted 8905 gene pairs whose REO patterns were significantly associated with the response outcomes (Exact binomial test, P < .01). From these gene pairs, we obtained a set of five gene pairs that reached the largest F‐score with the majority voting rule (see Section 2). These five gene pairs (Table 3) along with their REO patterns, denoted as 5‐GPS, were selected as the predictive signature.

Figure 2.

Flowchart for identification and validation of the relative expression orderings (REO)‐based signature in the present study. CRC, colorectal carcinoma; DEG, differentially expressed genes

Table 3.

Gene pairs in the five gene pair signatures (5‐GPS)

Gene pair	Gene 1	Gene 2
1	CALML5	IGFBP1
2	CCND2	GPR34
3	IRF6	WDR75
4	HOXB4	SMURF2
5	TRIM11	NT5DC3

Predictive model: a sample was determined to be a responder if there were at least three of five gene pairs with the specific relative expression orderings (REO) (gene 1 > gene 2). Otherwise, the sample was determined to be a non‐responder.

In training data CRC96, 62 of the 70 responders and 24 of the 26 non‐responders were correctly classified (sensitivity = 0.89 and specificity = 0.92) by the 5‐GPS signature. F‐score, accuracy and AUC were 0.90, 0.90 and 0.94, respectively (Figure 3A).

Figure 3.

Performance of our five gene pair signatures (5‐GPS) in identifying responder patients with 5‐fluorouracil, leucovorin and oxaliplatin (FOLFOX) treatment. A and B, Area under the receiver operating characteristic curve (AUC) in training datasets CRC96 and validation datasets CRC25. C‐H, Kaplan‐Meier curves of progression‐free survival (PFS) between different response groups predicted by 5‐GPS in datasets CRC55, CRC32, CCR93, all samples, stage III, and stage IV patients receiving FOLFOX treatment. Here, the data of all patients consists of patients from datasets CRC55, CRC32 and CRC93. CRC, colorectal carcinoma

3.3. Validation of the signature

The 5‐GPS was validated in one response dataset CRC25 and in three survival datasets CRC55, CRC32 and CRC93. First, in the validation data CRC25, 17 of the 20 responders (sensitivity = 0.85) and four of the five non‐responders were correctly classified (specificity = 0.80) by the signature, achieving an F‐score of 0.82. Accuracy and AUC were 0.84 and 0.82, respectively (Figure 3B). Further, three other datasets with PFS information were used to validate the association between the 5‐GPS signature and the PFS. In CRC55, Kaplan‐Meier curves and multivariate Cox analysis showed that the patients predicted as responders by 5‐GPS had significantly better PFS than the patients predicted as non‐responders after adjusting for stage, age, gender and tumor location (multivariate Cox, HR = 0.17, 95% CI = 0.06‐0.49, P = 9.09e‐04; Figure 3C). Similar results were also observed in CRC32 (multivariate Cox, HR = 0.29, 95% CI = 0.10‐0.84, P = .02; Figure 3D). As our signature is constructed in metastatic CRC samples, it might not be suitable for non‐metastatic CRC. In the combined data of http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE39582 and TCGA‐CRC (CRC93), part of samples did not have clear information about metastasis. However, stratification of the 93 CRC patients was still statistically significant (multivariate Cox, HR = 0.28, 95% CI = 0.10‐0.73, P = 9.81e‐03; Figure 3E). This indicates that our signature can predict the prognosis of non‐metastatic CRC, although our signature was not initially designed for these patients. Also, our signature indeed performs well in metastatic CRC even with some non‐metastatic CRC mixed in.

By pooling all the samples of the validation datasets of CRC55, CRC32 and CRC93 together, 5‐GPS could still predict two response groups with significantly different PFS (multivariate Cox, HR = 0.41, 95% CI = 0.25‐0.69, P = 9.18e‐04; Figure 3F). The 12‐month survival proportion was 71.8% for the predicted responder group and 40.7% for the predicted non‐responder group. Notably, we found that stage of patients was significantly correlated with PFS (multivariate Cox, HR = 3.25, 95% CI = 2.08‐5.06, P = 2.1e‐07; Table S3). Therefore, the performance of 5‐GPS was further tested separately in stage III and IV CRC patients. The results showed that 5‐GPS could discriminate PFS of responders and non‐responders in both stage III (multivariate Cox, HR = 0.29, 95% CI = 0.11‐0.77, P = .01; Figure 3G) and stage IV (multivariate Cox, HR = 0.29, 95% CI = 0.14‐0.61, P = 9.08e‐04; Figure 3H) CRC patients after adjusting for age, gender and tumor location. In particular, the 12‐month survival proportion of the predicted responder group (59.6%) was more than twice that of the predicted non‐responder group (28.6%) in stage IV CRC patients. Because 69 patients did not have metastatic information in the validation dataset of CRC93, we also tested the performance of 5‐GPS in the metastatic CRC samples from CRC55 and CRC32. It was found that 5‐GPS could still discriminate PFS of responders and non‐responders in the metachronous stage III patients (Figure S1A) and in the synchronous metastatic stage IV patients (Figure S1B), respectively. These results showed that 5‐GPS performed well in all the training and validation datasets.

3.4. Comparison of 5‐GPS with other published signatures

We also compared 5‐GPS with two other published FOLFOX signatures, a 27‐probe‐set signature7 and a 15‐probe‐set signature.8 Briefly, the prediction model of the two signatures were established using the k‐nearest‐neighbor method and random forests method, respectively, based on the expression measurement values of the gene probe sets. Notably, both of the two signatures consisted of the probe sets only from the Affymetrix platform, so we only used the datasets measured by this platform to compare the performance of the two signatures with our signature. Here, the processed profiles of 40 samples in http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE19860 and 54 samples in the training data of http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE28702, which were used as their training datasets in their original studies, were used as training datasets, respectively. Then, CRC25 and all the 52 CRC samples measured by Affymetrix and with PFS were used as independent validation data to compare these three signatures.

Results showed that all the three signatures performed well in their training datasets (Figure 4A‐C). In the validation dataset CRC25, 5‐GPS still performed well (AUC = 0.82; Figure 4D), but the 27‐probe‐set signature (AUC = 0.60; Figure 4E) and the 15‐probe‐set signature (AUC = 0.54; Figure 4F) showed poor classification ability. Similar results were also observed in the survival datasets. Patients predicted as responders by 5‐GPS had significantly better PFS than non‐responders (multivariate Cox, HR = 0.34, 95% CI = 0.16‐0.73, P = 6.27e‐03; Figure 4G). The 27‐probe‐set signature classified all the 52 patients as non‐responders, and the 15‐probe‐set signature showed a lower prognosis association than 5‐GPS (multivariate Cox, HR = 0.33, 95% CI = 0.13‐0.82, P = .02; Figure 4I). These results indicated that our 5‐GPS had a better predictive performance than the two published signatures across different datasets.

Figure 4.

Comparison of the performance of our five gene pair signatures (5‐GPS), 27‐probe‐set signature and 15‐probe‐set signature. A‐C, Area under the receiver operating characteristic curve (AUC) in corresponding training datasets for 5‐GPS, 27‐probe‐set signature and 15‐probe‐set signature. D‐F, AUC in datasets CRC25 for 5‐GPS, 27‐probe‐set signature and 15‐probe‐set signature. G‐I, Kaplan‐Meier curves of progression‐free survival (PFS) between different response groups predicted by 5‐GPS, 27‐probe‐set signature and 15‐probe‐set signature. (^a27‐probe‐set signature classified all the 52 patients as non‐responders, so there is only one line in this panel). CRC, colorectal carcinoma

As FOLFOX is one of the 5‐FU based chemotherapy regimens, we also tested the performance of four other 5‐FU‐based signatures, which were developed in our lab, in the FOLFOX datasets to develop the current signature. Song‐4GPS is a signature developed by Song et al23 for predicting the response to 5‐FU‐based adjuvant chemotherapy in stages II and III CRC with high relapse risk after curative surgery. Guo‐27GPS is a signature established by Guo et al24 for predicting pathological response to 5‐FU‐based neoadjuvant chemoradiation in locally advanced rectal cancers. Song‐3GPS and Song‐5GPS were also developed by Song et al19 for predicting the response to 5‐FU‐based adjuvant chemotherapy in stage II‐III right‐sided and left‐sided colon cancer, respectively. We found that all four signatures showed a poor classification performance in predicting the response to FOLFOX in CRC patients (Figures [Link], [Link], [Link], [Link]). These results suggested that although 5‐FU is one of the main components of FOLFOX, these 5‐FU‐based signatures are not suitable for predicting the response of CRC patients to FOLFOX. Each signature has its own applicable situation. The relationship between 5‐GPS and these four 5‐FU‐based signatures is further discussed in the following section.

3.5. Protein‐protein interaction network analysis for the genes in 5‐GPS

A regulatory network of protein‐protein interaction was constructed by linking the genes in 5‐GPS with genes related with 5‐FU or oxaliplatin transport, metabolism and other downstream effects (Figure 5). The PPI showed that four of the 10 genes in 5‐GPS could interact with 5‐FU‐ or oxaliplatin‐related genes though some commonly used genes. For example, IGFBP1, a gene in the 5‐GPS, can interact with many 5‐FU‐ or oxaliplatin‐related genes through gene HNF4A which can promote cell apoptosis.49 Downregulation of HNF4A is associated with tumor metastasis and poor prognosis in CRC.50 IGFBP1 is a tumor suppressor gene which can induce growth arrest in CRC cells.51 Similarly, CCND2, IRF6 and SMURF2 could also interact with 5‐FU‐ or oxaliplatin‐related genes indirectly (Figure 5). Studies showed that increased expression of CCND2 promoted cancer progression and metastasis in CRC,52, 53, 54 whereas 5‐FU treatment resulted in a decreased expression of CCND2.55, 56 SMURF2 is an interesting case, as the only gene included in 5‐GPS and four other signatures, including 27‐probe‐set signature, 15‐probe‐set signature, Song‐4GPS and Guo‐27GPS. Its expression is higher in non‐responders than in responders, in agreement with the observation that SMURF2 supports cancer cell migration and invasion in CRC,57 whereas oxaliplatin in combination with topotecan could decrease the expression of SMURF2.58 These results showed that the genes in 5‐GPS might play important roles in the response to FOLFOX chemotherapy.

Figure 5.

Protein‐protein interaction links between the genes in five gene pair signatures (5‐GPS) and drug‐related genes. 5‐FU, 5‐fluorouracil

4. DISCUSSION

In the present study, REO of five gene pairs (5‐GPS) were developed as a signature to accurately predict a CRC patient’s response to FOLFOX therapy. The FOLFOX responder groups predicted by 5‐GPS had significantly better PFS than the predicted non‐responder groups. The performance of 5‐GPS was compared with that of two published FOLFOX response signatures and four signatures for 5‐FU‐based chemotherapy regimens. Our signature was found to have better performance in different datasets than the two published FOLFOX response signatures and was more suitable for CRC patients treated with FOLFOX than the four signatures for 5‐FU‐based chemotherapy regimens.

In traditional CRC studies, CRC patients who achieved CR or PR were classified as responders, whereas SD or PD were classified as non‐responders. However, according to this classification scheme, the response information could not be well linked with the prognosis of CRC patients. In fact, it has been shown that RECIST often underestimates the response.59 Therefore, we trained the 5‐GPS using the datasets without SD patients. In the present study, we used F‐score and AUC which calculated both sensitivity and specificity to evaluate the performance of signatures, in consideration of unbalanced distribution in training and validation data caused by dropping the samples of SD patients. In the validation stage of this study, both the response status and survival information were used. The survival information (PFS) was used as an independent piece of evidence, which is a more reliable measure for the performance of a drug signature. PFS is associated with tumor burden and is a more accurate indication of the survival benefit of the first‐line treatments than the overall survival, which is often affected significantly by other factors, such as the second‐line or subsequent therapies.30

Our lab has previously developed several 5‐FU‐based signatures for predicting the response to 5‐FU‐based neoadjuvant chemoradiation in locally advanced rectal cancers (23), for predicting the response to 5‐FU‐based adjuvant chemotherapy in stage II and III CRC with high relapse risk after curative surgery (24) and in stage II‐III right‐sided and left‐sided colon cancer, respectively.19 However, these signatures might be unsuitable for predicting the response to FOLFOX in metastatic CRC. First, different signatures were focused on CRC patients in different stages which have different pathological characteristics and receive different therapeutic regimens. For example, stage II and III CRC patients usually receive curative surgery before 5‐FU‐based chemotherapy. Curative surgery would introduce bias to evaluate the response to the following 5‐FU‐based chemotherapy. Similarity, for locally advanced rectal cancers, radiotherapy always accompanies 5‐FU‐based chemotherapy. Second, our FOLFOX signature can theoretically perform better than 5‐FU‐based signature for FOLFOX‐treated CRC patients. Some patients might be misclassified due to the combinational nature of the regimens. For example, a patient who was sensitive to oxaliplatin but resistant to 5‐FU would be classified as a non‐responder by a 5‐FU‐based signature in theory, whereas the patient should indeed be a non‐responder to 5‐FU, FUFOL or FOLFIRI, but a responder to FOLFOX. Different from these studies, we focused on the FOLFOX regimen which is one of the first‐line therapies in metastatic CRC. We aimed to determine whether or not CRC patients respond to FOLFOX treatment. In principle, non‐responders should be resistant to both 5‐FU and oxaliplatin, whereas the responders comprise three subtypes according to the assumption of Tong et al:60 sensitive to 5‐FU only, sensitive to oxaliplatin only, and sensitive to both 5‐FU and oxaliplatin Therefore, our 5‐GPS signature should be more suitable for predicting pathological response of CRC patients treated with FOLFOX than these 5‐FU‐based signatures.

Previously, study has shown that the benefit of chemotherapy for colon cancer patients is influenced by tumor location.35 Therefore, we compared the performance of our signature in right colon, left colon and rectum cancer samples in the pooled datasets of metastatic CRC samples from CRC55 and CRC32, respectively. It was found that 5‐GPS could discriminate PFS of responders and non‐responders in the right colon (multivariate Cox, HR = 0.06, 95% CI = 0.01‐0.31, P = 8.51e‐04; Figure S6A) and left colon cancers (multivariate Cox, HR = 0.15, 95% CI = 0.02‐0.96, P = 4.56e‐02; Figure S6B), respectively. For rectum cancer, our signature could still perform well after one CRC patient, who showed partial response status but was misclassified as a non‐responder, was removed (multivariate Cox, HR = 0.11, 95% CI = 0.02‐0.52, P = 5.39e‐03; Figure S6C). It seemed that our signature works well in right colon cancer and left colon cancer, although a larger sample size is needed to further corroborate this conclusion in future, in particular for the performance of the signature in rectum cancers. Additionally, results of multivariate Cox proportional models in the validation of our signature show that the signature is an independent prognosis factor for FOLFOX‐treated metastatic CRC patients after adjusting for tumor location (Table S3). Moreover, although our signature was constructed in the training data including both unresectable and radically resected CRC patients, survival analysis showed that it performs well in the validation dataset including the resected CRC patients only. This might be due to the fact that only the response information was used in the training stage.

In conclusion, REO‐based 5‐GPS could identify those patients who may benefit from FOLFOX treatment. Furthermore, as the within‐sample REO are robust against experimental batch effects,13, 14 different measurement principles across platforms,61 sampling locations17 and RNA partial degradation,15 REO‐based 5‐GPS can be conveniently applied to clinical settings.

DISCLOSURE

Authors declare no conflicts of interest for this article.

Supporting information

 

 

 

 

 

 

 

 

 

He J, Cheng J, Guan Q, et al. Qualitative transcriptional signature for predicting pathological response of colorectal cancer to FOLFOX therapy. Cancer Sci. 2020;111:253–265. 10.1111/cas.14263