Multi-transcriptomics predicts clinical outcome in systemically untreated breast cancer patients with extensive follow-up

Abstract

Background

Prognostic tools for determining patients with indolent breast cancers (BCs) are far from optimal, leading to extensive overtreatment. Several studies have demonstrated mRNAs, lncRNAs and miRNAs to have prognostic potential in BC. Because mRNAs, lncRNAs, and miRNAs capture distinct transcriptomic information, we hypothesized that combining them would improve classification performance.

Methods

Our pair-matched design study included fresh frozen primary tumor samples from 160 lymph node negative and systemically untreated BC patients of which 80 developed recurrence while 80 remained recurrence-free (mean follow-up of 20.9 years). We integrated three classes of RNA and subsequently performed classification using seven machine learning methods followed by a voting scheme.

Results

Under the criteria of ≥ 90% sensitivity, individual classifications resulted in specificities ranging from 74–91% for the integrated dataset and 56–66%, 58–71% and 69–86% for mRNAs, lncRNAs and miRNAs individually. The specificity level for the multi-transcriptomic dataset was 85% after voting while it was 38%, 48% and 82% for mRNAs, lncRNAs and miRNAs, respectively. In the clinical setting, very high sensitivity may be requested. In the most stringent clinical setting with a sensitivity of 99%, the integrated dataset also outperformed the others with a specificity of 41% compared to 0%, 9% and 28% for mRNAs, lncRNAs and miRNAs, respectively.

Conclusion

Our results strongly suggest an improvement of prognostic power for classification using an integrated dataset compared to individual classes of RNA and thus encourage researches to opt for an integration of datasets rather than analyzing them separately.

Background

Improvements of advanced treatments and early diagnosis of breast cancer (BC) have markedly reduced mortality for BC patients [1]. However, with an estimated 2.3 million new cases (11.7% of total cancer cases) in 2020, female BC has exceeded lung cancer as the most commonly diagnosed cancer, and for women, BC is the leading cause of cancer death [2]. Clinicians lack optimal patient classification methods to identify patients with indolent tumors, which results in provision of adjuvant systemic treatments for more than 90% of all current BC patients in order to reduce risk of recurrence. As classical histopathological characteristics only can identify a small fraction of patients who are cured by local treatments [3, 4], a significant amount of patients are subjected to unnecessary systemic treatments and side effects.

Gene expression profiling using microarrays has allowed for significant advances in genomic studies. van’t Veer et al. demonstrated the prognostic role of gene expression profiling for primary breast tumors in 2002 and several mRNA-based profiles have been developed since. These genomic tests are in many countries routinely used in the clinic to guide treatment decisions considering chemotherapy needed to reduce risk of recurrence [5–7]. However, the profiles are rarely used for complete de-escalation of treatment including endocrine treatment. Moreover, optimal methods for predicting recurrence are still lacking, leading to overtreatment especially of lymph-node negative (LNN) patients [8–10]. Therefore, we need more efficient prognostic markers that reliably can identify patients not benefitting from any adjuvant systemic therapy.

Long non-coding RNAs (lncRNAs) are per definition more than 200 nucleotides (nt) long and have shown great prognostic potential in BC [11, 12]. We recently demonstrated superior classification performance of lncRNAs compared to mRNAs in systemically untreated BC [13]. microRNAs (miRNAs) are non-coding RNAs which are approximately 22 nt in length [14]. Based on differential miRNA expression, we could classify 37 out of 80 patients (46%) without recurrence after an average of over 20 years as ultralow-risk, and notably no misclassification was made in the group who experienced recurrence [15].

With several RNA molecules, all reflecting different transcriptomic aspects, it is reasonable to expect an improvement in patient classification performance, thus, motivating this study. We integrated mRNA, lncRNA and miRNA expression data into what we will refer as a ‘supermatrix’. We performed classification applying seven of the most commonly used machine learning methods for BC classification followed by a voting scheme. We investigated if differential RNA expression of the supermatrix in primary tumors from systemically untreated BC patients enabled reliable prediction of recurrence and compared it to single RNA classes. Our low-risk cohort included BC patients assigned to low-risk protocols at time of diagnosis, meaning that they did not receive any adjuvant systemic therapy. It is important to emphasize that the majority of these patients would be allocated to the high-risk group today, rendering a similar study challenging to perform due to the unique patient samples.

This study is the first to assess the ability of an integrated dataset consisting of mRNA, lncRNA and miRNA data to predict recurrence in systemically untreated BC patients with extensive follow-up.

Materials and methods

Description of materials

A total of 160 frozen tumor biopsies from LNN and systemically untreated BC patients were included in the study. The patients did not receive any adjuvant systemic therapy (chemotherapy nor endocrine therapy) where surgical treatment either was mastectomy without radiotherapy or breast conserving surgery with radiotherapy to the residual breast. None of the patients received radiotherapy to regional lymph nodes and all patients were LNN as determined by excision of four or more nodes by axillary dissection or sampling or by at least one sentinel node (introduced in 2001) [4, 16]. The patients were diagnosed from 1980 to 2003 on the island of Funen, Denmark and the tumor biopsies were collected as previously described [17]. This is notably a pair-matched design study where half of the patients developed recurrence i.e., regional or distant metastasis within 10 years after diagnosis while the remaining half were recurrence-free (mean follow-up of 20.9 years). The included BC patients were treated using low-risk protocols [4], thus, they did not receive any adjuvant systemic therapy as they were allocated to the low-risk group at time of diagnosis. Do note that most of these patients would be allocated to the high-risk group today and therefore receive adjuvant systemic therapy.

The tumor biopsies were pairwise matched according to various clinicopathological characteristics including: age (range: 33–88 years), tumor type, tumor diameter (range: 4–50 mm), year of surgery, receptor status (ER, PR, n/a) and histological grade (grade 1–3 or n/a) (Table 4 in Appendix). All clinicopathological information used for sample matching was acquired from the Danish Breast Cancer Cooperative Group (DBCG) database, the Funen pathology database, or the nationwide pathology database.

Microarray analysis and re-annotation

Total RNA was extracted from freshly frozen tumor biopsies as previously described [11]. A modified standard design of the SurePrint G3 Human GE 8 × 60 k oligonucleotide slides (G4102A) by Agilent Technologies (Santa Clara, USA) was applied for mRNA and lncRNA gene expression analysis. For miRNAs, the miRCURY LNA microRNA array ready-to-spot probe-set by Exiqon containing 1289 miRNAs was utilized as previously described [18]. The gene expression data have previously been deposited in NCBI's Gene Expression Omnibus [19] and are accessible through GEO Series accession number GSE48408 for lncRNAs and mRNAs [17], GSE103161 and GSE126125 for miRNAs [18].

To select the probes covering mRNAs, chromosomal positions of the probes in the annotation file from Agilent were matched to the positions in the Human RNA catalog from GENCODE v. 16 (https://www.gencodegenes.org/human/release_16.html) as previously described [17, 20]. Re-annotation of lncRNA probes was performed using the databases LNCipedia [21], GENCODE v. 41 (https://www.gencodegenes.org/human/release_41.html) and ENSEMBL v. 106 (https://ftp.ensembl.org/pub/release-106/gtf/homo_sapiens/). For GENCODE v. 41, the gencode.v41.long_noncoding_RNAs.gtf file was utilized. For lncRNAs that were not covered in these databases, BLASTn [22] was used to align the probe sequences with non-coding and coding transcripts from GENCODE v. 41. We considered the transformed data (lincRNA, antisense, sense_overlapping, 3prime_overlapping_ncrna, ncrna_host, bidirectional_promoter_lncrna, ambiguous_orf, retained_intron and sense_intronic) as lncRNAs. The miRNA capture-probes were based on miRBase v. 16 as previously described [18]. Re-annotation resulted in identification of 21,858 mRNA and 7,148 lncRNA probes on the Agilent arrays. Further analysis was performed using all probes for mRNA and miRNA as well as 4702 unique lncRNAs [23].

The combined dataset, designated as the ‘supermatrix’, containing mRNA, lncRNA and miRNA data, was constructed by standardizing each dataset and subsequently combining the three standardized datasets into one.

Data processing, classification and voting

Array scanning and data preprocessing were performed as previously described [11]. In the selection of machine learning methods applied for this study, a literature search regarding the most common machine learning methods for BC classification was performed, revealing Linear Discriminant Analysis (LDA), Support Vector Machines based on a radial kernel (R-SVM) or linear kernel (L-SVM), Random Forest (RF), Naïve Bayes (NB), COX risk score (COX-RS) and Least Absolute Shrinkage and Selection Operator Logistic Regression (LASSO LR) as being the most frequently used. We therefore chose to apply these for this study. We assessed classification performance for each machine learning method by the sensitivity and specificity metrics using a leave-one-pair-out cross-validation approach, with the optimal performance defined as achieving the highest specificity under the criteria of having a sensitivity of ≥ 90%. In this approach, a single pair of matched samples served as test samples while the remaining 79 sample pairs served as a training set. The differentially expressed RNAs were identified on the training set within each cross-validation iteration. This was repeated until all pairs had been left out once where the accuracy of the classifiers was determined by correctly classified samples. Leave-one-pair-out cross-validation provides an unbiased estimate of classifier performance and has furthermore proven to be optimal for analysis of smaller datasets [24, 25]. To improve classification performance, we applied a voting scheme with a final sensitivity of 90% after classification by the individual machine learning methods (Fig. 1) [23]. We also examined the most stringent cutoff of 1 as the sensitivity preferably might be higher in a clinical setting.

Fig. 1.

Overview of the applied classification strategy. Classification was performed based on the differentially expressed mRNAs, lncRNAs and miRNAs using seven machine learning methods, C-RS: COX Risk Score; L-SVM: Linear kernel Support Vector Machine; LDA: Linear Discriminant Analysis; L-LR: Least Absolute Shrinkage and Selection Operator Logistic Regression Logistic Regression; NB: Naïve Bayes; R-SVM: Radial kernel Support Vector Machine; RF: Random Forest. The individual methods were optimized using an approach that provided ≥ 90% sensitivity while maximizing specificity followed by voting with a final sensitivity of 90%

All classification procedures and statistical analyses were performed using the R open-source environment (version 4.0.2, https://cran.r-project.org/). For SVM and NB based classification, we used the e1071 R-package (https://cran.r-project.org/web/packages/e1071/index.html), while the RF, COX-RS, LDA and LASSO LR based classification procedures were performed using the randomForest (https://cran.r-project.org/package=randomForest), survival (https://cran.r-project.org/package=survival), MASS (https://cran.r-project.org/package=MASS) and glmnet (https://cran.r-project.org/web/packages/glmnet/index.html) R-packages, respectively. The Receiver Operating Characteristic (ROC) curve and the associated area under curve (AUC) values with corresponding confidence interval were constructed and calculated using the roc and auc functions embedded in the pROC R-package (https://cran.r-project.org/web/packages/pROC/index.html).

We performed classification based on the differentially expressed RNA molecules with above mentioned machine learning methods, applying an approach of a fixed sensitivity of ≥ 90% while maximizing specificity for the individual classifiers followed by voting scheme with a final sensitivity of 90%. We have recently shown that this classification strategy improves classification performance [13].

The integrated voting results were obtained using seven distinctive cutoffs ranging from 1–7, representing different degrees of classification agreement between the seven methods in terms of recurrence votes. A cutoff of 1 meant that one or more votes for recurrence, placed the patient as high-risk whereas zero votes classified the patient as low-risk, while a cutoff of 7 meant that if all votes were assigned for recurrence, the patient was placed as high-risk, while six or less votes corresponded to low-risk.

A more detailed description of the applied procedures is provided in Appendix.

Statistical analysis

Paired Student’s two-tailed t-tests were performed for analysis of differential mRNA, lncRNA and miRNA expression where a false discovery rate (FDR) ≤ 0.05 was considered significant. As the specificity levels did not deviate significantly from a normal distribution as assessed using the Shapiro–Wilk test (p = 0.34), comparison of performance across the seven classification methods was based on a one-sided Student’s t-test. Additionally, we tested if the supermatrix-based classifier performed significantly better after voting at the optimal voting cutoff than the individual mRNA-, lncRNA- and miRNA-classifiers by using a one-sided two-proportion z-test.

Results

Gene expression data based on mRNAs, lncRNAs and miRNAs were analyzed in 160 freshly frozen primary tumors from systemically untreated patients with invasive BC. The three different RNA gene expression data were standardized individually and subsequently assembled into one integrated data matrix. The comparison of the two matched patient groups (recurrence vs. recurrence-free) using paired Student’s two-tailed t-tests identified 569 significantly differentially expressed RNAs in the supermatrix dataset comprising of 333, 122 and 114 mRNAs, lncRNAs and miRNAs, respectively (FDR ≤ 0.05).

Classification of outcome

Classification combined with leave-one-pair-out cross validation was conducted with a clinically relevant threshold of ≥ 90% sensitivity as a cumulative risk of recurrence of ≥ 10% within 10 years is regarded as a high risk in Denmark [26]. We refer to the sensitivity metric as the proportion of truly affected subjects who correctly are classified as positive while specificity denotes truly unaffected patients who correctly are identified as negative [27]. We analyzed the mRNA, lncRNA and miRNA gene expression data in our 160 BC patients. The patients were divided into subgroups with high or low risk of recurrence through classification by the seven individual classifiers.

Under the criteria of having a sensitivity of ≥ 90% while maximizing specificity, the highest overall classification accuracy for the supermatrix data was 91% (92% sensitivity, 89% specificity) using Least Absolute Shrinkage and Selection Operator Logistic Regression (LASSO LR) (Table 1). For mRNAs alone, Random Forest (RF) achieved the highest accuracy of 66% (91% sensitivity, 40% specificity) while Support Vector Machines based on a linear kernel (L-SVM) obtained 71% (90% sensitivity, 52% specificity) for lncRNAs alone. Using miRNA-based classification, RF achieved 86% (91% sensitivity, 81% specificity). The overall classification accuracy was consistently higher across all methods for the supermatrix compared to single RNA types. The classification results with corresponding p-values between the supermatrix and single RNA type data are shown in Table 1.

Table 1.

Classification with ≥ 90% sensitivity threshold using seven individual machine learning methods

	Supermatrix			mRNA			lncRNA			miRNA
Method	Sen	Spe	Acca	Sen	Spe	Acca	Sen	Spe	Acca	Sen	Spe	Acca
COX-RS	91	56	74	90	21	56	91	38	64	90	49	69
LDA	92	79	86	90	30	60	90	44	67	92	56	74
LASSO	92	89	91	90	21	56	90	30	60	92	75	84
L-SVM	90	79	84	90	34	62	90	52	71	90	75	82
NB	90	70	80	90	32	61	90	25	58	94	62	78
RF	91	86	89	91	40	66	92	46	69	91	81	86
R-SVM	90	78	84	90	32	61	90	48	69	90	76	83
P-valueb Supermatrix vs. RNA				1.206 × 10–5			0.00018			0.0084

Classification was conducted with a threshold that provided at least 90% sensitivity while maximizing specificity and performances assessed by leave-one-pair-out cross-validation using COX risk score (COX-RS), LDA Linear Discriminant Analysis, LASSO Least Absolute Shrinkage and Selection Operator Logistic Regression, L-SVM Support Vector Machines based on a linear kernel, NB Naïve Bayes, RF Random Forest and SVM based on a radial kernel (R-SVM). ^aAccuracy (acc) defined as mean of sensitivity (sen) and specificity (spe). All measures are stated in percent. ^bP-value determined by a one-sided Student’s t-test, testing the H₀: µ ≤ 0 and H_A: µ > 0, where µ = supermatrix specificity minus individual RNA type specificity

Meeting the requirement of a final sensitivity of 90% after individual classification, a supermatrix-based voting obtained a sensitivity of 90% with a corresponding specificity of 85% at a cutoff of 5 (Table 2). At the same cutoff, the optimal performance based on mRNAs, lncRNAs or miRNAs alone obtained a sensitivity of 91% for all three types with a specificity of 38%, 48% and 82% respectively. The voting results with corresponding p-values between the supermatrix and single RNA type data are shown in Table 2.

Table 2.

Classification performance followed by ensemble voting across seven machine learning methods at varying cutoff thresholds

	Supermatrix			mRNA			lncRNA			miRNA
Voting cutoff	Sen	Spe	Acca	Sen	Spe	Acca	Sen	Spe	Acca	Sen	Spe	Acca
≥ 1	99	41	70	100	0	50	98	9	53	99	28	63
≥ 2	98	59	78	100	3	51	96	20	58	96	49	72
≥ 3	98	72	85	100	22	61	96	32	64	95	64	79
≥ 4	94	80	87	96	30	63	96	41	69	91	70	81
≥ 5	90	85	88	91	38	64	91	48	69	91	82	87
≥ 6	85	92	89	80	48	64	84	59	71	88	88	88
≥ 7	65	96	81	52	70	62	72	72	72	76	95	86
P-valueb P-valuec Supermatrix vs. RNA				2.02 × 10–10 9.62 × 10–10			2.51 × 10–6 6.22 × 10–7			0.048 0.42

The voting results were obtained using seven distinctive cutoffs ranging from 1–7, representing different degrees of classification agreement between the seven machine learning methods in terms of recurrence votes. A cutoff of 1 meant that one or more votes for recurrence, placed the patient as high-risk. ^aAccuracy (acc) defined as mean of sensitivity (sen) and specificity (spe). All measures are stated in percent. ^bP-value determined by a one-sided two-proportion z-test comparing the specificity difference after voting using the supermatrix and single RNA type data. The p-values are shown for supermatrix vs. mRNA, supermatrix vs. lncRNA and supermatrix vs. miRNA at cufoff ≥ 1. ^cas in b but p-values are shown for cufoff ≥ 5

Reasoning that the sensitivity in a clinical setting preferably might be higher than 90%, meaning that misclassification in the recurrence group would be less than 10%, we examined the most stringent cutoff of ≥ 1 after voting. Based on the supermatrix, a sensitivity of 99% was obtained with a specificity of 41% (Table 2). For the individual RNA types, the sensitivity ranged from 98–100% while the specificity was 0%, 9% and 28% for mRNAs, lncRNAs and miRNAs respectively. The voting results with corresponding p-values between the supermatrix and single RNA type data are shown in Table 2. The discriminatory power and diagnostic ability for the supermatrix voting classification model was additionally evaluated using Receiver Operating Characteristic (ROC) curve. The achieved ROC curve had an area under curve (AUC) of 0.95 (95% CI 0.91–0.98) (Fig. 3 in Appendix).

Top ten most consensual RNAs across the seven individual classifiers

As deregulated mRNAs, lncRNAs and miRNAs have crucial roles in cancer development and progression, we subsequently examined the top ten selected mRNAs, lncRNAs and miRNAs for classification. The highly consensual RNAs across the seven individual classifiers were analyzed to see if their deregulation in BC had been demonstrated by other studies. The top ten consensual mRNAs across all classifiers were C13orf43 (alias for SPART-AS1), NPY6R, GATS (alias for CASTOR3P), GSTTP1 (alias for GSTT4), LOC401233, LOC728903, SPHKAP, OGT, LOC339290 and OR13C9 (annotated December 2022). For lncRNAs, the ten most informative features was lnc-MEX3B-4 followed by lnc-ZNF131-2, lnc-LTBP3-2, lnc-NSUN6-1, lnc-HNRNPA0-4, RNF139-AS1, LINC02086, lnc-MRFAP1-2, lnc-SLC16A11-7 and lnc-SGSM2-1 (LNCipedia gene ID, annotated December 2022). The ten highest consensual miRNAs were miR-499-5p, miR-525-5p, miR-640, miR-212, miR-519e*, miR-942, miR-576-5p, miR-569, miRPlus-I320a* and miR-1299. For three mRNAs, two lncRNAs and seven miRNA candidates, supportive literature was found where an overview of the RNA candidates and their reported roles in BC-related studies is depicted in Table 3.

Table 3.

Overview of the RNA candidates and their reported roles in breast cancer-related studies

Study	mRNA	Reported role	Regulatory axis
Feng et al. [28]	GATS	Highly expressed in TNBC cells. Promote proliferation, migration and clonogenicity
Nagahashi et al. [29]	SPHKAP	Regulatory role in modulation of SPHK1 which is associated with obesity-related BC progression and metastasis
Liu et al. [30]	OGT	Increased levels of O-GlcNAcylation are associated with poor patient prognosis
	lncRNA
Gomes et al. [31]	lnc-LTBP3-2 (TALAM1)	TALAM1 synergizes with MALAT1 on regulation of the migratory ability of BC cells
Han et al. [32]	LINC02086	Upregulated in BC and associated with poorer prognosis. Knockdown decreases cell viability, promoted cell apoptosis and suppressed tumor growth	LINC02086 by miR-6757-5p/EPHA2
	microRNA
Yu et al. [33]	miR-525-5p	Involved in promotion of TNBC cell malignancy and tumorigenesis	LINC01234 by miR-525-5p/MEIS2
Tang et al. [34]	miR-640	Suppresses tumorigenesis of BC cells where overexpression inhibits proliferation and migration
Hadavi et al. [35]	miR-576-5p	Predicted to target PI3KCA. A reverse correlation was shown in TNBC clinical samples and cell lines
Chaluvally-Raghavan et al. [36]	miR-569	Amplification of 3q26.2 is associated with increased expression of miR-569. Overexpression alters cell survival and proliferation
Li et al. [37] Zhang et al. [38]	miR-942	Involved in reduced progression of BC Involved in promotion of TNBC metastasis	Circ_0001785 by miR-942/SOCS3 PAI-1 by SOX2-OT miR-942/PIK3CA
Liu et al. [39] Ma et al. [40]	miR-1299	Involved in promotion of proliferation, metastasis and PTX resistance Involved in regulation of the malignant phenotype of TNBC	Circ_0006528 by miR-1299/CDK8 CircEGFR by miR-1299/EGFR
Xie et al. [41] Ding et al. [42] Li et al. [43]	miR-212	Involved in development of doxorubicin resistance Involved in acceleration of malignancy and glycolysis of TNBC cells Important regulator of angiogenesis, proliferation, migration and invasion in BC	CircWHSC1 by miR-212/AKT3 ZLM-7 by miR-212-3p/Sp1/VEGFA

AKT3 AKT Serine/Threonine Kinase 3, BC breast cancer, CDK8 Cyclin Dependent Kinase 8, EGFR Epidermal Growth Factor, EPHA2 Ephrin type-A receptor 2, GATS also known as Stromal Antigen 3 Opposite Strand, MALAT1 Metastasis-associated lung adenocarcinoma transcript-1, MEIS2 Meis Homeobox 2, OGT O-Linked N-Acetylglucosamine (GlcNAc) Transferase, PAI-1 Plasminogen Activator Inhibitor-1, PIK3CA Phosphatidylinositol-4,5-bisphosphate 3-kinase Catalytic Subunit Alpha, PTX Paclitaxel, SOCS3 Suppressor Of Cytokine Signaling 3, Sp1 Specific Protein 1, SPHK1 Sphingosine Kinase 1, SPHKAP SPHK1 Interactor, AKAP Domain Containing, TALAM1 antisense transcript of MALAT1, TNBC triple negative breast cancer, VEGFA Vascular Endothelial Growth Factor A, ZLM-7 a combretastatin A-4 derivative

Discussion

In our study, we found a marked improvement of patient classification performance when established on differential expression of mRNAs, lncRNAs and miRNAs compared to single RNA datasets. A significant larger fraction of patients who did not experience recurrence (mean follow-up of 20.9 years) was correctly identified as low-risk. For each of the classifiers, the overall accuracy was consistently higher when established from the integrated dataset compared to single RNA, suggesting a clear improvement of prognostic accuracy.

To further improve our classification, we applied a voting scheme with a final sensitivity of 90%. We assessed the clinically relevant cutoff where the highest specificity was achieved while having a sensitivity of ≥ 90%. The highest specificity obtained for the integrated dataset was 85% while the highest levels were 38%, 48% and 82% for mRNAs, lncRNAs and miRNAs respectively (Table 2). This suggested that by combining the three RNA classes, we could achieve markedly better prognostic predictors than applying the RNA types individually. In terms of clinical translation, 68 out of 80 (85%) recurrence-free patients could appropriately forego unnecessary systemic therapy while correctly classifying 72 out of 80 (90%) patients with recurrence as high-risk. Compared to mRNA-based voting, a considerably smaller fraction of indolent patients, 30 out of 80 (38%), were identified while correctly classifying 73 out of 80 (91%) patients with recurrence as high-risk (Table 2). Consequently, these results demonstrated that by combining mRNA, lncRNA and miRNA expressions, we could reliably identify a much larger subgroup of BC patients not benefitting from any adjuvant systemic therapy while correctly classifying ≥ 90% of patients with recurrence as high-risk.

The sensitivity of ≥ 90% might not be high enough in a stringent clinical setting as it allows for ≤ 10% misclassification in the recurrence group. Examining the most stringent cutoff of 1, meaning that the patient would be placed as high-risk if only one individual classifier voted for recurrence, demonstrated a definite superiority of the supermatrix. In this case, 79 out of 80 patients would correctly be classified as high-risk. But even under these very stringent criteria 33 out of 80 patients would correctly be classified as low-risk by the supermatrix and could be spared adjuvant systemic therapy (Table 2). As emphasized earlier, the majority of these 33 patients would today be classified as high-risk and thus, receive unnecessary systemic therapy, making these results striking. In a future clinical setting, our profile could be used upfront to identify a subgroup of BC patients who could be spared any adjuvant systemic treatment. For the remaining patients, profiling tools such as MammaPrint [5], Oncotype DX [44] and PAM50 [45] could then be used to stratify treatment.

Our results were in consistency with a study by Xiong et al. who also analyzed the joint effects of mRNAs, lncRNAs and miRNAs in colorectal cancer and found that an integrated multi-RNA classifier was more effective for survival prediction than the RNA types alone [46].

We acknowledge that validation of these results is crucial. However, proper validation will be challenging as collection of suitable samples is difficult due to our extremely unique cohort of freshly frozen primary tumors from systemically untreated BC patients with more than 20 years of follow-up. These are patients who in most countries today would receive adjuvant treatment. The scarce availability of other miRNA expression datasets is another challenge which will render optimal validation difficult [47]. Applying a different integration approach of the three classes of RNA might also be a possible objective for future studies when a deeper understanding of the interactions and functions within the transcriptome have been further elaborated.

It is well established that deregulated mRNAs, lncRNAs and miRNAs have important roles in cancer development and progression ([48], reviewed in [49]). We examined the top ten selected mRNAs, lncRNAs and miRNAs features for classification defined as highly consensual RNAs across the seven individual classifiers to examine if their deregulation in BC had been demonstrated by other studies. For mRNAs, BC supportive literature was found for the genes GATS, SPHKAP and OGT. An overview of their reported roles in BC is depicted in Table 3. The remaining mRNAs were not found to be reported in BC related studies. A limited number of studies involves the top ten consensual lncRNAs, especially related to BC. However, for two lncRNAs, lnc-LTBP3-2 and LINC02086, BC related literature was found (Table 3). On the contrary, for miRNAs, supportive literature was available for miR-525-5p, miR-640, miR-576-5p, miR-569, miR-942, miR-1299 and miR-212. Several of these have been reported to be involved in relevant mechanisms such as proliferation, metastasis and survival (Table 3). It will be exciting for future studies to further investigate these various RNAs in terms of prognostic prospective and potential candidates for therapeutic application in BC.

Conclusions

In summary, classification with a fixed sensitivity of ≥ 90% for the individual machine learning methods followed by voting, obtained consistently higher specificities when based on supermatrix data compared to individual classes of RNA. Our results strongly suggest that added prognostic value could be achieved by combining all three RNA types where the superiority of the supermatrix was striking at the most stringent clinically relevant cutoff of 1.

Abbreviations

BC

Breast cancer

C-RS

COX Risk Score

LASSO

Least Absolute Shrinkage and Selection Operator

LDA

Linear Discriminant Analysis

lncRNA

Long non-coding ribonucleic acid

LNN

Lymph-node negative

L-SVM

Linear kernel Support Vector Machine

microRNA

Micro ribonucleic acid

NB

Naïve Bayes

RF

Random Forest

R-SVM

Radial-based kernel Support Vector Machine

TNBC

Triple negative breast cancer

Appendix

Methods

We performed classification based on the significantly differentially expressed RNA molecules using seven machine learning methods and applied an approach that provided ≥90% sensitivity while maximizing specificity, assessed by leave-one-pair-out cross validation (LOPOCV).

Feature selection is necessary in the training set to avoid a small sample-per-feature ratio and it has been shown to provide better classification [25]. The feature selection process in this study, comprised for all machine learning methods apart from LASSO LR of three steps: 1) Testing all three RNA molecules (mRNA, lncRNA and miRNA) in the training set (79 sample pairs as one pair served as test pair) for significant differential expression using paired Student’s two-tailed t-test (FDR ≤0.05); 2) Ranking the differentially expressed RNAs according to their random-forest importance value, which reports the standardized drop in prediction accuracy when the class labels are permuted [50]; 3) By subsequently adding one mRNA, lncRNA or miRNA at a time in a top-down forward-wrapper approach starting with top two RNAs upon the ranked list followed by classification accuracy assessment, the optimal number of RNAs were determined by each of the six machine learning methods. At each addition, the classification accuracy of the training samples was evaluated using LOPOCV in a nested inner loop [23]. The pipeline for these classification steps is shown in Figure 2 in Appendix (blue arrows).

The input for LASSO LR were the RNAs in the training set found to be significantly differentially expressed according to the paired Student’s two-tailed t-test (FDR ≤0.05). Next, based on these genes, we determined the optimal λ-value, which controls model complexity and retains important variables by reducing the coefficients of insignificant variables in the original model to zero. The optimal λ-value was, like the six other machine learning methods, defined as the value maximizing specificity under the criteria ≥90% sensitivity within the nested inner loop of the LOPOCV procedure. An overview of this procedure is shown in Fig. 2 in Appendix (green arrows).

Fig. 2.

Prediction of outcome using leave-one-pair-out cross-validation. In this procedure,a single pair of matched samples served as test samples while the remaining 79 sample pairs served as a training set. This was repeated until all pairs had been left out once where the accuracy of the classifiers was determined by correctly classified samples. For feature selection in the training set, Student’s two-tailed t-test was used to identify differentially expressed mRNAs, lncRNAs and miRNAs (FDR ≤0.05). These differential expressed RNAs were re-ranked according to their random-forest importance value for the six individual classifiers (blue arrows) while they were used for determination of the λ-value for LASSO Logistic Regression (LASSO LR, green arrows). Using leave-one-out-cross-validation (LOPOCV), each of the six machine learning methods identified the optimal RNA set while LASSO LR identified the optimal λ-value (determining the most informative set of genes), by predicting outcome in one selected pair where the remaining 78 pairs served as training set (loop, blue). The final optimal/informative RNA set was used to predict outcome in the initially left out test pair. The entire procedure was repeated until all pairs had been left out as a test pair and their outcome independently predicted by the inner training loop

Fig. 3.

Receiver Operating Characteristic (ROC) curve for the supermatrix voting classification model. The achieved ROC curve had an area under curve (AUC) of 0.95 (95% CI 0.91-0.98)

Table 4.

Patient and tumor characteristics

	Recurrence development	Recurrence-free
No. of patients (%)	80 (50)	80 (50)
Age at diagnosis (range: 33-88 years)
≤50 years	14 (8.8)	10 (6.3)
>50 years	66 (41.3)	70 (43.8)
Tumor size
<2 cm	27 (16.9)	30 (18.8)
2-5 cm	53 (33.1)	49 (30.6)
n/a	-	1 (0.6)
Estrogen receptor statusa
Positive	52 (32.5)	50 (31.3)
Negative	22 (13.8)	24 (15)
n/a	6 (3.8)	6 (3.75)
Tumor type
Invasive ductual carcinoma (IDC)	62 (38.8)	65 (40.6)
Invasive lobular carcinoma (ILC)	9 (5.6)	9 (5.6)
Mucinous carcinoma	2 (1.3)	2 (1.3)
Papillary carcinoma	3 (1.9)	2 (1.3)
Carcinoma with metaplasia	2 (1.3)	2 (1.3)
n/a	2 (1.3)	-
Histologic grade
1 (good)	12 (7.5)	15 (9.4)
2 (intermediate)	28 (17.5)	25 (15.6)
3 (poor)	22 (13.8)	24 (15)
n/a	18 (11.3)	16 (10)
Median year of surgery (range 1980-2003)	1993	1994
Mean time to recurrence (months)	58.5	n/a
Mean follow-up (months)	88.3	250.35
Alive at end of follow-up	1	48

n/a not available/applicable; ^aas defined by immunohistochemistry

Authors’ contributions

TAK and MT conceptualized and supervised the project. They also contributed to the methodology of the project together with QT and MB. MB handled the formal analyses including software application. AMBJ, BE and AVL contributed to the resources of the project while TAK, MT and KPS administered the funding acquisition. MB, MJL, IB and KPS curated the data. TTND and IB visualized and prepared the original draft and wrote, reviewed and edited it together with TAK, MT and MB. All authors have read and agreed to the published version of the manuscript.

Funding

Open access funding provided by University of Southern Denmark This research was funded by the Danish Cancer Society, the Danish Council for Independent Research (grants: 7016-00346B/FSS and 09–061677/FSS), the Danish Ministry of the Interior, the University of Southern Denmark (Horizon2020, grant: DAWN2020), the Danish Council for Strategic Research (grant: DBCG-TIBCAT) and Regionernes Medicinpulje, Dansk Kræftforskningsfond, Breast Friends, Fonden til Lægevidenskabens Fremme, Meta & Håkon Baggers Fond, A. J. Andersen & hustrus Fond, Inge & Jørgen Larsens Mindelegat, Overlægerådets Legatudvalg, Direktør Jacob Madsens & Hustru Olga Madsens Fond, Fru Ingeborg Anna Albinus Larsens Mindelegat, Fonden af 1870, Harboefonden, the Odense University Hospital Fund for Free Research, Lundbeckfonden (grant: Center of Excellence NanoCAN) and Frimodt-Heineke Fonden.

Data availability

The datasets analyzed during the current study are available in the National Center of Biotechnology Information’s Gene Expression Omnibus, and are accessible through GEO Series accession number GSE48408 for lncRNAs and mRNAs, GSE103161 and GSE126125 for miRNAs.

Declarations

Ethics approval and consent to participate

Patient consent was waived because the tumor samples included in the study were collected from established biobanks at Odense University Hospital, Denmark where utilization of these surplus samples has been approved by the Danish National Committee on Health Research Ethics. Patients who objected to the use of their tumor material via the Danish Tissue Utilization Register were excluded from the study.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.