Skip to main content
Heliyon logoLink to Heliyon
. 2024 Feb 22;10(5):e26859. doi: 10.1016/j.heliyon.2024.e26859

Bioinformatics analysis of MMP14+ myeloid cells affecting endothelial-mesenchymal transformation and immune microenvironment in glioma

Wei Luo a,1, Qi Quan a,1, Zihao Xu a, Jinju Lei b, Roujun Peng a,
PMCID: PMC10904238  PMID: 38434278

Abstract

Background

Gliomas, known for their complex and aggressive characteristics, are deeply influenced by the tumor microenvironment. Matrix metalloproteinases (MMPs) play a vital role in shaping this environment, presenting an opportunity for novel treatment strategies.

Methods

We collected six bulk RNA datasets, one single-cell RNA sequencing (scRNA-seq) dataset, and gene sets related to Matrix Metalloproteinases (MMPs), Endothelial-Mesenchymal Transformation (EndMT), and sprouting angiogenesis. We computed enrichment scores using Gene Set Variation Analysis (GSVA) and Single-sample Gene Set Enrichment Analysis (ssGSEA). To analyze immune infiltration, we employed the CIBERSORT method. Data analysis techniques included the log-rank test, Cox regression, Kruskal-Wallis test, and Pearson correlation. For single-cell data, we utilized tools such as Seurat and CellChat for dimensionality reduction, clustering, and cell communication analysis.

Results

1. MMP14 was identified as an independent prognostic marker, highly expressed in myeloid cells in recurrent glioblastoma, highlighting these cells as functionally significant. 2. C–C Motif Chemokine Ligand (CCL) signaling from MMP14+ myeloid cells was identified as a critical immune regulatory pathway, with high C–C Motif Chemokine Receptor 1 (CCR1) expression correlating with increased M2 macrophage infiltration and PD-L1 expression. 3. Patients with high MMP14 expression showed better responses to bevacizumab combined chemotherapy. 4. Signaling pathways involving Visfatin, VEGF, and TGFb, emanating from myeloid cells, significantly impact endothelial cells. These pathways facilitate EndMT and angiogenesis in gliomas. 5. Nicotinamide Phosphoribosyltransferase (NAMPT) showed a strong link with angiogenesis and EndMT, and its association with chemotherapy resistance and differential sensitivity to bevacizumab was evident.

Conclusions

MMP14+ myeloid cells are critical in promoting tumor angiogenesis via EndMT and in mediating immunosuppression through CCL signaling in glioblastoma. MMP14 and NAMPT serve as vital clinical indicators for selecting treatment regimens in recurrent glioma. The study suggests that a combined blockade of CCR1 and CD274 could be a promising therapeutic strategy.

Keywords: EndMT, CCR1, NAMPT, Bevacizumab, PD-L1

1. Introduction

Gliomas, the most common primary intracranial tumors, are known for their significant heterogeneity and aggressiveness [1,2]. The interactions between tumor cells and the tumor microenvironment (TME) play a crucial role in driving this heterogeneity and invasiveness. Non-tumor cells within gliomas, such as microglia, macrophages, vascular endothelial cells, fibroblasts, and lymphocytes, not only shape the tumor's characteristics but also contribute to the development of drug resistance [[3], [4], [5]]. Despite the potential of manipulating the TME to enhance treatment effectiveness, most efforts in recent decades have been unsuccessful. Only therapies targeting angiogenesis and immunotherapy have made limited strides.

Bevacizumab, a human monoclonal antibody, targets tumor angiogenesis by binding to vascular endothelial growth factor A (VEGFA). Phase III clinical trials have shown that while bevacizumab significantly increases progression-free survival (PFS), it does not improve overall survival (OS) in patients [6,7]. PD-1, a critical immunosuppressive molecule, leads to T cell exhaustion and impairs antitumor immunity by interacting with PD-1 ligands on tumor or host cells. Pembrolizumab, by binding to the PD-1 receptor, disrupts this immunosuppressive signaling and activates antitumor immunity [8]. Although it is effective in various cancers, its efficacy in gliomas is limited [9]. Benefits from pembrolizumab are observed mainly in patients receiving neoadjuvant therapy and those with mismatch repair deficiencies during second-line therapy [10,11]. Nivolumab, another PD-1 inhibitor, has not shown improved efficacy when combined with bevacizumab or chemoradiotherapy in newly diagnosed glioma cases [12]. The challenge remains to identify patients who will benefit from anti-angiogenic and anti-PD-1 therapies and to enhance the effectiveness of these treatments.

MMPs, a group of zinc- and calcium-dependent endopeptidases, play a significant role in the TME. They degrade basement membrane and extracellular matrix molecules, impacting angiogenesis and antitumor immunity [13]. Research has shown that angiogenic factors like VEGF, Basic Fibroblast Growth Factor (bFGF), Transforming Growth Factor Beta (TGF-β), and angiopoietin can induce MMP production, thereby promoting angiogenesis [14]. MMP14, MMP2, and MMP9 have been found to increase TGF-β release by reducing the expression of interleukin-2 receptor α on T lymphocytes, weakening their antitumor response [15]. This study delves into the influence of MMPs, related cells, and signaling pathways on angiogenesis and antitumor immunity, aiming to lay a foundation for informed decision-making and potential new therapeutic strategies in the treatment of glioma.

2. Materials and methods

2.1. RNA expression data and clinical data acquisition

For our research, we acquired bulk RNA-seq and clinical data for 152 patients with glioblastoma and 496 with low-grade gliomas (LGG) from The Cancer Genome Atlas (TCGA) database (https://portal.gdc.cancer.gov/). All gene expression data underwent log 2 (x+1) normalization. We excluded rows and columns with over 50% missing values. Additionally, we gathered five bulk RNA datasets from the Gene Expression Omnibus (GEO) database (https://www.ncbi.nlm.nih.gov/geo/), specifically GSE43388 (43 cases, including 15 from GSE43388-GPL570 and 28 from GSE43388-GPL8542), GSE61335 (62 cases), GSE72951 (112 cases), GSE74187 (60 cases), and GSE107850 (195 cases). Within GSE43388, the datasets GSE43388-GPL570 and GSE43388-GPL8542 were merged using the R package inSilicoMerging, followed by batch correction using the COMBAT method.

Moreover, we downloaded a single-cell dataset, GSE182109, from the GEO database. From this dataset, we selected two subsets each of newly diagnosed glioblastoma (nGBM - GSM5518602_ndGBM-01-D and GSM5518601_ndGBM-01-C), relapsed glioblastoma (rGBM - GSM5518615_rGBM-02-5 and GSM5518613_rGBM-02-3), and low-grade glioma (LGG - GSM5518632_LGG-04-3 and GSM5518638_LGG-03) for in-depth single-cell analysis. The single-cell sequencing was performed using the 10× Genomics single-cell sequencing technology platform.

2.2. Analysis of RNA-seq data

In line with the most recent classification criteria, we categorized G2 patients from the TCGA dataset as LGG (240 cases) and those with G3/G4 as high-grade gliomas (HGG, 408 cases). We utilized the GSVA package to compute Gene Set Variation Analysis (GSVA) scores for the MMPs gene set in each patient. For assessing immune cell infiltration in TCGA and GEO samples, the CIBERSORT method was implemented.

Furthermore, we downloaded a gene set focused on the positive regulation of angiogenesis (GOBP_POSITIVE_REGULATION_OF_CELL_MIGRATION_INVOLVED_IN_SPROUTING_ANGIOGENESIS) from the GSEA website (https://www.gsea-msigdb.org/gsea/index.jsp). The single-sample Gene Set Enrichment Analysis (ssGSEA) method was used to calculate scores for this gene set in the samples. We identified 26 marker genes for Endothelial-to-Mesenchymal Transition (EndMT) based on studies by Nicolas Clere et al. and Lukas S. Tombor et al. [16,17]and applied the ssGSEA method to compute gene set scores in the samples.

For survival analysis, we used the R survival package. Univariate and multivariate analyses were conducted using the logrank test and the COX method, respectively. The Kruskal-Wallis test was employed to evaluate differences in gene expression and enrichment scores across different groups. We used Pearson correlation analysis to ascertain correlation coefficients between two genes or between gene expressions and gene set enrichment scores. For Kyoto Encyclopedia of Genes and Genomes (KEGG) and Gene Ontology (GO) enrichment analysis, the R clusterProfiler package was employed. We considered a p-value of less than 0.05 as statistically significant.

2.3. Expression characteristics of the independent prognostic gene at the single-cell level

R packages (Seurat, tidyverse, dplyr, patchwork, harmony, devtools) were employed for this section. We used the CreateSeuratObject function to create six subsets as Seurat objects. The genes expressed in at least three cells and the cells expressing more than 200 genes were preserved. Subsequently, we merged and batch-corrected the six subsets. The PercentageFeatureSet function was used to calculate the mitochondrial gene ratio in cells. We extracted cells with gene expression numbers between 200 and 6000 and with a mitochondrial gene expression ratio below 25%. A total of 53,915 cells were obtained for further analysis. The JackStraw function and ScoreJackStraw function determined the dimensionality, the FindNeighbors function identified cell neighbors, the FindClusters function clustered cells using the Louvain algorithm (resolution parameter set at 0.3), and the RunUMAP function mapped high-dimensional clustering information into two-dimensional space. We plotted the prognostic gene expression in LGG, nGMB, rGMB, and whole cells using the FeaturePlot function and VlnPlot function. Lastly, we renamed the clusters using the RenameIdents function.

2.4. Cell communication analysis

For cell communication analysis, we used R packages (CellChat, Seurat, dplyr, igraph, ggalluvial, patchwork). We employed the createCellChat function to extract and create a CellChat object from a Seurat object. We used the subsetDB function to extract “Secreted Signaling” information from the CellChatDB.human database. The CellChat package workflow was utilized to calculate interaction probability for signaling pathways. Subsequently, we employed the netVisual_circle function to create a network diagram illustrating these pathways. The heat map of interaction pathways was generated using the netAnalysis_signalingRole_heatmap function. For signaling pathways of interest, we conducted importance analysis using the netAnalysis_signalingRole_network function, and we examined the contribution of ligand receptor pairs within the signaling pathways of interest with the netAnalysis_contribution function.

2.5. Data analysis platform

For RNA-seq data processing, we used Sangerbox (http://vip.sangerbox.com/home.html), an efficient and powerful platform based on R. The R software (version 4.2.3) was employed for single-cell data processing.

3. Results

3.1. MMP14 is an independent prognostic factor for LGG and HGG

To examine the impact of MMPs on the prognosis of glioma patients, we computed Gene Set Variation Analysis (GSVA) scores based on a set of 29 MMPs genes. We then carried out survival analyses and generated Kaplan-Meier (K-M) curves for both LGG and HGG patients. The results indicated a significant association between elevated GSVA scores and reduced overall survival in both LGG (P = 2.0e-3, HR = 3.97) and HGG (P = 4.9e-13, HR = 2.72) cohorts (Fig. 1A and B). Univariate analysis identified MMP14, MMP11, MMP9, and several others as adverse prognostic factors for HGG, whereas MMP16, MMP24, MMP15, and MMP17 were associated with a better prognosis (Fig. S1 A). For LGG, MMP14 and MMP11 were found to be detrimental, whereas MMP15 was a favorable prognostic marker (Fig. S1 B).

Fig. 1.

Fig. 1

A. In LGG, higher MMPs scores correlate with shorter overall survival; B. In HGG, higher MMPs scores also correspond with shorter overall survival; C. Multivariate analysis of MMPs genes in HGG; D. Multivariate analysis of MMPs genes in LGG.

Subsequent multivariate analysis revealed MMP14 as the sole independent prognostic factor affecting both HGG and LGG survival outcomes (Fig. 1C and D). Elevated MMP14 expression was linked to a 2.54-fold higher risk of death in LGG and a 4.52-fold higher risk in HGG (Fig. 2A and B). We extended our findings by validating the prognostic significance of MMP14 expression across several external datasets: GSE43388, GSE61335, and GSE74187, all HGG datasets, where MMP14 consistently emerged as a harmful prognostic factor (Fig. 2C–H). In the predominantly IDH1-mutated LGG dataset GSE107850, higher MMP14 levels suggested a trend towards poorer prognosis, yet without achieving statistical significance (Fig. 2I–K).

Fig. 2.

Fig. 2

A-B. Kaplan-Meier (K–M) curves of MMP14 in TCGA LGG and HGG; C. K-M curve of MMP14 in the GSE43388 dataset; D-E. K-M curves of MMP14 in the GSE43388 radiotherapy group and the radiotherapy combined with chemotherapy group; F. K-M curve of MMP14 in the GSE61335 dataset; G-H. K-M curve of MMP14 in the GSE74187 dataset, where high MMP14 expression indicates shorter PFS and OS; I–K. K-M curve of MMP14 in the GSE107850 dataset. Although the MMP14 high-expression group exhibited a tendency toward poor prognosis, it did not reach statistical significance.

3.2. MMP14 is predominantly expressed in rGMB and mainly in myeloid cell subsets

We proceeded to conduct dimensionality reduction and clustering on 53,915 cells, resulting in the delineation of 16 distinct cell subsets (Fig. 3A). Feature and Violin plots showed MMP14 expression was predominantly found in relapsed glioblastoma (rGBM) and Cluster 6 (Fig. 3B). For cell type identification, we consulted the study by Nourhan Abdelfattah et al. [18] and the CellMarker database (http://xteam.xbio.top/CellMarker/search.jsp). Clusters 0, 3, 5, 7, 9, 10, 11, 14, 15, and 16 demonstrated expression of glioma cell markers including SOX2, OLIG1, GFAP, and S100B. Clusters 3, 9, 10, 11, 15, and 16 were designated as Glioma_Cell(LGG) due to their derivation from low-grade glioma samples; Clusters 0, 5, and 14 were termed Glioma_Cell(nGBM), derived from newly diagnosed GBM samples; and Cluster 7 was defined as Glioma_Cell(rGBM), originating from relapsed GBM samples. Cluster 2, characterized by the expression of microglial markers P2RY12 and TMEM119, was annotated as Microglia_Cell. Clusters 4 and 8 showed expression of myeloid cell markers like PTPRC/CD45, ITGAM/CD11B, and CD68, but did not express MMP14, hence named MMP14-_Myeloid_Cell(1) and MMP14-_Myeloid_Cell(2), correspondingly. Cluster 6 also expressed myeloid markers PTPRC/CD45, ITGAM/CD11B, and CD68, but with notable MMP14 expression, and was thus labeled as MMP14+_Myeloid_Cell. Cluster 1, which exhibited T cell markers including PTPRC, CD3E, CD4, and CD8A, was classified as T_Cell. Cluster 12 showed expression of the endothelial cell marker PECAM1, and was denoted as Endothelial_Cell. Lastly, Cluster 13, expressing B cell markers such as PTPRC, CD79A, CD19, and MS4A1, was identified as B_Cell (Fig. 3C).

Fig. 3.

Fig. 3

A. UMAP divides cells into 16 clusters; B. MMP14 was mainly expressed in rGBM and primarily distributed in Cluster 6; C. Naming of the cell clusters.

3.3. MMP14+_Myeloid_Cell is the most important cell subset for sending and receiving signals

For the cellular communication analysis, we chose to focus on “Secreted Signaling” and mapped out the corresponding cellular interaction network. This network revealed extensive intercellular communication, prominently showcasing the MMP14+_Myeloid_Cell population exerting a substantial influence on myeloid cells, T cells, microglia, and endothelial cells (Fig. 4A and B). Additionally, we delved into the patterns of signal transmission and reception among various cell subsets. Our findings indicated that MMP14+_Myeloid_Cell was central to both emitting and receiving interaction signals, with CCL signals being primarily sent by this cell subset. The MMP14+_Myeloid_Cell also emerged as a key source of signals for VISFATIN, VEGF, and TGF-beta, which were mainly received by endothelial cells (Fig. 4C). Our data suggests that MMP14+_Myeloid_Cell is a crucial cellular entity in the modulation of glioblastoma immunity and angiogenesis.

Fig. 4.

Fig. 4

A-B. Cell interaction networks show that MMP14+_Myeloid_Cell mainly acts on myeloid cells, T cells, and endothelial cells; C. Signaling and reception modes were demonstrated. MMP14+_Myeloid_Cells were the most critical signaling and receiving cell populations, with CCL being the most contributing signaling pathway, and VISFATIN, VEGF, and TGFb representing signaling pathways sent by myeloid cells acting on endothelial cells.

3.4. The CCL signaling pathway has a significant correlation with the immunosuppressive microenvironment of glioblastoma

We began by exploring how various cell populations contribute to CCL signaling. Our analysis showed that the MMP14+_Myeloid_Cell was the dominant emitter of CCL signals, while MMP14-_Myeloid_Cell(2), MMP14+_Myeloid_Cell, and T_Cell were the main receivers of these signals (Fig. 5 A). Within the CCL signaling axis, CCL3-CCR5, CCL4-CCR5, and CCL3-CCR1 stood out as the three most prominent ligand-receptor pairs (Fig. 5B). We constructed a specific cellular interaction map for these pairs, revealing that the CCL3-CCR1 interaction was chiefly propagated by MMP14+_Myeloid_Cell targeting both myeloid cells and T cells (Fig. 5C–E).

Fig. 5.

Fig. 5

A. The primary transmitter of CCL signal is MMP14+_Myeloid_Cell, and the main recipients are myeloid cells and T cells; B. The most significant contributors to the CCL signaling pathway are CCL3-CCR5, CCL4-CCR5, and CCL3-CCR1; C-E. Analysis of the interaction intensity of CCL3-CCR5, CCL4-CCR5, and CCL3-CCR1 in cell-cell communication reveals that CCL3-CCR1 is mainly sent by MMP14+_Myeloid_Cell, acting on MMP14-_Myeloid_Cell(2) and T cells; F-J. The correlation between MMP14, CCR1, CD274, and CTLA4 was validated in RNA-seq data. K–N. Correlation between CCR1 and CD274; O. Calculation of the proportion of immune-infiltrating cells in samples using the CIBERSORT method. The CCR1-high expression group had a higher proportion of M2 cells and a lower proportion of CD8+ T cells.

Turning to gene expression analysis within the TCGA and GSE43388 datasets, we observed that the group with high MMP14 expression had increased levels of CCR1, CD274, and CTLA4. Similarly, elevated CCR1 expression was distinguished in the high-expression group in GSE61335 and GSE74187. However, no significant difference in CCR1, CD274, and CTLA4 expression was found between the groups in the GSE107850 dataset (Fig. 5F–J). A notable correlation between CCR1 and CD274 was detected in the TCGA, GSE43388, and GSE107850 datasets (r = 0.62, r = 0.51, r = 0.23), although this correlation was absent in the GSE74187 dataset (Fig. 5K–N).

Further investigating CCR1's association with immune cell presence, we used the CIBERSORT method to estimate immune cell infiltration in RNA-seq datasets. Within the CCR1 high-expression cohorts, there was a significantly higher proportion of M2 macrophages in TCGA, GSE61335, GSE74187, and GSE107850. This was coupled with a decreased fraction of CD8+ T cells and CD4+ T helper cells in TCGA, GSE61335, and GSE43388, and a reduced percentage of regulatory T cells in GSE61335 and GSE74187 (Fig. 5 O).

From these results, we propose that the CCL3-CCR1 pathway may foster M2 macrophage differentiation and play a vital role as an immunosuppressive mechanism that complements glioma immune checkpoint pathways. Therefore, targeting CCR1, along with immune checkpoint signals, could emerge as an effective immunotherapeutic approach.

3.5. MMP14+_Myeloid_Cell induces EndMT through VISFATIN, VEGF, and TGFb signaling pathways, promoting glioma angiogenesis

Our analysis revealed that within the signaling pathways impacting endothelial cells, the VISFATIN, VEGF, and TGF-beta pathways stood out as the most influential, with the MMP14+_Myeloid_Cell population playing a pivotal role as the principal sender (Fig. 6 A). Examining the key ligand-receptor interactions for each pathway, we uncovered that NAMPT-(ITGA5+ITGB1) notably drove the VISFATIN signaling pathway, VEGFB-VEGFR1 dominated the VEGF signaling pathway, and TGFB1-(TGFBR1+TGFBR2) was central to the TGF-beta signaling pathway (Fig. 6B–D). A closer look at the interaction dynamics revealed that NAMPT-(ITGA5+ITGB1) was primarily transmitted by myeloid cells and targeted endothelial cells (Fig. 6 E). Myeloid and endothelial cells engaged through VEGFB-VEGFR1, with endothelial cells also having strong interactions with one another via the same pathway (Fig. 6 F). Both the MMP14+_Myeloid_Cell and MMP14-_Myeloid_Cell(2) communicated with endothelial cells through TGFB1-(TGFBR1+TGFBR2); interestingly, this interaction was bidirectional with endothelial cells also targeting MMP14+_Myeloid_Cell (Fig. 6 G).

Fig. 6.

Fig. 6

A. The transmission and reception modes of VISFATIN, VEGF, and TGFb signals are presented; B-D. The contributions of the ligand-receptor pairs in the three signaling pathways are demonstrated; E-G. Interaction networks of the most contributing ligand-receptor pairs are displayed; H. Endothelial cells concurrently highly express both endothelial cell markers (CDH5, PECAM1, TIE1, TEK, and VWF) and mesenchymal cell markers (TAGLN, VIM, FGFR1, TGFBR2, FN1, POSTN, MGP, BGN, COL4A1, COL4A2, and TIMP1); I-J. KEGG and GO enrichment analysis of endothelial cell marker genes.

VISFATIN and TGF-beta are known to be closely linked to EndMT and may encourage glioma angiogenesis by driving the EndMT process. In endothelial cells, high expression levels of endothelial markers (CDH5, PECAM1, TIE1, TEK, and VWF) were matched with equally high levels of mesenchymal markers (TAGLN, VIM, FGFR1, TGFBR2, FN1, POSTN, MGP, BGN, COL4A1, COL4A2, and TIMP1), implying the occurrence of EndMT in the endothelial cell population (Fig. 6H).

A gene function enrichment analysis on the 146 avg_log2FC > 1 marker genes identified in endothelial cells pinpointed the Focal adhesion and PI3K-AKT signaling pathways as the most significantly enriched pathways via KEGG. The GO terms were specially related to vascular formation (Fig. 6I and J). Both the Focal adhesion and PI3K-AKT signaling pathways have been implicated in the mesenchymal transformation process [19], solidifying the connection to the observed EndMT in endothelial cells responding to myeloid cell interaction.

The relationship of NAMPT and TGFB1 with angiogenesis and EndMT was further validated in RNA-seq datasets. We determined ssGSEA scores for the EndMT and sprouting angiogenesis gene sets. In the TCGA, GSE61335, and GSE74187 datasets, strong correlations emerged for NAMPT and TGFB1 with both angiogenesis and EndMT (Fig. 7A–L). In the GSE107850 dataset, NAMPT maintained a high correlation with these processes, whereas TGFB1 correlation was not observed (Fig. 7M–P). These findings suggest that NAMPT demonstrated notable expression variability in glioblastomas and in IDH1-mutant LGG. A survival analysis focusing on NAMPT within the GSE107850 dataset revealed that patients with high NAMPT expression had significantly shorter PFS times (p = 1.1e-3), corroborating NAMPT as an adverse prognostic indicator in cohorts treated with radiotherapy as well as those receiving temozolomide (TMZ) (p = 3.8e-3, p = 7.0e-3).

Fig. 7.

Fig. 7

A-D. Correlations between NAMPT and TGFB1 with angiogenesis and EndMT in the TCGA dataset; E-H. Correlations between NAMPT and TGFB1 with angiogenesis and EndMT in the GSE61335 dataset; I-L. Correlations between NAMPT and TGFB1 with angiogenesis and EndMT in the GSE74187 dataset; M-P. Correlations between NAMPT and TGFB1 with angiogenesis and EndMT in the GSE107850 dataset; Q-S. In GSE107850, patients with high NAMPT expression exhibited worse responses to both radiotherapy and chemotherapy.

3.6. NAMPT and MMP14 can guide treatment strategy for recurrent glioma

The GSE72951 dataset, originating from a clinical study on recurrent gliomas, included 100 patients. Among them, 43 underwent a combination treatment of bevacizumab and lomustine (BC group), 33 received only bevacizumab (Beva group), and 34 were treated with lomustine monotherapy (CCNU group). The analysis showed no significant differences in overall survival across these three groups (Fig. 8 A). Additionally, NAMPT expression levels did not significantly influence survival (Fig. 8 B). Considering NAMPT's association with glioma angiogenesis, patients were categorized into high-expression (NAMPT_H group) and low-expression (NAMPT_L group) groups to examine the impact of NAMPT expression on bevacizumab efficacy.

Fig. 8.

Fig. 8

A. In the GSE72951 dataset, there were no significant overall survival differences between the Beva group, CCNU group, and BC group; B. NAMPT expression levels did not affect overall survival in GSE72951 patients; C-D. In NAMPT low-expression patients, the efficacy of Beva was significantly lower than that of BC (p = 0.01) and CCNU (p = 0.09), while in NAMPT high-expression patients, there were no significant differences in efficacy among the three groups; E-G. In the CCNU treatment group, patients with high NAMPT expression had worse treatment responses, while in the Beva group, high NAMPT expression was associated with better treatment responses; H. In the Beva group, VEGFA expression levels did not affect treatment responses; I-J. No significant survival differences were found among the three treatment modalities when MMP14 expression was low, but the BC group demonstrated better treatment outcomes with high MMP14 expression; K-L. There was no significant difference in survival between the three treatment modalities when CCR1 expression was low, but there was a trend toward benefit for the BC group when CCR1 was highly expressed.

In the NAMPT_L group, overall survival (OS) was shorter for patients in the Beva group compared to the other two groups (BC vs Beva: p = 0.01, CCNU vs Beva: p = 0.09) (Fig. 8C). For the NAMPT_H group, there was no significant survival difference among the three treatment groups (Fig. 8 D). Within the CCNU group, a notably higher risk of death was associated with high NAMPT expression (HR = 2.18, p = 0.04) (Fig. 8 E). In contrast, in the Beva group, high NAMPT expression significantly reduced the risk of death (HR = 0.49, p = 0.05) (Fig. 8 F). In the BC group, higher NAMPT expression tended to increase the risk of death, although this was not statistically significant (HR = 1.41, p = 0.29) (Fig. 8 G). Despite the known association of VEGFA with angiogenesis, bevacizumab did not significantly benefit patients with high VEGFA expression (Fig. 8H). These results suggest that high NAMPT expression may indicate resistance to chemotherapy but increased sensitivity to bevacizumab, potentially serving as a predictive marker for bevacizumab responsiveness.

We also analyzed MMP14 and CCR1 expression in relation to treatment outcomes. Patients with high MMP14 expression showed improved survival in the BC group (Beva vs BC: p = 0.02, CCNU vs BC: p = 0.04), while no significant survival difference was observed among the treatment groups for patients with low MMP14 expression (Fig. 8I and J). In patients with high CCR1 expression, there was an observable trend of improved outcomes in the BC group, but it did not reach statistical significance (Fig. 8K and L).

4. Discussion

Tumor progression and metastasis are driven by intricate interactions among malignant tumor cells and the non-malignant stromal components, including the extracellular matrix (ECM), mesenchymal cells such as endothelial cells, fibroblasts, and various infiltrating immune cells [20,21]. MMPs, zinc-dependent endopeptidases produced by both non-malignant mesenchymal cells and tumor cells, play a critical role in degrading and remodeling the ECM. This process is pivotal in regulating tumor growth, invasion, angiogenesis, and metastasis, ultimately facilitating tumor progression [[22], [23], [24]]. In our study, we calculated the GSVA scores for MMPs in the samples and conducted survival analyses. Our findings indicated that patients with higher MMP levels had shorter survival times. Subsequent univariate and multivariate analyses revealed that while most MMPs were associated with poor prognosis, a few (MMP16, MMP24, MMP15, and MMP17) were indicative of a better prognosis. Notably, MMP14 emerged as the sole MMP acting as an independent prognostic factor in both LGG and HGG.

MMP14, uniquely a membrane-bound collagenase with various extracellular activities [25], plays a significant role in degrading ECM proteins, regulating cytokine release, and altering the expression of cell surface receptors. These activities critically influence the integrity of the ECM and the phenotype and behavior of cells within the matrix, thereby promoting metastasis [26]. MMP14 is also known for its ability to activate other MMPs, including MMP2, MMP9, and MMP13 [27,28], and is broadly expressed in various cell types, particularly in malignant cancer cells, where its presence correlates with a poor prognosis [29]. In this study, we confirmed the prognostic significance of MMP14 across multiple independent datasets. However, in the GSE107850 dataset, although the high-expression MMP14 group showed a tendency towards a worse prognosis, the results were not statistically significant. This might be attributed to the dataset's focus on LGG samples with IDH1 mutations. Further single-cell analysis revealed that MMP14 was predominantly expressed in myeloid cells of recurrent glioblastoma, with notably lower levels in LGG. This suggests that the heterogeneity of MMP14 expression in LGG samples might be limited, thereby rendering it challenging to discern prognostic differences.

CC chemokines play a pivotal role in cell-to-cell communication and significantly impact the functionality of the tumor microenvironment (TME) [30]. Chemokines such as CCL2, 3, 4, 5, 7, and 21 are prevalent in most malignancies, where they regulate the composition of tumor-infiltrating myeloid cells [31]. Studies have identified CCR2 as the primary receptor for myeloid cell recruitment [32], while CCL3 and CCL4 receptors (CCR1 and CCR5) are linked with increased Myeloid-derived Suppressor Cell (MDSC) levels [[33], [34], [35]]. Additionally, the CCL21 receptor, CCR7, is known to facilitate the formation of intratumor-tolerated lymph node nodule structures [36]. In our study, we found that the most significant cell-to-cell communication pathway was the CCL signaling pathway, primarily mediated by MMP14+_Myeloid_Cell. Further analysis pinpointed CCL3-CCR5, CCL4-CCR5, and CCL3-CCR1 as the key ligand-receptor pairs in this pathway, suggesting that MMP14+_Myeloid_Cell might exhibit MDSC characteristics. Our investigation into the correlation between MMP14, CCR1, and PD-L1 revealed a positive association among these factors across multiple datasets. High MMP14 expression often coincided with elevated CCR1 levels; similarly, CCR1 and PD-L1 showed a positive correlation. In our immune cell infiltration analysis, M2 macrophages emerged as the predominant immune cells in gliomas, with an increased proportion in samples expressing high CCR1 levels. These findings underscore the critical role of immunosuppressive myeloid cells in the glioma immune microenvironment and highlight CCR1 as a potential therapeutic target. Consequently, we propose that a combined inhibition of PD-L1 and CCR1 could be an effective clinical treatment strategy.

Bevacizumab, widely used in recurrent glioma treatment, requires precise selection of responsive patient populations. During angiogenesis, tip endothelial cells undergoing the sprouting process lose their apical-basal polarity, leading to diminished cell interactions and augmented degradation of the extracellular matrix. Concurrently, these cells transition towards a mesenchymal phenotype (EndMT), crucial for blood vessel formation, while retaining some endothelial characteristics for maintaining cell interactions [37,38]. Our study discovered that in gliomas, the most potent signals influencing endothelial cells—VISFATIN, VEGF, and TGFb—primarily originate from myeloid cells. Visfatin, also known as NAMPT, is synthesized and secreted by tumor myeloid cells, promoting angiogenesis by targeting endothelial cells [39]. In breast cancer, NAMPT has been shown to facilitate EndMT [40]. TGFb signaling, a critical inducer of tumor EndMT [41], has been proven to trigger EndMT in various cancers, including melanoma, esophageal, colon, and lung cancers [[42], [43], [44], [45]]. Our hypothesis that these myeloid cell-regulated endothelial cells might be in a state of EndMT was confirmed upon finding that these cells highly express both endothelial and mesenchymal cell-specific genes. In multiple independent samples, we established a strong link between NAMPT and TGFB1 with angiogenesis and EndMT. Interestingly, NAMPT showed high correlations with these processes even in the GSE107850 dataset, representing LGG with IDH1 mutations and exhibiting lower heterogeneity than other datasets. Survival analysis indicated NAMPT's potential in predicting prognosis in these patients, underscoring the clinical importance of angiogenesis in LGG with IDH1 mutations. Although bevacizumab is an anti-VEGFA monoclonal antibody, high VEGFA expression in the GSE72951 dataset did not enhance its efficacy. However, patients with elevated NAMPT expression exhibited chemotherapy resistance but benefited more from bevacizumab. In contrast, in the low NAMPT expression group, bevacizumab's efficacy was significantly lower than chemotherapy. These observations suggest NAMPT as a clinical indicator for guiding bevacizumab use in glioma.

From a clinical perspective, treatment for more aggressive tumors often demands greater intensity. With this in mind, our examination of the GSE72951 dataset focused on the strategic use of bevacizumab in conjunction with chemotherapy. Our investigation primarily sought to determine if MMP14 and CCR1 levels could inform the decision to pursue combination therapies. Notably, both MMP14 and CCR1 show strong associations with MMP14+_Myeloid_Cell, a cell type predominantly arising from recurrent glioblastoma and representing the most functional subset within cell populations. Our findings revealed that patients expressing high levels of MMP14 achieved longer overall survival when subjected to the combined regimen rather than monotherapy. Leveraging both NAMPT and MMP14, we devised a streamlined and viable decision-making framework. For patients with recurrent glioblastoma, the initial consideration should be the level of MMP14: those with high MMP14 expression are suggested to receive a combination of bevacizumab and chemotherapy, whereas those exhibiting low MMP14 but high NAMPT levels should be considered for either bevacizumab or chemotherapy alone, and chemotherapy is the recommended course for patients with low levels of NAMPT (Fig. S2).

In this study, although we recognize the potential importance of CCR1 in the realm of immunotherapy, confirming its clinical value in glioma treatment has been challenged by the scarcity of immunotherapy data specific to gliomas.

Funding

This work was supported by the Basic and Applied Basic Research Fund of Guangdong Province (2022A1515012387).

Data availability statement

Data is included in article/supp. material/referenced in article.

CRediT authorship contribution statement

Wei Luo: Writing – original draft, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. Qi Quan: Writing – review & editing, Supervision. Zihao Xu: Funding acquisition. Jinju Lei: Writing – review & editing. Roujun Peng: Writing – review & editing, Supervision, Funding acquisition.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

The authors would like to thank the staff members of TCGA Research Network and GEO database and all the authors for making their valuable research data public.

Abbreviation

MMPs

Matrix metalloproteinases

EndMT

Endothelial-mesenchymal Transformation

GSVA

Gene Set Variation Analysis

ssGSEA

Single-sample Gene Set Enrichment Analysis

CCL

C–C Motif Chemokine Ligand

CCR1

C–C Motif Chemokine Receptor 1

PD-L1

Programmed Death Ligand 1

TGFb

Transforming Growth Factor Beta

VEGF

Vascular Endothelial Growth Factor

NAMPT

Nicotinamide Phosphoribosyltransferase

TME

Tumor Microenvironment

VEGFA

Vascular Endothelial Growth Factor A

PFS

Progression-free Survival

OS

Overall Survival

bFGF

Basic Fibroblast Growth Factor

TGF-β

Transforming Growth Factor Beta

LGG

Low-grade Gliomas

HGG

High-grade Glioma

TCGA

The Cancer Genome Atlas

GEO

Gene Expression Omnibus

nGBM

Newly Diagnosed Glioblastoma

rGBM

Relapsed Glioblastoma

KEGG

Kyoto Encyclopedia of Genes and Genomes

GO

Gene Ontology

K-M

Kaplan-Meier

CTLA4

Cytotoxic T-Lymphocyte Associated Protein 4

ECM

Extracellular Matrix

MDSC

Myeloid-derived Suppressor Cells

TMZ

Temozolomide

Footnotes

Appendix A

Supplementary data to this article can be found online at https://doi.org/10.1016/j.heliyon.2024.e26859.

Appendix A. Supplementary data

The following are the Supplementary data to this article:

figs1.

figs1

figs2.

figs2

References

  • 1.Hadjipanayis C.G., Van Meir E.G. Tumor initiating cells in malignant gliomas: biology and implications for therapy. J. Mol. Med. (Berl.) 2009;87(4):363–374. doi: 10.1007/s00109-009-0440-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Osuka S., Van Meir E.G. Overcoming therapeutic resistance in glioblastoma: the way forward. J. Clin. Invest. 2017;127(2):415–426. doi: 10.1172/JCI89587. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Goenka A., et al. The many facets of therapy resistance and tumor recurrence in glioblastoma. Cells. 2021;10(3) doi: 10.3390/cells10030484. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Vitale I., et al. Macrophages and metabolism in the tumor microenvironment. Cell Metabol. 2019;30(1):36–50. doi: 10.1016/j.cmet.2019.06.001. [DOI] [PubMed] [Google Scholar]
  • 5.Cole A.P., et al. Microglia in the brain tumor microenvironment. Adv. Exp. Med. Biol. 2020;1273:197–208. doi: 10.1007/978-3-030-49270-0_11. [DOI] [PubMed] [Google Scholar]
  • 6.Chinot O.L., et al. Bevacizumab plus radiotherapy-temozolomide for newly diagnosed glioblastoma. N. Engl. J. Med. 2014;370(8):709–722. doi: 10.1056/NEJMoa1308345. [DOI] [PubMed] [Google Scholar]
  • 7.Weller M., et al. EANO guidelines on the diagnosis and treatment of diffuse gliomas of adulthood. Nat. Rev. Clin. Oncol. 2021;18(3):170–186. doi: 10.1038/s41571-020-00447-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Yi M., et al. Combination strategies with PD-1/PD-L1 blockade: current advances and future directions. Mol. Cancer. 2022;21(1):28. doi: 10.1186/s12943-021-01489-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Blumenthal D.T., et al. Pembrolizumab: first experience with recurrent primary central nervous system (CNS) tumors. J. Neuro Oncol. 2016;129(3):453–460. doi: 10.1007/s11060-016-2190-1. [DOI] [PubMed] [Google Scholar]
  • 10.Bouffet E., et al. Immune checkpoint inhibition for hypermutant glioblastoma multiforme resulting from germline biallelic mismatch repair deficiency. J. Clin. Oncol. 2016;34(19):2206–2211. doi: 10.1200/JCO.2016.66.6552. [DOI] [PubMed] [Google Scholar]
  • 11.Cloughesy T.F., et al. Neoadjuvant anti-PD-1 immunotherapy promotes a survival benefit with intratumoral and systemic immune responses in recurrent glioblastoma. Nat. Med. 2019;25(3):477–486. doi: 10.1038/s41591-018-0337-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Reardon D.A., et al. Effect of nivolumab vs bevacizumab in patients with recurrent glioblastoma: the CheckMate 143 phase 3 randomized clinical trial. JAMA Oncol. 2020;6(7):1003–1010. doi: 10.1001/jamaoncol.2020.1024. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Murphy G., Nagase H. Progress in matrix metalloproteinase research. Mol. Aspect. Med. 2008;29(5):290–308. doi: 10.1016/j.mam.2008.05.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Niland S., Riscanevo A.X., Eble J.A. Matrix metalloproteinases shape the tumor microenvironment in cancer progression. Int. J. Mol. Sci. 2021;23(1) doi: 10.3390/ijms23010146. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Khokha R., Murthy A., Weiss A. Metalloproteinases and their natural inhibitors in inflammation and immunity. Nat. Rev. Immunol. 2013;13(9):649–665. doi: 10.1038/nri3499. [DOI] [PubMed] [Google Scholar]
  • 16.Tombor L.S., et al. Single cell sequencing reveals endothelial plasticity with transient mesenchymal activation after myocardial infarction. Nat. Commun. 2021;12(1):681. doi: 10.1038/s41467-021-20905-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Clere N., Renault S., Corre I. Endothelial-to-Mesenchymal transition in cancer. Front. Cell Dev. Biol. 2020;8:747. doi: 10.3389/fcell.2020.00747. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Abdelfattah N., et al. Single-cell analysis of human glioma and immune cells identifies S100A4 as an immunotherapy target. Nat. Commun. 2022;13(1):767. doi: 10.1038/s41467-022-28372-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Luo J., et al. 14, 15-EET induces breast cancer cell EMT and cisplatin resistance by up-regulating integrin alphavbeta3 and activating FAK/PI3K/AKT signaling. J. Exp. Clin. Cancer Res. 2018;37(1):23. doi: 10.1186/s13046-018-0694-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Hanahan D., Weinberg R.A. Hallmarks of cancer: the next generation. Cell. 2011;144(5):646–674. doi: 10.1016/j.cell.2011.02.013. [DOI] [PubMed] [Google Scholar]
  • 21.Hanahan D., Coussens L.M. Accessories to the crime: functions of cells recruited to the tumor microenvironment. Cancer Cell. 2012;21(3):309–322. doi: 10.1016/j.ccr.2012.02.022. [DOI] [PubMed] [Google Scholar]
  • 22.Roy R., Yang J., Moses M.A. Matrix metalloproteinases as novel biomarkers and potential therapeutic targets in human cancer. J. Clin. Oncol. 2009;27(31):5287–5297. doi: 10.1200/JCO.2009.23.5556. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.de Almeida L.G.N., et al. Matrix metalloproteinases: from molecular mechanisms to physiology, pathophysiology, and pharmacology. Pharmacol. Rev. 2022;74(3):712–768. doi: 10.1124/pharmrev.121.000349. [DOI] [PubMed] [Google Scholar]
  • 24.Kessenbrock K., Plaks V., Werb Z. Matrix metalloproteinases: regulators of the tumor microenvironment. Cell. 2010;141(1):52–67. doi: 10.1016/j.cell.2010.03.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Knapinska A.M., Fields G.B. The expanding role of MT1-MMP in cancer progression. Pharmaceuticals. 2019;12(2) doi: 10.3390/ph12020077. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Shimoda M., et al. Stromal metalloproteinases: crucial contributors to the tumor microenvironment. Pathol. Int. 2021;71(1):1–14. doi: 10.1111/pin.13033. [DOI] [PubMed] [Google Scholar]
  • 27.Yamahana H., et al. TGF-beta 1 facilitates MT1-MMP-mediated proMMP-9 activation and invasion in oral squamous cell carcinoma cells. Biochem Biophys Rep. 2021;27 doi: 10.1016/j.bbrep.2021.101072. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Knauper V., et al. Cellular mechanisms for human procollagenase-3 (MMP-13) activation. Evidence that MT1-MMP (MMP-14) and gelatinase a (MMP-2) are able to generate active enzyme. J. Biol. Chem. 1996;271(29):17124–17131. doi: 10.1074/jbc.271.29.17124. [DOI] [PubMed] [Google Scholar]
  • 29.Gifford V., Itoh Y. MT1-MMP-dependent cell migration: proteolytic and non-proteolytic mechanisms. Biochem. Soc. Trans. 2019;47(3):811–826. doi: 10.1042/BST20180363. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Korbecki J., et al. CC chemokines in a tumor: a review of pro-cancer and anti-cancer properties of the ligands of receptors CCR1, CCR2, CCR3, and CCR4. Int. J. Mol. Sci. 2020;21(21) doi: 10.3390/ijms21218412. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Nagarsheth N., Wicha M.S., Zou W. Chemokines in the cancer microenvironment and their relevance in cancer immunotherapy. Nat. Rev. Immunol. 2017;17(9):559–572. doi: 10.1038/nri.2017.49. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Flores-Toro J.A., et al. CCR2 inhibition reduces tumor myeloid cells and unmasks a checkpoint inhibitor effect to slow progression of resistant murine gliomas. Proc. Natl. Acad. Sci. U.S.A. 2020;117(2):1129–1138. doi: 10.1073/pnas.1910856117. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Blattner C., et al. CCR5(+) myeloid-derived suppressor cells are enriched and activated in melanoma lesions. Cancer Res. 2018;78(1):157–167. doi: 10.1158/0008-5472.CAN-17-0348. [DOI] [PubMed] [Google Scholar]
  • 34.Dyer D.P., et al. Chemokine receptor redundancy and specificity are context dependent. Immunity. 2019;50(2):378–389 e5. doi: 10.1016/j.immuni.2019.01.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Yang L., et al. Blockade of CCR5-mediated myeloid derived suppressor cell accumulation enhances anti-PD1 efficacy in gastric cancer. Immunopharmacol. Immunotoxicol. 2018;40(1):91–97. doi: 10.1080/08923973.2017.1417997. [DOI] [PubMed] [Google Scholar]
  • 36.Shields J.D., et al. Induction of lymphoidlike stroma and immune escape by tumors that express the chemokine CCL21. Science. 2010;328(5979):749–752. doi: 10.1126/science.1185837. [DOI] [PubMed] [Google Scholar]
  • 37.Welch-Reardon K.M., Wu N., Hughes C.C. A role for partial endothelial-mesenchymal transitions in angiogenesis? Arterioscler. Thromb. Vasc. Biol. 2015;35(2):303–308. doi: 10.1161/ATVBAHA.114.303220. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Potenta S., Zeisberg E., Kalluri R. The role of endothelial-to-mesenchymal transition in cancer progression. Br. J. Cancer. 2008;99(9):1375–1379. doi: 10.1038/sj.bjc.6604662. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Pylaeva E., et al. NAMPT signaling is critical for the proangiogenic activity of tumor-associated neutrophils. Int. J. Cancer. 2019;144(1):136–149. doi: 10.1002/ijc.31808. [DOI] [PubMed] [Google Scholar]
  • 40.Rajput P.K., Sharma J.R., Yadav U.C.S. Cellular and molecular insights into the roles of visfatin in breast cancer cells plasticity programs. Life Sci. 2022;304 doi: 10.1016/j.lfs.2022.120706. [DOI] [PubMed] [Google Scholar]
  • 41.Principe D.R., et al. TGF-beta: duality of function between tumor prevention and carcinogenesis. J. Natl. Cancer Inst. 2014;106(2):djt369. doi: 10.1093/jnci/djt369. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Krizbai I.A., et al. Endothelial-mesenchymal transition of brain endothelial cells: possible role during metastatic extravasation. PLoS One. 2015;10(3) doi: 10.1371/journal.pone.0123845. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Nie L., et al. Endothelial-mesenchymal transition in normal human esophageal endothelial cells cocultured with esophageal adenocarcinoma cells: role of IL-1 beta and TGF-beta 2. Am. J. Physiol. Cell Physiol. 2014;307(9):C859–C877. doi: 10.1152/ajpcell.00081.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Kim S.H., Song Y., Seo H.R. GSK-3 beta regulates the endothelial-to-mesenchymal transition via reciprocal crosstalk between NSCLC cells and HUVECs in multicellular tumor spheroid models. J. Exp. Clin. Cancer Res. 2019;38(1):46. doi: 10.1186/s13046-019-1050-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Wawro M.E., et al. Invasive colon cancer cells induce transdifferentiation of endothelium to cancer-associated fibroblasts through microtubules enriched in tubulin-beta 3. Int. J. Mol. Sci. 2018;20(1) doi: 10.3390/ijms20010053. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

Data is included in article/supp. material/referenced in article.


Articles from Heliyon are provided here courtesy of Elsevier

RESOURCES