Skip to main content
International Journal of Molecular Sciences logoLink to International Journal of Molecular Sciences
. 2020 Oct 15;21(20):7646. doi: 10.3390/ijms21207646

Combinatorial Effect of Cold Atmosphere Plasma (CAP) and the Anticancer Drug Cisplatin on Oral Squamous Cell Cancer Therapy

Chang-Min Lee 1,, Young-IL Jeong 1,, Min-Suk Kook 2,*, Byung-Hoon Kim 1,*
PMCID: PMC7588898  PMID: 33076565

Abstract

Cold atmospheric plasma (CAP) has been extensively investigated in the local treatment of cancer due to its potential of reactive oxygen species (ROS) generation in biological systems. In this study, we examined the synergistic effect of combination of CAP and cisplatin-mediated chemotherapy of oral squamous cell carcinoma (OSCC) in vitro. SCC-15 OSCC cells and human gingival fibroblasts (HGF-1) cells were treated with cisplatin, and then, the cells were irradiated with CAP. Following this, viability and apoptosis behavior of the cells were investigated. The viability of SCC-15 cells was inhibited by cisplatin with a dose-dependent manner and CAP treatment time. HGF-1 cells also showed decreased viability by treatment with cisplatin and CAP. Combination of 1 μM cisplatin plus 3 min of CAP treatment or 3 μM cisplatin plus 1 min of CAP treatment showed a synergistic anticancer effect with appropriate cytotoxicity against normal cells. ROS generation and dead cell staining were also increased by the increase in CAP treatment time. Furthermore, tumor-suppressor proteins and apoptosis-related enzymes also increased according to the treatment time of CAP. We showed the synergistic effect of cisplatin and CAP treatment against SCC-15 cells with low cytotoxicity against normal cells.

Keywords: cold atmospheric plasma, reactive oxygen species, squamous cell carcinoma, cisplatin, combination therapy

1. Introduction

Cold atmospheric plasma (CAP) has been extensively studied in the biomedical field in recent decades due to its biological and physicochemical properties [1,2,3,4]. Notably, CAP has recently been spotlighted in the local treatment of cancer because CAP in the biological system generates reactive oxygen species (ROS) and/or reactive nitrogen species (RNS), i.e., excessive ROS or RNS generation by CAP treatment induces oxidative stress against cancer cells and subsequently results in apoptotic death [5,6,7,8]. These unique properties of CAP are regarded as a safe treatment option for cancer patients because it can be applied to the specific body site of action without systemic side-effects against the human body [9,10,11,12]. For example, Kurake et al. reported that hydrogen peroxide and nitrous ion in the plasma-irradiated culture medium selectively induced the apoptotic death of glioblastoma tumor cells, but they did not significantly affect to the viability of normal mammary epithelial cells [7]. ROS/RNS generated by CAP treatment triggers molecular signaling pathways and promotes mitochondrial perturbation, leading to apoptosis [8]. Otherwise, plasma-activated medium (PAM) is known to induce immunogenic cell death by activation of the innate immune system [13]. Furthermore, Shi et al. reported that CAP treatment against cancer cells induces enriched p53 pathway in SCC-15 cells [14]. Despite these advantages of CAP treatment, some drawbacks have been reported [15,16,17]. For example, CAP treatment aroused focal mucosal erosion with superficial ulceration and necrosis with inflammatory reaction, even though these mucosal damages healed within a few days [15]. Since the free radicals induced by non-thermal plasma (NTP) have the danger of the hemolytic activity against red blood cells, the mild condition of ROS/RNS induction by CAP is required for safe treatment on human diseases [16]. Furthermore, cellular mutagenic changes induced by CAP treatment should be also considered to find precise conditions for safe treatment modality [17].

Otherwise, the efficacy of cancer cell treatment with CAP can be improved by co-treatment with gold nanoparticles, since gold nanoparticles have a synergistic effect on the production of ROS/RNS in the treatment of the cancer cells by the NTP system [18,19]. Moniruzzaman et al. reported that co-treatment of CAP and X-irradiation using sulfasalazine generates hydroxyl radicals 24 times higher than X-irradiation only, and then accelerates apoptotic death of human leukemia Molt-4 cells [20]. Furthermore, Lee et al. reported that CAP treatment sensitizes tamoxifen-resistant MCF-7 cells against tamoxifen treatment and overcomes drug resistance [21]. The efficacy of CAP should be improved to maximize the curative value of human diseases with minimized side-effects against normal tissues.

Cisplatin, which is a metal-based anticancer drug, has been extensively used as a chemotherapy medication for various kinds of cancers such as ovarian cancer, head and neck cancer, cervical cancer, breast cancer, bladder cancer, esophageal cancer, lung cancer, brain tumors and neuroblastoma [22,23,24,25,26,27,28,29]. Cisplatin works by interfering with DNA replication and inhibits the growth of cancer cells [22]. Although cisplatin has been used for cancer patients for several decades, severe side-effects such as bone marrow suppression, hearing problem, electrolyte disorders, nephrotoxicity and cardiotoxicity still inhibit its clinical application and degrade the quality of patients’ lives [30,31,32]. From these points of view, various regimens have been investigated to overcome these drawbacks [33,34,35,36]. For example, the electroporation significantly improves anticancer activity of cisplatin against metastatic pancreatic cancer [33,34]. Short electric pulses against cancer cells increase transitional permeabilization of the drug molecule through the cellular membrane and improve anticancer activity. Furthermore, polymeric micelles or nanoparticles incorporating cisplatin significantly improve the anticancer activity of cisplatin against cancer cells [35,36]. Nishiyama et al. reported that polymeric micelles remarkably improve blood circulation of cisplatin and effectively accumulated in a solid tumor following complete regression of the solid tumor [35].

In this study, we investigated the combinatorial effect of CAP and cisplatin against squamous cell carcinoma cells in vitro. Since cisplatin also induces oxidative damage both in normal cells and cancer cells, oxidative stress can be maximized by the combination of CAP and cisplatin against cancer cells. Furthermore, this treatment regimen may reduce the adverse effects against normal cells through local treatment of cancer cells. We studied the synergistic effect of the CAP/cisplatin combination on ROS generation, apoptosis and anticancer activity using squamous cell carcinoma cells and normal cells.

2. Results and Discussion

The ROS producing capacity of CAP is known to induce various biological activities [1,2,3,4,5]. The ROS/RNS generated in the biological system by CAP can be applied for specific killing of cancer cells. Notably, CAP is regarded as a promising candidate for local irradiation of superficial carcinoma such as squamous cell carcinoma because CAP application in cancer is limited to top cell layers, i.e., the effective depth of tissue penetration of CAP treatment is less than 100 μm [37]. Otherwise, the delivery of anticancer drugs via systemic administration against superficial carcinoma such as skin cancer and oral squamous cell carcinoma has limitations in clinical applications, and very low fractions of administered drug can be reached in the tumor region [38,39]. These delivery problems of anticancer drugs such as cisplatin amplify various adverse effects on the human body [28,29,30,31,32,33,34]. From these points of view, we designed a combination of CAP treatment and cisplatin to amplify the anticancer activity of cisplatin and to minimize the adverse effects against normal cells.

2.1. Optical Emission Spectroscopy (OES) Analysis

For the treatment of cancer cells using CAP, the distance between the nozzle and media in a 12-well plate was adjusted to 3 cm, and the flow rate of Ar gas was adjusted to 5 L/min (Figure 1A,B).

Figure 1.

Figure 1

(A) Schematic diagram of cold atmospheric plasma (CAP) equipment for the treatment of cells. (B) Photograph of the CAP nozzle. Flow rate of Ar gas was 5 L/min. Discharge power was adjusted at 45 W, at 8.5 kV, and 20 kHz radio frequency at atmospheric condition. The distance was 3 cm from the nozzle to the cells in a 12-well plate.

For analysis of the plasma jet plume from CAP devices, OES was employed, and the measurement result is shown in Figure 2. This result indicated that the specific peaks of Ar+, OH and O- ions were observed in the UV range (200 nm–400 nm) and visible range (690 nm–950 nm). Furthermore, this result indicates that CAP equipment in this study successively produces a plasma jet from Ar gas. Many scientists reported the curative efficacy of Ar plasma jets against various disease models, including wound healing, bacterial infection and cancer [40,41,42]. Cheng et al. reported that the Ar plasma jet efficiently decreases new blood vessels in streptozotocin-induced diabetic rats [40]. Furthermore, Lou et al. also reported that the Ar plasma jet efficiently destroys the bacteria cell wall and then induces successful killing of bacteria [41].

Figure 2.

Figure 2

Optical emission spectrum (OES) measurement of the CAP plume. An optical fiber was measured from the position of 3 mm along the axial direction of the CAP plume.

2.2. Single Treatment of Cisplatin or CAP against HGF-1 and SCC-15 Cells

The effect of sole treatment of cisplatin or CAP against SCC-15 cells and HGF-1 cells was examined in vitro, as shown in Figure 3. The viability of SCC-15 cells and HGF-1 cells was gradually decreased according to the increase in cisplatin concentration and CAP treatment time. Interestingly, SCC-15 cells showed lower viability in the sole treatment of cisplatin compared to HGF-1 cells, as shown in Figure 3A. That is, the viability of SCC-15 cells was lower than 50% at 3 μM cisplatin, while HGF-1 cells showed higher than 90% in cell viability at the same concentration. Moreover, SCC-15 cells resulted in lower viability by CAP treatment compared to HGF-1 cells (Figure 3B). As shown in Figure 3B, the viability of SCC-15 cells was less than 50% at a treatment time of 3 min (180 s), while more than 70% of HGF-1 cells were viable at the same concentration. These results indicated that HGF-1 cells properly tolerated the treatment of cisplatin or CAP when compared with SCC-15 cells. As shown in Table 1, the IC50 values of HGF-1 and SCC-15 cells by treatment of cisplatin were 16.0 μM and 2.8 μM, respectively. Furthermore, the viability of SCC-15 cells was also gradually decreased according to the CAP treatment time. As shown in Table 1, the IC50 value of HGF-1 and SCC-15 cells by treatment of CAP was approximately higher than 300 s and 173 s, respectively. Interestingly, the IC50 value of HGF-1 cells was higher than that of SCC-15 cells.

Figure 3.

Figure 3

The viability of SCC-15 and HGF-1 cells. (A) The effect of cisplatin treatment; (B) CAP treatment time.

Table 1.

IC50 of cisplatin and CAP treatment against SCC-15 cells.

IC50
HGF Cells SCC-15 Cells
Cisplatin 16.0 μM 2.8 μM
CAP treatment > 300 s 173 s

2.3. Combinatorial Treatment of Cisplatin and CAP against HGF-1 and SCC-15 Cells

From the results of the viability of cells, the optimal combination of cisplatin concentration and CAP treatment time was studied. As shown in Figure 4, 1 μM or 3 μM of cisplatin was combined with various treatment times of CAP to find the maximal anticancer activity against cancer cells and the minimal cytotoxicity against normal cells. Practically, higher than 80% of HGF-1 cells was viable at 5.0 μM (Figure 3A). As shown in Figure 4, the cells responded to CAP treatment time, and the viability of cells gradually decreased according to the increase in CAP treatment time when the various treatment times of CAP were combined with 1 μM or 3 μM cisplatin. Even though the viability of HGF-1 cells also decreased according to the treatment time of CAP, their viability was higher than those of SCC-15 cells at all treatment regimens. When 1 min treatment of CAP was combined with 1 μM cisplatin, less than 60% of SCC15 cells were viable, while more than 80% of HGF-1 cells were viable at same treatment regimen. One-minute treatment of CAP combined with 3 μM cisplatin induced a decrease in viability of SCC-15 cells of less than 40%, while more than 70% of HGF-1 cells were viable at same treatment modality. Furthermore, the viability of HGF-1 cells at 3 μM cisplatin was not significantly changed compared to the results of 1 μM cisplatin, while the viability of SCC-15 cells was significantly decreased at 3 μM cisplatin compared to 1 μM cisplatin. These results indicated that SCC-15 cells are more sensitive to cisplatin treatment compared to HGF-1 cells. Even though the higher dose of cisplatin in combination with CAP resulted in stronger antitumor activity, the lower dose of cisplatin combined with the longer treatment time of CAP is preferred for the treatment of epithelial cancer, because CAP brings only about local irritations in the normal tissues, while cisplatin induces systemic side-effects against the human body.

Figure 4.

Figure 4

The effect of CAP exposure time on the cell viability of HGF-1 (■) and SCC-15 (□). (A) Cisplatin, 1 μM; (B) cisplatin, 3 μM. Cisplatin at a dose of 1.0 μM or 3.0 μM was exposed to cells for 6 h, and then media was replaced with fresh media. These were then irradiated with CAP for various periods of time. *, **: p < 0.01.

Figure 5 also supports the results of Figure 3 and Figure 4. Even though the morphology and the density of HGF-1 cells were changed at 3 μM cisplatin plus 3 min of CAP treatment, the cell density and the morphology of HGF-1 cells were relatively maintained at 1 μM cisplatin, 1 μM cisplatin plus 3 min of CAP treatment and 3 μM cisplatin, as shown in Figure 5A. Otherwise, SCC-15 cells showed greater changes in the morphology and the cell density by treatment of cisplatin and/or CAP. At 3 μM cisplatin plus 3 min of CAP treatment, the cell density and the morphology of SCC-15 cells were significantly decreased and changed, as shown in Figure 5B. These results supported the results of Figure 4. Pasqual-Melo et al. also reported the anticancer efficacy of plasma jets using a kINPen argon plasma jet [42]. In their report, the plasma jet revealed higher cytotoxicity in malignant cells such as SCC-13 and A431 cells but not in non-malignant HaCaT keratinocytes. We also obtained similar results from Ar plasma treatment against non-malignant HGF-1 cells and malignant SCC-15 cells, as shown in Figure 4 and Figure 5.

Figure 5.

Figure 5

Observation of changes in morphology and density. (A) HGF-1 cells; (B) SCC-15 cells. The effect of 1 μM and 3 μM cisplatin concentration with 3 min of CAP exposure. (Magnification: 100×)

The live/dead cell staining of SCC-15 cells supported the results of Figure 4 and Figure 5. As shown in Figure 6, dead cells with a red color was gradually increased with the combination of cisplatin and CAP treatment compared to control treatment. The higher treatment time of CAP significantly decreased the live cells, while dead cells simultaneously increased compared to control or cisplatin only. Otherwise, the live cell staining in HGF-1 cells with a green color was relatively higher than dead cell staining (Figure S1), indicating that CAP treatment efficiently influences the viability of SCC-15 cells rather than HGF-1 cells. Figure 4, Figure 5 and Figure 6 show that the combination of cisplatin and CAP treatment induces the synergistic death of cancer cells with alleviated cytotoxicity against normal cells.

Figure 6.

Figure 6

Live and dead cell staining. SCC-15 cells and HGF-1 cells were treated with cisplatin and CAP, as described in Figure 4. Cells treated with cisplatin/CAP were harvested and then stained with a live/dead cell staining solution. Green and red represent live and dead cells, respectively. (Magnification: 40×)

2.4. ROS Generation and Apoptosis/Necrosis Signals

Figure 7 shows the ROS generation from HGF-1 cells and SCC-15 cells by treatment with cisplatin and CAP. As expected, the higher CAP treatment time induced increased ROS production in both HGF-1 cells (Figure 7A,B) and SCC-15 cells (Figure 7C,D). Interestingly, ROS production of HGF-1 cells was not significantly changed between 1 μM and 3 μM cisplatin, while the ROS level was significantly changed in SCC-15 cells, as shown in Figure 7C,D. Notably, 1 μM cisplatin plus 1 min CAP or 3 μM cisplatin plus 1 min CAP resulted in higher than 300% or 500% increases in ROS levels in SCC-15 cells, respectively, while ROS generation in HGF-1 cells in same treatment option was controlled under 150% and 170%, respectively. These results indicate that CAP treatment against SCC-15 cancer cells is an efficient candidate for the generation of ROS and specific suppression of cancer cells.

Figure 7.

Figure 7

Synergistic combination effect of cisplatin and plasma on reactive oxygen species (ROS) generation. ROS generation of HGF-1 (A,B) and SCC-15 cells (C,D) after the CAP exposure (1, 2 and 3 min) with 1 mM (a and c) and 3 μM (b and d) of cisplatin.

Figure 8 shows the expression of apoptosis-related signals such as PTEN, cleaved caspase-9 (Cas-9) and p53 proteins of SCC-15 cells and HGF-1 cells. As shown in Figure 8, tumor-suppressor protein PTEN was gradually increased both in SCC-15 cells and HGF-1 cells, indicating that CAP treatment increases anti-tumor activity with expression of tumor-suppressor genes. Furthermore, apoptosis-related enzyme Cas-9 in SCC-15 cells was remarkably changed according to the CAP treatment of both 1 μM and 3 μM cisplatin, i.e., the higher treatment time of CAP gradually increases the expression intensity of Cas-9. Otherwise, the changes in Cas-9 expression intensity in HGF-1 cells were relatively not significant compared to SCC-15 cells. The expression intensity of p53 protein was also gradually increased by treatment with CAP in both SCC-15 cells and HGF-1 cells. These results indicated that CAP treatment in combination with cisplatin induces apoptosis of tumor cells and increases the tumor-suppressor protein, resulting in an increase in antitumor activity. Figure 9 shows the flow cytometric analysis of apoptosis/necrosis of SCC-15 cells. Apoptosis/necrosis of SCC-15 cells was gradually increased according to the increase in treatment time of CAP. Furthermore, the early/late apoptosis of cancer cells at 3 μM of cisplatin combined with CAP treatment was higher than 1 μM of cisplatin. Notably, late apoptosis was relatively higher at 3 min CAP treatment, while early apoptosis was evidently higher than late apoptosis at less than 2 min CAP treatment. With sole treatment of cisplatin, early apoptosis was dominant at 3 μM of cisplatin rather than late apoptosis, while its value at 1 μM of cisplatin was lower than late apoptosis. These results might be due to the fact that SCC-15 cells respond to cisplatin treatment during the initial 6 h, and then apoptosis/necrosis is accelerated following cell death. Xu et al. also reported that CAP increased caspase-9 activity of multiple myeloma and induced apoptosis/necrosis of cells [43]. Saadati et al. also reported that indirect/direct treatment of B16F10 melanoma cells using CAP induces apoptosis-related protein expression, such as BAX/BCL-2 [44]. They reported that CAP treatment greatly inhibited the viability of malignant cells in vitro and in vivo through apoptosis.

Figure 8.

Figure 8

Western blot for the analysis of apoptosis behavior of cells. SCC-15 cells and HGF-1 cells were treated with cisplatin and CAP as described in Figure 4. Cells treated with cisplatin/CAP were then used to analyze tumor suppressor proteins and apoptosis proteins using Western blotting.

Figure 9.

Figure 9

Flow cytometry of apoptosis/necrosis analysis of SCC-15 cells. SCC-15 cells treated with cisplatin/CAP as described in Figure 4 were then stained with fluorescein isothiocyanate (FITC)–annexin V for apoptosis and propidium iodide (PI) for necrosis. Apoptosis/necrosis analysis of SCC-15 cells using flow cytometry under CAP exposure and cisplatin.

The results of apoptosis analysis of cells indicated that CAP treatment efficiently induces apoptotic death of cancer cells. Kurake et al. also reported that plasma-activated medium (PAM) selectively killed glioblastoma cells but not in normal mammary epithelial cells [7]. They reported that ROS such as hydrogen peroxide (H2O2) and nitrous ion (NO2) were detected in the plasma-irradiated culture medium, and these components contributed to inducing apoptotic death of glioblastoma tumor cells, while the viability of normal mammary epithelial cells was properly maintained. Furthermore, Ikeda et al. also reported that PAM shows anticancer activity through inhibition of cancer-initiating cells [45]. They observed that PAM selectively induces apoptotic death of cancer cells but not in normal cells. Furthermore, they also obtained positive results from PAM/cisplatin combination therapy, i.e., the combination of PAM and cisplatin has synergistic cancer cell-killing activity when compared to PAM or cisplatin only. Our results also indicated that anticancer activity against SCC-15 cells can be emphasized by the combination of CAP and cisplatin. Notably, this combinatorial treatment dominantly inhibited viability of malignant SCC-15 cells with reduced cytotoxicity against non-malignant HGF-1 cells. Even though many scientists investigated the anticancer efficacy of CAP, we discovered the optimal treatment modality of the cisplatin/CAP combination for therapy of gingival carcinoma in vitro. Our treatment modality using cisplatin chemotherapy and CAP treatment may provide the optimal treatment strategy for gingival cancer, i.e., local treatment of CAP against gingival cancer must have a maximized cancer cell-killing capacity and minimized systemic side-effects of cisplatin. The increased expression of tumor-suppressor proteins and apoptosis-related proteins, resulting in apoptotic death of cancer cells, also verified the synergistic anticancer activity of the combination of cisplatin and CAP, as shown in Figure 8.

Even though the maximum anticancer activity was observed for the combination of 5 μM cisplatin plus 3 min of CAP treatment, the viability of HGF-1 cells also decreased, and, thus, the optimal combination of cisplatin/CAP should be determined. From these points of view, appropriate combinations can be selected to maximize anticancer activity with minimized cytotoxicity against normal cells. In our opinion, 1 μM cisplatin plus 3 min of CAP treatment or 3 μM cisplatin plus 1 min of CAP treatment are the preferred combination options rather than other treatment regimens, because the viability of cancer cells was sufficiently inhibited, while the viability of HGF-1 cells remained higher than 60%. Furthermore, local treatment of disease in the periodontal region is required to improve therapeutic efficacy because it is hard to reach the periodontal region with the systemic delivery of drugs [46]. Thus, the local treatment modality should be developed to maximize drug efficacy against the periodontal and gingival region [47,48]. Our finding provides an appropriate treatment strategy of gingival cancer using a local application of CAP with systemic chemotherapy of cisplatin.

3. Materials and Methods

3.1. Materials

Cisplatin, 3-(4,5-dimethylthiazol-2-yl)-2,5-diphenyltetrazolium bromide (MTT), and 2`,7`-dichlorofluorescein diacetate (DCFH-DA) were purchased from Sigma-Aldrich Chemical Co. (St. Louis, MO, USA). All organic solvents and other chemicals including dimethyl sulfoxide (DMSO) were used as extra-pure grade. The live/dead cell staining kit was purchased from Biovision (Milpitas, CA, USA). Dulbecco’s modified Eagle medium (DMEM) and DMEM nutrient mixture F-12 (DMEM/F12, 1:1) were purchased from Life Tech. Co. (Grand Island, NY, USA). Human gingival fibroblasts (HGF) and SCC-15 human squamous cell carcinoma cell lines were obtained from American Type Culture Collection (ATCC, Manassas, VA 20108, USA).

3.2. CAP Apparatus for Treatment of Cells

CAP apparatus P500-SM (Sakikake Co. Ltd., Kyoto, Japan) was used to treat cancer cells, as shown in Figure 1. CAP apparatus is consisted of a gas supply system, a mass flow controller (MFC), a plasma jet and a high-voltage AC power supply, as shown in Figure 1A. This apparatus was employed to produce a stable discharge with a 20 kHz radio frequency at atmospheric conditions, and the discharge power was adjusted at 45 W and at 8.5 kV. From the gas supply system, 5 L/min of Ar gas was supplied for plasma treatments, and the purity of gas was higher than 99.9%. A 12-well plate containing cells was mounted on the XY stage. The distance between the plasma jet nozzle and the media in the 12-well plate was adjusted to 3 cm, as shown in Figure 1B.

3.3. OES Measurement in the Plasma Jet Flame

A fiber optic spectral analyzer (Avantes AvaSpec-ULS2048CL-EVO-RS, spectra range from 200 to 1100 nm, Apeldoorn, Netherlands) was used to measure optical emission spectra of the CAP. An optical fiber (Avantes, Fiber-optic cable, FC-UV400-2, Apeldoorn, Netherlands) was located at a distance of 10 mm from the nozzle of the plasma jet.

3.4. Cell Culture and CAP Treatment

HGF-1 cells and SCC-15 cells were maintained in DMEM and DMEM/F12 medium, respectively. Each medium was supplemented with 10% heat-inactivated FBS and 1% penicillin/streptomycin at 37 °C in a 5% CO2 incubator. To maintain cells, media were exchanged and/or sub-cultured at intervals of every 2–3 days.

For CAP treatment, SCC-15 or HGF-1 cells were seeded in 12 wells (at a density of 1 × 105 cells in each well), and these cells were cultured in the 5% CO2 incubator overnight before treatment of CAP. Following this, cells were treated with CAP equipment (Sakigake; Kyoto, Japan, P500-SM, Figure 1A) with 5 kV for various periods of time (10 s–5 min) (flow rate: 5 L/min, Argon gas) at a 3 cm distance from the media (Figure 1B).

3.5. Cell Viability Assay

Cell viability was checked with an MTT assay. Cells seeded in 12 wells (1 × 105 cells in each well) were primarily treated with various concentrations of cisplatin. Six hour later, cells were washed with phosphate-buffered saline (PBS, 0.01M, pH 7.4) and replaced with fresh serum-free media. Following this, cells were treated again with CAP. To examine the treatment period, cells were exposed to CAP for various periods of time. Cells were placed in the 5% CO2 incubator for 24 h. After that, the MTT solution (500 μg/mL PBS, pH 7.4) was then added to the each well, which was followed by incubation for 2 h in the 5% CO2 incubator at 37 °C. The supernatant was discarded and replaced with 100 μL DMSO. The viability of cells was measured absorbance at 560 nm with a microplate reader (EPOCH, BioTek Instruments Inc., Winooski, VT., USA). All the values in the MTT assay are average ± S.D. (standard deviation) from four wells.

3.6. ROS Assay

ROS generation by the treatment of cisplatin and CAP was monitored with the DCFH-DA method. Briefly, cells seeded in 12 wells (1 × 105 cells in each well) were treated with cisplatin and/or CAP as described for the MTT assay. Briefly, cells were treated with cisplatin for 6 h, washed with PBS, replaced with fresh serum-free media and then treated with CAP. After that, DCFH-DA (final concentration, 20 μM) in the serum-free media was added and further incubated in the 5% CO2 incubator for 1 h. Intracellular ROS was fluorescently measured with an Infinite M200 pro microplate reader (Tecan, Mannedorf, Switzerland) (excitation wavelength: 485 nm, emission wavelength: 520 nm). This procedure was performed under dark conditions. ROS values are average ± S.D. (standard deviation) from four wells.

3.7. Live and Dead Cell Staining

To observe the viability behavior of the cells, 1 × 105 cells in 12 wells were treated with cisplatin for 6 h followed by CAP treatment as described above. Then, cells were stained with live/dead cell dye as follows: 1 × 105 cells in 12 wells were treated with cisplatin and/or CAP as described for the MTT assay. Briefly, cells were treated with cisplatin for 6 h followed by CAP treatment. Then, cells were harvested using trypsin–EDTA. The harvested cells were washed with PBS twice and then incubated for 15 min with 1 mL of the live/dead cell staining solution (1 μL live dye (1 mM) and 1 μL of the dead dye solution (1 mg/mL propidium iodide) (PI) in staining buffer, 1 mL) at 37 °C. Following this, cells were observed with a fluorescence microscope under dark conditions (Eclipse Ni-U, Nikon, Tokyo, Japan).

3.8. Western Blot Assay

To monitor the expression of apoptotic proteins in cells, SCC-15 cells were treated with cisplatin and CAP as described above. These cells were further incubated for 24 h. After that, cells were harvested from all wells and then washed with PBS twice. These cells were harvested by centrifugation. Cell pellets were lysed with 30 μL RIPA lysis buffer (150 mM NaCl, 1.0% IGEPAL® CA-630, 0.5% sodium deoxycholate, 0.1% SDS, 50 mM Tris, pH 8.0) containing protease and dephosphatase inhibitors (Thermo Fisher Scientific; Waltham, Massachusetts, USA) for 30 min at 4 °C. Following this, the lysed cells were centrifuged at 16,000× g for 20 min, and then supernatants were removed. Protein concentration was measured with a BCA protein assay kit at 562 nm (Pierce Biotechnology, IL 61105, USA) using a UV–vis spectrophotometer (Genesis G10S UV-VIS spectrophotometer, Thermo Scientific, USA). Then, 50 μg of protein was loaded to SDS polyacrylamide gel electrophoresis (SDS-PAGE). This was transferred to a polyvinyl difluoride (PVDF) membrane, blocked with 5% skim milk in TBS-T and then probed with a primary antibody such as caspase 9 (1:1000), PTEN (1:1000), P53 (1:1000) or β-actin (1:40,000) (Abcam; Cambridge, UK). Following this, the protein was treated with a secondary antibody (donkey anti rabbit (1:4000), donkey anti mouse (1:10,000) (Thermo Fisher Scientific)) for 1 h. The immunoblots were detected by chemiluminescence and then quantified with digital analyses using the ImageJ software program.

3.9. Flow Cytometry for Analysis of Apoptosis/Necrosis of Cancer Cells

The apoptosis and necrosis behavior of cells after cisplatin/CAP treatment was monitored using propidium iodide (PI) for necrosis and FITC–annexin V (sc-4252 FITC, Santa Cruz Biotech., Inc., Dallas, Texas, USA) for apoptosis. SCC-15 cells were treated with cisplatin for 6 h followed by CAP treatment as described above. Cells were further incubated in the 5% CO2 incubator at 37 °C for 24 h. Cells were washed with PBS and harvested by trypsinization. Then, collected pellets were resuspended into the binding buffer (10 mM HEPES pH 7.4, 150 mM NaCl, 5 mM KCl, 1 mM MgCl2, and 1.8 mM CaCl2) containing FITC–annexin V (1 µg/mL) to stain apoptotic cells, which was followed by incubation for 20 min in the 5% CO2 incubator at 37 °C. After that, PI (10 µg/mL) was added to this solution to stain the necrotic cells, which were incubated for a further 10 min in the 5% CO2 incubator at 37 °C. Following this, the apoptosis/necrosis behavior of cells was analyzed with a FACScan flow cytometer (Becton Dickenson Biosciences, San Jose, CA, USA) equipped with an excitation laser line at 488 nm for FITC–annexin and a 575 ± 15 nm band pass filter for PI. All procedures of apoptosis/necrosis dye staining were performed in dark conditions.

3.10. Statistical Analysis

Statistics of the experimental results were evaluated with the Student t-test, and p values lower than 0.05 were considered as a value of statistical significance.

4. Conclusions

The synergistic anticancer effect of cisplatin and CAP treatment was examined using SCC-15 and HGF-1 cells in vitro. Viability of SCC-15 cells was inhibited by cisplatin with a dose-dependent manner and/or CAP treatment time. HGF-1 cells also showed decreased viability by treatment with cisplatin and CAP. A combination of 1 μM cisplatin plus 3 min of CAP treatment or 3 μM cisplatin plus 1 min of CAP treatment showed a synergistic anticancer effect with appropriate cytotoxicity against normal cells. ROS generation and dead cell staining were also increased by the increase in CAP treatment time. Furthermore, tumor-suppressor proteins and apoptosis-related enzymes also increased according to the treatment time of CAP. We showed a synergistic effect of cisplatin and CAP treatment against SCC-15 cells with low cytotoxicity against normal cells.

Supplementary Materials

The following are available online at https://www.mdpi.com/1422-0067/21/20/7646/s1, Figure S1: Live and dead cell staining of cells

Author Contributions

Conceptualization, B.-H.K. and M.-S.K.; methodology, Y.-I.J., C.-M.L.; validation, B.-H.K. and M.-S.K.; formal analysis, Y.-I.J., C.-M.L.; investigation, M.-S.K., C.-M.L.; data curation, Y.-I.J., C.-M.L.; writing—original draft preparation, M.-S.K.; writing—review and editing, B.-H.K.; supervision, B.-H.K.; project administration, B.-H.K. All authors have read and agree to the published version of the manuscript.

Funding

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2020R1A2C2008540). This study was supported (in part) by research funds from Education and Cultural Foundation of College of Dentistry, Chosun University, 2018.

Conflicts of Interest

The authors declare no conflict of interest.

Footnotes

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  • 1.Isbary: G., Shimizu T., Li Y.F., Stolz W., Thomas H.M., Morfill G.E., Zimmermann J.L. Cold Atmospheric Plasma Devices for Medical Issues. Expert Rev. Med. Devices. 2013;10:367–377. doi: 10.1586/erd.13.4. [DOI] [PubMed] [Google Scholar]
  • 2.Izadjoo M., Zack S., Kim H., Skiba J. Medical applications of cold atmospheric plasma: State of the science. J. Wound Care. 2018;27(Suppl. 9):S4–S10. doi: 10.12968/jowc.2018.27.Sup9.S4. [DOI] [PubMed] [Google Scholar]
  • 3.Kaushik N.K., Ghimire B., Li Y., Adhikari M., Veerana M., Kaushik N., Jha N., Adhikari B., Lee S.J., Masur K., et al. Biological and medical applications of plasma-activated media, water and solutions. Biol. Chem. 2018;400:39–62. doi: 10.1515/hsz-2018-0226. [DOI] [PubMed] [Google Scholar]
  • 4.Bernhardt T., Semmler M.L., Schäfer M., Bekeschus S., Emmert S., Boeckmann L. Plasma Medicine: Applications of Cold Atmospheric Pressure Plasma in Dermatology. Oxid. Med. Cell Longev. 2019;2019:3873928. doi: 10.1155/2019/3873928. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Girard F., Peret M., Dumont N., Badets V., Blanc S., Gazeli K., Noël C., Belmonte T., Marlin L., Cambus J.P., et al. Correlations between gaseous and liquid phase chemistries induced by cold atmospheric plasmas in a physiological buffer. Phys. Chem. Chem. Phys. 2018;20:9198–9210. doi: 10.1039/C8CP00264A. [DOI] [PubMed] [Google Scholar]
  • 6.Girard P.M., Arbabian A., Fleury M., Bauville G., Puech V., Dutreix M., Sousa J.S. Synergistic effect of H2O2 and NO2 in cell death induced by cold atmospheric He plasma. Sci. Rep. 2016;6:29098. doi: 10.1038/srep29098. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Kurake N., Tanaka H., Ishikawa K., Kondo T., Sekine M., Nakamura K., Kajiyama H., Kikkawa F., Mizuno M., Hori M. Cell survival of glioblastoma grown in medium containing hydrogen peroxide and/or nitrite, or in plasma-activated medium. Arch. Biochem. Biophys. 2016;605:102–108. doi: 10.1016/j.abb.2016.01.011. [DOI] [PubMed] [Google Scholar]
  • 8.Ahn H.J., Kim K.I., Hoan N.N., Kim C.H., Moon E., Choi K.S., Yang S.S., Lee J.S. Targeting cancer cells with reactive oxygen and nitrogen species generated by atmospheric-pressure air plasma. PLoS ONE. 2014;9:e86173. doi: 10.1371/journal.pone.0086173. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Babington P., Rajjoub K., Canady J., Siu A., Keidar M., Sherman J.H. Use of cold atmospheric plasma in the treatment of cancer. Biointerphases. 2015;10:029403. doi: 10.1116/1.4915264. [DOI] [PubMed] [Google Scholar]
  • 10.Attri P. Cold atmospheric plasma activated solution: A new approach for cancer treatment. Anticancer Agents Med. Chem. 2018;18:768. doi: 10.2174/187152061806181112124717. [DOI] [PubMed] [Google Scholar]
  • 11.Yan D., Sherman J.H., Keidar M. The application of the cold atmospheric plasma-activated solutions in cancer treatment. Anticancer Agents Med. Chem. 2018;18:769–775. doi: 10.2174/1871520617666170731115233. [DOI] [PubMed] [Google Scholar]
  • 12.Aryal S., Bisht G. New paradigm for a targeted cancer therapeutic approach: A short review on potential synergy of gold nanoparticles and cold atmospheric plasma. Biomedicines. 2017;5:E38. doi: 10.3390/biomedicines5030038. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Azzariti A., Iacobazzi R.M., Di Fonte R., Porcelli L., Gristina R., Favia P., Fracassi F., Trizio I., Silvestris N., Guida G., et al. Plasma-activated medium triggers cell death and the presentation of immune activating danger signals in melanoma and pancreatic cancer cells. Sci. Rep. 2019;9:4099. doi: 10.1038/s41598-019-40637-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Shi L., Yu L., Zou F., Hu H., Liu K., Lin Z. Gene expression profiling and functional analysis reveals that p53 pathway-related gene expression is highly activated in cancer cells treated by cold atmospheric plasma-activated medium. Peer J. 2017;5:e3751. doi: 10.7717/peerj.3751. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Jablonowski L., Kocher T., Schindler A., Müller K., Dombrowski F., von Woedtke T., Arnold T., Lehmann A., Rupf S., Evert M., et al. Side effects by oral application of atmospheric pressure plasma on the mucosa in mice. PLoS ONE. 2019;14:e0215099. doi: 10.1371/journal.pone.0215099. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Baik K.Y., Huh Y.H., Kim Y.H., Kim J., Kim M.S., Park H.K., Choi E.H., Park B. The Role of free radicals in hemolytic toxicity induced by atmospheric-pressure plasma jet. Oxid. Med. Cell Longev. 2017;2017:1289041. doi: 10.1155/2017/1289041. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Wende K., Bekeschus S., Schmidt A., Jatsch L., Hasse S., Weltmann K.D., Masur K., von Woedtke T. Risk assessment of a cold argon plasma jet in respect to its mutagenicity. Mutat. Res. Genet. Toxicol. Environ. Mutagen. 2016;798–799:48–54. doi: 10.1016/j.mrgentox.2016.02.003. [DOI] [PubMed] [Google Scholar]
  • 18.Irani S., Shahmirani Z., Atyabi S.M., Mirpoor S. Induction of growth arrest in colorectal cancer cells by cold plasma and gold nanoparticles. Arch. Med. Sci. 2015;11:1286–1295. doi: 10.5114/aoms.2015.48221. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Matulionyte M., Dapkute D., Budenaite L., Jarockyte G., Rotomskis R. Photoluminescent gold nanoclusters in cancer cells: Cellular uptake, toxicity, and generation of reactive Oxygen species. Int. J. Mol. Sci. 2017;18:E378. doi: 10.3390/ijms18020378. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Moniruzzaman R., Rehman M.U., Zhao Q.L., Jawaid P., Mitsuhashi Y., Imaue S., Fujiwara K., Ogawa R., Tomihara K., Saitoh J.I., et al. Roles of intracellular and extracellular ROS formation in apoptosis induced by cold atmospheric helium plasma and X-irradiation in the presence of sulfasalazine. Free Radic. Biol. Med. 2018;129:537–547. doi: 10.1016/j.freeradbiomed.2018.10.434. [DOI] [PubMed] [Google Scholar]
  • 21.Lee S., Lee H., Jeong D., Ham J., Park S., Choi E.H., Kim S.J. Cold atmospheric plasma restores tamoxifen sensitivity in resistant MCF-7 breast cancer cell. Free Radic Biol Med. 2017;110:280–290. doi: 10.1016/j.freeradbiomed.2017.06.017. [DOI] [PubMed] [Google Scholar]
  • 22.Dilruba S., Kalayda G.V. Platinum-based drugs: past, present and future. Cancer Chemother Pharmacol. 2016;77:1103–1124. doi: 10.1007/s00280-016-2976-z. [DOI] [PubMed] [Google Scholar]
  • 23.Singhal P., Lele S. Intraperitoneal chemotherapy for ovarian cancer: where are we now? J. Natl. Compr. Canc. Netw. 2006;4:941–946. doi: 10.6004/jnccn.2006.0077. [DOI] [PubMed] [Google Scholar]
  • 24.Homma A., Onimaru R., Matsuura K., Robbins K.T., Fujii M. Intra-arterial chemoradiotherapy for head and neck cancer. Jpn. J. Clin. Oncol. 2016;46:4–12. doi: 10.1093/jjco/hyv151. [DOI] [PubMed] [Google Scholar]
  • 25.Serrano-Olvera A., Cetina L., Coronel J., Dueñas-González A. Emerging drugs for the treatment of cervical cancer. Expert Opin. Emerg. Drugs. 2015;20:165–182. doi: 10.1517/14728214.2015.1002768. [DOI] [PubMed] [Google Scholar]
  • 26.Egger S.J., Willson M.L., Morgan J., Walker H.S., Carrick S., Ghersi D., Wilcken N. Platinum-containing regimens for metastatic breast cancer. Cochrane Database Syst. Rev. 2017;2017:CD003374. doi: 10.1002/14651858.CD003374.pub4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Schardt J., Roth B., Seiler R. Forty years of cisplatin-based chemotherapy in muscle-invasive bladder cancer: are we understanding how, who and when? World J. Urol. 2019;37:1759–1765. doi: 10.1007/s00345-018-2544-8. [DOI] [PubMed] [Google Scholar]
  • 28.Ku G.Y. Systemic therapy for esophageal cancer: chemotherapy. Chin. Clin. Oncol. 2017;6:49. doi: 10.21037/cco.2017.07.06. [DOI] [PubMed] [Google Scholar]
  • 29.Reveiz L., Rueda J.R., Cardona A.F. Chemotherapy for brain metastases from small cell lung cancer. Cochrane Database Syst. Rev. 2012;2012:CD007464. doi: 10.1002/14651858.CD007464.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Wei M., Yuan X. Cisplatin-induced ototoxicity in children with solid tumor. J. Pediatr. Hematol. Oncol. 2019;41:e97–e100. doi: 10.1097/MPH.0000000000001282. [DOI] [PubMed] [Google Scholar]
  • 31.Oun R., Moussa Y.E., Wheate N.J. The side effects of platinum-based chemotherapy drugs: a review for chemists. Dalton Trans. 2018;47:6645–6653. doi: 10.1039/C8DT00838H. [DOI] [PubMed] [Google Scholar]
  • 32.Oronsky B., Caroen S., Oronsky A., Dobalian V.E., Oronsky N., Lybeck M., Reid T.R., Carter C.A. Electrolyte disorders with platinum-based chemotherapy: mechanisms, manifestations and management. Cancer Chemother. Pharmacol. 2017;80:895–907. doi: 10.1007/s00280-017-3392-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Michel O., Kulbacka J., Saczko J., Mączyńska J., Błasiak P., Rossowska J., Rzechonek A. Electroporation with Cisplatin against Metastatic Pancreatic Cancer: In Vitro Study on Human Primary Cell Culture. Biomed. Res. Int. 2018;2018:7364539. doi: 10.1155/2018/7364539. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Gothelf A., Mir L.M., Gehl J. Electrochemotherapy: Results of cancer treatment using enhanced delivery of bleomycin by electroporation. Cancer Treatm. Rev. 2003;29:371–387. doi: 10.1016/S0305-7372(03)00073-2. [DOI] [PubMed] [Google Scholar]
  • 35.Nishiyama N., Okazaki S., Cabral H., Miyamoto M., Kato Y., Sugiyama Y., Nishio K., Matsumura Y., Kataoka K. Novel cisplatin-incorporated polymeric micelles can eradicate solid tumors in mice. Cancer Res. 2003;63:8977–8983. [PubMed] [Google Scholar]
  • 36.Lee K.D., Jeong Y.I., Kim D.H., Lim G.T., Choi K.C. Cisplatin-incorporated nanoparticles of poly(acrylic acid-co-methyl methacrylate) copolymer. Int. J. Nanomedicine. 2013;8:2835–2845. doi: 10.2147/IJN.S48367. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Partecke L.I., Evert K., Haugk J., Doering F., Normann L., Diedrich S., Weiss F.U., Evert M., Huebner N.O., Guenther C., et al. Tissue tolerable plasma (TTP) induces apoptosis in pancreatic cancer cells in vitro and in vivo. BMC Cancer. 2012;12:473. doi: 10.1186/1471-2407-12-473. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Vyas A., Das S.K., Singh D., Sonker A., Gidwani B., Jain V., Singh M. Recent nanoparticulate approaches of drug delivery for skin cancer. Trends Appl. Sci. Res. 2012;7:620–635. doi: 10.3923/tasr.2012.620.635. [DOI] [Google Scholar]
  • 39.Zhu Y., Wen L.M., Li R., Dong W., Jia S.Y., Qi M.C. Recent advances of nano-drug delivery system in oral squamous cell carcinoma treatment. Eur. Rev. Med. Pharmacol. Sci. 2019;23:9445–9453. doi: 10.26355/eurrev_201911_19438. [DOI] [PubMed] [Google Scholar]
  • 40.Cheng K.Y., Lin Z.H., Cheng Y.P., Chiu H.Y., Yeh N.L., Wu T.K., Wu J.S. Wound Healing in Streptozotocin-Induced Diabetic Rats Using Atmospheric-Pressure Argon Plasma Jet. Sci Rep. 2018;8:12214. doi: 10.1038/s41598-018-30597-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Lou B.S., Lai C.H., Chu T.P., Hsieh J.H., Chen C.M., Su Y.M., Hou C.W., Chou P.Y., Lee J.W. Parameters Affecting the Antimicrobial Properties of Cold Atmospheric Plasma Jet. J. Clin. Med. 2019;8:1930. doi: 10.3390/jcm8111930. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Pasqual-Melo G., Nascimento T., Sanches L.J., Blegniski F.P., Bianchi J.K., Sagwal S.K., Berner J., Schmidt A., Emmert S., Weltmann K.D., et al. Plasma Treatment Limits Cutaneous Squamous Cell Carcinoma Development In Vitro and In Vivo. Cancers (Basel) 2020;12:1993. doi: 10.3390/cancers12071993. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Xu X., Xu Y., Cui Q., Liu D., Liu Z., Wang X., Yang Y., Feng M., Liang R., Chen H., et al. Cold atmospheric plasma as a potential tool for multiple myeloma treatment. Oncotarget. 2018;9:18002. doi: 10.18632/oncotarget.24649. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Saadati F., Mahdikia H., Abbaszadeh H.A., Abdollahifar M.A., Khoramgah M.S., Shokri B. Comparison of direct and indirect cold atmospheric-pressure plasma methods in the B16F10 melanoma cancer cells treatment. Sci. Rep. 2018;8:7689. doi: 10.1038/s41598-018-25990-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Ikeda J.I., Tanaka H., Ishikawa K., Sakakita H., Ikehara Y., Hori M. Plasma-activated medium (PAM) kills human cancer-initiating cells. Pathol. Int. 2018;68:23–30. doi: 10.1111/pin.12617. [DOI] [PubMed] [Google Scholar]
  • 46.Payer M., Tan W.C., Han J., Ivanovski S., Mattheos N., Pjetursson B.E., Zhuang L., Fokas G., Wong M.C.M., Acham S., et al. International Team for Implantology (ITI) Antibiotic Study Group. The effect of systemic antibiotics on clinical and patient-reported outcome measures of oral implant therapy with simultaneous guided bone regeneration. Clin. Oral Implants Res. 2020;31:442–451. doi: 10.1111/clr.13580. [DOI] [PubMed] [Google Scholar]
  • 47.Da Rocha H.A., Silva C.F., Santiago F.L., Martins L.G., Dias P.C., De Magalhães D. Local drug delivery systems in the treatment of periodontitis: A literature review. J. Int. Acad. Periodontol. 2015;17:82–90. [PubMed] [Google Scholar]
  • 48.Kalsi R., Vandana K.L., Prakash S. Effect of local drug delivery in chronic periodontitis patients: A meta-analysis. J. Indian Soc. Periodontol. 2011;15:304–309. doi: 10.4103/0972-124X.92559. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials


Articles from International Journal of Molecular Sciences are provided here courtesy of Multidisciplinary Digital Publishing Institute (MDPI)

RESOURCES