Skip to main content
Nutrients logoLink to Nutrients
. 2011 Oct 13;3(10):839–857. doi: 10.3390/nu3100839

Hydroxytyrosol Protects against Oxidative DNA Damage in Human Breast Cells

Fernando Warleta 1,2, Cristina Sánchez Quesada 1,2, María Campos 1,2, Yosra Allouche 1,2,3, Gabriel Beltrán 2,3, José J Gaforio 1,2,*
PMCID: PMC3257739  PMID: 22254082

Abstract

Over recent years, several studies have related olive oil ingestion to a low incidence of several diseases, including breast cancer. Hydroxytyrosol and tyrosol are two of the major phenols present in virgin olive oils. Despite the fact that they have been linked to cancer prevention, there is no evidence that clarifies their effect in human breast tumor and non-tumor cells. In the present work, we present hydroxytyrosol and tyrosol’s effects in human breast cell lines. Our results show that hydroxytyrosol acts as a more efficient free radical scavenger than tyrosol, but both fail to affect cell proliferation rates, cell cycle profile or cell apoptosis in human mammary epithelial cells (MCF10A) or breast cancer cells (MDA-MB-231 and MCF7). We found that hydroxytyrosol decreases the intracellular reactive oxygen species (ROS) level in MCF10A cells but not in MCF7 or MDA-MB-231 cells while very high amounts of tyrosol is needed to decrease the ROS level in MCF10A cells. Interestingly, hydroxytyrosol prevents oxidative DNA damage in the three breast cell lines. Therefore, our data suggest that simple phenol hydroxytyrosol could contribute to a lower incidence of breast cancer in populations that consume virgin olive oil due to its antioxidant activity and its protection against oxidative DNA damage in mammary cells.

Keywords: breast cancer, Mediterranean diet, olive oil minor compounds, hydroxytyrosol, tyrosol, phenols, oxidative stress, reactive oxygen species, DNA damage

1. Introduction

Olive oil is the major source of fats in the Mediterranean diet and is considered to be responsible for the health benefits associated with this diet. In fact, it has been demonstrated that people who consume virgin olive oil (VOO) present a lower incidence of several cancers, including breast cancer [1]. This effect has previously been attributed to the high content of monounsaturated fatty acids. However, more recently, the importance of the minor constituents of olive oil has been considered [2]. Over the last five decades, several publications have firmly established that ingestion of small quantities of certain compounds isolated from plants can lower the risk of cancer in mammals exposed to carcinogens, including polyphenols [3].

VOO contains relatively high amounts of minor compounds compared to other oils (refined olive oil or seed oils). Among these, phenolic compounds are present at levels between 200 and 1500 mg/kg [4] depending on the olive tree variety, climatic and agronomic conditions, degree of maturation at harvest, and the manufacturing process [4]. At present, there are many studies reporting biological activities in vitro, in vivo and in clinical assays of phenolic compounds naturally present in VOO. Between them, anti-inflammatory, cardioprotective antioxidant and chemopreventive effects in breast and other types of cancers have been defined [5]. The major phenols identified in olive oils include the simple phenols hydroxytyrosol (HT) and tyrosol (TY), secoiridoids and lignans [2]. The concentration of TY is always higher than of HT [6]. Hydrolysis of secoiridoid during olive oil storage results in the formation of HT and TY [7].

It has been well established that HT is a potent antioxidant because of its marked antioxidant activity, its ability to scavenge oxygen and nitrogen free radicals, to inhibit Low Density Lipoprotein (LDL) oxidation, platelet aggregation and endothelial cell activation and its protection against DNA damage [2,8]. HT was able to reduce the synthesis of prostaglandin E2 blocking the transcription of COX-2 and 5-lipooxygenase, thereby reducing the chronic influence associated with diseases such as cancer [9]. TY has been described as exerting a weak antioxidant activity, although it is able to scavenge peroxynitrite and superoxide radicals, inhibit LDL oxidation in Caco2 cells and inhibit LPS-induced cytokines release from human monocytes [10,11].

It has been suggested that HT and TY compounds might have preventive activity against breast cancer, but, at present, the exact role played by these phenols in breast cancer prevention is still unknown. In this sense, despite epidemiological evidence, in vitro experiments have not been conducted to check if there are different effects of the simple phenols HT and TY between human breast cancer cells and human breast non-cancer cells.

The present study attempts to provide new insights into the antioxidant capacity of HT and TY and the in vitro effects on proliferation, cell cycle progression, apoptosis, reactive oxygen species (ROS) production and oxidative DNA damage in the human breast epithelial MCF10A cell line and the human breast MCF7 and MDA-MB-231 cancer cell lines.

2. Experimental Section

2.1. Chemicals and Materials

The following were purchased from Sigma-Aldrich Co. (St Louis, MO, USA): Hepes Buffer; Sodium Pyruvate; Non-Essential Amino Acids mixture 100× (NEAA); 2′,7′-dichlorofluorescein diacetate (DCFH-DA); Dimethyl sulfoxide (DMSO); 2,3-Bis(2-methoxy-4-nitro-5-sulfophenyl)-2H-tetrazolium-5-carboxanilide inner salt (XTT sodium salt) purity ≥90%; N-Methylphenazonium methyl sulfate (PMS) purity ~98%; 2-hydroxyphenyl ethanol (Tyrosol, CAS 501-94-0 (TY)) purity 98%; DL-all-rac-α-Tocopherol (Vitamin E, CAS 10191-41-0 (TOC)) purity ≥96%; 6-Hydroxy-2,5,7,8-tetramethylchroman-2-carboxylic acid (Trolox™ CAS 53188-07-1 (TR)) purity ≥97%; 2,2′-Azobis (2-methylpropionamidine) dihydrochloride (AAPH) purity ~97%; 2,2-Diphenyl-1-picrylhydrazyl (DPPH) purity ~90%, (S)-(+)-camptothecin (CAS 7689-03-4 (CPT)) purity ~95%; 2,2′-azino-bis(3-ethylbenzthiazoline-6-sulphonic acid) diammonium salt tablets (CAS 30931-67-0 (ABTS)); PBS; HBSS. 2-(3,4-dihydroxyphenyl) ethanol (Hydroxytyrosol, CAS 10597-60-1 (HY)) purity ≥98% was obtained from Cayman Chemical (Ann Arbor, MI, USA). Minimum essential medium with Eagle’s salts (MEM), Fetal Bovine Serum (FBS) and Phenol-Red-free Roswell Park Memorial Institute 1640 medium (RPMI) were obtained from PAA Laboratories GmbH (Pasching, Austria). TrypLE Express, HuMEC Ready Medium kit and Fluorescein (FL) were obtained from Invitrogen (Eugene, OR, USA). K2S2O8 (CAS 7727-21-1) was obtained from Panreac Quimica S.A.U. (Barcelona, Spain). Culture plates were obtained from NUNC™ (Roskilde, Denmark). The PI/RNase Staining Buffer kit, FITC-conjugated Annexin V and Binding Buffer were obtained from BD Biosciences Pharmigen (San Diego, CA, USA). The Comet assay kit was obtained from Trevigen, Inc. (Helgerman CT, Gaithersburg, MD, USA).

2.2. DPPH Assay

The antioxidant activity of HT and TY against the stable radical DPPH was measured as previously reported [12] with some modifications. Briefly, 100 µM ethanolic solution of DPPH was mixed with different ethanolic solutions of HT or TY in 96-well plates at 0.06, 0.13, 0.25, 0.5 and 1 (moles of antioxidant/moles of DPPH). (±)-α-tocopherol (TOC) was used as a positive control and a sample without antioxidant was also measured as a blank control. The decrease in absorbance at 520 nm was determined immediately and every 5 min for 2 h in a microplate reader (TECAN, GENios Plus). Measurements were performed in triplicate.

The inhibition of the DPPH radical was calculated according to the following percentage of Radical Scavenging Activity (% RSA) formula:

% RSA = [(AC(0)AA(t))/AC(0)] × 100

where AC(0) is the absorbance of the control at t = 0 min and AA(t) is the absorbance of the antioxidant at t = 50 min.

2.3. ABTS Assay

ABTS cation radical scavenging activity was determined using a previously reported procedure [13]. ABTS radicals (ABTS•+) were obtained by ABTS/H2O 0.5 mM reaction with K2S2O8 for 16 h in the dark at room temperature. ABTS•+ was diluted in ultrapure water until absorbance at 734 nm was 0.7 (±0.1). HT, TY and Trolox™ (TR) (as antioxidant reference) was dissolved in ethanol to yield a 10 mM stock solution and diluted with ultrapure water to the assayed concentrations. Twenty microliters of each concentration of HT, TY, standard (TR), blank (ultrapure water) or ethanol control (8%) were added to a 96-well plate. The reaction was initiated by the addition of 280 µL of ABTS•+. Absorbance readings were taken every 5 min at 30 °C for 2 h in a microplate reader (TECAN, GENios Plus). All determinations were carried out in triplicate.

The inhibition of ABTS•+ was calculated according to the percentage of Radical Scavenging Activity (% RSA) described above (at t = 30 min).

2.4. ORAC Assay

Peroxyl radical scavenging activity was measured by the ORACFL assay as previously described [14]. A stock solution of HT or TY were reconstituted in DMSO and then diluted in PBS. A stock solution of TR, as antioxidant standard, was also diluted in DMSO and diluted in PBS. The assay was carried out in 96-well plates with a final volume of 160 μL. Samples were run in triplicate. Fluorescein (48 nM) was mixed with various concentrations of SQ, standard (TR) or blank (PBS) containing at final volume 1% (v/v) DMSO. Plates were incubated for 15 min at 37 °C. The assay was initiated by the addition of AAPH (100 mM) and fluorescence readings (Ex: λ485/Em: λ520 nm) were taken every 5 min at 37 °C for 160 min in a microplate reader (TECAN GENios Plus). Final results were calculated based on the difference in the Area Under the fluorescence decay Curve (AUC) between the blank and each sample. The AUC formula was:

AUC = 1 + f1/f0 + f2/f0 + f3/f0 + f4/f0 + … + f20/f0

Results were expressed as micromolar TR equivalents (TE) calculated using the line equation from the standard curve:

TE = (Yb)/m

where Y is the net AUC (AUCsample − AUCcontrol), m is the slope and b is the Y-intercept.

2.5. Cell Culture

Highly invasive MDA-MB-231 human breast cancer cells (estrogen and progesterone receptor-negative), minimally invasive MCF7 human breast cancer cells (estrogen and progesterone receptor-positive) and immortalized non-tumorigenic MCF10A human breast epithelial cells, were obtained from American Type Culture Collection (ATCC, Rockville, MD, USA). Breast tumor cells were grown as a monolayer culture in Minimum Essential Medium with Eagle’s salts (MEM) supplemented with 10% Fetal Bovine Serum (FBS), 1% Hepes Buffer 1 M, 1% Sodium Pyruvate 100 mM and 1% Non-Essential Amino Acids mixture 100×. MCF10A cells were cultivated in HuMEC Ready Medium. All cell lines were maintained at 37 °C in a humidified incubator with 5% CO2. Cells were routinely sub-cultured using TrypLE Express solution. Cells in the exponential growth phase were used for all experiments.

2.6. Cell Proliferation Assay

Cell proliferation, measured as the cellular growth of treated cells vs. untreated controls, was measured using an XTT-based assay as described by Scudiero et al. [15] with some modifications. Briefly, cells were seeded at 2 × 103 cells/well (MCF7) or 1 × 103 cells/well (MDA-MB-231 and MCF10A) into 96-well culture plates (flat bottom) (100 μL of cell suspension/well). At 24 h after plating, 100 μL of fresh culture medium, with different concentrations of HT or TY was added in triplicate to the wells. Plates were incubated for 24 h or 24 h followed by a 48 h proliferation period with fresh medium at 37 °C and 5% CO2. At these time points, medium was removed and 200 μL of fresh RPMI medium without phenol red that contained XTT (200 μg/mL) and PMS (20 μg/mL) was added. Plates were incubated for 3 h at 37 °C in 5% CO2 and absorbance was measured at 450 nm wavelength (620 nm as reference) in a plate reader (TECAN GENios Plus). Viability was calculated using the formula:

viable cells (%) = (ODtreated cells/ODcontrol) × 100

where OD is the difference in absorbance between optical density units (OD = OD450 − OD620).

All measurements were performed in triplicate and each experiment was repeated at least three times.

2.7. Cell Cycle Assay

Cells were seeded in 12-well culture plates at 1 × 105 cells/well for MCF7 and MDA-MB-231 or at 5 × 104 cells/well for MCF10A for 48 h. Cells were then treated with different doses of HT or TY for 24 h. After incubation, cells were washed in cold PBS, fixed with cold 70% ethanol and stored at −20 °C for at least 24 h. At least 1 × 104 cells per sample were analyzed on an EPICS XL-MCL (Beckman Coulter, Spain) flow cytometer after propidium iodide labeling (PI/RNase Staining Buffer kit). The percentage of cells in G0/G1, S and G2/M phases were calculated using FlowJo program (v5.7.2). Each experiment was repeated three independent times.

2.8. Apoptosis

The percentage of apoptosis was determined using a double staining assay with FITC-conjugated Annexin V and propidium iodide (PI). Briefly, after 24 h of cell exposure to the previously indicated doses of HT or TY in 12-well culture plates, cells were harvested, washed twice in cold PBS and resuspended in 100 μL 1× Annexin Binding Buffer. Cells were then stained with 5 μL Annexin V-FITC and 1 μL PI solution, gently vortexed, and incubated for 15 min at room temperature in the dark before flow cytometric analysis. As a positive control, cells were treated with 1 μM camptothecin (CPT). Each experiment was repeated three independent times.

2.9. Reactive Oxygen Species Detection

Intracellular reactive oxygen species (ROS) level was measured using a cell-permeable fluorescent probe, 2′,7′-dichlorofluorescein diacetate (DCFH-DA) as we described previously [16]. In brief, cells were seeded into 96-well culture plates at 1 × 104 cells/well (MCF7, MDA-MB-231 cells) or 5.5 × 103 cells/well (MCF-10A cells). After 24 h at 37 °C and 5% CO2, cells were treated with different doses of HT or TY for 24 h. Cells were then washed twice with Hank’s Buffered Salt Solution (HBSS) and incubated with fresh DCFH-DA (100 μM) in HBSS for 30 min at 37 °C in 5% CO2. DCFH-DA stock solution (20.5 mM) was prepared in DMSO and stored at −20 °C for maximum one month. After that, cells were washed twice in HBSS, and wells were filled with 100 µL HBSS before fluorescence acquisition in a plate reader (TECAN GENios Plus) (Ex: λ485/Em: λ535 nm, Gain 60). Intracellular ROS level percentage was calculated as follows:

F = [(Ft30 − Ft0)/Ft0] × 100

where Ft0 is the fluorescence at t = 0 min and Ft30 the fluorescence at t = 30 min.

It has been reported that the addition of H2O2 increases oxidative stress in cultured cells [17]. Therefore, in order to evaluate the protective capacity of HT or TY against induced oxidative stress, H2O2 (500 μM) was added to the wells after removal of assay medium. This allows avoiding a direct reaction in the medium between these compounds and the oxidant source. After 30 min at 37 °C, fluorescence was quantified as described above.

All tests were run in triplicate for each experimental condition and each experiment was repeated at least three times. All experiments were conducted using iron-free media (MEM and HuMEC).

2.10. Alkaline Single-Cell Gel Electrophoresis (Comet Assay)

At 24 h, cells treated with HT or TY were scraped into 12-well culture plates, washed twice (300× g 10 min, 4 °C) with cold 1× PBS (Ca2+/Mg2+ free) and resuspended in 1 mL of cold 1× PBS. In order to evaluate the ability of HT and TY to prevent oxidative DNA damage, cell suspensions were exposed for 10 min to 50 µM H2O2 at 4 °C. After that, cells were washed twice and frozen in FBS-DMSO (90:10, v/v) at −80 °C until the Comet assay procedure.

DNA single strand break by alkaline microgel electrophoresis was performed according to Singh et al. [18] with some modifications. Cells were thawed in a bath at 37 °C, centrifuged (300× g 10 min, 4 °C) in cold MEM with 25% FBS and resuspended in cold 1× PBS to a density of 1.65 × 105 cells/mL. Cells were then suspended in melted and cooled (at 40 °C) low melting point agarose (LMA). Cell suspensions (50 µL) were spread over a sample area of pre-warmed 1% normal melting point agarose (NMA) precoated CometSlide™ slides. After 15 min at 4 °C in the dark, slides were immersed in cold Lysis Solution (Trevigen, Inc.) at 4 °C for 30 min to dissolve lipids and proteins. In order to separate the two DNA strands, slides were then immersed in fresh Alkaline Solution (pH > 13) for 30 min at room temperature in the dark. Electrophoresis was performed in an Ebony acrylic electrophoresis tank with a cooled platform containing cold Alkaline Electrophoresis Solution (300 mM NaOH, 1 mM EDTA, pH > 13) at 25V (1 V/cm) and 300 mA for 40 min. The slides were washed twice with distilled water for 10 min and neutralized with 10 mM Tris-HCl, pH 7.5 for 5 min, followed by immersion in 70% ethanol for 5 min and air-dried overnight at room temperature. Slides were stained with Sybr® green before scoring.

2.11. Slide Scoring and Analysis

DNA strand breaks were examined using a fluorescence microscope (Zeiss Axiovert 200) equipped with a Luca EMCCD camera (Andor Technology, Belfast, UK) under 494 nm excitation and 521 nm emission wavelength using the Komet 5.5 software package (Kinetic Imaging Ltd., Liverpool, UK). Fifty cell images were randomly characterized per sample using 20× magnification. Relative fluorescence between head and tail through the olive tail moment (Olive_TM) was used to determine DNA damage. Olive_TM is defined as the product of the Tail Moment Length and the fraction of DNA in the tail.

Olive_TM = [(Tail (mean) − Head (mean)) × Tail (% DNA)]/100

2.12. Statistical Analysis

Results are presented as mean (±SEM), except for cell proliferation results. For this assay, results are presented as mean (±SD). Results are expressed as a percentage relative to the control, which was defined as 100%. Statistical analysis was performed using one-way analysis of variance (ANOVA) followed by Fisher’s least significant difference (LSD) test. Values of p < 0.05 were considered significant. Statgraphics Plus 5.1 statistical software (Statpoint Technologies, Inc., Warrenton, VA, USA) was used for the statistical analysis.

3. Results

3.1. Effect of HT and TY on Radical Scavenging Activity

The antiradical activity of HT and TY, measured by scavenging activity in the DPPH radical assay, indicated that HT at up to 10 mole ratio (mole antioxidant/mole DPPH) exerts a slightly higher scavenging activity than TOC while TY does not possess a radical scavenger activity (Table 1(a)).

The ABTS antiradical assay showed that HT was more effective than TR in scavenging the ABTS cationic radical while TY exhibited a maximum 85% RSA at 800 µM (Table 1(b)).

The peroxyl radical scavenging activity of HT and TY, measured by the ORACFL assay, showed a protective effect against AAPH-induced peroxyl radical activity for both phenols. Both exerted higher protection against the peroxyl radical than TR for low concentrations up to 100 µM (Table 1(c)).

3.2. Cell Proliferation

To investigate the effect of HT and TY on human breast cell growth, cells were treated with concentrations of HT or TY ranging from 1 to 100 μM for 24 h. Neither HT nor TY had significant effects on the cell proliferation rates of MCF7, MDA-MB-231 and MCF10A cells (Figure 1(a)), even after an additional 48 h with fresh medium (Figure 1(b)). We also investigated the potential antiproliferative effect of these compounds at high, non-physiological concentrations up to 1000 or 5000 μM of HT or TY, respectively. HT showed a dose-dependent reduction of cell proliferation in the three cell lines from a concentration of 200 μM with an absence of viability observed at 1000 μM, while TY did not affect cell viability at any concentration assayed (data not shown).

No marked changes in cell morphology were observed by light microscopy in any of the cell lines tested when concentrations between 1 and 100 μM of HT or TY were used (data not shown).

Table 1.

Antioxidant activity of hydroxytyrosol (HT) or tyrosol (TY) quantified as Radical Scavenging Activity (RSA) by (a) DPPH assay (% RSA at 50 min) and (b) ABTS assay (% RSA at 30 min); (c) Antioxidant activity quantified as Trolox Equivalent (TE) by ORACFL assay. Trolox™ (TR) and α-tocopherol (TOC) were used as antioxidant references.

(a)
mole AH/ mole DPPH HT TY TOC
n.d.: not determined.
0 3.96 3.64 3.63
0.06 36.24 n.d. 23.21
0.13 71.47 5.20 48.16
0.25 96.40 4.25 75.85
0.5 97.80 3.02 90.17
1 98.07 2.59 95.98
2.5 n.d. 2.34 n.d.
5 n.d. 3.44 n.d.
10 n.d. 3.37 n.d.
(b)
µM HT TY TR
n.d.: not determined.
6 5.07 10.31 n.d.
12.5 5.45 14.32 n.d.
25 8.42 20.14 n.d.
50 18.09 31.81 16.21
100 36.63 44.55 29.39
200 69.53 58.25 50.07
400 96.30 73.82 88.03
800 99.47 85.04 99.48
(c)
TE (µM) HT (µM) TY (µM)
3.12 14.82 1.64
6.25 30.97 8.30
12.5 50.37 20.84
25 92.24 58.35
50 150.43 106.76
100 262.76 205.57

Figure 1.

Figure 1

Cell proliferation assay measured with XTT tetrazolium salt (a) after 24 h of HT or TY exposure, or (b) after 24 h of HT or TY exposure followed by 48 h with fresh medium. Data are the mean (±SD) relative to an untreated control of three independent assays carried out in triplicate.

3.3. Cell Cycle and Apoptosis

To evaluate whether HT or TY interfered with the cell cycle or the induction of apoptosis, MCF7, MDA-MB-231 and MCF10A cells were treated for 24 h with increasing concentrations of HT or TY (between 10 and 200 μM). The results revealed that HT and TY did not alter the cell cycle in any of the cell lines studied (data not shown).

Flow cytometric analysis of apoptosis revealed that treatment with HT or TY for 24 h did not induce apoptosis in MCF10A cells or in MCF7 or MDA-MB-231 cells when compared to the controls (data not shown).

3.4. Intracellular ROS Level

Intracellular reactive oxygen species (ROS) were quantified by the dichlorofluorescein diacetate (DCFH-DA) assay using a microplate reader. Results showed a dose-dependent decrease in ROS level of MCF10A cells treated for 24 h with either HT or TY. However, HT and TY failed to significantly decrease intracellular ROS level in either MCF7 or MDA-MB-231 cells (Figure 2(a)). While HT reduced ROS level by up to 20% in MCF7 cells, this reduction was not considered statistically significant (p = 0.34).

H2O2 effectively induced oxidative stress in both, human breast cancer cells and human breast epithelial cells (Figure 2(b)). In order to investigate the in vitro preventive effect of HT or TY against H2O2-mediated oxidative stress, we measured the intracellular ROS level in cells treated with HT or TY for 24 h. As can be seen in Figure 2(c), MCF10A cells treated with HT or TY showed a significant dose-dependent decrease in ROS production compared to the control. In addition, HT was also able to decrease the ROS level in MCF7 and MDA-MB-231 cells induced by H2O2 exposure. It is worth mentioning that the decrease in ROS level was greater in the breast epithelial cell line than in the breast cancer cell lines. On the other hand, TY did not decrease ROS level in MCF7 or MDA-MB-231 cells at concentrations up to 5000 μM (Figure 2(c)).

Figure 2.

Figure 2

(a) Intracellular reactive oxygen species (ROS) in breast cells treated for 24 h with HT or TY; (b) Increase of the cellular ROS level after an oxidative burn with H2O2; (c) Intracellular ROS in breast cells treated for 24 h with HT or TY followed by an oxidative burst with H2O2. Inhibitory effects of HT and TY are shown as percent inhibition of untreated or H2O2-stimulated fluorescence and represented as the mean ± SEM of three independent replicates carried out in triplicate. MCF7; MDA-MB-231; * MCF10A indicates significant differences.

IC20 and IC50 values were defined as the values for 20% and 50% antioxidant inhibition of basal or H2O2-stimulated fluorescence in DCFH-DA probes. The Relative Antioxidant Value (RAV) ratio was found to be a good parameter for the determination of oxidative inhibition profiles.

RAV= [(IC20 (PH)/IC20 (TOC)) + (IC50 (PH)/IC50 (TOC))]/2

where PH is the compound (simple phenol) and TOC is the reference (α-tocopherol).

Our results showed that TY has a RAV about 46-fold higher than TOC for MCF10A cells whereas HT only has 1.44-fold higher. This indicated much high antioxidant activity of HT compared with TY in normal breast cells, but less than TOC (Table 2). In MCF7 and MDA-MB-231 cells, a 50% antioxidant inhibition was not observed; therefore, RAV ratios were not determined in these cell lines.

Interestingly, in H2O2-stimulated MCF10A cells, the RAV ratio of TY was 42-fold higher than TOC, indicating a very low antioxidant capacity in normal breast cells. The 0.67-fold difference between RAV ratio of HT and TOC is of particular interest, due to the high antioxidant activity of HT in H2O2-stimulated MCF10A cells (Table 2).

Table 2.

Oxidative inhibition in MCF10A cells. IC20 and IC50 values defined as the values for antioxidant inhibition of basal or H2O2-stimulated fluorescence in DCFH-DAassays and the Relative Antioxidant Value (RAV) as a parameter for the relative determination of oxidative inhibition profiles compared to α-tocopherol.

Basal H2O2-stimulated
HT (µM) TY (µM) TOC (µM) HT (µM) TY (µM) TOC (µM)
IC20 3.52 4.04 4.33 2.66 65.36 2.49
IC50 65.64 2942.60 31.89 20.66 4244.40 73.50
RAV 1.44 46.60 1.00 0.67 42.00 1.00

3.5. Effect of HT and TY on Oxidative DNA Damage

The ability of H2O2 to induce DNA strand breaks in these human breast epithelial cell lines was examined using the Comet assay. In untreated cells, DNA does not migrate far from the origin when examined by alkaline microgel electrophoresis (Figure 3(a)). Following H2O2 exposure, control and pretreated breast cells with damaged DNA have the shape of a comet, the tail length and fluorescent intensity of which are related to the number of DNA strand breaks induced by the DNA-damaging agent (Figure 3(b,c)).

Figure 3.

Figure 3

Representative images of Comet assay analysis of MCF10A cells. (a) Untreated cell, showing a circular shape indicating absence of DNA damage; (b) 10 min. H2O2 exposed cell, exhibiting a long and bright tail related to DNA strand breaks, indicating DNA oxidative damage; (c) 10 min. H2O2 exposed cell after 24 h of 100 µM HT pretreatment, illustrating the reduction of tail length and fluorescent intensity indicative of reduced DNA damage.

Breast cells exposed to H2O2 were effectively DNA damaged and the mean olive tail moment (Olive_TM) was determined by the Comet assay. Breast epithelial cells were the most sensitive to the H2O2-induced DNA damage (Figure 4(a)).

In unexposed cells, HT reduced DNA damage significantly in MCF7, MDA-MB-231 and MCF10A cells whereas TY only reduced it in MCF10A cells (Figure 4(b)). In H2O2-exposed cells, HT showed a preventive DNA damage effect in the three cell lines whereas TY was unable to reduce Olive_TM in any of the cell lines; indeed, in MDA-MB-231 cells, TY increased Olive_TM significantly.

Figure 4.

Figure 4

Olive Tail Moment (Olive_TM) as the mean ± SEM for three independent assays. (a) After an H2O2 injury; (b) after 24 h of HT or TY treatment, and (c) after 24 h of HT or TY treatment followed by an H2O2 injury.

4. Discussion

There is some scientific evidence relating Mediterranean dietary pattern with a lower incidence of cardiovascular diseases and cancer, among other diseases. Virgin olive oils (VOOs) represent the main source of fats in this diet and it has been demonstrated that consumption of VOOs reduces human arterial hypertension, lipid peroxidation of membranes, tumor incidence and number of tumors [19,20]. Minor compounds play a key role in VOOs’ healthy properties. Among them, phenols have demonstrated healthy bioactivity properties. Interest in phenolic compounds has increased greatly, with attention being focused on finding naturally occurring antioxidants for foods or medical uses to replace synthetic antioxidants that, in some cases, have been reported to be carcinogenic [21].

HT and TY are two of the major simple phenols present in VOOs as simple form or conjugates [2].Bioavailability studies have demonstrated that they are dose-dependently absorbed in animals and humans after olive oil ingestion [22], accumulated in the body and, finally, systemically exert biological effects [23].

The present work describes the antioxidant capacity of HT and TY molecules using chemical and cellular assays and their relationship with proliferation of human breast tumor vs. normal cells.

HT and TY are structurally identical except that HT has an extra -OH group forming a catechol group, which is considered responsible for its higher antioxidant activity. This catechol group is able to stabilize free radicals through the formation of intermolecular hydrogen bonds [8]. In our chemical analysis, the catechol phenol HT exhibited a strong antioxidant activity in DPPH, ABTS and ORAC assays, while TY, without a catechol group, showed a weak antioxidant activity in DPPH assay. Remarkably, TY acts as an efficient scavenger against ABTS and AAPH radicals, although to a lesser extent than HT, indicating the minor importance of the cathecol group in cationic or peroxylic radicals’ scavenging activities. These results are in agreement with those previously reported by Visioli et al. [8] affirming that HT and, to a lesser degree, TY are more potent scavengers of free radicals than vitamin E.

Although nowadays there is no scientific evidence relating to the physiological concentrations of HT or TY after olive oil ingestion, some authors have suggested it could be between 10 and 100 μM [24]. Cell treatment with HT or TY in the range of their possible physiological concentrations (1-100 μM) did not have any effect on cell proliferation in any of the cell lines studied, independently of the exposure times. However, HT dramatically reduced the viability of MCF7, MDA-MB-231 and MCF10A cell lines when used at concentrations from 200 μM to 1000 μM. Fabiani et al. described such an effect in colon adenocarcinoma HT29 cells [25]. Furthermore, HT and TY did not alter the cell cycle or induce apoptosis in these cell lines. Although these results are in agreement with those achieved in LLC-PK1 renal cells, they are in contrast with results in human promyelocytic leukaemia HL60 cells with a noticeable antiproliferative, cell cycle arrest and apoptotic effect of HT. Otherwise, TY showed no antiproliferative effect in HL60 cells [7,25].

HT or TY’s inability to inhibit breast cancer cell proliferation at the assayed times and concentrations, suggests that they cannot protect against breast cancer once developed. Quiles et al. [24] described the lack of inhibition of HT or TY in PC3 cells treated with 10 to 250 μM, as did Menendez et al. [26] in SKBR3 and MCF7 cells after 5 days of HT or TY treatments in the range of 6.25 to 100 μM. Moreover, Sirianni et al. [27] recently described the dose-dependent inhibition of MCF7 cell proliferation by HT and oleuropein (OL) with treatments of 1 to 100 μM; cell growth was induced by 17-β-estradiol (E2). In addition, HT and OL are not able to interfere with estrogen action through competition with estrogen receptors (ER), which are responsible for activation of the gene expression involved in cell proliferation.

In order to clarify how nutritional antioxidants are able to prevent or treat oxidative damage, Berger [28] affirmed that nutrients cannot treat an installed disease, such as gastrointestinal cancer, but that they may prevent its promotion. Indeed, the answer to the question: “Can installed damage caused by ROS be treated by antioxidant nutrients?” is “probably not”, but the answer to the question: “Can oxidative damage be treated nutritionally?” is “yes” [28].

Growing evidence supports the hypothesis that risk factors such as lifestyle, age, environment, diet, drinking, smoke, etc. are determinants in breast neoplastic transformation, and are closely associated with a chronic increase in the basal level of oxidative stress. A decrease in oxidative stress state could prevent the development of tumors and, potentially, cancer. In fact, serum markers for oxidative DNA damage have been shown to increase in women diagnosed with breast cancer [29]. On the other hand, it has been suggested that consumption of VOOs, which are particularly rich in phenolic antioxidants, such as HT and TY, should afford considerable protection against breast cancer by inhibiting oxidative stress [2]. In our study we demonstrated that HT and TY reduce basal and H2O2-induced ROS level in breast epithelial MCF10A cells, whereas TY failed to reduce both in MCF7 or MDA-MB-231 cells and HT only reduced H2O2-induced ROS level slightly in breast cancer cells. These results point to a differential antioxidant activity of both compounds between normal breast and tumor cells as we described for squalene [16]. Thus, we suggest that HT and TY could prevent oxidative stress in normal breast cells, thereby preventing the initiation of a chain of reactions to transform normal cells into cancer cells. Noticeably, it is necessary to use a much larger amount of TY to obtain the same ROS reduction level as HT in MCF10A cells (Table 2). Up to 100 μM concentrations of HT and TY used in the present study are probably within the physiological range. However, 500 to 5000 μM of TY exceed this range and could be regarded as being in the pharmacological range.

Di Bendeto et al. [11] described differences between HT and TY in inhibiting cell-mediated oxidation of LDL (100% HT vs. 40% TY) in J774 A.1 macrophage cells due to its intracellular presence. Thus, time-dependent TY, accumulated inside the cell was effective only at later time-points (24 h) or at higher concentrations than HT, which was rapidly found inside the cells and disappeared within 18 h. Thus, we can presume a quick antioxidant defense by HT followed by a slower defense by TY upon VOO intake.

Estrogens, known human breast pro-carcinogens, exert their actions by two mechanisms; the ER-dependent mechanism, involving the activation of ER and subsequent stimulation of cell growth and proliferation [30] or the ER-independent mechanism, involving the generation of genotoxic estrogen metabolites, which are highly reactive and damage DNA by the formation of free radicals and consequently ROS [30]. In accordance with Sirianni et al. [27], HT inhibition of E2-induced MCF7 proliferation does not involve the ER-dependent mechanism but points to an inhibition of the E2 signaling pathway. Felty et al. [31] identified mitochondria as a major source of E2-induced ROS (mtROS) in breast cancer cells and described mtROS as a messenger involved in signaling pathways of cell proliferation control, increasing the transcription of cell cycle genes. These authors found the same amount of mtROS in ER-negative MDA-MB-468 cells and in ER-positive MCF7 or T47D cells, suggesting that mtROS production does not depend on the presence of ER in breast cancer cells. If mtROS acts as a messenger in breast cancer proliferation, it could explain why an antioxidant such as HT reduces E2-induced cell proliferation, as described by Sirianni et al., whereas in the same concentrations without E2 stimulation we do not detect any significant growth alteration.

Cellular protection against oxidative stress is provided by two types of antioxidants; direct antioxidants with a redox activity; and indirect antioxidants (redox active or not) which activated the Nrf2/ARE pathway resulting in transcription of phase II enzymes such as glutathione S-transferase, NAD(P)H: quinone oxidoreductase 1 or glutathione reductase [32]. In addition to the fact that HT and TY act as direct antioxidants, they could also be indirect antioxidants activating the nuclear factor-like 2 (Nrf2). Nrf2, considered a key factor in the cellular defense mechanisms against oxidative stress, might be induced more strongly in MCF10A cells than in MCF7 and might have little or no effect in MDA-MB-231 cells, explaining the differential protection effect of HT and TY on intracellular ROS level. To the best of our knowledge, until now, only Liu et al. [33] have described the protection of HT on ARPE-19 human retinal pigment epithelial cell line from oxidative stress induced by acrolein, a major component of cigarette smoke. Further studies will be necessary to elucidate the possible Nrf2/ARE pathway intervention of HT and TY differential antioxidant activity in breast cell lines.

HT has been described as preventing DNA damage beyond its antioxidant capacity, as it can affect a range of enzymes, including cyclooxygenase and NAD(P)H oxidase while TY has no protective effect [8]. In accordance with these authors, our findings point to a protective effect of HT against basal and H2O2-induced DNA damage regardless of the breast cellular type, whereas TY only has a protective effect on ROS basal level in non-tumoral breast cells.

Both compounds reduce intracellular ROS level and oxidative DNA damage in normal breast cells. This could protect against cellular mutations, preventing carcinogenesis. However, when the disease has occurred, oxidative status in the malignancy place is altered. In this condition, while HT still protects non-tumor breast cells against DNA damage, TY fails to protect them at physiological concentrations. Although HT contributes to reduce DNA damage in normal breast cells, it protects breast tumor cells too. Accordingly, our results must be interpreted carefully, because a reduction of DNA damage in cancer cells might promote cell growth and might inhibit the action of anthracycline chemotherapeutic agents, such as doxorubicin, which induces apoptosis of cancer cells by the oxidative damage resulting from enhanced oxidative state of the cells or, in contrast, might reduce ROS messenger signaling of proliferation resulting in a reduction of tumor cell growth. In any case, we have not detected any modulation of the growth activity in vitro in breast cancer cells after HT or TY treatment at the assayed times.

In this paper, HT has been described as an antioxidant compound with higher activity than TY and related to the prevention of breast cancer, but we must not forget that VOO´s minor compounds can interact with each other, potentiating or inhibiting the effects described for each component alone. Our results indicate some healthy properties of these two simple phenols which may be of interest in pharmacology or as a nutritional supplement or could even lead to establishing the ideal concentrations of each component in VOOs in order to label it as a healthy oil. However, we must be prudent about extrapolating these results regarding epidemiological olive oil health impacts. Future work is needed to investigate these synergetic or inhibitory effects.

5. Conclusions

The simple phenol HT could contribute to the preventive cancer activity attributed to VOOs due to the reduction of oxidative stress and oxidative DNA protection in normal breast cells at physiological concentrations, whereas TY is needed at pharmacological concentrations to reduce oxidative stress and fails to protect DNA damage against an oxidative burst.

Both phenols exert a selective antioxidant defense, preventing oxidation in normal breast cells but not in breast cancer cells, which could be helpful to cancer therapies that increase oxidative stress. HT also prevents induced DNA damage in cancer cells, so it might interfere with these therapies.

Although in vitro studies have pointed to a preventive role of HT against human breast cancer, the precise mechanisms of action remain to be clarified. Further studies are necessary to elucidate the cellular signaling events that HT and TY target in oxidativestress protection and subsequent breast cancer prevention.

Acknowledgments

This study was supported by the “Instituto Andaluz de Biotecnología” (BIOÁNDALUS 08/22/L5.3); Ministerio de Ciencia e Innovación-FEDER (RTA2008-00066-C03-03); and “Centro de Excelencia en Investigación sobre Aceite de Oliva y Salud” (CEAS).

Conflict of Interest

The authors declare no conflict of interest.

References

  • 1.Escrich E., Ramírez-Tortosa M.C., Sánchez-Rovira P., Colomer R., Solanas M., Gaforio J.J. Olive oil in cancer prevention and progression. Nutr. Rev. 2006;64:S40–S52. [Google Scholar]
  • 2.Owen R.W., Giacosa A., Hull W.E., Haubner R., Würtele G., Spiegelhalder B., Bartsch H. Olive-oil consumption and health: The possible role of antioxidants. Lancet Oncol. 2000;1:107–112. doi: 10.1016/s1470-2045(00)00015-2. [DOI] [PubMed] [Google Scholar]
  • 3.Zhang D.D. Mechanistic studies of the Nrf2-Keap1 signaling pathway. Drug Metab. Rev. 2006;38:769–789. doi: 10.1080/03602530600971974. [DOI] [PubMed] [Google Scholar]
  • 4.Allouche Y., Jiménez A., Gaforio J.J., Uceda M., Beltrán G. How heating affects extra virgin olive oil quality indexes and chemical composition. J. Agric. Food Chem. 2007;55:9646–9654. doi: 10.1021/jf070628u. [DOI] [PubMed] [Google Scholar]
  • 5.Cicerale S., Conlan X.A., Sinclair A.J., Keast R.S.J. Chemistry and health of olive oil phenolics. Crit. Rev. Food Sci. Nutr. 2009;49:218–236. doi: 10.1080/10408390701856223. [DOI] [PubMed] [Google Scholar]
  • 6.Brenes M., García A., García P., Rios J.J., Garrido A. Phenolic compounds in spanish olive oils. J. Agric. Food Chem. 1999;47:3535–3540. doi: 10.1021/jf990009o. [DOI] [PubMed] [Google Scholar]
  • 7.Loru D., Incani A., Deiana M., Corona G., Atzeri A., Melis M.P., Rosa A., Dessì M.A. Protective effect of hydroxytyrosol and tyrosol against oxidative stress in kidney cells. Toxicol. Ind. Health. 2009;25:301–310. doi: 10.1177/0748233709103028. [DOI] [PubMed] [Google Scholar]
  • 8.Visioli F., Poli A., Gall C. Antioxidant and other biological activities of phenols from olives and olive oil. Med. Res. Rev. 2002;22:65–75. doi: 10.1002/med.1028. [DOI] [PubMed] [Google Scholar]
  • 9.Cornwell D.G., Ma J. Nutritional benefit of olive oil: The biological effects of hydroxytyrosol and its arylating quinone adducts. J. Agric. Food Chem. 2008;56:8774–8786. doi: 10.1021/jf8015877. [DOI] [PubMed] [Google Scholar]
  • 10.Covas M.-I., Ruiz-Gutiérrez V., de la Torre R., Kafatos A., Lamuela-Raventós R.M., Osada J., Owen R.W., Visioli F. Minor components of olive oil: Evidence to date of health benefits in humans. Nutr. Rev. 2006;64:S20–S30. [Google Scholar]
  • 11.Di Benedetto R., Varì R., Scazzocchio B., Filesi C., Santangelo C., Giovannini C., Matarrese P., D’Archivio M., Masella R. Tyrosol, the major extra virgin olive oil compound, restored intracellular antioxidant defences in spite of its weak antioxidative effectiveness. Nutr. Met. Cardiovasc. Dis. 2007;17:535–545. doi: 10.1016/j.numecd.2006.03.005. [DOI] [PubMed] [Google Scholar]
  • 12.Brand-Williams W., Cuvelier M.E., Berset C. Use of a free radical method to evaluate antioxidant activity. LWT Food Sci. Technol. 1995;28:25–30. [Google Scholar]
  • 13.Re R., Pellegrini N., Proteggente A., Pannala A., Yang M., Rice-Evans C. Antioxidant activity applying an improved abts radical cation decolorization assay. Free Radic. Biol. Med. 1999;26:1231–1237. doi: 10.1016/s0891-5849(98)00315-3. [DOI] [PubMed] [Google Scholar]
  • 14.Prior R.L., Hoang H., Gu L., Wu X., Bacchiocca M., Howard L., Hampsch-Woodill M., Huang D., Ou B., Jacob R. Assays for hydrophilic and lipophilic antioxidant capacity (Oxygen Radical Absorbance Capacity (ORAC(FL))) of plasma and other biological and food samples. J. Agric. Food Chem. 2003;51:3273–3279. doi: 10.1021/jf0262256. [DOI] [PubMed] [Google Scholar]
  • 15.Scudiero D.A., Shoemaker R.H., Paull K.D., Monks A., Tierney S., Nofziger T.H., Currens M.J., Seniff D., Boyd M.R. Evaluation of a soluble tetrazolium/formazan assay for cell growth and drug sensitivity in culture using human and other tumor cell lines. Cancer Res. 1988;48:4827–4833. [PubMed] [Google Scholar]
  • 16.Warleta F., Campos M., Allouche Y., Sanchez-Quesada C., Ruiz-Mora J., Beltran G., Gaforio J.J. Squalene protects against oxidative DNA damage in MCF10A human mammary epithelial cells but not in MCF7 and MDA-MB-231 human breast cancer cells. Food Chem. Toxicol. 2010;48:1092–1100. doi: 10.1016/j.fct.2010.01.031. [DOI] [PubMed] [Google Scholar]
  • 17.Lee D.H., Lim B.-S., Lee Y.-K., Yang H.-C. Effects of hydrogen peroxide (H2O2) on alkaline phosphatase activity and matrix mineralization of odontoblast and osteoblast cell lines. Cell Biol. Toxicol. 2006;22:39–46. doi: 10.1007/s10565-006-0018-z. [DOI] [PubMed] [Google Scholar]
  • 18.Singh N.P., McCoy M.T., Tice R.R., Schneider E.L. A simple technique for quantitation of low levels of DNA damage in individual cells. Exp. Cell Res. 1988;175:184–191. doi: 10.1016/0014-4827(88)90265-0. [DOI] [PubMed] [Google Scholar]
  • 19.Lasekan J.B., Clayton M.K., Gendron-Fitzpatrick A., Ney D.M. Dietary olive and safflower oils in promotion of DMBA-induced mammary tumorigenesis in rats. Nutr. Cancer. 1990;13:153–163. doi: 10.1080/01635589009514056. [DOI] [PubMed] [Google Scholar]
  • 20.Perona J.S., Arcemis C., Ruiz-Gutierrez V., Catalá A. Effect of dietary high-oleic-acid oils that are rich in antioxidants on microsomal lipid peroxidation in rats. J. Agric. Food Chem. 2005;53:730–735. doi: 10.1021/jf048374p. [DOI] [PubMed] [Google Scholar]
  • 21.Leopoldini M., Marino T., Russo N., Toscano M. Antioxidant properties of phenolic compounds: H-Atom versus electron transfer mechanism. J. Phys. Chem. A. 2004;108:4916–4922. [Google Scholar]
  • 22.Visioli F., Galli C., Bornet F., Mattei A., Patelli R., Galli G., Caruso D. Olive oil phenolics are dose-dependently absorbed in humans. FEBS Lett. 2000;468:159–160. doi: 10.1016/s0014-5793(00)01216-3. [DOI] [PubMed] [Google Scholar]
  • 23.Weinbrenner T., Fitó M., de la Torre R., Saez G.T., Rijken P., Tormos C., Coolen S., Albaladejo M.F., Abanades S., Schroder H., et al. Olive oils high in phenolic compounds modulate oxidative/antioxidative status in men. J. Nutr. 2004;134:2314–2321. doi: 10.1093/jn/134.9.2314. [DOI] [PubMed] [Google Scholar]
  • 24.Quiles J.L., Farquharson A.J., Simpson D.K., Grant I., Wahle K.W.J. Olive oil phenolics: Effects on DNA oxidation and redox enzyme mRNA in prostate cells. Br. J. Nutr. 2002;88:225–234. doi: 10.1079/BJN2002620. [DOI] [PubMed] [Google Scholar]
  • 25.Fabiani R., de Bartolomeo A., Rosignoli P., Servili M., Montedoro G.F., Morozzi G. Cancer chemoprevention by hydroxytyrosol isolated from virgin olive oil through g1 cell cycle arrest and apoptosis. Eur. J. Cancer Prev. 2002;11:351–358. doi: 10.1097/00008469-200208000-00006. [DOI] [PubMed] [Google Scholar]
  • 26.Menendez J.A., Vazquez-Martin A., Colomer R., Brunet J., Carrasco-Pancorbo A., Garcia-Villalba R., Fernandez-Gutierrez A., Segura-Carretero A. Olive oil’s bitter principle reverses acquired autoresistance to trastuzumab (Herceptin™) in HER2-Overexpressing breast cancer cells. BMC Cancer. 2007;7:80. doi: 10.1186/1471-2407-7-80. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Sirianni R., Chimento A., de Luca A., Casaburi I., Rizza P., Onofrio A., Iacopetta D., Puoci F., Andò S., Maggiolini M., et al. Oleuropein and hydroxytyrosol inhibit MCF-7 breast cancer cell proliferation interfering with ERK1/2 activation. Mol. Nutr. Food Res. 2010;54:833–840. doi: 10.1002/mnfr.200900111. [DOI] [PubMed] [Google Scholar]
  • 28.Berger M.M. Can oxidative damage be treated nutritionally? Clin. Nutr. 2005;24:172–183. doi: 10.1016/j.clnu.2004.10.003. [DOI] [PubMed] [Google Scholar]
  • 29.Musarrat J., Arezina-Wilson J., Wani A.A. Prognostic and aetiological relevance of 8-hydroxyguanosine in human breast carcinogenesis. Eur. J. Cancer. 1996;32A:1209–1214. doi: 10.1016/0959-8049(96)00031-7. [DOI] [PubMed] [Google Scholar]
  • 30.Yager J.D., Davidson N.E. Estrogen carcinogenesis in breast cancer. N. Engl. J. Med. 2006;354:270–282. doi: 10.1056/NEJMra050776. [DOI] [PubMed] [Google Scholar]
  • 31.Felty Q., Xiong W.-C., Sun D., Sarkar S., Singh K.P., Parkash J., Roy D. Estrogen-induced mitochondrial reactive oxygen species as signal-transducing messengers. Biochemistry. 2005;44:6900–6909. doi: 10.1021/bi047629p. [DOI] [PubMed] [Google Scholar]
  • 32.Dinkova-Kostova A.T., Talalay P. Direct and indirect antioxidant properties of inducers of cytoprotective proteins. Mol. Nutr. Food Res. 2008;52:S128–S138. doi: 10.1002/mnfr.200700195. [DOI] [PubMed] [Google Scholar]
  • 33.Liu Z., Sun L., Zhu L., Jia X., Li X., Jia H., Wang Y., Weber P., Long J., Liu J. Hydroxytyrosol protects retinal pigment epithelial cells from acrolein-induced oxidative stress and mitochondrial dysfunction. J. Neurochem. 2007;103:2690–2700. doi: 10.1111/j.1471-4159.2007.04954.x. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Nutrients are provided here courtesy of Multidisciplinary Digital Publishing Institute (MDPI)

RESOURCES