Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2011 Nov 15.
Published in final edited form as: Cancer Biol Ther. 2010 Jun 26;9(11):853–861. doi: 10.4161/cbt.9.11.11632

2-deoxy-D-glucose induces oxidative stress and cell killing in human neuroblastoma cells

Damon C Shutt 1,2,*, M Sue O'Dorisio 2, Nukhet Aykin-Burns 1, Douglas R Spitz 1
PMCID: PMC3215774  NIHMSID: NIHMS335886  PMID: 20364116

Abstract

Malignant cells have a demonstrably greater sensitivity to glucose deprivation-induced cytotoxicity than normal cells (Biochem J 418:29). This has been hypothesized to be due to a higher level of reactive oxygen species (ROs) production in cancer cells leading to the increased need for reducing equivalents, produced by glucose metabolism, to detoxify hydroperoxides. Because complete glucose deprivation cannot be achieved in vivo, it has been proposed that agents that antagonize glucose metabolism, such as 2-deoxy-D-glucose (2DG), can mimic in vitro glucose deprivation that selectively kills cancer cells by oxidative stress. To test this hypothesis, neuroblastoma cell lines were treated with 2DG and the effects on clonogenic survival and the distribution of cellular phenotypes among surviving colonies was determined. The results showed that all three major cell types found in neuroblastoma (schwann, Neuronal and Intermediate) were sensitive to 2DG-induced clonogenic cell killing. Furthermore, treatment with the thiol antioxidant, N-acetyl cysteine, or with polyethylene glycol-conjugated superoxide dismutase and catalase, protected neuroblastoma cells from 2DG-induced cell killing. Finally normal non-immortalized neural precursor cells were relatively resistant to 2DG-induced cell killing when compared to neuroblastoma cell lines. These results support the hypothesis that inhibitors of glucose metabolism could represent useful adjuvants in the treatment of neuroblastoma by selectively enhancing metabolic oxidative stress.

Keywords: glucose deprivation, oxidative stress, clonogenic survival, N-acetyl cysteine, superoxide dismutase, catalase, I-cells, N-cells, S-cells, SK-N-SH, SH-SY5Y

Introduction

Neuroblastoma is the most common extracranial solid tumor of childhood, accounting for up to 15% of childhood cancer deaths.1 Relapse-free survival rates for high-risk neuroblastoma have remained low due to the escape and eventual outgrowth of treatment-resistant primitive precursor cells within the heterogeneous tumor population. Relative to other cancer types, neuroblastoma and other neural crest-derived tumors rank among the most sensitive to radiotherapy, but are often diagnosed after extensive metastasis;2 this makes an effective external beam radiation regimen to all satellite tumors very difficult, and elevates the risk of extensive damage to normal tissue. Because of this, current approaches to neuroblastoma therapy rely upon on the selective toxicities of chemotherapeutics instead of radiosensitivity. Furthermore, because the relatively hypoxic bone marrow is a sanctuary for metastatic tumor initiating cells in children with advanced disease,3 reduced oxygen tension also reduces the potential efficacy of radiotherapy. We were therefore interested in determining the sensitivity of neuroblastoma cells to chemotherapeutic agents which promote oxidative stress through inhibition of normal metabolic pathways.

Because neuroblastoma cell lines and tumors exhibit a range of cell morphologies that are believed to be indicative of treatment responsiveness, we were interested in determining whether one or more of the cell types present are preferentially sensitive to inhibitors of glucose metabolism. It is well accepted that there are three basic neuroblastoma cell types, commonly termed I (intermediate), N (neuroblastic) and S (Schwann-like)4 appearing at a variable percentage between cell lines. Among these, it is hypothesized that the I-type has many characteristics of stem cells, the N-type cells are more differentiated but still tumorigenic primitive neural cells, and the S-type cells are the most differentiated and the least tumorigenic of the three.5 An extensive literature exists describing the clinical, histological and molecular characteristics of neuroblastoma that are predictive of disease progression (the Shimada classification),6,7 including such factors such as age at diagnosis, degree of differentiation, N-MYC amplification and mitotic index.8,9 While the two descriptive systems (Shimada and I-type/N-type/S-type) have a different emphasis, they describe many of the same characteristics that can both predict tumorigenicity in cell lines and can predict disease outcome based on tumor histology.10,11 Little is currently known about the relative efficacy of chemotherapeutics on the different cell types found in neuroblastoma tumors and cell lines.

It has been increasingly recognized that malignant cells generate higher basal levels of reactive oxygen species (ROS) than normal tissue cells, and that they consume glucose at a higher rate than untransformed cells.12,13 One of the hypotheses to explain these observations is that cancer cells produce more ROS due to defects in oxidative metabolism, and glucose metabolism is upregulated to generate increased levels of reducing equivalents for detoxifying the resulting ROS, particularly hydroperoxides.13-15 This theory has been tested in in vitro glucose deprivation experiments, where cancer cells were found to be much more susceptible to oxidative stress-induced cell death than their normal tissue counterparts as a result of removing glucose.13,16,17 One explanation for these observations is that glucose metabolism generates pyruvate, which can scavenge hydroperoxides directly, and also regenerates NADPH, which provides electrons for detoxification of hydroperoxides via glutathione- and thioredoxin-dependent peroxidases.13,18-24

Cancer cells are hypothesized to be more dependent on these glucose-dependent hydroperoxide metabolizing pathways in order to avoid the oxidative damage caused by higher steady-state levels of ROS produced as by-products of metabolism.13,25,26 In the current study, the hypothesis that inhibition of glucose metabolism using 2DG could kill cells via oxidative stress was tested in two human neuroblastoma cell lines as well as non-immortalized neural precursor cultures derived from embryonic stem cells. The results indicated that both neuroblastoma cell lines, as well as all the subpopulations of I-type, N-type and S-type cells were sensitive to 2DG-induced clonogenic cell killing, apparently via metabolic oxidative stress. The non-immortalized neural precursor cells were relatively resistant to the toxicity of 2DG. This is the first report of its kind that examines the phenotypic diversity among surviving neuroblastoma clonogens after in vitro treatment, and further suggests that 2DG could represent an effective adjuvant for treating all neuroblastoma subtypes with chemo-therapeutic agents that induce oxidative stress.

Results

Colony phenotype and cell type determination

The initial experiments prior to assessing the sensitivity of SK-N-SH neuroblastoma to 2DG treatment revealed at least three distinct morphologies of clonogenic colonies (Fig. 1A–C). A similar range of cell morphologies was observed for SH-SY5Y as well (Fig. 1D–F). The three basic cell morphologies within these colonies can be described as (1) small and spiky cells (I-type), as seen in (Fig. 1A and D); (2) medium sized, spherical cells (N-type) as seen in Figure 1B and E; and (3) large, spread cells (S-type), as shown in Figure 1C and F. These observations correlate with the three subtypes of cells found within human neuroblastoma cell lines as reported by Biedler and others.4,5,11,27 When examining expression of bcl-2 and tissue transglutaminase II (tTGII) in SK-N-SH, Balster et al. found a high expression of bcl-2 in N-type cells (moderate-sized, spherical cells), and enrichment for the tTGII marker in S-type cells (large, spread cells), while the I-type cells were found to express both.27 According to this classification, ~46% of the surviving colonies in SK-N-SH were I-type, ~39% were N-type, and ~15% of surviving clones were S-type (Table 1). The related SH-SY5Y line showed a different distribution, with markedly fewer I-type cell colonies and concomitantly more of the N-type (Table 1).

Figure 1.

Figure 1

Colony morphology in SK-N-SH and SH-SY5Y cell lines. In the clonogenic survival experiments, different surviving colony morphologies arose from single cells during the 17 day cloning period and were fixed and stained with Coomassie Blue. On the left, the cells in colonies are small and spiky (I-type), in the middle panels there are moderate-sized cells that are more rounded (N-type), and on the right side are cells with a large spread morphology (S-type). Photos were taken using an inverted Nikon microscope with a 4× objective.

Table 1.

Percent of surviving colonies in control plating efficiency experiments

Cell line % small, spiky (Intermediate) % medium, round (Neuroblast) % large, spread (Schwann)
SK-N-SH (n = 9) 46 ± 13 39 ± 9 15 ± 7
SH-SY5Y (n = 9) 36 ± 6 54 ± 7 10 ± 4
p-value 0.04 0.03 0.06

Immunophenotyping neuroblastoma cell subtypes

Based on this initial set of observations, we used antibodies to examine proteins expressed in cultures of SK-N-SH and SH-SY5Y using Tissue transglutaminase II (tTGII, a Schwann cell marker), Tuj1 (against neuronal-specific class III β-tubulin) and CD133 (a stem cell marker). While all cells in these cultures expressed the class III β-tubulin at some level, only a subset expressed signifi-cant amounts of either tTGII or CD133. tTGII was expressed at the highest level in the large spread cells such as those shown in Figure 1C and F (Fig. 2, red arrows). SH-SY5Y cultures displayed fewer of the tTGIIhigh cells than SK-N-SH, agreeing with Table 1, with the difference made up by having more of the Tuj1+ CD133low and tTGIIlow cells (Fig. 2 and green arrows), also agreeing with Table 1. Both cell lines displayed a relatively low concentration of the CD133high stem cell-like population, at 1–3% of the overall population. As a marker for I cells, this level of CD133high cells falls short when compared to the distribution in Table 1. This originally suggested that CD133 might not be a good marker to correlate with the number of I-type colonies shown in Figure 1 and quantified in Table 1. However, because expression of CD133 is overall somewhat low in many of the I-type cells, by increasing the exposure relative to the signal for tTGII and Tuj1, it becomes clearly detectable. When counting all the CD133+ cells of SK-N-SH (bluish-green and purple in Fig. 2A) a good agreement with the data in Table 1 is observed. For SH-SY5Y, the relative expression between the three antigens is different than for SK-N-SH, as assessed by the color mixture (pinkish-orange and purple) seen in the cells that are neither green nor red in Figure 2B relative to 2A. However, it is clear that there are relatively more green cells (N-type) in 2B than 2A, and concomitantly more bluish or mixed-color cells (I-type) in 2A than 2B, while a roughly equivalent proportion of red cells (S-type) are seen in the microscopic fields of the two cell lines (Fig. 2A and B). This provides evidence in favor of the assignment of the (untreated) colony morphologies to the systematic cell types previously observed8-11 also as demonstrated in Figure 1, and is furthermore consistent with the proportional differences found between the two cell lines as quantified in Table 1.

Figure 2.

Figure 2

Indirect immunofluorescence images of SKNSH (A) and SHSY5Y (B). Shown in red is rabbit polyclonal anti-tissue transglutaminase (a marker of Schwann cells), detected with goat anti-rabbit Alexa 647; Shown in green is anti-class III b-tubulin (neuron-specific), detected with goat anti-chicken Alexa 555; Shown in blue is anti-CD133, a stem cell marker, detected with goat anti-mouse Alexa 488. Arrows pointing to cells showing as red are Schwann-type, green are neuroblast-type, and blue are CD133high stem cell-like (I-type) neuroblastoma cells. Smaller white arrows designate cells with mixed expression, neither red nor green, which are intermediate (I-type) cells. 20× objective.

In counting the clonogenic colonies according to their cell type, it was also appreciated that there are mixtures of cell types within a subset of these colonies. In the examples shown in Figure 1, there are a few small spiky cells (I-type) that can be discerned in B and E (designated N-type colonies), and some medium-sized spherical cells in the colonies shown in C and F (cells designated as S-type). In the occasional more diverse cases, a colony was scored according to the type representing the majority of the total (>10% greater than the second most prevalent), of cells in a given colony. In the relatively few numbers of colonies that exhibited significant numbers of all three morphologies, and where no cell type was clearly prevalent, it was scored as an N-type. These, although rare, always had a significant proportion of the N-type cells, and a similar rationale was used that designates the N-type cells as “somewhat differentiated” in the tumor histology classification systems.6-9 Interestingly, another relatively rare subset of colonies displayed evidence of “switching” in cellular phenotype as suggested by colony sectoring (e.g., Fig. 3A and B), providing evidence that a given clone is capable of generating more than one cellular phenotype within a given colony.

Figure 3.

Figure 3

Evidence of switching by colony sectoring of a neuroblastoma clonogen. To generate colonies derived from a single cell, 200–400 cells were plated in a 60 mm tissue culture plastic dish and grown in normal medium for 17 days before fixation and staining with Coomassie Blue (as in Fig. 1). At this density, discrete colonies form from surviving single cells of the original population, a measure of maintenance of cellular reproductive integrity. In both examples, a white border separates cells which have changed morphology during clonal expansion; in one case (A) between the I-type and S-type, and in the other example (B) between the I-type and the N-type. Photos were taken as in Figure 1.

The best fit of the data presented in Figures 1 and 2 with the clinically accepted neuroblastoma cell classifications observed within tumors, suggests that the small spiky (I-type) correspond to the undifferentiated histology of the most primitive neural crest cells, classified as “undifferentiated”. The medium-sized spherical cells (N-type) fit best with the “somewhat differentiated” histology, and the large flat cells (S-type) best fit the category of “differentiated”. The latter can alternatively be described as having a large stromal cell component, according to the Shimada and other histopathological classification systems.4-9,11,27

2DG induces cell killing among all three major neuroblastoma cell phenotypes

Knowing the distribution of cell type found within surviving clonogens without treatment, experiments were conducted to determine the sensitivity of the different neuroblastoma subtypes to 2DG-induced clonogenic cell killing. Two different physiologically relevant concentrations of 2DG were tested for an effect on clonogenic survival in time course experiments for both cell lines.28 SK-N-SH cells were treated for 1, 2 and 3 days with 5.5 mM or 11 mM 2DG and scored for the proportion of surviving colony phenotypes after treatment. All three clonal subtypes in SK-N-SH showed statistically significant clonogenic cell killing in response to both 5.5 mM as well as 11 mM 2DG at both 48 and 72 hours of treatment when compared to their untreated controls (Fig. 4A). No significant differences were discernable in the degree of clonogenic cell killing between the three colony phenotypes (I vs. N, I vs. S or N vs. S).

Figure 4.

Figure 4

Clonogenic survival of SK-N-SH neuroblastoma categorized according to the predominant cell morphology of each colony of 50 or more cells. After 24, 48 or 72 hours of treatment with either 5.5 mM or 11 mM 2DG, plates were harvested, counted, and 200–400 cells per 60 mm plate were grown undisturbed for 17 days in normal complete medium. Each point represents data from three independent experiments, with each data point per experiment done in triplicate. Culture morphology was scored as suggested in Figure 1, with “stem” or I-type cells shown in (1A and D) neuroblast or N-type cells shown in (1B and E) and schwann or S-type cells shown in (1C and F) for each cell line. (A) result for SK-N-SH cell line, (B) for SH-SY5Y line. Data is presented as the average with errors graphed as + the standard error of the mean (SEM) for three experiments each done in triplicate. Asterisks indicate statistical significance (p < 0.05).

In general the SH-SY5Y line, a common model for immature neurons in vitro,29 is somewhat more resistant (relative to SK-N-SH) to 2DG-induced clonogenic cell killing at 24 hours of exposure (Fig. 4B). However, significant clonogenic cell killing (p < 0.05) occurred in all subtypes of the SH-SY5Y with 5.5 mM 2DG at 48 and 72 hours (40–78%). Likewise, significant (p < 0.05) clonogenic killing occurred in all SH-SY5Y subtypes with 11 mM 2DG at all exposure time points (40–90% clonogenic cell killing; Fig. 4B p < 0.05). Therefore both the more resistant SH-SY5Y and the more sensitive SK-N-SH neuroblastoma cell lines were clonogenically inactivated by 2DG-exposure for 48 and 72 hours.

A comparison of the 2DG-induced clonogenic cell killing between the two cell lines, when combining all three cell subtypes, also reveals a similar pattern of increased sensitivity in SK-N-SH relative to SH-SY5Y (Fig. 5). However, with the exception of the 24 hr 5.5 mM 2DG treatment group in SH-SY5Y cells, there are significant reductions (p < 0.05) in clonogenic survival in all groups. The difference in overall 2DG susceptibility between the two neuroblastoma cell lines is not unexpected because SH-SY5Y is a neuroblastic subclone of SK-N-SH and neuroblastic cells are thought to more tumorigenic.30 In addition the differences in the proportion of each of the subtypes in SK-N-SH and SH-SY5Y shown in Table 1 may account for some of the overall differences in 2DG sensitivity observed in Figure 5. However, the major finding is that both the more resistant SH-SY5Y and the more sensitive SK-N-SH neuroblastoma cell lines are clonogenically killed by 2DG-exposure at 48 and 72 hours.

Figure 5.

Figure 5

Effect of 2DG on clonogenic cell killing in two neuroblastoma cell lines with no distinction between colony morphology. The data for both cell lines is the same as presented in Figure 4, but all colonies are counted as one category instead of three. The cells were treated as described in Methods, with either no treatment (control) or treatment with 5.5 mM 2DG or 11 mM 2DG. After 24, 48 or 72 hours of incubation with the indicated drug concentration, cells were harvested, plated and processed for colony counting as described. Significance at the <0.05 p value was calculated using Student's paired t-test and is indicated by an asterisk above the data point. Data is presented as the mean ± SEM. Asterisks indicate data points that vary significantly from control, and significant differences between the two dosage groups are shown by a line and asterisk above the pair.

Normal neural precursor cells are less susceptible to 2DG-induced cell killing than neuroblastoma cells

To determine if differential sensitivity to 2DG-induced toxicity could be demonstrated between neuroblastoma vs. normal neural cells, changes in trypan blue dye exclusion and propidium iodide uptake were measured in normal embryonic stem cell-derived neural precursor cells (NPC) and SK-N-SH neuroblastoma cells exposed to 2DG (Table 2). These endpoints of cytotoxicity were used instead of clonogenic cell survival because NPC cells have very low cloning efficiencies. After 48 or 72 hours of treatment with 5.5 mM 2DG, there was no significant cytotoxicity as determined by trypan blue dye exclusion in normal NPC cells. In contrast, there was significant 2DG-induced cytotoxicity detected using trypan blue dye exclusion in SK-N-SH neuroblastoma cells at both 48 and 72 hours of exposure (Table 2, p < 0.01). Furthermore there was also significantly more cytotoxicity detected in SK-N-SH cells (relative to NPC) at 48 and 72 hours of 2DG exposure using the propidium iodide uptake assay (Table 2, p < 0.01). These results demonstrate that 2DG is differentially cytotoxic to normal neural vs. neuroblastoma cells in vitro and provide support for the hypothesis that 2DG might be useful in therapy protocols for neuroblastoma.

Table 2.

Toxicity of 2-deoxy-D-glucose in normal (NPC) vs. cancer (SK-N-SH) cells

Normalized Surviving Fraction (%) (to untreated cells at the same time point) p-values
Trypan Blue Prop. Iodide Range (T.B.) (P.I.)
48 hr NPC 92.5 ± 3.4 94.5 ± 5.4 89.5–102.0 (NS)* (NS)*
48 hr SK-N-SH 70.5 ± 4.1 74.3 ± 4.7 66.1–79.2 p < 0.003** p < 0.009**
72 hr NPC 97.9 ± 4.4 88.3 ± 3.7 84.0–101.7 (NS)* p < 0.02*
72 hr SK-N-SH 54.3 ± 5.0 63.5 ± 12.9 47.4–74.7 p < 0.001** p < 0.005**
*

relative to untreated cells from the same cell type, student's t-test.

**

relative to 2DG-treated NPC, students t-test.

Treatment with antioxidants protects neuroblastoma cells from 2DG-induced cell killing

To directly test the hypothesis that the toxicity of 2DG was mediated by metabolic oxidative stress, experiments were conducted in which the effects of adding either a non-specific thiol antioxidant, N-acetylcysteine (10 mM NAC), or 50 U/mL pegylated super-oxide dismutase (to scavenge O2•-) plus 50 U/mL pegylated catalase (to scavenge H2O2) were accomplished in SK-N-SH and SH-SY5Y human neuroblastoma cell lines (Fig. 6). When normalizing the 2DG treated cell data to the respective untreated control, significant 2DG-induced cell killing was again observed in both neuroblastoma cell lines. However, when NAC and 2DG were added simultaneously, a significant increase in clonogenic cell survival was noted relative to 2DG alone in both cell lines, indicating that the non-specific thiol antioxidant was capable of protecting both human neuroblastoma lines from 2DG-induced toxicity (Fig. 6). When comparing the SK-N-SH and SH-SY5Y cell lines, NAC was capable of nearly completely rescuing SK-N-SH, while only partially rescuing SH-SY5Y from 2DG-induced clonogenic cell killing. When 2DG was administered in combination with PEG-SOD/CAT, only the SH-SY5Y cells were significantly protected from 2DG-induced toxicity (Fig. 6). These results provide strong support for the hypothesis that 2DG-induced cell killing is mediated by oxidative stress in both human neuroblastoma cell lines. These results also support the hypothesis that thiol oxidation could be a common mechanism of 2DG-induced cell killing between the two cell lines, whereas superoxide and hydrogen peroxide may only play a significant role in 2DG-induced cell killing in SH-SY5Y cells.

Figure 6.

Figure 6

Protection from 2DG-induced neuroblastoma cell killing by the addition of antioxidants. Experiments as in Figures 4 and 5 were repeated, but also included treatments with either 10 mM NAC or 50 U/mL PEG-SOD and PEG-CAT with and without 11 mM 2DG. Data were normalized to the no treatment (control) for 2DG, normalized to NAC for 2DG + NAC, and normalized to SOD + CAT for the 2DG + SOD + CAT group. Data is presented as the mean + SEM for these experiments. Asterisks indicate data points that vary significantly from the experimental control (p < 0.05).

Discussion

The clonogenic survival assay is a commonly used approach in cancer biology for testing the efficacy of a therapy in vitro.31 In order to determine the therapeutic potential of agents that elicit metabolic oxidative stress, we exposed neuroblastoma to 2DG. 2DG was effective in killing human neuroblastoma cells at therapeutically achievable levels (5–10 mM),44 based on the results of this clonogenic survival assay. The protection of neuroblastoma cells against 2DG-induced cell death by antioxidants is further evidence that 2DG-induced killing is occurring, at least in part, through increased metabolic oxidative stress caused by restricted glucose metabolism.

Interestingly, the broadly distributed small molecule thiol-based antioxidant, N-acetyl cysteine (NAC), was able to significantly rescue both neuroblastoma cells lines from 2DG-induced clonogenic cell killing while treatment with PEG-SOD/CAT was only able to rescue the SH-SY5Y cell line. Since PEG-SOD and PEG-catalase are believed to be specific for superoxide and hydrogen peroxide scavenging as well as having limited access to some subcellular organelles, these results support the speculation that either the source of ROS [or reactive nitrogen species (RNS)] production in the case of SK-N-SH may not be accessible to PEG-SOD/CAT, or the ROS/RNS involved in 2DG-induced killing in SK-N-SH may not be detoxified by PEG-SOD/CAT. These results are in contrast to those with the broadly distributed small molecule antioxidant, NAC, which was able to significantly protect both neuroblastoma cell lines from 2DG-induced cytotoxicity, and which suggests thiol oxidation or alkylation resulting from reactions of ROS or RNS is a common and important mechanism of toxicity in both cell lines. Overall these results with antioxidants suggest that when compared with SK-N-SH, SH-SY5Y is likely to have a differential compartmentalization of the mechanisms responsible for 2DG-induced cytotoxicity and oxidative stress but both neuroblastoma cell lines are sensitive to 2DG-induced oxidative stress.

Clearly, antagonizing glucose metabolism with 2DG may cause toxicity by more than one mechanism in different cancer cells, depending on the type of metabolic oxidative defect that is present in a given cell type or cell line. As 2DG is believed to cause disruptions in thiol metabolism,32,40 the observed ability of NAC to rescue both neuroblastoma cell lines from 2DG-induced toxicity in the current study is consistent with thiol oxidation (or alkylation) being a key component of 2DG-induced cell killing in neuroblastoma. Because PEG-SOD plus PEG-CAT (which are most likely localized to the cytoplasm) did not effectively rescue the SK-N-SH line from 2DG toxicity, it raises the possibility that the increases in reactive oxygen species (or reactive nitrogen species) in SK-N-SH may occur inside the mitochondria, which are not accessible to cytoplasmic enzymes. Alternatively, steady-state levels of thioredoxin- and/or glutathione-dependent decomposition of organic hydroperoxides may be relatively more important to 2DG-induced toxicity in SK-N-SH, while superoxide and hydrogen peroxide may have a more dominant role in 2DG-induced toxicity for SH-SY5Y. It is also possible the reactive nitrogen species such as nitric oxide and peroxynitrite could contribute to 2DG-induced cytotoxicity since they would be expected to be detoxified by NAC and have been suggested to contribute to cytotoxicity during glucose deprivation of glioma cells.45 The current studies point out that the biochemical characteristics of the metabolic oxidative stress induced by inhibition of glucose metabolism in neuroblastoma cell types may differ significantly as well as contribute to sensitivity to 2DG-induced cytotoxicity. However, our results continue to support the overall conclusion that inhibitors of glucose metabolism appear to exacerbate cell killing and oxidative stress in all the neuroblastoma cell lines and cell subtypes we tested, irrespective of the predominant source of reactive species that contributes to cell killing. These results further suggest that inhibitors of glucose and hydroperoxide metabolism may represent a useful adjuvant for sensitizing neuroblastomas to conventional chemo-therapy agents thought to induce oxidative stress.33-35,40

Because induction of metabolic oxidative stress and manipulation of intracellular antioxidants is easily achieved in the context of many cancer therapies, and neuroblastoma cellular subtypes including a stem-like I-type36-38 appear to be sensitive to 2DG-induced cytotoxicity and oxidative stress, neuroblastoma may represent a useful model for testing the effects of novel chemo-therapy combinations based on taking advantage of defects in cancer cell oxidative metabolism.33-35,39-41 As with all cancer cell types studied thus far, neuroblastoma cells also demonstrate significantly greater susceptibility to the cytotoxic effects of 2DG, relative to non-transformed cells of a similar developmental derivation. These results continue to support the hypothesis that the biochemical approach of antagonizing glucose and hydroperoxide metabolism for selectively enhancing oxidative stress for the purpose of improving cancer therapy responses,13-17,24,32,40,42-44,46 appears to represent a promising new strategy that can be applied to developing novel combined modality cancer therapies for the treatment of neuroblastoma.27,33-35,41

Materials and Methods

Cells and culture

Human neuroblastoma cell lines SK-N-SH and SH-SY5Y were maintained in MEM and low glucose DMEM, respectively, both supplemented with penicillin-streptomycin, MEM non-essential amino acids, L-glutamine, and 15% fetal bovine serum (Invitrogen). Cultures were grown in a 5% CO2 humidified incubator and split when reaching 80% confluency. Fresh cells were thawed for use after 6 weeks of growth to maintain consistency of passage number between experimental replicates.

Normal non-immortalized neural precursor cells, derived from the NIH-approved and karyotypically normal WA09 hESC cell line, were purchased from Neuromics, Inc., (Minneapolis, MN). Cells were expanded for 10 days on Matrigel (BD Biosciences)—coated culture dishes in supplemented serum-free medium supplied by the company with the following recommended additions: penicillin-streptomycin, 2 mM L-glutamine (Invitrogen) and 20 ng/ml of bFGF (R&D Systems, Minneapolis, MN). Cultures during expansion were maintained in humidified 5% CO2 incubator at 37°C, splitting at 1:2 or 1:3 when 90–100% confluency was achieved and feeding the cultures with fresh medium every 2 days. After 10 days, two 35 mm treatment dishes and two control dishes for each time point were plated and grown for 24 hours in MEM prior to addition of 5.5 mM 2-DG, as was done in parallel for SK-N-SH. A second isolate if the neural precursor cells was obtained, cultured as described and the experiment was repeated in triplicate.

Drug treatments

2-deoxy-D-glucose was purchased from Sigma Chemical Co., (St. Louis, MO). The glucose concentration in cell culture medium was 5.5 mM, so a 200× stock of 2DG was made by dissolving in PBS at a 1.1 M concentration, and sterile filtered. Cells were plated in treatment dishes and allowed to attach and recover for 36–48 hours before drug treatment. Sterile drug was added to fresh medium at the indicated concentration and the medium in the culture dish was replaced with drug and incubated under the standard culture condition for 24, 48 or 72 hours before harvest, as indicated. For the antioxidants, a 1 M N-acetyl cysteine (100x; Sigma) stock solution was made in PBS with 1 M sodium bicarbonate and used in cultures at 10 mM, with bicarbonate alone used as the treatment control. Pegylated SOD and pegylated catalase were both purchased from Sigma and dissolved in sterile PBS at 5 U per μl or 50 U per μl for stock solutions, respectively, and each were used to treat cultures at a concentration of 50 U per ml. In the SOD/CAT experiments, polyethylene glycol alone was used as a control.

Immunofluorescence

After seeding at low density on glass chamber slides in normal complete medium, cells were grown for 4–7 days, replenishing medium every 2 days. Cultures were removed from the incubator, the medium was removed and replaced with freshly prepared 4% paraformaldehyde (Electron Microscopy Sciences) in 1× Dulbecco's Phosphate Buffered Saline (D-PBS, Invitrogen) for 15 minutes at room temperature. Two post-fixation rinses were performed, incubating for 5 min each in fresh PBS. Finally, Fresh PBS was added and the chamber slides with fixed cells, then sealed and stored at 4°C until processing for indirect immunofluorescence microscopy.

When samples were processed, the glass chamber slides were dismantled leaving only the glass slide for further processing. All subsequent steps were performed at room temperature unless otherwise noted. The slides were placed in a humidity chamber and the PBS storage buffer was removed and replaced with fresh for 5 min. Cells were permeabilized with a 0.2% deoxycholate, 0.2% Triton X-100 (Sigma) solution in PBS for 15 min, followed by a 30 min blocking step with 5% heat inactivated normal goat serum (Sigma) and 2% w/v of Fraction V BSA in PBS. The primary antibodies were diluted into a 1% BSA, 0.1% Tween-20 PBS (antibody diluent) mixture as follows: mouse monoclonal anti-CD133 (Miltenyi) at 1:20; rabbit polyclonal anti Tissue Transglutaminase II (LabVision) at 1:50; and chicken polyclonal anti-class II β-tubulin (Tuj1, Neuromics) at 1:1,000. After addition of the antibodies, slides were incubated overnight at 4°C in the humidity chamber. The following day, slides were rinsed 3 × 5 min each in antibody diluent, followed by incubating in the dark with Alexa 647 goat anti-rabbit, Alexa 555 goat anti-chicken, and Alexa 488 goat anti-mouse secondary antibodies (Molecular Probes) for 2–3 hours. Prior to mounting, slides were rinsed in antibody diluent, PBS-T, and PBS for 5 min each, dipped into ddH2O very briefly, dried and coverslips mounted with Prolong Gold antifade with DAPI (Molecular Probes). After the mountant had set for at least 16 hours, slides were examined with a Nikon BX-51 fluorescence microscope and photographed with a 20× objective using a SPOT Camera and the manufacturer's supplied acquisition software. Images obtained from the different channels were combined to generate an overlay of fluorescence signals with Adobe Photoshop CS4 software.

Clonogenic survival assay

Cultures were harvested by first collecting all non-adherent cells in the dish, trypsinization of the adherent cells, combining, pelleting and then resuspending in fresh medium, followed by counting with a Coulter counter. Cells were plated at a low density (200–300 per dish) in 60 mm clonogenic expansion dishes, in triplicate for each treatment and time point. Each experiment was independently repeated a minimum of three times for analyzing significance through statistical analysis (described below). After the expansion of the surviving cells for 17 days, cultures were fixed with 70% ethanol and stained with Coomassie blue for clonogenic cell survival analysis as described.26,31

Normal neuronal cell survival assays

To determine the survival of normal vs. cancer cells using trypan blue exclusion or lack of propidium iodide staining, the 2 ml of culture supernatant, including non-adherent cells in each 35 mm plate was harvested into 15 ml conical tubes along with adherent cells released by 1 ml of 0.25% trypsin-EDTA, setting aside a 100 μl aliquot prior to centrifugation at 400 ×g for 10 minutes. The 100 μl aliquot was mixed with an equal volume of 0.04% trypan blue in normal saline and immediately examined at 100× magnification with an inverted microscope, counting a minimum of 100 blue (dead) and clear (live) cells. After centrifugation, the cell pellets were resuspended in 0.5 ml Dulbecco's PBS (Invitrogen), placed on ice, stained with 10 μl of 5 μg/ml propidium iodide for 5 minutes and analyzed with a Becton-Dickinson FACS-Scan instrument using CellQuest® software. The untreated survival data from five replicates of experiments were averaged and normalized to 100%, and results with each of five treated cultures for both trypan blue exclusion and propidium iodide staining measurements were used in generating the data set presented in Table 2.

Statistical analysis

Statistical analysis was performed using embedded data analysis software in MS Excel. All statistical analyses were done at the p < 0.05 level of significance using 2-tailed Student's t-test.

Acknowledgements

This work was supported by NIH R01CA100045 and R01CA133114 (D.R.S.), by an administrative supplement to promote reentry R01CA100045-S1 (D.C.S.), by grants from the Aiming for a Cure Foundation (D.C.S.), the Children's Miracle Network (M.S.O.), and by the Radiation and Free Radical Research Core in the Holden Comprehensive Cancer Center (NIH P30CA086862).

References

  • 1.National Cancer Institute PDQ®, Neuroblastoma Treatment Accessed on the web at http://www.cancer.gov/cancertopics/pdq/treatment/neuroblastoma/healthprofessional/allpages.
  • 2.Weinstein JL, Katzenstein HM, Cohn SL. Advances in the diagnosis and treatment of neuroblastoma. Oncologist. 2003;8:278–92. doi: 10.1634/theoncologist.8-3-278. [DOI] [PubMed] [Google Scholar]
  • 3.Ara T, Song L, Shimada H, Keshelava N, Russell HV, Metelitsa LS, et al. Interleukin-6 in the bone marrow microenvironment promotes the growth and survival of neuroblastoma cells. Cancer Res. 2009;6:329–37. doi: 10.1158/0008-5472.CAN-08-0613. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Ciccarone V, Spengler B, Meyers M, Bieldler J, Ross R. Phenotypic diversification in human neuroblastoma cells: Expression of distinct cell lineages. Cancer Research. 1989;49:219–25. [PubMed] [Google Scholar]
  • 5.Ross RA, Biedler JL, Spengler BA. A role for distinct cell types in determining malignancy in human neuroblastoma cell lines and tumors. Cancer Letters. 2003;197:35–9. doi: 10.1016/s0304-3835(03)00079-x. [DOI] [PubMed] [Google Scholar]
  • 6.Shimada H, Umehara S, Monobe Y, Hachitanda Y, Nakagawa A, Goto S, et al. International neuroblastoma pathology classification for prognostic evaluation of patients with peripheral neuroblastic tumors: a report from the Children's Cancer Group. Cancer. 2001;92:2451–61. doi: 10.1002/1097-0142(20011101)92:9<2451::aid-cncr1595>3.0.co;2-s. [DOI] [PubMed] [Google Scholar]
  • 7.Shimada H, Ambros IM, Dehner LP, Hata J, Joshi VV, Roald B, et al. The International Neuroblastoma Pathology Classification (the Shimada system). Cancer. 1999;86:364–72. [PubMed] [Google Scholar]
  • 8.Bowman LC, Castleberry RP, Cantor A, Joshi V, Cohn SL, Smith EI, et al. Genetic staging of unresectable or metastatic neuroblastoma in infants: a Pediatric Oncology Group study. J Natl Cancer Inst. 1997;89:373–80. doi: 10.1093/jnci/89.5.373. [DOI] [PubMed] [Google Scholar]
  • 9.Goto S, Umehara S, Gerbing RB, Stram DO, Brodeur GM, Seeger RC, et al. Histopathology (International Neuroblastoma Pathology Classification) and MYCN Status in Patients with Peripheral Neuroblastic Tumors. Cancer. 2001;92:2699–708. doi: 10.1002/1097-0142(20011115)92:10<2699::aid-cncr1624>3.0.co;2-a. [DOI] [PubMed] [Google Scholar]
  • 10.George BA, Yanik G, Wells RJ, Martin LW, Soukup S, Ballard ET, et al. Growth patterns of human neuroblastoma xenografts and their relationship to treatment outcome. Cancer. 1993;72:3331–9. doi: 10.1002/1097-0142(19931201)72:11<3331::aid-cncr2820721132>3.0.co;2-#. [DOI] [PubMed] [Google Scholar]
  • 11.Walton JD, Kattan DR, Thomas KT, Spengler BA, Guoy H-F, Biedler JL, et al. Characteristics of Stem Cells from Human Neuroblastoma Cell Lines and in Tumors. Neoplasia. 2004;6:838–45. doi: 10.1593/neo.04310. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Warburg O, Posener K, Negelein E. Ueber den Stoffwechsel der Tumoren. Biochemische Zeitschrift. 1924;152:319–44. [Google Scholar]
  • 13.Aykin-Burns N, Ahmad IM, Zhu Y, Oberley LW, Spitz DR. Increased levels of superoxide and H2O2 mediate the differential susceptibility of cancer cells versus normal cells to glucose deprivation. Biochem J. 2009;418:29–37. doi: 10.1042/BJ20081258. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Jelluma N, Yang X, Stokoe D, Evan GI, Dansen TB, Haas-Kogan DA. Glucose withdrawal induces oxidative stress followed by apoptosis in glioblastoma cells but not in normal human astrocytes. Mol Cancer Res. 2006;4:319–30. doi: 10.1158/1541-7786.MCR-05-0061. [DOI] [PubMed] [Google Scholar]
  • 15.Spitz DR, Sim JE, Ridnour LA, Galoforo SS, Lee YJ. Glucose deprivation-induced oxidative stress in human tumor cells. A fundamental defect in metabolism? Ann N Y Acad Sci. 2000;899:349–62. doi: 10.1111/j.1749-6632.2000.tb06199.x. [DOI] [PubMed] [Google Scholar]
  • 16.Ahmad IM, Aykin-Burns N, Sim JE, Walsh SA, Higashikubo R, Buettner GR, et al. Mitochondrial O2•- and H2O2 mediate glucose deprivation-induced stress in human cancer cells. J Biol Chem. 2005;280:4254–63. doi: 10.1074/jbc.M411662200. [DOI] [PubMed] [Google Scholar]
  • 17.Lee YJ, Galoforo SS, Berns CM, Chen JC, Davis BH, Sim JE, et al. Glucose deprivation induced cytotoxicity and alterations in mitogen-activated protein kinase activation are mediated by oxidative stress in multidrug-resistant human breast carcinoma cells. J Biol Chem. 1998;273:5294–9. doi: 10.1074/jbc.273.9.5294. [DOI] [PubMed] [Google Scholar]
  • 18.Wang X, Perez E, Liu R, Yan LJ, Mallet RT, Yang SH. Pyruvate protects mitochondria from oxidative stress in human neuroblastoma SK-N-SH cells. Brain Res. 2007;1132:1–9. doi: 10.1016/j.brainres.2006.11.032. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Aw TY, Rhoads CA. Glucose regulation of hydroperoxide metabolism in rat intestinal cells. Stimulation of reduced nicotinamide adenine dinucleotide phosphate supply. J Clin Invest. 1994;94:2426–34. doi: 10.1172/JCI117610. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Roudier E, Bachelet C, Perrin A. Pyruvate reduces DNA damage during hypoxia and after reoxygenation in hepatocellular carcinoma cells. FEBS J. 2007;274:5188–98. doi: 10.1111/j.1742-4658.2007.06044.x. [DOI] [PubMed] [Google Scholar]
  • 21.Chung SJ, Lee SH, Lee YJ, Park HS, Bunger R, Kang YH. Pyruvate protection against endothelial cytotoxicity induced by blockade of glucose uptake. J Biochem Mol Biol. 2004;37:239–45. doi: 10.5483/bmbrep.2004.37.2.239. [DOI] [PubMed] [Google Scholar]
  • 22.Cullen JJ, Hinkhouse MM, Grady M, Gaut AW, Liu J, Zhang YP, et al. Dicumarol inhibition of NADPH:quinone oxidoreductase induces growth inhibition of pancreatic cancer via a superoxide-mediated mechanism. Cancer Res. 2003;63:5513–20. [PubMed] [Google Scholar]
  • 23.Biaglow JE, Miller RA. The thioredoxin reductase/thioredoxin system: novel redox targets for cancer therapy. Cancer Biol Ther. 2005;4:6–13. doi: 10.4161/cbt.4.1.1434. [DOI] [PubMed] [Google Scholar]
  • 24.Coleman MC, Asbury CR, Daniels D, Du J, Aykin-Burns N, Smith BJ, et al. 2-deoxy-D-glucose causes cytotoxicity, oxidative stress and radiosensitization in pancreatic cancer. Free Radic Biol Med. 2008;44:322–31. doi: 10.1016/j.freeradbiomed.2007.08.032. [DOI] [PubMed] [Google Scholar]
  • 25.Nicole A, Santiard-Baron D, Ceballos-Picot I. Direct evidence for glutathione as mediator of apoptosis in neuronal cells. Biomed Pharmacother. 1998;52:349–55. doi: 10.1016/S0753-3322(99)80001-8. [DOI] [PubMed] [Google Scholar]
  • 26.Ross RA, Spengler BA, Domènech C, Porubcin M, Rettig WJ, Biedler JL. Human neuroblastoma I-type cells are malignant neural crest stem cells. Cell Growth Differ. 1995;6:449–56. [PubMed] [Google Scholar]
  • 27.Balster DA, O'Dorisio MS, Albers AR, Park SK, Qualman SJ. Suppression of tumorigenicity in neuroblastoma cells by upregulation of human vasoactive intestinal peptide receptor type 1. Regul Pept. 2002;109:155–65. doi: 10.1016/s0167-0115(02)00199-4. [DOI] [PubMed] [Google Scholar]
  • 28.Vijayaraghavan R, Kumar D, Dube SN, Singh R, Pandey KS, Bag BC, et al. Acute toxicity and cardiorespiratory effects of 2-deoxy-D-glucose: a promising radio sensitiser. Biomed Environ Sci. 2006;19:96–103. [PubMed] [Google Scholar]
  • 29.Chan KY, Baxter CF. Compartments of tubulin and tubulin-like proteins in differentiating neubroblastoma cells. Brain Res. 1979;174:135–52. doi: 10.1016/0006-8993(79)90809-6. [DOI] [PubMed] [Google Scholar]
  • 30.Ross RA, Spengler BA, Biedler JL. Coordinate morphological and biochemical interconversion of human neuroblastoma cells. J Natl Cancer Inst. 1983;71:741–7. [PubMed] [Google Scholar]
  • 31.Spitz DR, Malcolm RR, Robert RJ. Cytotoxicity and metabolism of 4-hydroxy-2-nonenol and 2-nonenol in H2O2-resistant cell lines. Do aldehydic by-products of lipid peroxidation contribute to oxidative stress? Biochem J. 1990;267:453–9. doi: 10.1042/bj2670453. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Lin X, Zhang F, Bradbury CM, Kaushal A, Li L, Spitz DR, et al. 2-Deoxy-D-glucose induced cytotoxicity and radiosensitization in tumor cells is mediated via disruptions in thiol metabolism. Cancer Res. 2003;63:3413–7. [PubMed] [Google Scholar]
  • 33.Altun ZS, Günes D, Aktas S, Erbayrktar Z, Olgun N. Protective Effects of Acetyl-L: -Carnitine on Cisplatin Cytotoxicity and Oxidative Stress in Neuroblastoma. Neurochem Res. 2009 doi: 10.1007/s11064-009-0076-8. Epub ahead of print. [DOI] [PubMed] [Google Scholar]
  • 34.Novotny NM, Grosfeld JL, Turner KE, Rescorla FJ, Pu X, Klaunig JE, et al. Oxidative status in neuroblastoma: a source of stress? J Pediatr Surg. 2008;43:330–4. doi: 10.1016/j.jpedsurg.2007.10.040. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Izaki S, Goto H, Yokota S. Increased chemosensitivity and elevated reactive oxygen species are mediated by glutathione reduction in glutamine deprived neuroblastoma cells. J Cancer Res Clin Oncol. 2008;134:761–8. doi: 10.1007/s00432-007-0338-2. [DOI] [PubMed] [Google Scholar]
  • 36.Ross R, Spengler B. Human neuroblastoma stem cells. Semin. Cancer Biol. 2007;17:241–7. doi: 10.1016/j.semcancer.2006.04.006. [DOI] [PubMed] [Google Scholar]
  • 37.Warnat P, Oberthuer A, Fischer M, Westermann F, Eils R, Brors B. Cross-study analysis of gene expression data for intermediate neuroblastoma identifies two biological subtypes. BMC Cancer. 2007;7:89. doi: 10.1186/1471-2407-7-89. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Acosta S, Lavarino C, Paris R, Garcia I, de Torres C, Rodríguez E, et al. Comprehensive characterization of neuroblastoma cell line subtypes reveals bilineage potential similar to neural crest stem cells. BMC Dev Biol. 2009;9:12. doi: 10.1186/1471-213X-9-12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Hansford LM, McKee AE, Zhang L, George RE, Gerstle JT, Thorner PS, et al. Neuroblastoma cells isolated from bone marrow metastases contain a naturally enriched tumor-initiating cell. Cancer Res. 2007;67:11234–43. doi: 10.1158/0008-5472.CAN-07-0718. [DOI] [PubMed] [Google Scholar]
  • 38.Mahller YY, Williams JP, Baird WH, Mitton B, Grossheim J, Saeki Y, et al. Neuroblastoma cell lines contain pluripotent tumor initiating cells that are susceptible to a targeted oncolytic virus. PLoS One. 2009;4:4235. doi: 10.1371/journal.pone.0004235. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Marengo B, De Ciucis C, Verzola D, Pistoia V, Raffaghello L, Patriarca S, et al. Mechanisms of BSO (L-buthionine-S,R-sulfoximine)-induced cytotoxic effects in neuroblastoma. Free Radic Biol Med. 2008;44:474–82. doi: 10.1016/j.freeradbiomed.2007.10.031. [DOI] [PubMed] [Google Scholar]
  • 40.Simons AL, Ahmad IM, Mattson DM, Dornfeld KJ, Spitz DR. 2-Deoxy-D-glucose combined with cisplatin enhances cytotoxicity via metabolic oxidative stress in human head and neck cancer cells. Cancer Res. 2007;67:3364–70. doi: 10.1158/0008-5472.CAN-06-3717. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Jiang Y, Guo C, Fishel ML, Wang ZY, Vasko MR, Kelley MR. Role of APE1 in differentiated neuroblastoma SH-SY5Y cells in response to oxidative stress: use of APE1 small molecule inhibitors to delineate APE1 functions. DNA Repair. 2009;8:1273–82. doi: 10.1016/j.dnarep.2009.08.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Simons AL, Fath MA, Mattson DM, Smith BJ, Walsh SA, Graham MM, et al. Enhanced response of human head and neck cancer xenograft tumors to Cisplatin combined with 2-deoxy-D-glucose correlates with increased 18F-FDG uptake as determined by PET imaging. Int J Radiat Oncol Biol & Phys. 2007;69:1222–30. doi: 10.1016/j.ijrobp.2007.07.2343. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Liu H, Hu YP, Savaraj N, Priebe W, Lampidis TJ. Hypersensitization of tumor cells to glycolytic inhibitors. Biochemistry. 2001;40:5542–7. doi: 10.1021/bi002426w. [DOI] [PubMed] [Google Scholar]
  • 44.Mohanti BK, Rath GK, Anantha N, Kannan V, Das BS, Chandramouli BA, et al. Improving cancer radiotherapy with 2-deoxy-D-glucose: phase I/II clinical trials on human cerebral gliomas. Int J Radiat Oncol Biol Phys. 1996;35:103–11. doi: 10.1016/s0360-3016(96)85017-6. [DOI] [PubMed] [Google Scholar]
  • 45.Wang ZF, Tang XC. Huperzine A protects C6 rat glioma cells against oxygen-glucose deprivation-induced injury. FEBS Lett. 2007;581:596–602. doi: 10.1016/j.febslet.2007.01.016. [DOI] [PubMed] [Google Scholar]
  • 46.Shenoy MA, Singh BB. Non-nitro radiation sensitizers. Int J Radiat Biol Relat Stud Phys Chem Med. 1985;48:315–26. doi: 10.1080/09553008514551311. [DOI] [PubMed] [Google Scholar]

RESOURCES