Abstract
Hypoxia plays a critical role in the tumor microenvironment of high-grade gliomas by promoting the glioma stem cell (GSC)-like phenotype, which displays resistance to standard therapies. We tested three glioblastoma multiforme xenograft lines (xenolines) against γ134.5-deleted recombinant oncolytic herpes simplex virus (oHSV) C101 under 1% (hypoxia) and 20.8% (normoxia) oxygen tension for effects on oHSV infectivity, replication, and cytotoxicity in all tumor cells and CD133+ GSCs. Expression levels of CD133, a putative GSC marker, and CD111 (nectin-1), an adhesion molecule that is the most efficient method for HSV entry, increased significantly under hypoxia in all three xenolines. Despite increased CD111 expression under hypoxic conditions, oHSV infectivity, cytotoxicity and viral recovery were not improved or were diminished in all three xenolines under hypoxia. In contrast, wild-type HSV-1 equally infected xenoline cells in normoxia and hypoxia, suggesting that the 34.5 mutation plays a role in the decreased C101 infectivity in hypoxia. Importantly, CD133+ cells were not more resistant to oHSV than CD133- tumor cells regardless of oxygen tension. Furthermore, CD133 expression decreased as viral dose increased in two of the xenolines suggesting that up-regulation of CD133 in hypoxia was not the cause of reduced viral efficacy. Our findings that oHSV infectivity and cytotoxicity were diminished under hypoxia in several GBM xenolines likely have important implications for clinical applications of oHSV therapies, especially considering the vital role of hypoxia in the microenvironment of GBM tumors.
Introduction
Intracranial malignant primary brain tumors experience a physiologic hypoxia with in situ oxygen concentrations typically ranging between 1% and 10%; oxygen tension in most high-grade gliomas is less than 2.5% but can be as low at 0.1% [1–3]. Importantly, hypoxia is a frequently occurring condition during maintenance and proliferation of normal neural stem cells—cells that, through acquisition of mutations, are thought to give rise to brain tumor stem cells. These have been described as a small subpopulation of radiation-resistant and chemotherapy-resistant cells that are thought to be responsible for maintaining the neoplastic clone and for tumor recurrence [4–7]. CD133, the most common marker used to identify glioma stem cells (GSCs), increases significantly in hypoxia, which seems to promote a more “stem-like” phenotype in glioma cells. Hypoxia is believed to be a central driver of growth and aggressiveness of human glial brain tumors in situ, inducing complex survival responses that contribute to treatment resistance [8–11]. If not adequately considered when testing novel therapeutics in vitro, the hypoxic phenotype may result in a therapeutic effect that is not replicated with tumors in situ.
One such novel therapy is oncolytic herpes simplex virus (oHSV) that uses a genetically engineered virus to selectively kill cancer cells while sparing normal cells. Selectivity is achieved through the deletion of a diploid gene (e.g., the “neurovirulence” gene, γ134.5), which is essential for replication in quiescent normal cells but is nonessential for replication in tumor cells [12]. Two different γ134.5-deleted viruses, G207 and HSV1716, have been used safely in phase 1 trials in patients with recurrent high-grade gliomas [13,14]. Whereas radiographic antitumor activity was demonstrated in a significant proportion of patients, with some long-term survivors (>5 years), overall responses were more modest compared with those seen in earlier in vitro and in vivo murine studies [12,15]. Decreased viral efficacy because of oxygen tension-induced changes in glioma cells or GSCs is one potential explanation for this divergence because in vitro studies have traditionally been performed under normoxic, high-nutrient conditions.
Recent studies in breast cancer and glioblastoma multiforme (GBM) cells suggest that hypoxia may enhance oHSV replication and effectiveness [16,17]. These studies were performed with one or two established tumor cell lines, which have undergone potent selection for growth in normoxic, high-nutrient, high-growth factor-supplemented culture medium over many passages to a point where they may no longer be representative of their originating tumor. Moreover, there are conflicting data that demonstrate improved oncolysis in human GBM cell lines in normoxia compared with hypoxia using a hypoxia-inducible factor-activated HSV [18]. Because of these variabilities, we sought to determine the effect of oxygen tension on oHSV infectivity, replication, and oncolytic activity and on the ability of oHSV to infect CD133+ GSCs in a panel of human GBM xenolines that have never been passaged in tissue culture.
Materials and Methods
Human Glioma Xenolines and Genetically Engineered HSVs
GBM-X12 and GBM-X6 are serially transplantable xenolines established initially by direct implantation of freshly resected human GBM tissues into the flanks of immunocompromised athymic nude (nu/nu) mice. We prefer to call these “xenolines” because these are established tumor cell lines in mice as opposed to cell lines grown in culture that are xenografted into mice. These tumors (gift from C. David James, PhD, and Jann Sarkaria, MD, Mayo Clinic) were maintained by serial transplantation in athymic nude mice and have been determined to fit the classic or proliferative molecular subtype [19,20] (data not shown). GBM-XD456 (gift from Darell D. Bigner, MD, PhD, Duke University Medical Center) was derived from a human pediatric frontoparietal GBM, which was directly implanted into the flank of immunocompromised mice [21]. The molecular subtype of this tumor was determined to be proneural (data not shown). Transfer of these tumor samples to the University of Alabama at Birmingham (UAB) Brain Tissue Core (X050415007), which provides anonymized coded tissues from properly consented patients, was approved by the UAB institutional review board. All animal studies were approved by the UAB Institutional Animal Care and Use Committee under the animal project number 100608793.
C101, a γ134.5-deleted recombinant virus, has been described previously [22]. Briefly, the virus was constructed by inserting the gene-encoding enhanced green fluorescent protein (EGFP) under the control of the CMV IE promoter in the UL3-UL4 intergenic region of Δγ134.5 HSV, R3616. Expression of green fluorescent protein (GFP) was used to distinguish infected and uninfected cells by fluorescence-activated cell sorter (FACS) analysis. M2001, an HSV-1 wild-type virus, was constructed by inserting the gene encoding EGFP under the control of the CMV IE promoter in the UL3-UL4 intergenic regions of wild-type strain F.
Tumor Disaggregation and Tissue Culture
Xenoline tumors were harvested aseptically from the flank of mice and washed five times with PBS to remove excess blood. Tumors were separately minced finely with no. 11 scalpel blades and dis-aggregated in an enzyme solution (5 mg of collagenase I (Worthington Biochemical Co, Lakewood, NJ), 0.5% trypsin/0.53 mM EDTA (Invitrogen, Grand Island, NY), 2.5 mg DNAse I (Worthington Biochemical Co) with four to five sequential harvests, pooling cells at each harvest on ice. Cells were washed twice by centrifugation (200g for 8 minutes at 4°C) in serum-less Dulbecco modified Eagle medium mixed 50:50 with Ham's Nutrient Mixture F-12 (Dulbecco modified Eagle medium/F12; MediaTech, Manassas, VA). Cells were resuspended in serum-less NeuroBasal medium (Invitrogen) prepared with fibroblast growth factor β (Invitrogen) and epidermal growth factor (Invitrogen) at 10 ng/ml each, 2% B-27 supplement without vitamin A (Invitrogen), 2 mM l-glutamine, amphotericin B (250 µg/ml), and gentamicin (50 µg/ml). NeuroBasal medium was used to promote growth of GSCs and retard their rapid differentiation in response to wound-fluid-type growth factors (platelet-derived growth factor, transforming growth factor β, etc). Cell cultures were evenly divided and maintained (37°C, 95% humidity, 5% CO2) in either normoxia (20.8%) or hypoxia (1%) in a BioSpherix hypoxia chamber (Lacona, NY) for 7 days before performing additional studies. All media and solutions were allowed to equilibrate to the 1% oxygen tension before being exposed to the cells. All procedures with hypoxic cells were conducted within the hypoxia chamber, which is fitted with glove ports. Experiments were performed in 1% oxygen tension to recapitulate the severe hypoxia in high-grade gliomas as demonstrated by Evans et al. [1,2]. Culture medium was exchanged every 3 to 7 days as needed. Characteristically, xenoline-derived cells in NeuroBasal medium do not attach to the plastic substratum under either hypoxia or normoxia conditions but form aggregates of cells, variously termed spheroids, tumorspheres, or neurospheres.
FACS Analyses
Expression levels of CD133 and CD111 were determined by two-color FACS on glioma cells cultured for 7 days in normoxia or hypoxia. Glioma cell spheroids were harvested from culture and dissociated (37°C, 10–15 minutes) into a single-cell suspension using a solution of Accutase (Innovative Cell Technologies, San Diego, CA) supplemented with 2.5 µg/ml of DNAse I (Worthington). Cells were gently triturated with a wide bore pipette to completely dissociate the spheroids. Cells were filtered through a 40-µm pore Cell Strainer (BD Biosciences, San Jose, CA) to remove any clumps, washed in medium, and counted by hemacytometer with trypan blue viability determination. Viable cells (106) were resuspended in 80 µl of FACS buffer containing ice-cold PBS with 5% fetal bovine serum, 0.1% of sodium azide, and Fc receptor blocking reagent (20 µl; Miltenyi Biotec, Auburn, CA). Fluorochrome-conjugated monoclonal antibodies were added: phycoerythrin (PE)-poliovirus receptor-related 1 protein (PRR1; CD111) (Immunotech, Marseille, France), allophycocyanin (APC)-CD133 (Miltenyi Biotec). Cells were incubated (10–30 minutes, 4°C), washed twice with iced FACS buffer, and resuspended in 1 ml of ice-cold PBS with 0.1% sodium azide. Labeled cells were then fixed in ice-cold fresh 4% paraformaldehyde and then stored in the dark at 4°C until analysis by the UAB Flow Cytometry Core Facility. Side scatter versus forward scatter profiles of unstained cells were used to set gates for each individual xenoline. Data were analyzed using Flow-Jo software (Tree Star, Inc, Ashland, OR), and results were expressed as a percentage of gated cells for each cell type identified by antibody binding. Mean values from multiple determinations were calculated for each xenograft tested. Paired Student's t tests were used to determine significance.
In Vitro Infectivity Assay
To determine the ability of a Δγ134.5 recombinant virus or wild-type virus to infect glioma tumor and GSCs, we dissociated spheroids cultured in normoxia or 1% hypoxia into a single-cell suspension (vide supra) and replated 5 x 105 cells/0.8 ml of NeuroBasal medium in a 12-well flat-bottom plate (Corning Incorporated, Corning, NY). After overnight culture, C101 or M2001 was added at various multiplicities of infection (MOI): 0, 1, 3.3, and 10 plaque-forming units per cell (PFU/cell) in 200 µl of medium. At 30 hours after infection, cells and spheroids were collected from each well, dissociated with Accutase-DNAse I (vide supra) and prepared for FACS analysis with APC-CD133 and PE-PPR1 monoclonal antibodies. Dead cells were excluded with SytoxBlue Dead Cell Stain (Invitrogen). Unfixed cells were tested for expression of GFP, APC, and PE. In a limited number of studies to assess graded reduction in oxygen tension, cells were cultured at 3% and 5% hypoxia. Percentages of infected cells at each level of oxygen tension were normalized to percentages of cells infected under normoxia (20.8%) at each MOI.
In Vitro Cytotoxicity Assay
Cells cultured in normoxia and hypoxia were dissociated, and 104 cells/well were dispensed into 96-well plates overnight. Graded doses of C101, ranging from 0 to 100 PFU/cell, were added to each row. After 72 hours, 25 µl of alamarBlue was added to each well and incubated for 6 to 8 hours to detect metabolic activity as evidenced by reduction of the dye from dark blue to pink. Color changes were quantified with a BioTek microplate spectrophotometer (Winooski, VT). OD595-562nm values were used to calculate numbers of PFU/cell required to kill 50% of the cells in a 3-day period, an LD50. Whereas hypoxia may change the extent to which the dye is reduced, each experiment, whether under hypoxia or normoxia, always included a mock-treated control, to which the extent of dye reduction was normalized for each experimental condition. Thus, the effect of graded doses of HSV was internally compared with a set of “control” wells that represented 100% survival of the tumor cells for that set of environmental conditions.
Viral Recovery Assay
Disaggregated xenoline cells (3 x 105) were subjected to normoxic or hypoxic conditions for 7 days and then infected in parallel with C101 or M2001 virus at an MOI of 0.1. The virus inocula were removed after 2 hours and replaced with medium and maintained under normoxic or hypoxic conditions. At 48 hours after infection, the infected cultures were frozen (-80°C), and virus recovery from the normoxic and hypoxic cultures was measured by limiting plaque dilution on Vero cells as previously described [22].
Statistical Analyses
Student's t test analyses for significance of mean differences were performed using Microsoft Excel (Microsoft Corp, Redmond, WA). P ≤ .05 was considered significant.
Results
CD133 and CD111 Expression
We assessed the percentages of CD133 positivity, a surrogate marker for GSCs, present in three freshly disaggregated xenolines (GBMXD456, GBM-X12, and GBM-X6) in tissue culture under normoxic and 1% hypoxic environments. Similar to other reports, CD133 expression increased significantly in all three xenolines under hypoxia (Table 1) [8,9]. GBM-XD456, which had the lowest percentage of CD133+ cells in normoxia (16.5%), had the greatest relative increase in CD133 expression (nearly four-fold) under hypoxia. GBM-X6 CD133 expression likewise increased almost three-fold under hypoxia, whereas GBM-X12, which had the highest CD133% in normoxia (60.6%), showed only a modest increase in CD133 expression under hypoxia (76.6%).
Table 1.
Xenograft | CD133% | CD111% | ||||
Normoxia | Hypoxia | P | Normoxia | Hypoxia | P | |
GBM-XD456 | 16.5 ± 4.2 | 64.4 ± 2.1 | <.0001 | 78.9 ± 14.9 | 88.4 ± 10.8 | .03 |
GBM-X12 | 60.6 ± 10.5 | 74.3 ± 6.2 | .002 | 42.8 ± 17.6 | 60.3 ± 20.0 | .02 |
GBM-X6 | 26.1 ± 13.0 | 76.6 ± 4.1 | .01 | 48.9 ± 10.4 | 78.3 ± 6.4 | .0007 |
Next, we examined CD111 (nectin-1) expression in the xenolines under normoxia and hypoxia. HSV entry is mediated primarily via CD111, a cell surface adhesion molecule broadly expressed in epithelial and neuronal cells [23,24]. We have reported that low CD111 expression (<20%) in glioma cells results in ineffective oHSV infectivity and killing [25]. All three cell lines had greater than 40% expression in normoxia (Table 1). Similar to CD133, CD111 expression increased significantly in all three xenolines under hypoxic conditions; greater than 60% of cells expressed the HSV entry molecule. As shown below, this increase in CD111 expression, however, did not result in improved infectivity or cytotoxicity in any of the xenolines.
Viral Infectivity
To determine whether increased CD111 expression in hypoxia led to improved oHSV infection, xenoline cells (GBM-XD456, GBM-X12, and GBM-X6) grown in normoxia and 1% hypoxia were infected with C101 using serial three-fold MOI concentrations and the proportion of infected cells were analyzed by FACS at 30 hours. This time point was chosen to allow the infection process to be well underway yet before progression to cell death and lysis. The recombinant GFP-tagged virus enabled us to discriminate infected and uninfected individual cells based on CD133 and CD111 expression detected by multicolor FACS analyses. The highest percentage of infection was seen in GBM-X12 cells in both normoxia (65.5 ± 2.0) and hypoxia (48.2 ± 0.8) followed by GBM-XD456. GBM-X6 was the most resistant to oHSV infection with less than 21% of cells infected at 10 PFU/cell at 30 hours. Importantly, all three xenolines showed significantly lower infection with C101 in hypoxia compared with normoxia despite an increase in CD111 expression (Table 2).
Table 2.
% Infected | GBM-XD456 | GBM-X12 | GBM-X6 |
Total % infected normoxia | 41.5 ± 0.9 | 65.5 ± 2.0 | 20.4 ± 1.3 |
Total % infected hypoxia | 26.4 ± 0.9 | 48.2 ± 0.8 | 10.0 ± 0.2 |
P | <.0001 | .002 | .0001 |
CD133+ % infected normoxia | 42.1 ± 1.6 | 61.3 ± 1.7 | 15.8 ± 0.9 |
CD133+ % infected hypoxia | 27.7 ± 0.8 | 44.1 ± 1.2 | 7.5 ± 0.1 |
P | .0007 | .006 | <.0001 |
Likewise, more CD133+ GSCs were infected in normoxia compared with hypoxia in each of the xenolines. Similar percentages of CD133- tumor cells and CD133+ GSCs were infected in either normoxic or hypoxic conditions with differences in infection ranging from only 0.6% to 4.6%. Whereas the percentage of CD133+ cells infected was similar to the total percentage of cells infected, the overall proportion of cells that expressed CD133 was significantly decreased in GBM-X12 and GBM-X6 with increasing viral dose (Figure 1). GBMX12 CD133 expression decreased on average from 73.5% ± 1.3% to 49.5% ± 1.3% (P = .0003) in normoxia and 85.2% ± 3.4% to 60.6% ± 1.4% (P = .0005) in hypoxia with increasing virus dose (0–10 PFU/cell). GBM-X6 CD133 expression similarly fell from 22.0% ± 1.2% to 16.0% ± 0.9% (P = .003) in normoxia and 77.2% ± 8.7% to 59.6% ± 1.4% (P = .009) in hypoxia. No appreciable change in overall CD133 expression was seen with increasing virus dose in GBM-XD456 in either cell culture condition.
CD111 expression was assessed after infection with increasing MOI of C101. In all three xenolines, CD111 expression fell significantly with increasing viral dose and cell infectivity irrespective of oxygen tension (Figure 2). This was not unexpected because CD111 is down-regulated after trans-interaction with glycoprotein D [26]. The greatest drop in CD111 expression was seen in GBM-X12, which also had the highest percentage of cells infected (71.3% ± 8.4% to 6.1% ± 1.4% CD111+ in hypoxia and 56.5% ± 0.4% to 1.9% ± 0.5% in normoxia). The total number of cells infected for each xenoline was less than the overall percentage of CD111 expression at baseline, suggesting that not all cells expressing CD111 were infected at 30 hours.
To determine whether the decreased infectivity seen in hypoxia may be related to the 34.5 deletion, we examined the ability of wild-type HSV-1 to infect the most resistant xenoline, GBM-X6, in normoxia and hypoxia. Compared with the 34.5-deleted virus, the wild-type virus infected significantly more cells in both normoxia (44.8 ± 5.1 vs 20.4 ± 1.3, P = .002) and hypoxia (48.9 ± 3.4 vs 10.0 ± 0.2, P = .0003). Notably, there was no difference (P =.3) in the ability of wild-type virus to infect GBM-X6 cells in hypoxia and normoxia, suggesting that the 34.5 mutation plays a role in the diminished C101 infectivity in hypoxia.
To establish the effect of the degree of hypoxia on infectivity, differences in the percent of cells infected in normoxia and hypoxia at 1%, 3%, or 5% oxygen tension were compared in GBM-X12, the xenoline having the greatest infectivity differential in normoxia versus 1% hypoxia. Similar percentages of GBM-X12 cells were infected regardless of the level of hypoxia, suggesting that even mild to moderate hypoxia (5%) limits oHSV infectivity (Figure 3).
Viral Recovery and Cytotoxicity
To confirm our finding that oHSV infection was diminished in hypoxia, we tested the amount of C101 virus recovered after 48 hours after infection in each xenoline under both normoxia and hypoxia, and we measured the cytotoxic effect of C101 72 hours after infection. The amount of virus recovered was highest in GBM-X12, the xenoline with the most cells infected, followed by GBM-XD456, and lowest in GBM-X6 in both normoxia and hypoxia. For GBM-XD456, significantly more virus was recovered in normoxia compared with hypoxia (Figure 4). Although there was a trend toward increased viral recovery in normoxia for GBM-X12, this was not statistically significant. In GBM-X6, viral recovery was equivalent in normoxia and hypoxia. Compared with C101, significantly more M2001 virus was recovered in all three xenolines (Figure 4). Similar to C101, an equivalent amount of M2001 was recovered in GBM-X12 and GBM-6 in normoxia and hypoxia, but less virus was recovered in GBM-XD456 in hypoxia.
Importantly, significantly more cells were killed by C101 in normoxia compared with hypoxia at multiple-dose levels in GBM-XD456 and GBM-X12 (Figure 5). No difference in cell killing was seen at any dose level in normoxia compared with hypoxia in GBM-X6. The lethal dose of C101 required to kill 50% of the cells (LD50) was lowest in GBM-X12 and highest in GBM-X6 in both normoxia and hypoxia. This was consistent with both the infectivity and viral recovery data, which demonstrated that GBM-X12 was the most sensitive and GBM-X6 was the most resistant xenoline. Significantly higher doses of C101 were required to kill 50% of the cells in hypoxia compared with normoxia for GBM-XD456 (10.7 ± 0.8 vs 7.0 ± 2.0 PFU/cell, P = .04) and GBM-X12 (3.2 ± 0.2 vs 1.9 ± 0.1 PFU/cell, P = .006). A 52.9% and 68.4% higher dose in hypoxia was required to kill the same number of cells as in normoxia, respectively. Although a higher dose of GBM-X6 was also required in hypoxia (47.1 ± 2.9 vs 44.8 ± 3.0 PFU/cell), this was not statistically significant (P = .36). Taken together, these data demonstrate that oHSV infectivity and cytotoxicity in GBM xenolines is not enhanced by hypoxia, and in fact, hypoxia may diminish efficacy.
Discussion
Gliomas experience a physiologic hypoxia with hypoxic gradients within tumors ranging from mild (10%) to moderate (2.5%) to severe (0.1%), which tends to mark regions of necrosis [1]. High-grade tumors contain more areas of moderate to severe hypoxia compared with low-grade gliomas [2,3]. Hypoxia plays a critical role in the tumor microenvironment of high-grade gliomas [1–3]. Recent evidence suggests that hypoxia is involved in GBM development, invasion, induction of angiogenesis, loss of apoptotic potential, and resistance to chemotherapy and radiation [27]. Moreover, the degree of hypoxia correlates with a more clinically aggressive phenotype and more rapid tumor recurrence [2]. Many of these features that make GBM tumors so difficult-to-treat may well be attributed to hypoxia's effect on promoting the GSC phenotype. Recently, oHSV was demonstrated to be efficacious against GSCs under normoxic conditions in vitro [25,28]. This novel, targeted therapy has been successfully translated to clinical trials, and new trials are forthcoming with “second-generation” viruses that were more effective in animal studies than the viruses previously used in human studies. As next-generation viruses are created, an understanding of hypoxia's effect on oHSV infectivity and replication in tumor cells and GSCs will be critical to ensure the most beneficial viruses make it to clinical trials. We chose to conduct studies to determine oHSV efficacy in vitro at 1% oxygen tension, the lower limit of cell viability, to model the most stringent hypoxic cellular microenvironment in which cells live in high-grade gliomas because these areas seem to be critical to the overall aggressiveness and therapeutic resistance of these tumors. Mutant HSV must be able to infect and replicate in cells under this harsh environment to be most effective.
HSV enters mammalian cells both through direct cell membrane fusion and through endocytosis. HSV has broad cellular tropism involving the interaction of multiple HSV glycoproteins with cellular surface receptors. The process follows three stages: 1) binding or tethering, 2) triggering, and 3) fusion of the membranes [29]. The first step in oHSV infection is virus entry, which is primarily mediated in glioma cells by the adhesion molecule CD111 (nectin-1). CD111 and CD112 (nectin-2) are one of three recognized classes of cell membrane receptors important for HSV-1 entry, along with herpes virus entry mediator and heparan sulfate proteoglycan, which is believed to be ubiquitously expressed [24,29,30]. Herpes virus entry mediator, a member of the tumor necrosis factor receptor family, is mainly expressed on T and B lymphocytes and not neuroglial cells to a significant degree. Whereas CD111 and CD112 (nectin-2) are expressed in a variety of tissues including neuronal cells, CD112 is more important for HSV-2 entry. CD111 is the most efficient method for HSV-1 virus entry, and its expression level predicts sensitivity to oHSV in several tumor types including thyroid cancer, squamous cell carcinoma, and other invasive carcinomas [24,31–33]. In brain tumors, Rueger et al. [34] found that, whereas only tumor cells that expressed CD111 were transducible with an HSV-1 amplicon vector, not all cells expressing CD111 were transduced. Previously, we showed variable infectability of glioma xenolines without a direct correlation between CD111 expression and oHSV infectivity and cytotoxicity; however, we demonstrated that oHSV was unable to efficiently enter and therefore infect and kill glioma xenolines with low (<20%) CD111 expression [25].
In this report, one novel finding was the increased expression of CD111 in GBM xenolines under hypoxia. Koike et al. [35] showed that several cell adhesion molecules, like CD111, are induced by hypoxia in colon cancer cells and result in enhanced adhesion to vascular endothelial cells, which may facilitate tumor angiogenesis. The up-regulation of CD111 in hypoxia may play a similar role in GBM tumors. Paradoxically, despite the increased CD111 expression under hypoxic conditions, oHSV infectivity, cytotoxicity, and viral recovery were diminished in several xenolines. Moreover, GBM-X12, the tumor with the lowest CD111 expression in both normoxia and hypoxia, was the most efficiently killed by oHSV. This suggests that a minimum level of CD111 is necessary for a productive infection, but other cellular factors can limit oHSV infectivity, even if there are ample entry molecules. Furthermore, these data suggest that other less efficient mechanisms of cell entry are used in sensitive tumors with lower CD111 expression. It is important to note that whereas approximately 40% of GBM-X12 cells were considered negative with regard to fluorescence threshold for detection in FACS analysis, these cells may express CD111 to a limited extent that may enable infection. As might be expected when oHSV enters and infects a cell, we observed that CD111 expression was lost. This likely represents the internalization or change in the conformation of the molecule as a function of oHSV entry and viral gene expression [26].
Similar to others, we found CD133 expression increased significantly under hypoxia. Importantly, the CD133+ fraction was infected in similar proportions to all tumor cells regardless of oxygen tension, confirming our earlier findings that CD133+ cells are as sensitive to oHSV as other tumor cells and that oHSV may be an attractive therapy to eradicate difficult to treat GSCs. Similar to all tumor cells, CD133+ GSC infectivity was enhanced in normoxia. Moreover, CD133 expression declined in two of the three xenolines with increasing viral doses. The mechanism of this novel finding is unknown and requires further study. Because we previously showed that CD133+ and CD133- cells are similarly killed by various oHSVs, it is unlikely that the virus selectively kills the CD133+ cells in greater proportions to CD133- cells [25]. Thus, the significance of this phenomenon remains to be determined with regard to the overall efficacy of oncolytic virotherapy. One possible explanation is that CD133+ cells are more fragile and die faster or undergo apoptosis at a more rapid rate than CD133- cells. However, given the modulation of the cell membrane by virus entry, a more likely explanation is that the conformation of the CD133 molecule is being altered or that CD133 is being internalized or its expression downregulated. HSV is able to modify the autophagic and apoptotic properties of cells under stress from infection so that the virus may complete its life cycle [36]. Because CD133 expression is known to be regulated by bioenergetic stress, it is possible that the virus is interfering with cellular response to stress resulting in a change to the cellular phenotype and down-regulation of CD133 [37]. Whether this decrease in CD133 expression actually alters the “stem cell-like” properties of the glioma cell requires further investigation.
Notably, hypoxia decreased or did not enhance oHSV infectivity, viral replication, and cytotoxicity in all three xenolines tested, whereas wild-type HSV-1 infectivity was not limited by hypoxia. A similar percentage of GBM-X12 cells were infected at 1%, 3%, and 5% oxygen tension, suggesting that the decreased infectivity is not limited to severe hypoxic conditions but also occurs at more modest degrees of hypoxia. These findings are likely clinically significant and strongly suggest that oxygen tension must be considered when preparing next-generation viruses for clinical application. Understanding the mechanism(s) behind the decreased viral efficacy in hypoxia will be important for the development of more effective viruses that can evade the mechanism(s) or limit the effects. The decreased GFP expression seen under hypoxia in the infectivity assay suggests that the effect on the virus occurs early because GFP is produced along with early viral genes.
There are many potential mechanisms that require investigation including, but not limited to, reduced viral entry or transport, enhanced activation of antiviral host cell response(s), increased autophagy, and limited infected cell polypeptide 0 (ICP0) protein production or activity. Although hypoxia increases expression of CD111, it may alter the conformation of the adhesion molecule such that HSV entry is impeded. Alternatively, the low-energy hypoxic conditions may preclude efficient fusion of the viral envelope with the cell membrane or actin-dependent intracellular transport of the virus. Once the virus has entered the cell, hypoxia may augment the activation of antiviral host cell responses, which occurs in three phases: sensitization, induction, and amplification [38]. Sensitization involves phosphorylation of interferon-regulatory factor 3 (IRF3), which upregulates expression of interferon β1. Recently, hypoxia has been shown to enhance IRF3 activation in microglia cells [39]. Interferon β1 activates signal transducers and activators of transcription family of proteins which results in production of interferon α in the induction phase. In breast cancer cells, Lee et al. [40] demonstrated that hypoxia is a critical stimulator for activation of signal transducers and activators of transcription proteins. The amplification phase consists of increased production of antiviral proteins including protein kinase R (PKR), which results in translation inhibition in the infected cell to limit viral replication. Hypoxia may increase production of PKR, a known regulator of hypoxia-inducible factor 1α, thereby decreasing efficacy of oHSV [41]. PKR is positioned upstream of Beclin-1, an autophagy protein that the γ134.5 gene product (ICP34.5) binds to confer neurovirulence [42]. Recently, Song et al. [43] demonstrated that decreased hepatocellular carcinoma cell death in hypoxia was mainly attributed to decreased apoptosis, which was dependent on Beclin-1 activity. Thus, the γ134.5-deleted virus may be limited by autophagy in hypoxia. Lastly, ICP0 (product of the HSV-1 α0 gene) plays a critical role in HSV infection by inactivating and disrupting host cell antiviral responses [44]. If ICP0 expression is decreased or blocked in hypoxia, HSV replication would be decreased.
Our result that hypoxia decreased oHSV efficacy is different from that reported in one previous study involving the U87 glioma cell line [16]. A trivial explanation is that our in vitro studies used three different short-term in vivo passaged human gliomas that were never cultured long term, in contrast to U87-MG, which has been in almost continuous normoxic tissue culture because it was established in the late 1960s by Dr Jan Ponten. Thus, these long-term cultured glioma cells may be quite distinct from the glioma xenolines in many respects, given entirely different selection pressures. Another distinction lies in the basic experimental design that could account for the differences; for example, this could be virus dose related in that a single dose of 0.4 MOI was used in the U87 cells, whereas we used three larger doses (1, 3.3, and 10 MOI) in our infectivity assays and five higher doses (1–100 MOI) in our cytotoxicity assays. Moreover, our defined “stem cell” culture medium did not contain fetal bovine serum and the entire hypoxic culture procedure and infection manipulations were performed within a BioSpherix hypoxia chamber with glove ports that enclosed a Forma CO2 incubator within the workspace maintained at a constant oxygen tension. Only after the cells were fixed in the hypoxia chamber were any procedures performed under normal atmospheric conditions. Our observation has been that only a brief exposure to 20.8% oxygen is necessary to activate an entirely distinct transcriptomic response in cells that have been maintained under hypoxia. Another possible distinction is that the U87 cell line may contain unique features such as the up-regulation of GADD34, which promoted virus replication under lower oxygen tension. We have not examined GADD34 expression in our three GBM-X lines.
Our findings that oHSV infectivity and cytotoxicity are diminished under hypoxia in several GBM xenolines likely have important implications for future oHSV-based clinical trials, especially considering the vital role of hypoxia in the microenvironment of GBM tumors. The impact of oxygen tension must be considered when testing “next-generation” viruses in vitro. Strategies that may be used to enhance oHSV efficacy in hypoxia include viruses that disrupt the microenvironment through protein production, create an enhanced immune response through cytokine production, or preferentially replicate in hypoxic areas through a hypoxia-responsive promoter [18,45,46]. Further studies will be necessary to examine the efficacy of oHSV in hypoxic areas of GBM xenolines in vivo and to determine mechanisms resulting in decreased efficacy of oHSV in hypoxic conditions so that these mechanisms may be circumvented by further manipulation of the virus genome, if possible.
Acknowledgments
The authors thank Enid Keyser and the Analytic and Preparative Core Facility (supported by the National Institutes of Health P30 no. AR48311) for performing the FACS analyses. The authors thank Drs Darell Bigner, Jann Sarkaria, and C. David James for sharing the human xenolines for these studies.
Footnotes
This work was funded by the Kaul Pediatric Research Institute grants to G.K.F. and K.A.C. and grants provided by the National Institutes of Health, National Cancer Institute (CA071933, CA097247, and CA151129). The authors report no conflicts of interest.
References
- 1.Evans SM, Judy KD, Dunphy I, Jenkins WT, Nelson PT, Collins R, Wileyto EP, Jenkins K, Hahn SM, Stevens CW, et al. Comparative measurements of hypoxia in human brain tumors using needle electrodes and EF5 binding. Cancer Res. 2004;64:1886–1892. doi: 10.1158/0008-5472.can-03-2424. [DOI] [PubMed] [Google Scholar]
- 2.Evans SM, Judy KD, Dunphy I, Jenkins WT, Hwang WT, Nelson PT, Lustig RA, Jenkins K, Magarelli DP, Hahn SM, et al. Hypoxia is important in the biology and aggression of human glial brain tumors. Clin Cancer Res. 2004;10:8177–8184. doi: 10.1158/1078-0432.CCR-04-1081. [DOI] [PubMed] [Google Scholar]
- 3.Collingridge DR, Piepmeier JM, Rockwell S, Knisely JP. Polarographic measurements of oxygen tension in human glioma and surrounding peritumoural brain tissue. Radiother Oncol. 1999;53:127–131. doi: 10.1016/s0167-8140(99)00121-8. [DOI] [PubMed] [Google Scholar]
- 4.Morrison SJ, Csete M, Groves AK, Melega W, Wold B, Anderson DJ. Culture in reduced levels of oxygen promotes clonogenic sympathoadrenal differentiation by isolated neural crest stem cells. J Neurosci. 2000;20:7370–7376. doi: 10.1523/JNEUROSCI.20-19-07370.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Studer L, Csete M, Lee SH, Kabbani N, Walikonis J, Wold B, McKay R. Enhanced proliferation, survival, and dopaminergic differentiation of CNS precursors in lowered oxygen. J Neurosci. 2000;20:7377–7383. doi: 10.1523/JNEUROSCI.20-19-07377.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Diabira S, Morandi X. Gliomagenesis and neural stem cells: key role of hypoxia and concept of tumor “neo-niche”. Med Hypotheses. 2008;70:96–104. doi: 10.1016/j.mehy.2007.04.024. [DOI] [PubMed] [Google Scholar]
- 7.Dirks PB. Brain tumor stem cells: bringing order to the chaos of brain cancer. J Clin Oncol. 2008;26:2916–2924. doi: 10.1200/JCO.2008.17.6792. [DOI] [PubMed] [Google Scholar]
- 8.McCord AM, Jamal M, Shankavaram UT, Lang FF, Camphausen K, Tofilon PJ. Physiologic oxygen concentration enhances the stem-like properties of CD133+ human glioblastoma cells in vitro. Mol Cancer Res. 2009;7:489–497. doi: 10.1158/1541-7786.MCR-08-0360. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Heddleston JM, Li Z, McLendon RE, Hjelmeland AB, Rich JN. The hypoxic microenvironment maintains glioblastoma stem cells and promotes reprogramming towards a cancer stem cell phenotype. Cell Cycle. 2009;8:3274–3284. doi: 10.4161/cc.8.20.9701. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Kaur B, Khwaja FW, Severson EA, Matheny SL, Brat DJ, Van Meir EG. Hypoxia and the hypoxia-inducible-factor pathway in glioma growth and angiogenesis. Neuro Oncol. 2005;7:134–153. doi: 10.1215/S1152851704001115. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Brat DJ, Mapstone TB. Malignant glioma physiology: cellular response to hypoxia and its role in tumor progression. Ann Intern Med. 2003;138:659–668. doi: 10.7326/0003-4819-138-8-200304150-00014. [DOI] [PubMed] [Google Scholar]
- 12.Mineta T, Rabkin SD, Yazaki T, Hunter WD, Martuza RL. Attenuated multi-mutated herpes simplex virus-1 for the treatment of malignant gliomas. Nat Med. 1995;1:938–943. doi: 10.1038/nm0995-938. [DOI] [PubMed] [Google Scholar]
- 13.Markert JM, Medlock MD, Rabkin SD, Gillespie GY, Todo T, Hunter WD, Palmer CA, Feigenbaum F, Tornatore C, Tufaro F, et al. Conditionally replicating herpes simplex virus mutant, G207 for the treatment of malignant glioma: results of a phase I trial. Gene Ther. 2000;7:867–874. doi: 10.1038/sj.gt.3301205. [DOI] [PubMed] [Google Scholar]
- 14.Rampling R, Cruickshank G, Papanastassiou V, Nicoll J, Hadley D, Brennan D, Petty R, MacLean A, Harland J, McKie E, et al. Toxicity evaluation of replication-competent herpes simplex virus (ICP 34.5 null mutant) in patients with recurrent malignant glioma. Gene Ther. 2000;7:859–866. doi: 10.1038/sj.gt.3301184. [DOI] [PubMed] [Google Scholar]
- 15.McKie EA, MacLean AR, Lewis AD, Cruickshank G, Rampling R, Barnett SC, Kennedy PG, Brown SM. Selective in vitro replication of herpes simplex virus type 1 (HSV-1) ICP34.5 null mutants in primary human CNS tumours—evaluation of a potentially effective clinical therapy. Br J Cancer. 1996;74:745–752. doi: 10.1038/bjc.1996.431. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Aghi MK, Liu TC, Rabkin S, Martuza RL. Hypoxia enhances the replication of oncolytic herpes simplex virus. Mol Ther. 2009;17:51–56. doi: 10.1038/mt.2008.232. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Fasullo M, Burch AD, Britton A. Hypoxia enhances the replication of oncolytic herpes simplex virus in p53? breast cancer cells. Cell Cycle. 2009;8:2194–2197. doi: 10.4161/cc.8.14.8934. [DOI] [PubMed] [Google Scholar]
- 18.Longo SL, Griffith C, Glass A, Shillitoe EJ, Post DE. Development of an oncolytic herpes simplex virus using a tumor-specific HIF-responsive promoter. Cancer Gene Ther. 2011;18:123–134. doi: 10.1038/cgt.2010.62. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Giannini C, Sarkaria JN, Saito A, Uhm JH, Galanis E, Carlson BL, Schroeder MA, James CD. Patient tumor EGFR and PDGFRA gene amplifications retained in an invasive intracranial xenograft model of glioblastoma multiforme. Neuro Oncol. 2005;7:164–176. doi: 10.1215/S1152851704000821. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Verhaak RG, Hoadley KA, Purdom E, Wang V, Qi Y, Wilkerson MD, Miller CR, Ding L, Golub T, Mesirov JP, et al. Integrated genomic analysis identifies clinically relevant subtypes of glioblastoma characterized by abnormalities in PDGFRA, IDH1, EGFR, and NF1. Cancer Cell. 2010;17:98–110. doi: 10.1016/j.ccr.2009.12.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Friedman HS, Dolan ME, Moschel RC, Pegg AE, Felker GM, Rich J, Bigner DD, Schold SC., Jr Enhancement of nitrosurea activity in medulloblastoma and glioblastoma multiforme. J Natl Cancer Inst. 1992;84:1926–1931. doi: 10.1093/jnci/84.24.1926. Erratum in: J Natl Cancer Inst86, 1027. [DOI] [PubMed] [Google Scholar]
- 22.Cassady KA. Human cytomegalovirus TRS1 and IRS1 gene products block the double-stranded-RNA-activated host protein shutoff response induced by herpes simplex virus type 1 infection. J Virol. 2000;79:8707–8715. doi: 10.1128/JVI.79.14.8707-8715.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Geraghty RJ, Krummenacher C, Cohen GH, Eisenberg RJ, Spear PG. Entry of alphaherpesviruses mediated by poliovirus receptor-related protein 1 and poliovirus receptor. Science. 1998;280:1618–1620. doi: 10.1126/science.280.5369.1618. [DOI] [PubMed] [Google Scholar]
- 24.Krummenacher C, Baribaud F, Ponce de Leon M, Baribaud I, Whitbeck JC, Xu R, Cohen GH, Eisenberg RJ. Comparative usage of herpesvirus entry mediator A and nectin-1 by laboratory strains and clinical isolates of herpes simplex virus. Virology. 2004;322:286–299. doi: 10.1016/j.virol.2004.02.005. [DOI] [PubMed] [Google Scholar]
- 25.Friedman GK, Langford C, Coleman J, Cassady KA, Parker JN, Markert JM, Gillespie GY. Engineered herpes simplex virus efficiently infects and kills CD133+ glioma cells that express CD111. J Neurooncol. 2009;95:199–209. doi: 10.1007/s11060-009-9926-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Stiles KM, Milne RS, Cohen GH, Eisenberg RJ, Krummenacher C. The herpes simplex virus receptor nectin-1 is down-regulated after trans-interaction with glycoprotein D. Virology. 2008;373:98–111. doi: 10.1016/j.virol.2007.11.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Jensen RL. Brain tumor hypoxia: tumorigenesis, angiogenesis, imaging, pseudoprogression, and as a therapeutic target. J Neurooncol. 2009;92:317–335. doi: 10.1007/s11060-009-9827-2. [DOI] [PubMed] [Google Scholar]
- 28.Wakimoto H, Kesari S, Farrell CJ, Curry WT, Jr, Zaupa C, Aghi M, Kuroda T, Stemmer-Rachamimov A, Shah K, Liu TC, et al. Human glioblastoma-derived cancer stem cells: establishment of invasive glioma models and treatment with oncolytic herpes simplex virus vectors. Cancer Res. 2009;69:3472–3481. doi: 10.1158/0008-5472.CAN-08-3886. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Spear PG. Herpes simplex virus: receptors and ligands for cell entry. Cell Microbiol. 2004;6:401–410. doi: 10.1111/j.1462-5822.2004.00389.x. [DOI] [PubMed] [Google Scholar]
- 30.Shukla D, Liu J, Blaiklock P, Shworak NW, Bai X, Esko JD, Cohen GH, Eisenberg RJ, Rosenberg RD, Spear PG. A novel role for 3-O-sulfated heparan sulfate in herpes simplex virus 1 entry. Cell. 1999;99:13–22. doi: 10.1016/s0092-8674(00)80058-6. [DOI] [PubMed] [Google Scholar]
- 31.Huang YY, Yu Z, Lin SF, Li S, Fong Y, Wong RJ. Nectin-1 is a marker of thyroid cancer sensitivity to herpes oncolytic therapy. J Clin Endocrinol Metab. 2007;92:1965–1970. doi: 10.1210/jc.2007-0040. [DOI] [PubMed] [Google Scholar]
- 32.Yu Z, Adusumilli PS, Eisenberg DP, Darr E, Ghossein RA, Li S, Liu S, Singh B, Shah JP, Fong Y, et al. Nectin-1 expression by squamous cell carcinoma is a predictor of herpes oncolytic sensitivity. Mol Ther. 2007;15:103–113. doi: 10.1038/sj.mt.6300009. [DOI] [PubMed] [Google Scholar]
- 33.Yu Z, Chan MK, O-charoenrat P, Eisenberg DP, Shah JP, Singh B, Fong Y, Wong RJ. Enhanced nectin-1 expression and herpes oncolytic sensitivity in highly migratory and invasive carcinoma. Clin Cancer Res. 2005;11:4889–4897. doi: 10.1158/1078-0432.CCR-05-0309. [DOI] [PubMed] [Google Scholar]
- 34.Rueger MA, Winkeler A, Miletic H, Kaestle C, Richter R, Schneider G, Hilker R, Heneka MT, Ernestus RI, Hampl JA, et al. Variability in infectivity of primary cell cultures of human brain tumors with HSV-1 amplicon vectors. Gene Ther. 2005;12:588–596. doi: 10.1038/sj.gt.3302462. [DOI] [PubMed] [Google Scholar]
- 35.Koike T, Kimura N, Miyazaki K, Yabuta T, Kumamoto K, Takenoshita S, Chen J, Kobayashi M, Hosokawa M, Taniguchi A, et al. Hypoxia induces adhesion molecules on cancer cells: a missing link between Warburg effect and induction of selectin-ligand carbohydrates. Proc Natl Acad Sci USA. 2004;101:8132–8137. doi: 10.1073/pnas.0402088101. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Orvedahl A, Levine B. Autophagy and viral neurovirulence. Cell Microbiol. 2008;10:1747–1756. doi: 10.1111/j.1462-5822.2008.01175.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Griguer CE, Oliva CR, Gobin E, Marcorelles P, Benos DJ, Lancaster JR, Jr, Gillespie GY. CD133 is a marker of bioenergetic stress in human glioma. PLoS One. 2008;3:e3655. doi: 10.1371/journal.pone.0003655. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Cassady KA, Saunders U, Shimamura M. Δγ134.5 HSVs encoding HCMV IRS1 or TRS1 induce interferon regulatory factor 3 (IRF3) phosphorylation and interferon stimulated gene response. J Virol. 2011;86:610–614. doi: 10.1128/JVI.05099-11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Ock J, Jeong J, Choi WS, Lee WH, Kim SH, Kim IK, Suk K. Regulation of Toll-like receptor 4 expression and its signaling by hypoxia in cultured microglia. J Neurosci Res. 1995;85:1989–1995. doi: 10.1002/jnr.21322. [DOI] [PubMed] [Google Scholar]
- 40.Lee MY, Joung YH, Lim EJ, Park JH, Ye SK, Park T, Zhang Z, Park DK, Lee KJ, Yang YM. Phosphorylation and activation of STAT proteins by hypoxia in breast cancer cells. Breast. 2006;15:187–195. doi: 10.1016/j.breast.2005.05.005. [DOI] [PubMed] [Google Scholar]
- 41.Papadakis AI, Paraskeva E, Peidis P, Muaddi H, Li S, Raptis L, Pantopoulos K, Simos G, Koromilas AE. eIF2α kinase PKR modulates the hypoxic response by Stat3-dependent transcriptional suppression of HIF-1α. Cancer Res. 2010;70:7820–7829. doi: 10.1158/0008-5472.CAN-10-0215. [DOI] [PubMed] [Google Scholar]
- 42.Orvedahl A, Alexander D, Tallóczy Z, Sun Q, Wei Y, Zhang W, Burns D, Leib DA, Levine B. HSV-1 ICP34.5 confers neurovirulence by targeting the Beclin 1 autophagy protein. Cell Host Microbe. 2007;1:23–35. doi: 10.1016/j.chom.2006.12.001. [DOI] [PubMed] [Google Scholar]
- 43.Song J, Guo X, Xie X, Zhao X, Li D, Deng W, Song Y, Shen F, Wu M, Wei L. Autophagy in hypoxia protects cancer cells against apoptosis induced by nutrient deprivation through a beclin1-dependent way in hepatocellular carcinoma. J Cell Biochem. 2011;112:3406–3420. doi: 10.1002/jcb.23274. [DOI] [PubMed] [Google Scholar]
- 44.Smith MC, Boutell C, Davido DJ. HSV-1 ICP0: paving the way for viral replication. Future Virol. 2011;6:421–429. doi: 10.2217/fvl.11.24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Wojton J, Kaur B. Impact of tumor microenvironment on oncolytic viral therapy. Cytokine Growth Factor Rev. 2010;21:127–134. doi: 10.1016/j.cytogfr.2010.02.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Hellums EK, Markert JM, Parker JN, He B, Perbal B, Roizman B, Whitley RJ, Langford CP, Bharara S, Gillespie GY. Increased efficacy of an interleukin-12-secreting herpes simplex virus in a syngeneic intracranial murine glioma model. Neuro Oncol. 2005;7:213–224. doi: 10.1215/S1152851705000074. [DOI] [PMC free article] [PubMed] [Google Scholar]