Abstract
Tumors are characterized by a rigid, highly cross-linked extracellular matrix (ECM), which impedes homogeneous drug distribution and potentially protects malignant cells from exposure to therapeutics. Lysyl oxidases are major contributors to tissue stiffness and the elevated expression of these enzymes observed in most cancers might influence drug distribution and efficacy. We examined the effect of lysyl oxidases on drug distribution and efficacy in 3D in vitro assay systems. In our experiments elevated lysyl oxidase activity was responsible for reduced drug diffusion under hypoxic conditions and consequently impaired cytotoxicity of various chemotherapeutics. This effect was only observed in 3D settings but not in 2D-cell culture, confirming that lysyl oxidases affect drug efficacy by modification of the ECM and do not confer a direct desensitizing effect. Both drug diffusion and efficacy were strongly enhanced by inhibition of lysyl oxidases. The results from the in vitro experiments correlated with tumor drug distribution in vivo, and predicted response to therapeutics in murine tumor models. Our results demonstrate that lysyl oxidase activity modulates the physical barrier function of ECM for small molecule drugs influencing their therapeutic efficacy. Targeting this process has the potential to significantly enhance therapeutic efficacy in the treatment of malignant diseases.
Systemic therapy is one of the cornerstones in the management of cancer. It is of particular importance for the treatment of progressed metastatic disease when local therapeutic options like surgery and radiotherapy can only have limited effect. However, systemic treatment is rarely curative and it is estimated that in around 90% of cases chemotherapeutics are administered in a purely palliative setting, aiming to stabilize disease, prolong survival and improve quality of life1. There are two major reasons for this situation: first the considerable side effects and the small therapeutic window of most anti-neoplastic drugs, leads in a significant fraction of patients to a premature termination of the treatment. Second, development of resistance to the therapy is ultimately observed in almost all patients. Both of these problems are aggravated by the ineffective transport of systemic drugs into the tumor and to the malignant cells. It has been demonstrated for a variety of different drugs that their concentration after systemic application is significantly lower in the tumor tissue than in non-target organs2,3,4. Moreover, the drug distribution within solid tumors is heterogeneous, leaving large parts of the tumor protected from therapeutically effective drug concentrations5. The impaired drug transport necessitates higher dosage of the drugs, leading to higher systemic exposure and subsequently increased risk of side effects. On the other side, exposing tumor cells to low, sub-lethal drug levels as a direct result of the impaired, heterogeneous intratumoral distribution, effectively selects for resistance.
The defective tumor vasculature has been extensively discussed as a main reason for the impaireded tumor drug delivery6. The persistently strongly pro-angiogenic milieu in the tumor micro-environment keeps vascular endothelial cells in a constantly activated state, leading to permanent vessel remodeling, deficient support by pericytes and vascular permeability. Various anti-angiogenic strategies have been investigated with the aim to improve drug delivery7,8,9,10.
However, to be effective, anti-neoplastic drugs must not only be transported into the tumor, but they also have to reach the tumor cells at a critical concentration. The rate of diffusion within the tissue depends on the characteristics of the extracellular matrix (ECM), which forms a barrier that drugs first have to cross before reaching the individual tumor cells. Like the tumor vasculature, the ECM in tumors differs significantly from the ECM in normal tissue. The most apparent is the abundance of extracellular material in many tumors, with ECM often representing 30–60% of the tumor volume11. Also its composition, the ratio of different ECM contributing macromolecules, differs significantly from the ECM in the surrounding tissue of origin12,13,14,15,16. Finally, matrix modifying, stabilizing and degrading enzymes like metallo proteinases, and lysyl oxidases are strongly dysregulated in cancer17,18,19. The altered biochemical condition also influences the biomechanical characteristics of the pathological tumor microenvironment. Tumor tissue is more rigid than normal tissue, and the physical barrier function of the ECM impeding diffusion is increased, strongly inhibiting transport and efficacy of macromolecular drugs20,21,22,23. Experimental data on how ECM characteristics influence transport and efficacy of small molecule drugs however is still limited.
Stability and biomechanical properties of the ECM are strongly influenced by the activity lysyl oxidases, a family of five copper-enzymes (LOX and LOXL1-4), which catalyze the cross-linking of collagens and elastin24,25. The cross-linking not only stabilizes these proteins, but also increases tissue stiffness and in all probability reduces diffusion through the ECM. Lysyl oxidases have been consistently found to be up-regulated in solid tumors26,27,28. In contrast to the structural components of the ECM, lysyl oxidases as enzymes are well targetable by pharmacological means. Thus, inhibition of lysyl oxidases or modulation of their activity might be an interesting approach to reduce tumor stiffness and potentially improve tumor drug distribution.
To get a better understanding of the influence of ECM crosslinking on drug delivery, the effect of lysyl oxidases on the diffusion and efficacy of small molecular drugs was studied in 3D in vitro assay systems. Our results demonstrate that lysyl oxidases activity indeed strongly influences drug diffusion and thereby cytotoxicity of chemotherapeutic drugs. This effect is restricted to 3D assay conditions, while lysyl oxidases do not affect drug efficacy towards tumor cells grown in 2D. Moreover, our results show that the efficacy of various cytotoxic drugs is differently affected by increased lysyl oxidase activity, a finding we were able to corroborate in vivo.
Results
Increased lysyl oxidase activity reduces drug diffusion
We have shown recently that lysyl oxidase inhibition improved response to cytotoxic drugs in vivo. The results indicated that lysyl oxidase catalyzed ECM-crosslinking impedes tumor drug diffusion, thereby forming a physical barrier that protects tumor cells from exposure to the systemically applied drugs. However, in vivo matrix changes caused by inhibition of lysyl oxidases also have secondary effects on the tumor microenvironment, on stroma cell populations and on the tumor vasculature29,30. These effects might also affect drug delivery and efficacy. Consequently, to test the effect of lysyl oxidases on drug diffusion unaffected from these secondary effects, we established a tumor spheroid-based in vitro assay system (Fig. 1). Spheroids were generated by culturing tumor cells in absence of a suitable attachment surface, using the liquid overlay technique31,32. With the compact spheroids formed after six days of culture it was possible to measure the diffusion of fluorescent drugs, like doxorubicin (DOX), via laser scanning confocal microscopy in real time. We used spheroids generated from four different murine tumor cell lines—lewis lung carcinoma (LLC), a fibrosarcoma line (MT6) and two breast carcinoma lines (4T1, EMT6). The diffusion rate differed significantly between spheroids generated from the various lines (Fig. 2A,B). To examine the effect of lysyl oxidase on drug diffusion, we cultivated the spheroids in presence of 500 μM 3-amino propionitrile (βAPN), an inhibitor of all five lysyl oxidase family members33,34,35. Even at this high concentration βAPN did not affect formation or size of the generated spheroids (Supplemental Fig. 1). When tumor spheroids were cultivated at standard levels of 20% O2, βAPN-treatment did not significantly affect diffusion of doxorubicin (DOX). However, when the spheroids were cultivated at reduced oxygen levels (2% O2), mimicking hypoxic conditions seen in tumors, DOX diffusion was significantly reduced in the spheres generated from all four cell lines. Under the reduced oxygenation levels βAPN-treatment was able to at least partially restore DOX diffusion to levels observed under high oxygen exposure. To validate that βAPN treatment indeed reduces collagen crosslinking we treated tumor bearing mice for 10 days with βAPN, and evaluated the ECM from the treated tumors by interferences reflection microscopy36. βAPN treatment significantly reduced the number of crosslinked colagen fibers (Supplemental Fig. 2).
It has been demonstrated that the lysyl oxidase family members LOX, LOXL2 and LOXL4 are regulated by the Hif1α/β transcription factors28,36,37. We analyzed the promoter regions of the LOX proteins for the HIF binding site consensus sequence 5′-(R)CGTG-3′38. In the promoter regions of the human genes lox, loxl1 and loxl4, and of the murine genes lox and loxl1 two copies of the consensus sequences as inverted repeats could be found (Supplemental Fig. 3). Thus, cells cultivated under reduced oxygen levels can be expected to secrete elevated levels of lysyl oxidases. However, it has been shown previously that depending on the analyzed cancer cell line, different members of the lysyl oxidase family are up-regulated in response to reduced oxygen supply36. We therefore tested the four cell lines in our experiments for changes in the mRNA expression of the five lysyl oxidases under reduced oxygen levels (Fig. 3A). Most consistently mRNA-levels of LOX, LOXL2 and LOXL4 were up-regulated. LOXL2 was stronger expressed in all four cell lines, and also showed the strongest increase in mRNA-levels (9.5-fold in MT6, 7.6-fold in LLC cells). LOX and LOXL4 were up-regulated in all but EMT6 cells. In contrast LOXL1 and LOXL3 were only significantly up-regulated in 4T1 and MT6 cells respectively. The up-regulation of mRNA-levels also resulted in increased overall lysyl oxidase activity in the supernatant of cells cultivated at oxygen levels of 2% (Fig. 3B). In all four lines cultivation at reduced oxygen levels increased secreted lysyl oxidase activity within the supernatant by 43% to 127%.
In the spheroid assay, diffusion rates of DOX differed significantly between spheroids generated from different cell lines. We subsequently tested if this is also reflected in the corresponding tumors generated by implantation of the respective cell lines. Tumor bearing mice were injected with the fluorescent tracer Hoechst 33342 and fluorophor-labeled isolectin GS-B4 and sacrificed 20 min later. In the harvested tumors, diffusivity for small molecule drugs could be estimated by the penetration depth of Hoechst 33342 from the stained vessels into the tumor parenchyma (Fig. 3C). Penetration depth varied significantly in the various tumors and was correlated with the diffusion rates for DOX observed in vitro (Fig. 3D).
Recombinant overexpression of lysyl oxidases reduces drug diffusion
To further corroborate the concept that lysyl oxidases directly influence drug diffusity and efficacy, we constructed lentiviral vectors for the constitutive overexpression of hLOX and hLOXL2 in mammalian cells (Supplemental Fig. 4). Lentiviral pseudoparticles generated with these vectors in HEK293T cells were used to infect 4T1 cells, which were selected for these experiments because they displayed the lowest basal expression of lysyl oxidases. We first tested that the recombinant lysyl oxidases were secreted in active form into the culture supernatant (Fig. 4A). Tumor spheroids from control transfected and LOX/LOXL2 overexpressing 4T1 cells were generated and cultivated at 20% oxygen levels to keep interferences by endogenous lysyl oxidases at a minimum. Tumor spheroids generated from the overexpressing cells showed a strongly impaired diffusion of DOX (Fig. 4B,D). The effect was stronger in the hLOXL2 overexpressing 4T1 cells compared to spheroids generated from 4T1 cells expressing hLOX (Fig. 4C,E). Although βAPN significantly improved drug diffusion in the hLOXL2 spheroids, the reduced diffusivity in the 4T1 hLOXL2 spheroids could not completely be restored by incubation with βAPN.
Impaired drug diffusion after matrix crosslinking affects efficacy of cytotoxic drugs
The impaired diffusion observed in the spheroid assays should also reduce the efficacy of drugs to act as cytotoxic agents. Evidently, a possible effect of matrix crosslinking on cytotoxic activity should only be observable in a 3D setting, when cells are surrounded by a matrix that can act as a protective barrier. Furthermore, protection from exposure to drugs by impaired diffusion through a highly cross-linked matrix should not be a factor when tumor cells are exposed to the agents over several days—a standard practice for testing the cytotoxic potential of drugs in vitro39,40. Thus, to test the potential effect of a physical barrier function of the ECM on drug efficacy, 3D-embedded cells have to be exposed to tested drugs only for a short period of time. Limiting the time that drugs are allowed to reach the tumor cells by diffusion increases the impeding effect of a physical diffusion barrier and simultaneously reflects the situation in vivo more closely, where small molecule drugs are cleared relatively fast from the blood stream. Based on this considerations, we evaluated cell toxicity of two standard chemotherapeutic drugs, DOX and paclitaxel (PTX), on MT6, EMT6 and 4T1 cells, in three different settings: standard long-term exposure (72 h) of cells grown in 2D, short term exposure (30 min) followed gradual removal of the drug from the media in 2D, and short term exposure under the same conditions of 3D collagen-embedded tumor cells (Fig. 5A). As the spheroid diffusion experiments indicated that lysyl oxidases only significantly affect drug diffusion at reduced pO2, these experiments were performed with cells cultivated at 2% O2 levels. When tumor cells were exposed to the two potent cytotoxic drugs DOX and PTX for 72 h (Fig. 5B), we recorded EC50-values in the medium nanomolar range with sensitivities varying only by factor of four (from 19.0 ± 3.6 nM for 4T1/DOX to 67.8 ± 16.2 nM for EMT6/DOX). Not surprisingly, reduction to the short exposure time significantly reduced the toxicity of the drugs in all pairings (Fig. 5C). In the short-term system, we also tested if treatment with βAPN, either by lysyl oxidase inhibition or by an alternative mechanism, has an effect on the toxicity of the chemotherapeutics. Even at a concentration of 500 μM βAPN had no toxic effects on the tested tumor cells (Supplemental Fig. 5). Moreover, it also did not affect the toxicity of DOX or PTX (Fig. 5D). Embedding and treating the tumor cells in a 3D collagen matrix further reduced cell toxicity by at least one order of magnitude (Fig. 5E). The matrix embedding similarly reduced apparent cell toxicity of both drugs. Importantly, only in the 3D setting lysyl oxidase inhibition with βAPN decreased cell toxicity of the drugs, which further validated the concept that lysyl oxidases interfere with drug efficacy by reducing matrix diffusivity of small molecule drugs.
We next tested the effect of recombinant LOX and LOXL2 overexpression on the cytotoxic potential of DOX on 4T1 cells. Overexpression of hLOX or hLOXL2 did not affect sensitivity of 4T1 cells towards DOX when grown in 2D (Fig. 5E). However, after embedding into a 3D collagen-matrix, overexpression of hLOX and hLOXL2 significantly reduced cell toxic effects of DOX on the 4T1 cells.
An interesting finding of the in vitro studies was, that while the efficacy of the chemotherapeutics was gradually reduced when cells were shorter exposed to the drugs and better protected by embedding into a collagen matrix, the extent of this effect was drastically different for the various cell/line drug parings (Fig. 6A,B). Most remarkably, 4T1 cells, which were equally sensitive to PTX and DOX in long-term exposure experiments (EC50 = 26.4 ± 5.4 nM and 19.0 ± 3.6 nM, respectively), were much less affected in short-term exposure experiments by PTX, with EC50-values differing by two orders of magnitude from the sensitivity towards DOX (160 ± 10.6 μM in 2D, 1280 ± 40.2 μM in 3D for PTX; 2.19 ± 0.46 μM in 2D, 15.9 ± 1.07 μM in 3D for DOX). Conversely, in each of the assay settings the sensitivity of MT6 cells to both tested drugs was similar. If the short-term exposure settings indeed better reflects the situation in vivo, 4T1 tumors should be resistant to PTX abut sensitive to DOX, while MT6 tumors should be sensitive to both drugs. To test this hypothesis we implanted mice with 4T1 and MT6 cells respectively, and treated established tumors with either of the two chemotherapeutics. Treatment was performed using established protocols (5 mg/kg BW DOX or 20 mg/kg BW PTX)41. The MT6 fibrosarcoma indeed reacted with significantly reduced growth to treatment with either DOX or PTX (Fig. 6B,C), reflecting the similar sensitivity of the cells in vitro. Conversely, 4T1 tumors responded well to DOX with strongly reduced growth, but PTX at the given dosage was not able to affect a significant reduction in tumor growth (Fig. 6D,E). Thus, from the results of the 3D cell toxicity experiments we were able to predict the insensitivity of 4T1 tumors towards PTX. Importantly, this resistance would not have been predictable from in vitro assays under standard long-term exposure conditions.
Discussion
In this study we have demonstrated that lysyl oxidase activity affects drug efficacy in a direct way by increasing the barrier function of the ECM even for small molecule drugs. The results indicate that interfering with this process by inhibiting lysyl oxidase activity has the potential to increase the efficacy of systemic treatment with various drugs.
Central to our approach was the utilization of tumor spheroid diffusion assays. The spheroids allow growing cells in a compact 3D environment with tightly packed cells resembling the situation of tumor tissue. It has already been shown that tumor spheroids reflect many aspects of tumor tissue more closely than cells grown in 2D32,42,43,44,45. Transferring tumor cells in a 3D setting substantially alters their metabolism, proliferation rate and expression profile. These changes also affect drug sensitivity, and tumor cells are substantially more resistant to drugs in a 3D environment than in 2D culture. Imamura et al. have recently demonstrated that the reduced drug sensitivity in the 3D environment results from a lower proliferation rate and reduced expression of caspase-3, which is involved in the apoptotic cascade42. Also the impaired oxygenation in tumor spheroids leads to up-regulation of Hif1α and subsequently to increased expression of ABC transporters like MDR143. These studies focused on the changes in tumor cell behavior in the 3D environment. However, an important aspect of 3D-cultures is the ECM itself, whether supplied as an embedding media or directly produced by the cells as an anchoring matrix. In addition to previous work our study demonstrate the protective effect of the matrix as a physical barrier that impedes drug transport to the cells. Direct effects of lysyl oxidases on drug sensitivity can be excluded as inhibition with βAPN in our experiments only increased efficacy of the tested chemotherapeutics when cells were embedded in a collagen matrix. Without a surrounding modifiable matrix lysyl oxidase inhibition did neither influence cell proliferation by itself nor the sensitivity towards drugs. Equally, we were able to show that overexpression of hLOX or hLOXL2 did not alter the sensitivity to cytotoxic drugs, but protected the cells from exposure to the agents by modifying the surrounding matrix. Consequently, inhibition of lysyl oxidases increased the amount of drugs reaching the cells, significantly reducing the concentration needed to cause a cytotoxic effect. Improving drug distribution and cytotoxicity by lysyl oxidase inhibition was successful in multiple genetically different cell lines, which also originated in different organs, an important result as the heterogeneity between tumors often precludes successful implementation of a therapeutic strategies over a range of different tumors. The tested cell lines showed very different patterns of expression and induction by hypoxia of the LOX family members. That treatment with βAPN was able to significantly improved delivery and response in these diverse cell lines further underlines the functional redundancy of the LOX family members and simultaneously reveals the clinical potential of a potent pan-lysyl oxidase inhibitor as an auxiliary agent to improve treatment efficacy. βAPN itself has unfavorable pharmacological properties and is an only moderately active inhibitor, however due to the discovery that the lysyl oxidases are essential in the formation of a malignant tumor microenvironment several more promising inhibitors are currently under development.
It is important to realize that the different aspects that contribute to reduced drug sensitivity in both 3D culture and in vivo are interdependent. Effects that impair drug diffusion also reduce transport of oxygen. Previous studies showed that apparent drug diffusion constant and pO2 in tumors is correlated46,47. Subsequently, increased lysyl oxidase expression should lead to increased hypoxia. Hypoxic tissue is less sensitive to cytotoxic therapeutics. As mentioned above, one reason is the up-regulation of efflux pumps, as ABC transporters like MDR1 and BCRP that are positively regulated by Hif1α43,48,49. Hence, it is possible that treatment with βAPN in our experiments in addition to evidently improving drug diffusion also enhanced drug efficacy and drug accumulation in the spheroids’ cells by increasing oxygen supply and subsequent down-regulating ABC transporters. This hypothesis needs to be examined further in subsequent studies.
We only could detect a significant dependency of drug diffusion from lysyl oxidase expression in tumor spheroids at a reduced oxygen level of 2%. At this reduced O2 concentration lysyl oxidases activity was significantly up-regulated in all cell lines. Which of the five lysyl oxidase family members was up-regulated at reduced pO2 differed between the tested cell lines, confirming previously published results36. Importantly, cultivation of cells at 2% O2 mirrors the actual oxygenation status of solid tumors. In tumors pO2 ranges between 2.5 and 15 mmHg (0.5–2.0 × 103 Pa, or 0.5–2.0 × 10−2 Atm) and in normal organs between 15 and 75 mmHg46,50. In cell culture media pO2 levels in equilibrium are around 144 mmHg under standard conditions (19% O2, 80% N2, 5% CO2, 5.5% H2O), considerably higher than even in healthy well-perfused tissue. Thus, reduction to 2% O2 (pO2 = 15.2 mmHg) in a suitable incubator, reflects better the conditions within a still comparatively well-oxygenated tumor. Thus, the often-used terms “hypoxic conditions” for cell cultivation at less than 5% O2 and “normoxic conditions” for cell cultivation at 20% O2 are misleading. Indeed cultivation at 20% O2 subjects the cells to considerable oxidative stress and reflects the situation neither in a poorly supplied tumor nor in normal tissue.
In addition to oxygen concentration the outcome of the drug sensitivity assays relied strongly on the treatment time with the cytotoxic agents. While it is not only intuitive but long established that reducing the time that cells are subjected to a treatment decreases its effect51, both screening for new anti-neoplastic drugs and pharmacological characterization of tumor cell lines is still often performed by prolonged exposure of tumor cells to the test compounds. An exposure time of 72 h—which we also used in our experiments—appears to be established as the standard time frame for these experiments39,40. However, in our experiments the relative sensitivity of the tumor cells towards the tested chemotherapeutics after such a prolonged exposure did not reflect at all the situation we later observed in vivo. Small molecule drugs have only a short half-life in the circulation, before being removed from the blood stream by hepatic metabolism or renal clearance. After i.v. administration doxorubicin reaches were rapidly a peak concentration, which is also fast reduced by biliary excretion. The AUC(0→20 min) contributes up to 50% of the AUC, thus the main toxicity of the drug is caused within the very early phase after administration52. In our in vitro experiments we tried to emulate the short exposure time to drugs in vivo. As anticipated, this significantly increased the concentration necessary to cause toxicity. Reduction of exposure times affected EC50 levels for PTX stronger than those for DOX. This also could be expected, because PTX exhibits a cell-cycle specific activity, being most effective against cells in G2-M transition, thus being effective only to a subset of tumor cells at a given time point53. However, surprisingly the increase did not run in parallel for all drug/tumor cell line combinations. PTX displayed a very similar toxicity in all tested cell lines when the cells were exposed to it for a prolonged time (72 h). At shorter exposure times EC50 levels differed substantially between the cell lines. In particular 4T1 cells appeared in long term exposure experiments as sensitive to PTX as the other tested cell lines, and their relative resistance was only evident in short term exposure experiments. We were able to corroborate the differences in sensitivity towards PTX that is apparent only in the short-term exposure experiment in vivo. These findings challenge strongly the idea that the results obtained in standard screening processes that measure pharmacological effects after prolonged exposure are even useful for side-by-side comparisons, as the in vivo efficacy of the tested substances can no longer be simply extrapolated from the in vitro results. However, a simple change of the time that cells are subjected to the tested drugs could result in more meaningful results from in vitro assays.
In the last years the role of lysyl oxidases in enhancing tumor angiogenesis, invasiveness and metastasis, have been extensively examined and is meanwhile established17,28,29,34,35,54. Together with our results this indicates that a potent lysyl oxidase inhibitor or therapeutic modulation of lysyl oxidase activity could serve a double purpose in the management of cancer, by simultaneously reducing malignancy of tumors and improving sensitivity towards concomitant treatment with anti-neoplastic drugs.
Methods
If not otherwise indicated, chemicals were purchased from Sigma Aldrich (Munich, Germany) or Carl Roth (Karlsruhe, Germany).
Cell culture
MT6 (CRL-2805), 4T1 (CRL-2539), LLC (CRL-1642) cells were obtained from ATCC (Manassas, VA). EMT6 cells have been purchased from NCI Tumor Repository (http://ncifrederick.cancer.gov/Services/NcifRepositories.aspx). All tumor cells were maintained in DMEM (Gibco, Lifetechnologies) with Penicillin/Streptomycin at 37 °C, 5% CO2. Cell lines were tested for mycoplasma contamination.
D-Cell Spheroid drug penetration assay
Tumor spheroids were generated by the liquid overlay technique, using the protocol from Walser et al. with slight modifications31. Wells of a 96-well culture plate were coated with 45 μL of 1% agarose. After the agarose had solidified, 1000 tumor cells were seeded in 200 μL medium (supplemented with 500 μM βAPN were applicable) on top of the coating. Cells were incubated at 37 °C, 5% CO2 for six days. To test the effect of hypoxia, the forming spheroids were transferred after 96 h to an incubator with oxygen levels set to 2%. Spheroids were transferred to an 8 well cover slip slide (μ-slide, Ibidi, Martinsried, Germany) in 180 μL of medium. 60 μL of 40 μg/mL doxorubicin hydrochloride in PBS were added, and drug penetration was measured using an inverted laser-scanning confocal microscopy (Nikon A1, 20× objective), taking images at 30 min time intervals for four hours. 12 spheroids/assay condition were measured in parallel. The focal plane was identical for spheres. To ensure viability throughout the spheroid in the focal plane, the gfp signal was also recorded.
Acquired images were evaluated using the software ImageJ (http://imagej.nih.gov/ij/). All images of spheroids in a single experiment were transformed into binary images using the same threshold. Average penetration depth was calculated from the doxorubicin stained area and the total area of the spheroids.
Cell toxicity studies
For 2D studies cells were seeded into 96-well dishes at 2500 cells/well in 150 μL full media (supplemented with 500 μM βAPN were applicable) and incubated at 37 °C, 5% CO2, 2% O2. After 48 h 50 μL of DMEM containing four times the indicated concentration of cytotoxic chemotherapeutics were added to each well without prior removal of medium. Each concentration was tested in a 6-fold replicate. Cells were incubated with the therapeutics for 72 h before media was removed. To study the long-term effect of the therapeutics, cells were incubated with the therapeutics for 72 h before media were removed. For short-term exposure, media was removed after 60 min and cells were carefully washed with three changes of media for 30 min each. Afterwards cells were incubated for 72 h.
For 3D studies, tumor cells were suspended at 1.25 × 105 cells/mL in 60% (v/v) full medium and 40% (v/v) of bovine collagen I solution (Purecol 3 mg/mL, Advanced Biomatrix, Carlsbad, CA) were added. pH was adjusted with 0.1 M NaOH (10 μL/mL suspension) and NaHCO3 (5 μL/mL suspension). 40 μL suspension/well were transferred to a 96-well culture plate and incubated for 60 min at 37 °C, 5% CO2 to let the collagen set. 160 μL full medium were added (supplemented with 625 μM βAPN were applicable) and cells were incubated at 37 °C, 5% CO2, 2% O2. After 48 h 50 μL of DMEM containing five times the indicated concentration of cytotoxic chemotherapeutics were added to each well without prior removal of medium. Each concentration was tested in a 6-fold replicate. Media was removed after 60 min and cells were carefully washed with three changes of media for 30 min each. Afterwards embedded cells were incubated for 72 h.
Relative remaining viable cells were determined using the resazurin assay. EC50-values were determined by using a non-linear regression based model using the Prism software (Prism 5, GraphPad, LaJolla, CA).
Lysyl oxidase activity assay
A modified form of the fluorometric microwell assay described by Fogelgren was used55. In short: In a black 96-well microtiterplate was added to 120 μL of an freshly prepared assay solution consisting of 75 μL 2× PBS, 15 μL N-Acetyl-Resorufin (100 μM, Ampliflu-Red, Sigma-Aldrich), 15 μL 2,5-diaminopentane (100 mM, cadaverine a substrate of all five lysyl oxidases55,56,57) and 15 μL horseradish-peroxidase (5 U/mL, Sigma-Aldrich). For each sample the assay solution was prepared in two triplicate rows. To one triplicate 30 μL water, to the other 30 μL βAPN (3.5 mM in water) was added. Finally 50 μL of cell culture supernatant or protein containing lysate was added and the fluorescence of released resorufin was recorded continuously over 3 h in a fluorescence plate reader (PerkinElmer, Wallac II; Ex: 530 nm, Em: 570 nm). Slope of the fluorescence signal in the linear range was calculated for both βAPN-inhibited and non-inhibited samples; the difference was used as a measure of lysyl oxidase activity.
Promotor Analysis
Sequences were retrieved from Ensembl (http://www.ensembl.org), including the 1500 bp 5′-flanking region and the 900 bp downstream of the transcription start point. HRE consensus sequences (5′-RCGTG-3′)36 were located using JASPER (http://jaspar.genereg.net/)58,59.
mRNA expression analysis
Tumor cells were seeded at 104 cells/well in 48 well plates in DMEM with 10% FBS. The cells were cultivated at either 20% or 2% oxygen (5% CO2) for 48 h and harvested by passive lysis. RNA was isolated using the PeqGold RNA kit (Peqlab, Erlangen, Germany) according to the manufactures instructions.
mRNA-Expression was quantified by qRT-PCR on an 7900HT Thermal cycler (Applied Biosystems, Darmstadt, Germany) using SYBR green mix (Fermentas, Darmstadt, Germany). Sequences of primers used in qRT-PCR are given in supplemental table 1. All primer pairs were designed to span at least one intron to avoid amplification of contaminating gDNA. Expression levels of lox-family members were normalized against expression of rsp2960.
Production of lentivirial particles and generation of stable cell lines expressing hLOX and hLOXL2
The entire CDS of both hLOX and hLOXL2 CDS including the signal peptide was amplified from HUVEC cDNA. The amplified DNA was cloned behind the IRES sequence into the lentiviral vector pLVX-luc-IRES-puro (Clontech, Mountain View, CA). Lentiviral particles were generated in HEK 293T cells by co-transfection with the pCMV-dR8.9 and pCMV-VSV-G61 (both plasmids were obtained from Addgene, Cambridge, MA), using a standard CaCl2-based transfection method. Supernatant was used to transfect 4T1 tumor cells. Stable cells selected with puromycin (5 μg/mL). To generate a control cell line, 4T1 cells were thransfected with lentiviral particles produced in HEK 293T cells using the pLVX-luc-IRES-puro plasmid.
Tumor models and treatment
All experiments involving animals were reviewed and approved by the Regional Administration of Unterfranken, Würzburg, and were performed in accordance with relevant guidelines and regulations.
Tumor engraftment
MT6 (106 cells in PBS) fibrosarcomas were generated by subcutaneous injection in the dorsal region of female C57Bl/6J mice. 4T1 (105 cells in PBS) breast adenocarcinomas were generated by injection of cells into the inguinal mammary fat pad of female Balb/c mice. Balb/c mice were purchased from Charles River, Sulzfeld Germany, Balb/c and C57Bl/6J from Jackson Labs, Bar Harbor, ME.
All animals in the individual experiments were of the same age and sex. For each experiment tumor bearing mice were randomly assigned to the different treatment groups just prior to the start of treatment. In treatment studies were tumor growth was a critical outcome assessment of tumor size was performed blinded by a second researcher.
Exclusion of data
Animals that never developed tumors due to take rate lower than 100% were excluded from the studies. All data from animals that died or had to be sacrificed prior to the scheduled termination of the experiment was excluded.
Tumor treatment
Treatment of fully established tumors started on day 12 post implantation. Doxorubicin (5 mg/kg BW) was administered by intra peritoneal injection on indicated days. Control substance for doxorubicin was 0.9% NaCl. Paclitaxel (20 mg/kg BW) was diluted in 0.9% NaCl to a final volume of 200 μL from a stock solution (6 mg/mL PTX in cremophor/EtOH 1:1) and administered by intra peritoneal injection on indicated days. Control substance for PTX was 0.9% NaCl.
3-aminopropionitrile fumarate was administered at 100 mg/kg or 30 mg/kg BW in 0.9% NaCl by daily intra peritoneal injection. Control substance was a corresponding volume of cremophor/EtOH 1:1 diluted to 200 μL in 0.9% NaCl.
Tumor growth was followed by measuring perpendicular diameters of the tumors with a vernier calliper. Tumor volume was calculated using the equation V = π/6 × l × w2. In addition tumors were excised post mortem and weighted. Only tumors that could be excised completely without additional invaded tissue were used for weight measurements.
To monitor intra-tumoral distribution of drugs, 50 μL of Hoechst 33342 (Sigma, 20 mg/mL in 0.9% NaCl) and 100 μL of Alexa 488 or Alexa 647-labeled isolectin GS-B4 (Life Technologies, Darmstadt, Germany. 500 μg/mL in 0.9% NaCl) were injected i.v. into tumor bearing mice 20 min before sacrificing the animal. Tumors were removed and flash frozen in OCT (Sakura Finetek Torrance, CA). Tissue was cut on a cryotom to 200 μm slices and mounted on glass slides. Z-stacks were acquired by confocal imaging in the blue and near infrared channel (Nikon A1, 20 × objective). Tissue penetration was measured as the maximal distance from the vessel surface (by Alexa 647 staining) that Hoechst 33342 staining was present using ImageJ. For this purpose the acquired z-stacks were evaluated at the same tissue depth for isolated, longitudinal cut blood vessels. The maximal distance of Hoechst 33342 staining was measured perpendicular to both sides of each blood vessel, the arithmetic mean of the two values was used. Each blood vessel was evaluated at several positions. At least 5 vessels per stack, and four stacks per biological sample were evaluated.
Collagen Crosslinking Analysis
ECM from βAPN treated and control tumors was obtained by high salt extraction of cellular components. The insoluble ECM was re-suspended in water and used to coat glass slides (angiogenesis μ-slides, Ibidi, Martinsried, Germany) at μg/well. Interferences reflection Images were acquired as z-stacks (30 slides, z-distance: 1.0 μm) on a Nikon A1 microscope in reflection mode using am 60× oil immersion objective and an 647 nm laser following a published protocol36. Identifiable collagen fibers in optical fields were manually counted.
Additional Information
How to cite this article: Schütze, F. et al. Inhibition of Lysyl Oxidases Improves Drug Diffusion and Increases Efficacy of Cytotoxic Treatment in 3D Tumor Models. Sci. Rep. 5, 17576; doi: 10.1038/srep17576 (2015).
Supplementary Material
Acknowledgments
The experimental work was supported by a research grants from the German Research Foundation (DFG, Grant-Nr. HE4365/2-1 and HE4365/3-1) to E.H.
Footnotes
Author Contributions F.S., F.R., S.V., S.G., A.H. and E.H. conducted the experiments. Figures were prepared by F.S. (Figs 2, 4 and 5), S.V. (Fig. 3), and E.H. (Figs 1,2,3,4,5,6). F.S., S.E. and E.H. planned the experiments and the manuscript was written by S.V., S.E. and E.H. All authors reviewed the manuscript.
References
- Kratz F. In Drug Delivery in Oncology − Challenges and Perspectives in Drug Delivery in Oncology – from Research Concepts to Cancer Therapy Vol. 1 (eds Kratz F., Steinhagen H. & Senter P. ) LIX–LXXXV (VCM, 2011). [Google Scholar]
- Gangloff A. et al. Estimation of paclitaxel biodistribution and uptake in human-derived xenografts in vivo with (18)F-fluoropaclitaxel. J Nucl Med 46, 1866–1871 (2005). [PubMed] [Google Scholar]
- Memon A. A. et al. Positron emission tomography (PET) imaging with [11C]-labeled erlotinib: a micro-PET study on mice with lung tumor xenografts. Cancer research 69, 873–878, 10.1158/0008-5472.CAN-08-3118 (2009). [DOI] [PubMed] [Google Scholar]
- Phillips P. G. et al. Increased tumor uptake of chemotherapeutics and improved chemoresponse by novel non-anticoagulant low molecular weight heparin. Anticancer Res 31, 411–419 (2011). [PubMed] [Google Scholar]
- Kato Y., Okollie B. & Artemov D. Noninvasive 1H/13C magnetic resonance spectroscopic imaging of the intratumoral distribution of temozolomide. Magn Reson Med 55, 755–761, 10.1002/mrm.20831 (2006). [DOI] [PubMed] [Google Scholar]
- Jain R. K. Normalization of tumor vasculature: an emerging concept in antiangiogenic therapy. Science 307, 58–62, 307/5706/58. [DOI] [PubMed] [Google Scholar]
- Winkler F. et al. Kinetics of vascular normalization by VEGFR2 blockade governs brain tumor response to radiation: role of oxygenation, angiopoietin-1, and matrix metalloproteinases. Cancer Cell 6, 553–563 (2004). [DOI] [PubMed] [Google Scholar]
- Tong R. T. et al. Vascular normalization by vascular endothelial growth factor receptor 2 blockade induces a pressure gradient across the vasculature and improves drug penetration in tumors. Cancer Res 64, 3731–3736 (2004). [DOI] [PubMed] [Google Scholar]
- Olive K. P. et al. Inhibition of Hedgehog signaling enhances delivery of chemotherapy in a mouse model of pancreatic cancer. Science 324, 1457–1461, 10.1126/science.1171362 (2009). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Escorcia F. E. et al. Selective killing of tumor neovasculature paradoxically improves chemotherapy delivery to tumors. Cancer Res 70, 9277–9286, 0008-5472.CAN-10-202. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nugent L. J. & Jain R. K. Extravascular diffusion in normal and neoplastic tissues. Cancer research 44, 238–244 (1984). [PubMed] [Google Scholar]
- Naba A. et al. The matrisome: in silico definition and in vivo characterization by proteomics of normal and tumor extracellular matrices. Molecular & cellular proteomics: MCP 11, M111 014647, 10.1074/mcp.M111.014647 (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Provenzano P. P. et al. Collagen density promotes mammary tumor initiation and progression. BMC medicine 6, 11, 10.1186/1741-7015-6-11 (2008). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kozono S. et al. Pirfenidone inhibits pancreatic cancer desmoplasia by regulating stellate cells. Cancer Res 73, 2345–2356, 10.1158/0008-5472.CAN-12-3180 (2013). [DOI] [PubMed] [Google Scholar]
- Oskarsson T. et al. Breast cancer cells produce tenascin C as a metastatic niche component to colonize the lungs. Nat Med 17, 867–874, 10.1038/nm.2379 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Toole B. P. & Slomiany M. G. Hyaluronan: a constitutive regulator of chemoresistance and malignancy in cancer cells. Semin Cancer Biol 18, 244–250, 10.1016/j.semcancer.2008.03.009 (2008). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Barker H. E. et al. LOXL2-mediated matrix remodeling in metastasis and mammary gland involution. Cancer research 71, 1561–1572, 10.1158/0008-5472.CAN-10-2868 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Coussens L. M., Tinkle C. L., Hanahan D. & Werb Z. MMP-9 supplied by bone marrow-derived cells contributes to skin carcinogenesis. Cell 103, 481–490 (2000). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tester A. M., Ruangpanit N., Anderson R. L. & Thompson E. W. MMP-9 secretion and MMP-2 activation distinguish invasive and metastatic sublines of a mouse mammary carcinoma system showing epithelial-mesenchymal transition traits. Clin Exp Metastasis 18, 553–560 (2000). [DOI] [PubMed] [Google Scholar]
- Davies Cde L., Berk D. A., Pluen A. & Jain R. K. Comparison of IgG diffusion and extracellular matrix composition in rhabdomyosarcomas grown in mice versus in vitro as spheroids reveals the role of host stromal cells. British journal of cancer 86, 1639–1644, 10.1038/sj.bjc.6600270 (2002). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Beyer I. et al. Controlled extracellular matrix degradation in breast cancer tumors improves therapy by trastuzumab. Molecular therapy: the journal of the American Society of Gene Therapy 19, 479–489, 10.1038/mt.2010.256 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Eikenes L., Tari M., Tufto I., Bruland O. S. & De Lange Davies C. Hyaluronidase induces a transcapillary pressure gradient and improves the distribution and uptake of liposomal doxorubicin (Caelyx) in human osteosarcoma xenografts. British journal of cancer 93, 81–88, 10.1038/sj.bjc.6602626 (2005). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Eikenes L., Bruland O. S., Brekken C. & Davies Cde L. Collagenase increases the transcapillary pressure gradient and improves the uptake and distribution of monoclonal antibodies in human osteosarcoma xenografts. Cancer Res 64, 4768–4773, 10.1158/0008-5472.CAN-03-1472 (2004). [DOI] [PubMed] [Google Scholar]
- Smith-Mungo L. I. & Kagan H. M. Lysyl oxidase: properties, regulation and multiple functions in biology. Matrix Biol 16, 387–398 (1998). [DOI] [PubMed] [Google Scholar]
- Molnar J. et al. Structural and functional diversity of lysyl oxidase and the LOX-like proteins. Biochim Biophys Acta 1647, 220–224 (2003). [DOI] [PubMed] [Google Scholar]
- Baker A. M. et al. The role of lysyl oxidase in SRC-dependent proliferation and metastasis of colorectal cancer. Journal of the National Cancer Institute 103, 407–424, 10.1093/jnci/djq569 (2011). [DOI] [PubMed] [Google Scholar]
- Buchinger B., Spitzer S., Karlic H., Klaushofer K. & Varga F. Lysyl oxidase (LOX) mRNA expression and genes of the differentiated osteoblastic phenotype are upregulated in human osteosarcoma cells by suramin. Cancer Lett 265, 45–54, 10.1016/j.canlet.2008.02.008 (2008). [DOI] [PubMed] [Google Scholar]
- Erler J. T. et al. Lysyl oxidase is essential for hypoxia-induced metastasis. Nature 440, 1222–1226, 10.1038/nature04695 (2006). [DOI] [PubMed] [Google Scholar]
- Zaffryar-Eilot S. et al. Lysyl oxidase-like-2 promotes tumour angiogenesis and is a potential therapeutic target in angiogenic tumours. Carcinogenesis 34, 2370–2379, 10.1093/carcin/bgt241 (2013). [DOI] [PubMed] [Google Scholar]
- Bignon M. et al. Lysyl oxidase-like protein-2 regulates sprouting angiogenesis and type IV collagen assembly in the endothelial basement membrane. Blood 118, 3979–3989, 10.1182/blood-2010-10-313296 (2011). [DOI] [PubMed] [Google Scholar]
- Walser R. et al. Generation of co-culture spheroids as vascularisation units for bone tissue engineering. European cells & materials 26, 222–233 (2013). [DOI] [PubMed] [Google Scholar]
- Ma H. L. et al. Multicellular tumor spheroids as an in vivo-like tumor model for three-dimensional imaging of chemotherapeutic and nano material cellular penetration. Molecular imaging 11, 487–498 (2012). [PubMed] [Google Scholar]
- Tang S. S., Trackman P. C. & Kagan H. M. Reaction of aortic lysyl oxidase with beta-aminopropionitrile. The Journal of biological chemistry 258, 4331–4338 (1983). [PubMed] [Google Scholar]
- Bondareva A. et al. The lysyl oxidase inhibitor, beta-aminopropionitrile, diminishes the metastatic colonization potential of circulating breast cancer cells. PLoS One 4, e5620, 10.1371/journal.pone.0005620 (2009). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Barry-Hamilton V. et al. Allosteric inhibition of lysyl oxidase-like-2 impedes the development of a pathologic microenvironment. Nature medicine 16, 1009–1017, 10.1038/nm.2208 (2010). [DOI] [PubMed] [Google Scholar]
- Wong C. C. et al. Hypoxia-inducible factor 1 is a master regulator of breast cancer metastatic niche formation. Proc Natl Acad Sci USA 108, 16369–16374, 10.1073/pnas.1113483108 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Levental K. R. et al. Matrix crosslinking forces tumor progression by enhancing integrin signaling. Cell 139, 891–906, 10.1016/j.cell.2009.10.027 (2009). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Semenza G. L. et al. Hypoxia response elements in the aldolase A, enolase 1, and lactate dehydrogenase A gene promoters contain essential binding sites for hypoxia-inducible factor 1. J Biol Chem 271, 32529–32537 (1996). [DOI] [PubMed] [Google Scholar]
- Garnett M. J. et al. Systematic identification of genomic markers of drug sensitivity in cancer cells. Nature 483, 570–575, 10.1038/nature11005 (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Barretina J. et al. The Cancer Cell Line Encyclopedia enables predictive modelling of anticancer drug sensitivity. Nature 483, 603–607, 10.1038/nature11003 (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shaked Y. et al. Rapid chemotherapy-induced acute endothelial progenitor cell mobilization: implications for antiangiogenic drugs as chemosensitizing agents. Cancer Cell 14, 263–273 (2008). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Imamura Y. et al. Comparison of 2D- and 3D-culture models as drug-testing platforms in breast cancer. Oncology reports, 10.3892/or.2015.3767 (2015). [DOI] [PubMed] [Google Scholar]
- Doublier S. et al. HIF-1 activation induces doxorubicin resistance in MCF7 3-D spheroids via P-glycoprotein expression: a potential model of the chemo-resistance of invasive micropapillary carcinoma of the breast. BMC Cancer 12, 4, 10.1186/1471-2407-12-4 (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Longati P. et al. 3D pancreatic carcinoma spheroids induce a matrix-rich, chemoresistant phenotype offering a better model for drug testing. BMC Cancer 13, 95, 10.1186/1471-2407-13-95 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shweiki D., Neeman M., Itin A. & Keshet E. Induction of vascular endothelial growth factor expression by hypoxia and by glucose deficiency in multicell spheroids: implications for tumor angiogenesis. Proc Natl Acad Sci USA 92, 768–772 (1995). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dunn J. F. et al. The apparent diffusion constant measured by MRI correlates with pO2 in a RIF-1 tumor. Magn Reson Med 34, 515–519 (1995). [DOI] [PubMed] [Google Scholar]
- Primeau A. J., Rendon A., Hedley D., Lilge L. & Tannock I. F. The distribution of the anticancer drug Doxorubicin in relation to blood vessels in solid tumors. Clin Cancer Res 11, 8782–8788, 11/24/8782. [DOI] [PubMed] [Google Scholar]
- Jogi A. et al. Human neuroblastoma cells exposed to hypoxia: induction of genes associated with growth, survival, and aggressive behavior. Experimental cell research 295, 469–487 (2004). [DOI] [PubMed] [Google Scholar]
- Wartenberg M. et al. Regulation of the multidrug resistance transporter P-glycoprotein in multicellular tumor spheroids by hypoxia-inducible factor (HIF-1) and reactive oxygen species. FASEB journal: official publication of the Federation of American Societies for Experimental Biology 17, 503–505, 10.1096/fj.02-0358fje (2003). [DOI] [PubMed] [Google Scholar]
- Helmlinger G., Yuan F., Dellian M. & Jain R. K. Interstitial pH and pO2 gradients in solid tumors in vivo: high-resolution measurements reveal a lack of correlation. Nat Med 3, 177–182 (1997). [DOI] [PubMed] [Google Scholar]
- Capri G., Tarenzi E., Fulfaro F. & Gianni L. The role of taxanes in the treatment of breast cancer. Seminars in oncology 23, 68–75 (1996). [PubMed] [Google Scholar]
- Schellens J. H. M., McLeod H. L. & Newell D. R. Cancer clinical pharmacology. (Oxford University Press, 2005). [Google Scholar]
- Dorr R. T. Pharmacology of the taxanes. Pharmacotherapy 17, 96S–104S (1997). [PubMed] [Google Scholar]
- Baker A. M. et al. Lysyl Oxidase Plays a Critical Role in Endothelial Cell Stimulation to Drive Tumor Angiogenesis. Cancer research 73, 583–594, 10.1158/0008-5472.CAN-12-2447 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fogelgren B. et al. Cellular fibronectin binds to lysyl oxidase with high affinity and is critical for its proteolytic activation. The Journal of biological chemistry 280, 24690–24697, 10.1074/jbc.M412979200 (2005). [DOI] [PubMed] [Google Scholar]
- Kim S. & Kim Y. Variations in LOXL1 associated with exfoliation glaucoma do not affect amine oxidase activity. Molecular vision 18, 265–270 (2012). [PMC free article] [PubMed] [Google Scholar]
- Rodriguez H. M. et al. Modulation of lysyl oxidase-like 2 enzymatic activity by an allosteric antibody inhibitor. J Biol Chem 285, 20964–20974, 10.1074/jbc.M109.094136 (2010). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sandelin A., Alkema W., Engstrom P., Wasserman W. W. & Lenhard B. JASPAR: an open-access database for eukaryotic transcription factor binding profiles. Nucleic Acids Res 32, D91–94, 10.1093/nar/gkh012 (2004). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mathelier A. et al. JASPAR 2014: an extensively expanded and updated open-access database of transcription factor binding profiles. Nucleic Acids Res 42, D142–147, 10.1093/nar/gkt997 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
- De Jonge H. J. et al. Evidence based selection of housekeeping genes. PLoS One 2, e898, 10.1371/journal.pone.0000898 (2007). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stewart S. A. et al. Lentivirus-delivered stable gene silencing by RNAi in primary cells. Rna 9, 493–501 (2003). [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.