Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2012 Mar 15.
Published in final edited form as: Cancer Res. 2011 Mar 15;71(6):2308–2317. doi: 10.1158/0008-5472.CAN-10-3213

Overcoming temozolomide resistance in glioblastoma via dual inhibition of NAD+ biosynthesis and base excision repair

Eva M Goellner 1,2, Bradford Grimme 2, Ashley R Brown 2, Ying-Chih Lin 2, Xiao-Hong Wang 2, Kelsey F Sugrue 2, Leah Mitchell 2, Ram N Trivedi 1,2, Jiang-bo Tang 2,3, Robert W Sobol 1,2,3,*
PMCID: PMC3077901  NIHMSID: NIHMS268045  PMID: 21406402

Abstract

Glioblastoma multiforme (GBM) is a devastating brain tumor with poor prognosis and low median survival time. Standard treatment includes radiation and chemotherapy with the DNA alkylating agent temozolomide (TMZ). However, a large percentage of tumors are resistant to the cytotoxic effects of the TMZ-induced DNA lesion O6-methylguanine (O6-MeG) due to elevated expression of the repair protein O6-methylguanine-DNA methyltransferase (MGMT) or a defect in the mismatch repair (MMR) pathway. Although a majority of the TMZ induced lesions (N7-methylguanine and N3-methyladenine) are base excision repair (BER) substrates, these DNA lesions are also readily repaired. However, blocking BER can enhance response to TMZ and therefore the BER pathway has emerged as an attractive target for reversing TMZ resistance. Our lab has recently reported that inhibition of BER leads to the accumulation of repair intermediates that induce energy depletion-mediated cell death via hyperactivation of poly(ADP-ribose) polymerase. Based on our observation that TMZ-induced cell death via BER inhibition is dependent on the availability of NAD+, we have hypothesized that combined BER and NAD+ biosynthesis inhibition will increase TMZ efficacy in glioblastoma cell lines greater than BER inhibition alone. Importantly, we find that the combination of BER and NAD+ biosynthesis inhibition significantly sensitizes glioma cells with elevated expression of MGMT and those deficient in MMR, two genotypes normally associated with TMZ resistance. Dual targeting of these two interacting pathways (DNA repair and NAD+ biosynthesis) may prove to be an effective treatment combination for patients with resistant and recurrent GBM.

Keywords: Glioblastoma multiforme, FK866, Base excision repair, temozolomide, methoxyamine

Introduction

Glioblastoma multiforme (GBM) is a devastating form of brain cancer with a dismal median survival time, a high level of resistance to current therapy and common recurrence after treatment (1). The current standard therapy for GBM includes maximum debulking surgery, radiation and treatment with the monofunctional alkylating agent temozolomide (TMZ), also referred to as Temodar® (2). Despite the current standard treatment regimen, including the addition of concomitant and adjuvant TMZ, the average survival expectancy is 14.6 months and the overall 5-year survival rate for GBM is 9.8% (3, 4). TMZ generates a spectrum of DNA lesions including O6-methylguanine, N3-methyladenine and N7-methylguanine (5). The O6-methylguanine lesion is responsible for most of the TMZ associated toxicity and is a substrate for direct repair by O6-methylguanine-DNA methyltransferase (MGMT) (6). In the absence of MGMT repair, O6-methylguanine is suggested to initiate a futile cycle of mismatch repair (MMR) or alternately to trigger ataxia telangiectasia and Rad3 related protein kinase (ATR) activation through the action of several MMR proteins (7), leading to apoptosis and cell death (810). Much of the resistance to TMZ observed clinically is due to high expression of MGMT (and subsequent repair of the lesion) or loss of MMR (therefore preventing the initiation of apoptotic signalling) (1113). Additionally, almost all patients eventually recur with the disease and the large majority of recurrent tumors are resistant to chemotherapy (14, 15). There are currently few alternate treatment options for patients with TMZ resistant tumors and adjuvant chemotherapy options are an area of intense research (1).

The N7-methylguanine and N3-methyladenine adducts make up the majority of the TMZ lesions and are base excision repair (BER) substrates. However, these DNA adducts induce little toxicity under current therapeutic strategies due to rapid repair by BER (16, 17). Hence, small molecules targeting BER have become an attractive option for enhancing TMZ toxicity independent of the cytotoxicity related to the O6-methylguanine DNA lesion. Repair of the alkylation induced DNA lesions N3-methyladenine and N7-methylguanine proceeds primarily through the short patch BER pathway (18). DNA polymerase β (Polβ) has been shown to be the rate-limiting step in BER and loss of Polβ leads to increased sensitivity to the N3-methyladenine and N7-methylguanine lesions due to the accumulation of repair intermediates (16, 19, 20). Alternatively, blocking the abasic site formed during the BER process with the chemical inhibitor methoxyamine (MX) (also referred to as TRC102) has also been shown to enhance alkylation sensitivity independently of the O6-methylguanine lesion (2123). Thus, strategic targeting of BER can enhance TMZ efficacy regardless of MGMT or MMR status.

We have recently reported that cell death due to incomplete BER (e.g., triggered by BER inhibition) is mediated through poly(ADP-ribose) polymerase (PARP) hyperactivation and subsequent NAD+ and ATP depletion (20). The PARP1 (ARTD1) and PARP2 (ARTD2) proteins (24) act as sensors of incomplete BER and become hyperactivated, consuming NAD+ as a substrate for poly(ADP-ribose) (PAR) synthesis (20). Consumption of NAD+ after DNA damage leads to ATP depletion, likely due to continued resynthesis of NAD+ as well as ongoing cellular utilization of NAD+ and ATP for metabolic functions (25, 26). Many reports on DNA-damage induced PARP hyperactivation suggest cell death occurs via PAR signalling and apoptosis inducing factor (AIF) translocation from the mitochondria to the nucleus leading to nuclear fragmentation (27, 28). Interestingly, the caspase-independent cell death observed after incomplete BER is independent of AIF translocation or PAR catabolite signalling (20). Further, alkylation hypersensitivity observed after incomplete BER can be rescued by supplementation with the NAD+ precursor NMN, supporting our conclusion that cell death after BER failure is an energy dependent phenotype (20).

Therefore, we hypothesized that dual targeting of the NAD+ biosynthesis pathway and the BER pathway would greatly enhance TMZ sensitivity regardless of MGMT or MMR status. NAD+ can be synthesized through the de novo pathway from L-tryptophan, but in most mammalian cells NAD+ content is maintained through the NAD+ salvage pathways (29). Nicotinamide phosphoribosyl transferase (NAMPT) controls the salvage-pathway rate-limiting step for the biosynthesis of NAD+ from nicotinamide (30). The small molecule inhibitor FK866 is a specific inhibitor of NAMPT with activity in the low nanomolar range that is highly effective in reducing cellular NAD+ levels (31). FK866 is currently in clinical trials (APO866; Topotarget USA, Inc.) as a monotherapy for the treatment of hematological cancers (32) and is reported to increase cell killing in combination with multiple cytotoxic agents (33). We report herein that combining FK866 with BER inhibition (using the small molecule BER inhibitor MX) potentiates TMZ tumor cell killing and strongly sensitizes glioma cells derived from chemotherapy resistant tumors to TMZ.

Materials and Methods

Chemicals and reagents

Alpha EMEM was from MediaTech (Manassas, VA). MEM, heat-inactivated fetal bovine serum (FBS), L-glutamine, antibiotic/antimycotic, and geneticin were from InVitrogen (Carlsbad, CA). Gentamycin was from Sigma (St. Louis, MO). FK866 (NIMH #F-901; IUPAC name: (E)-[4-(1-Benzyoylpiperidin-4-yl)butly]-3-(pyridin-3-yl) acrylamide; CAS number: 201034-75-5) was obtained from the National Institute of Mental Health Chemical Synthesis and Drug Supply Program (Bethesda, MD). FK866 was dissolved in DMSO to prepare a stock solution at a concentration of 1mM and stored at −30°C. Methyl methanesulfonate (MMS) and methoxyamine hydrochloride (MX) were from Sigma (St. Louis, MO). Temozolomide (NSC# 362856; IUPAC name: 3-methyl-2-oxo-1, 3, 4, 5, 8-pentazabicyclo[4.3.0]nona-4,6,8-triene-7-carboxamide; CAS number: 856622-93-1) was obtained from the National Cancer Institute Developmental Therapeutics Program (Bethesda, MD) and Sigma. Temozolomide (TMZ) was dissolved in DMSO to a concentration of 100mM.

Cell culture

All cells were cultured at 5% CO2 and 37°C. LN428 glioblastoma cells (a generous gift from Dr. Ian Pollack, University of Pittsburgh) and derived cell lines were cultured in alpha EMEM supplemented with 10% heat inactivated FBS, L-glutamine, antibiotic/antimytotic and gentamycin. LN428 is an established glioblastoma-derived cell line with mutations in p53, deletions in p14ARF and p16 and is WT for PTEN (34, 35). The LN428/MPG cells (over-expressing methylpurine DNA glycosylase; MPG) were supplemented with 6 mg/mL G418. LN428/MPG cell lines with stable knockdown of PARP1, PARP2, MLH1, MSH2 and MSH6 were generated via lentiviral transduction of shRNA as previously described (20) and cultured in growth media with 6mg/mL G418 and 1.0 μg/mL puromycin. Knockdown of the target gene was assed by reverse transcription PCR (RT-PCR) (see Supplemental Methods). The T98G glioblasoma cell line was purchased from American Type Culture Collection (ATCC) (obtained in 2004) and cultured in MEM supplemented with 10% heat inactivated FBS, non-essential amino acids, sodium pyruvate and antibiotic/antimytotic (36). Both cell lines were tested for cross species contamination and authenticated by RADIL cell check services as of 11/16/2010. The genetic profiles provided by RADIL were checked against the ATCC STR loci database to confirm the identity of the T98G cell line, and to ensure that the LN428 genetic profile was unique from any ATCC banked cell line.

Short-term cytotoxicity assay

Glioma cells were seeded 24 hours prior to treatment at 2000 cells/well in 96-well plates. For FK866 pre-treatment experiments, cells were treated with the indicated dose of FK866 or media control for 24 hours. Media was then removed and cells were treated with the indicated dose of MMS for 1 hour. Cells were then washed with media and allowed to recover for 48 hours, at which point cell survival was assayed utilizing the MTS assay (Promega), as previously described (16).

NAD+ measurements

Cells were seeded and cultured for 24 hours and then treated for 24 hours with the indicated dose of FK866. NAD+ was measured as reported previously using the Enzychrom NAD+/NADH assay kit (BioAssay Systems) (20). Absorbance was determined using a Molecular Devices VersaMax™ tuneable plate reader.

Quantitative PAR assay

Cells were seeded at 1.5 × 106 cells per 100 mm dish 24 hours prior to treatment. Cells were then treated with media (control) or MMS for the indicated time. Cells were immediately lysed in modified Laemmli buffer containing 1% SDS in the absence of bromophenol blue. Protein content was quantified using the DC protein assay (Bio-Rad). PAR content was quantified using a colorimetric PAR ELISA kit (Trevigen). Absorbance was determined using a Molecular Devices VersaMax™ tuneable plate reader.

ATP measurements

Cells were seeded in black 96-well plates (Perkin Elmer) 24 hours prior to treatment. Cells were treated with drugs as described above. One-hour post alkylation treatment, cells were washed and allowed to recover in normal media (1 hour). Cells were lysed and ATP content measured using the ATP-lite assay kit (Perkin Elmer). Luminescent readings were performed using a BioTek Synergy™ 2 Multi-Mode microplate reader.

Long-term cytotoxicity assay in modified LN428 cell lines

For long-term survival assays with FK866 pre-treatment, cells were seeded at 80,000 cells/well in 6-well plates 24 hours prior to treatment. Cells were treated with FK866 or media control for 24 hours. Cells were then trypsinized and plated into 60 mm dishes with or without FK866 and were allowed to attach for 6 hours prior to replacing media with TMZ containing media. All doses were done in triplicate 60 mm dishes. Cells were allowed to grow for 10–12 days in the presence of TMZ. Cells were then trypsinized and counted using a CASY automated cell counter. All experiments were performed in triplicate. Long-term survival experiments combining FK866 with MX pre-treatment were performed as above, but after the 6-hour attachment, cells were treated for 30 minutes with 10 mM MX that was diluted to 5 mM upon addition of TMZ. Cells were allowed to grow for 10–12 days in the presence of MX and TMZ prior to counting.

Long-term cytotoxicity assay in T98G cells

T98G cells (36) were seeded in 60mm dishes and allowed to adhere for 24 hours. Cells were then treated with FK866 or media control for 24 hours, followed by MX pre-treatment or media control as described above. Cells were treated with TMZ either alone or as a co-treatment with MX for 6 hours, at which time the drug treatments were removed and replaced with normal culture media. Cells were allowed to grow for an additional 7 days and were assayed for cell survival as described above. All doses were performed in triplicate.

Statistical Analysis

All data is shown as a mean +/− standard error from 3 independent experiments. Student’s t-test was used for comparisons between two groups. For multiple comparisons, one-way ANOVA followed by post-hoc test with Bonferroni correction was used. Statistical analysis was performed using GraphPad PRISM.

Additional methods

See Supplemental Data.

Results

MPG and PARP1 expression modulate BER dependent PAR generation and cell death after alkylation exposure

We have hypothesized that varying the expression of the initiating BER enzyme MPG will initiate an imbalance in the BER pathway, resulting in the accumulation of BER intermediates (e.g., 5′dRP) after alkylation base damage (19), leading to an increase in TMZ-induced PARP activation and cell death (20). To test this hypothesis, we have stably over-expressed MPG in the glioma cell line LN428 (noted as LN428/MPG). The parental LN428 cell line contains undetectable levels of MPG (Supplemental Fig. S1). This allows for a model system in which events related to incomplete BER can be dissected from other alkylation damage events such as effects on translation (37). In addition, by utilizing relatively low doses of alkylating agent that produce an effect only in the LN428/MPG cell line we can eliminate effects from other alkylation events such as the formation of DNA double-strand breaks or protein alkylation (38). For example, LN428/MPG cells are hypersensitive to the alkylating agent MMS at doses as low as 0.5 mM, whereas there is little or no cytotoxicity observed for the parental LN428 cell line at doses as high as 1.5 mM (Fig. 1A). Further, utilizing a quantitative ELISA method, we find that PAR generation is 10-fold higher in LN428/MPG cells compared to LN428 cells at an MMS dose of 1.5 mM (Fig. 1B), likely resulting from accumulation of BER intermediates (20).

Figure 1. Expression of MPG and PARP1 modulate BER dependent PAR generation and cell death after alkylation damage.

Figure 1

A, Cell viability of LN428 or LN428/MPG cells after 1 h MMS treatment, as measured by an MTS assay 48 hours after exposure. Plots show the % viable cells as compared to untreated (control) cells. Means are calculated from quadruplicate values in each experiment. Results indicate the mean ± S.E. of three independent experiments.

B, PAR generation in LN428 or LN428/MPG cells before or after 15 minute MMS exposure (15 and 30 min.; 1.5 mM), as measured by quantitative ELISA (see Materials & Methods).

C, Cell viability of LN428/MPG cells or LN428/MPG cells with stable knockdown of either PARP1 or PARP2 after 1 h MMS treatment, as measured by an MTS assay 48 hours after exposure. Plots show the % viable cells as compared to untreated (control) cells. Means are calculated from quadruplicate values in each experiment. Results indicate the mean ± S.E. of three independent experiments.

D, Cells were depleted of PARP1 or PARP2 via shRNA. PAR generation in LN428/MPG cells LN428/MPG/PARP1KD and LN428/MPG/PARP2KD is shown before or after MMS exposure (15 and 30 min.; 1.5 mM), as measured by quantitative ELISA (see Materials & Methods). All values are normalized to LN428/MPG signal at 15 min. exposure (comparable to Fig. 1B) and reported as fold change in PAR level.

There are multiple PARP family members (24), of which PARP1 (ARTD1) and PARP2 (ARTD2) have also been associated with the BER pathway (26, 39). To define the role of either PARP1 or PARP2 as the enzyme responsible for the increased generation of PAR after BER intermediate accumulation, stable PARP1 and PARP2 knockdown (KD) cell lines were generated via lentiviral shRNA expression in the LN428/MPG cells (Supplemental Fig. S2A). PARP1-KD provides almost complete rescue of the MMS-induced cytotoxicity and ATP depletion observed in the LN428/MPG/PARP1-KD cells and reduces PAR levels as compared to the LN428/MPG cell line after 15 minutes of alkylation exposure (Fig. 1C&D; Supplemental Fig S2B&C). Interestingly, PARP1-KD did not completely eliminate PAR generation, most likely from PARP2-mediated synthesis. Further, PARP1-KD had a significant effect on the kinetics of PAR catabolism, with the PAR signal remaining at the same level after 30 minutes exposure instead of the rapid PARG-mediated degradation observed in PARP1 proficient cells (Fig. 1D and Supplemental Fig. S2C). This leaves open the question of whether PARP2 generated PAR has different properties and consequences compared to PAR chains generated by PARP1 and whether PARP1 expression impacts PARG activity. Taken together, we suggest that the energy depletion-mediated cell death observed after incomplete BER (BER inhibition) is largely PARP1 mediated with only a small contribution by PARP2. Given our recent understanding that tumor cell death after BER inhibition is an energy depletion-mediated process (20), we sought to take advantage of this observation by targeting the intersecting pathways of BER and NAD+ biosynthesis to enhance cell death in chemotherapy resistant tumor cells.

Inhibition of NAD+ biosynthesis enhances alkylation induced cell death after incomplete BER

Inhibition of NAD+ biosynthesis (24 hours) with FK866 reduces cellular NAD+ content to 25% of control levels in both parental and LN428/MPG tumor cells (Fig. 2A). Consistent with previous reports in other cell types, this 24 hour window of NAD+ biosynthesis inhibition is not toxic when evaluated using a short term (48 hour) cell survival assay (Fig. 2B), but does exhibit significant toxicity in both the LN428 and LN428/MPG cells after 72 hours of continuous FK866 treatment independent of BER status (Supplemental Fig. S3)(31, 33). The lack of toxicity after 24 hours of FK866 treatment is attributed to recovery of NAD+ levels within 24 hours when cultured in normal media prior to assaying for cell survival (Fig. 2A). Importantly, combining the non-toxic 24 hour window of NAD+ depletion immediately followed by a minimally toxic dose of MMS (0.5 mM) dramatically sensitized glioma cells in a BER dependent manner (Fig. 2C). This effect was dependent on the dose of FK866 (Fig. 2C).

Figure 2. FK866 transiently decreases the level of NAD+ and enhances alkylation-induced cell death via a BER dependent mechanism.

Figure 2

A, Cellular NAD+ levels in LN428 and LN428/MPG cells before exposure, after exposure to FK866 (10nM; 24 hrs) and following an additional 24 hours of recovery from FK866 treatment. The bars are defined in the legend

B, Cytotoxicity profile of FK866. Viability of LN428 and LN428/MPG cells was measured by an MTS assay 48 hours after FK866 (24 hrs) treatment. Viable cells were determined as in Fig. 1A and reported as percentage relative to vehicle control treated cells (% Control).

C, Cell viability after FK866-mediated NAD+ depletion immediately followed by alkylation damage. LN428 and LN428/MPG cells were treated with FK866 (24 hours) at varying doses followed by 1 hour MMS treatment at 0.5 mM. Cells were allowed to recover in normal media for 48 hours prior to analysis for cell viability by the MTS assay. Viable cells were determined as in Fig. 1A.

Inhibition of NAD+ biosynthesis potentiates cell death by alkylating agents after BER inhibition

To test our hypothesis that NAD+ depletion would enhance BER inhibition-mediated cell death, we utilized MX, a small molecule BER inhibitor that can potentiate alkylation damage-induced cell death (2123) by covalently binding to the aldehydic form of the DNA abasic site formed after BER initiation (21). A 30 mM dose of MX (30 minute pre-treatment) and co-treatment with MMS potentiates cell death 10-fold, consistent with previous reports in colon and ovarian carcinomas (Fig. 3B; solid circle) (21, 23). This potentiation is dependent on MPG expression, as the initiating DNA glycosylase is required for abasic site formation after alkylation damage, and so we observe no MX-mediated potentiation in the LN428 cells but robust MX-mediated potentiation in the LN428/MPG cells (compare Fig. 3A and Fig. 3B). MX inhibition of BER, in combination with FK866-mediated inhibition of NAD+ biosynthesis, results in an IC50 for MMS of 150μM, a sensitizer enhancement ratio (SER) of 3.125. This also yields a 2-fold decrease in IC50 from either FK866 + MMS co-treatment or MX + MMS co-treatment (Fig. 3B).

Figure 3. Chemical inhibition of BER and NAD+ biosynthesis potentiates alkylation-induced cell death.

Figure 3

A, Cell viability of LN428 cells measured by an MTS assay after treatment with MMS alone, MMS in combination with 30 mM MX, MMS in combination with 10nM FK866, or with the 3-drug combination of MX, FK866 and MMS. Drug treatments were carried out as described in the Materials and Methods section. Viable cells were determined as in Fig. 1A and reported as percentage relative to vehicle control treated cells (% Control).

B, Cell viability measured by an MTS assay in the LN428/MPG cells after treatment with MMS alone, MMS in combination with 30 mM MX, MMS in combination with 10nM FK866, or with the 3-drug combination of MX, FK866 and MMS. Drug treatments were carried out as described in the Materials and Methods section. Viable cells were determined as in Fig. 1A and reported as percentage relative to vehicle control treated cells (% Control). SER was calculated as IC50 TMZ/ IC50 TMZ + sensitizer.

Both FK866 and MX enhance ATP depletion after alkylation damage

To further investigate the enhanced cell death observed when MX and FK866 are combined with alkylating agents, cellular ATP levels were measured after combination treatment. As expected (20), alkylation damage resulted in a decrease in the level of cellular ATP when measured two hours after MMS treatment of the LN428/MPG cells (Fig. 4A and 4B; solid bars). Further, FK866 pre-treatment prior to alkylation (MMS) damage resulted in lower levels of ATP two hours after MMS treatment compared to alkylation damage alone (Fig. 4A; dotted bars). However, 24 hours of NAD+ depletion alone had no significant effect on ATP levels (Fig. 4A; striped bars). Surprisingly, treatment with MX + MMS also induced loss of ATP beyond that observed by alkylation (MMS) damage alone (Fig. 4B; dotted bars). It is important to note that this enhanced ATP loss is only observed in the LN428/MPG cell line, supporting the notion that ATP loss after alkylation damage is dependent on the expression of MPG and PARP hyperactivation (an event not seen in the LN428 cell line) (Fig 1B). This suggests that both inhibitors (FK866 and MX) enhance cell death when combined with alkylation by enhancing ATP loss in a BER and PARP dependent manner.

Figure 4. Enhanced ATP depletion following MMS, MX and FK866 combined treatment.

Figure 4

A, Cellular ATP content two hours after 0.5mM MMS treatment (or media control) as measured by the ATP-lite luminescent kit. Results are reported as percent of untreated control. Cells were pre-treated for 24 hours with either 10nM FK866 or media control.

B, Cellular ATP content two hours after 0.5mM MMS treatment (or media control) as measured by the ATP-lite luminescent kit. Results are reported as a percentage of untreated control. Cells were pre-treated for 30 minutes with MX or media control followed by co-treatment with MMS or media and post treatment as described in the Materials and Methods section.

*Represents statistical significance with a p < 0.05.

Dual inhibition of BER and NAD+ biosynthesis sensitizes chemotherapy resistant glioma cells to TMZ

Given the strong potentiation of cell death when combining TMZ with inhibitors of BER and NAD+ biosynthesis in a 48-hour cytotoxicity assay, we investigated whether this three-drug combination could sensitize glioma cells harbouring established TMZ-resistant genotypes. TMZ resistance can result from either over-expression of MGMT or loss of MMR (1113). LN428 and LN428/MPG cells are both MMR positive and express low levels of MGMT. In a long-term survival assay (allowing multiple rounds of replication required for O6-methylguanine-mediated cell death), LN428 cells are highly sensitive to TMZ (IC50≈15μM)(Supplemental Fig. S4A). To convert these cells to a TMZ-resistant phenotype, MGMT was stably over-expressed in the LN428/MPG cell line, resulting in resistance of the LN428/MPG/MGMT cell line to TMZ in a long-term survival assay (Fig. 5A, and Supplemental Fig. 4A). MX, in combination with TMZ, sensitizes MPG positive cells with an IC50 of 160 μM TMZ (Supplemental Table S1). Similarly, FK866 pre-treatment plus TMZ sensitizes LN428/MPG/MGMT cells with an IC50 of 175 μM TMZ (Supplemental Table S1). However, combining FK866 pre-treatment with MX and TMZ results in an IC50 of 17 μM, about a 10-fold decrease in IC50 and an SER value of 15.88 (Fig. 5A, Supplemental Fig. 5A and Supplemental Table S1). Knockdown of the MMR proteins MLH1, MSH2 and MSH6 (Supplemental Fig. S4B) also renders LN428/MPG cells resistant to TMZ (Fig. 5B–D and Supplemental Fig. S5B–D). However, the dual inhibition of BER and NAD+ biosynthesis can overcome the TMZ-resistance that results from MMR deficiency, yielding an increase in the cytotoxic response to 75 μM TMZ (Fig. 5B–D and Supplemental Fig. 5B–D).

Figure 5. Combining NAD+ biosynthesis inhibition with BER inhibition sensitizes chemotherapy resistant glioma cells to TMZ.

Figure 5

A, Cell survival determined by a long-term survival assay in cells modified to over-express MGMT. LN428/MPG/MGMT cells were treated with either TMZ alone for 12 days, 24 hours of 10nM FK866 followed by 12 days of TMZ, a 30 minute pre-treatment of 10mM MX followed by MX (5mM) and TMZ co-treatment for 12 days, or the three-drug combination (a 24 hour pre-treatment of FK866, followed by a 30 minute pre-treatment with MX and a 12 day co-treatment with MX and TMZ. TMZ was used at a 75 μm dose.

B–D, Cell survival determined by a long-term survival assay in cells stably expressing shRNA to MLH1 (B), MSH2 (C) or MSH6 (D). LN428/MPG MMR knockdown cells were treated with either TMZ alone for 12 days, 24 hours of 10nM FK866 followed by 12 days of TMZ, a 30 minute pre-treatment of 10mM MX followed by MX (5mM) and TMZ co-treatment for 12 days, or the three-drug combination (a 24 hour pre-treatment of FK866, followed by a 30 minute pre-treatment with MX and a 12 day co-treatment with MX and TMZ. TMZ was used at a 75 μm dose.

*Represents statistical significance with a p < 0.05.

To further investigate the utility of this drug combination in treating GBMs with endogenous or acquired resistance to TMZ, we utilized T98G cells, a glioma cell line with endogenous over-expression of MGMT (36, 40). T98G cells have robust expression of BER proteins as well as NAMPT (data not shown) and have an approximate 2-fold increase in the basal level of NAD+ as compared to the LN428 and LN428/MPG cells (Supplemental Fig. S6A), requiring cell line specific titering of FK866 dosing. Twenty-four hour exposure to FK866 with doses up to 80nM had minimal effect on cell viability in a long-term cell survival assay (Supplemental Fig. S6B). FK866 enhances TMZ toxicity in a dose dependent manner (Fig. 6A), whereas MX, in combination with TMZ, yields an enhancement of TMZ toxicity (Fig. 6B), Most importantly, the combination of FK866 and MX, with TMZ, significantly enhanced cell death in an FK866 dose dependent manner beyond that of either combination alone (Fig. 6B). All together, these results support our proposed strategy of dual inhibition of the BER pathway and NAD+ biosynthesis. We suggest that such a strategy may be useful to overcome TMZ resistance in primary or recurrent TMZ-resistant GBM and we offer proof of principle data to validate future analysis in an in vivo preclinical model.

Figure 6. Combining NAD+ biosynthesis inhibition with BER inhibition sensitizes chemotherapy resistant T98G cells to TMZ.

Figure 6

A, Cell survival determined by a long-term survival assay in the T98G cells after a 24 hour pre-treatment with varying doses of FK866 followed by a 6 hour exposure to 500 μM TMZ.

B, Cell survival determined by a long-term survival assay in T98G cells after treatment with either TMZ alone for 6 hours, a 30 minute pre-treatment with 10mM MX followed by 5mM MX and TMZ co-treatment for 6 hours, or the three-drug combination with varying doses of FK866 as a 24 hour pre-treatment followed by a 30 minute pre-treatment with 10mM MX and a 6 hour co-treatment with 5mM MX and TMZ. A dose of 500μM TMZ was used in each treatment condition.

*Represents statistical significance with a p < 0.05 as compared to TMZ + MX.

Discussion

This study presents a novel strategy to elicit tumor cell toxicity from the N3-methyladenine and N7-methylguanine lesions induced by TMZ by targeting two interacting pathways: the BER and NAD+ biosynthesis pathways. The introduction of TMZ as a standard treatment for glioblastoma has resulted in a significant increase in response rate and median survival time over surgery and radiation therapy alone (4). However, 5-year survival rates still remain dismally low (3, 4), likely from inherent TMZ resistance and/or relapse with chemotherapy resistant tumors (14). Most of this resistance to TMZ is attributed to the loss of toxicity by the O6-methylguanine lesion via elevated expression of MGMT or a defect in the MMR pathway (12). However, more than 80% of the DNA lesions produced by TMZ are BER substrates, suggesting BER modulation is a promising therapeutic option (41). The explosion of clinical trials involving TMZ combined with PARP inhibitors and ongoing clinical trials of TMZ combined with MX highlights this point (42, 43).

We have previously observed that inhibition of BER (via genetic modulation) combined with alkylation treatment leads to necrotic cell death resulting from PARP hyperactivation with concomitant NAD+ and ATP depletion (20). Given this model, we sought to enhance TMZ response by inhibiting both BER and NAD+ biosynthesis. This novel, combinatorial, 3-drug strategy realized a reduction of TMZ IC50 greater than ten fold in multiple models of TMZ resistant tumor cells as well as an endogenously resistant GBM cell line (T98G). The induction of necrotic cell death as opposed to apoptotic cell death is a promising feature of this novel treatment combination, since a percentage of GBMs are also known to be resistant to apoptotic cell death (44), suggesting that this drug combination may be preferable over alternate chemotherapy options that induce apoptosis. While necrosis is primarily thought of as having a negative inflammatory impact on surrounding tissue, release of HMGB1 and other RAGE ligands after necrosis may aid in stimulation of the immune system and clearance of the tumor, thus potentially increasing the efficacy of this drug combination (45, 46).

MX, in combination with TMZ, is currently in Phase I clinical trials for the treatment of a variety of solid tumors and appears to be well tolerated (43). FK866 (also known as APO866) is also currently in clinical trials as a monotherapy and is well tolerated (32). The concern in combining BER inhibition and NAD+ biosynthesis inhibition is that the combination may critically lower energy levels and induce necrosis in healthy tissue. The high energy requirements and rapid NAD+ turnover of tumor cells may allow for selectivity for the FK866/MX/TMZ drug combination over normal tissue (47). Further, it has been suggested that NAD+ loss by PARP hyperactivation is cytotoxic by creating a glycolytic block (48, 49). Tumor selectivity may be realized from the reliance of glioblastomas on glycolysis for energy, whereas normal brain tissue exhibits higher levels of oxidative phosphorylation (50). Interestingly, in one report, up to 50% of GBMs tested were deficient in the NAPRT1-mediated NAD+ biosynthesis pathway responsible for generating NAD+ from nicotinic acid (NA) and in a separate report it was shown that NA supplementation can rescue the toxic effects of high dose FK866 on normal tissue (47, 51). This also suggests an opportunity for a synthetic lethal approach using our novel 3-drug combination, in which healthy tissue is supplemented with NA while selectively killing tumor cells unable to utilize the NAPRT1 arm of the pathway.

Targeted NAD+ biosynthesis inhibition combined with BER inhibition plus TMZ gives a promising decrease in IC50 in resistant glioblastoma tumor cells. TMZ is also currently in clinical trials for use with malignant melanoma, another chemotherapy resistant tumor with few treatment options (52). This drug combination may also prove to be useful in increasing efficacy of TMZ in melanoma and potentially in other tumor types that have shown little response to TMZ. For example, we have reported that genetic inhibition of BER in breast cancer cells shows similar sensitization, suggesting that this drug combination would also be useful for the treatment of triple-negative tumors (19). In conclusion, this study illustrates a novel therapeutic option for tumors resistant to the TMZ-induced O6-methylguanine lesion, by utilizing clinically available chemical inhibitors of the interacting pathways of BER and NAD+ biosynthesis.

Supplementary Material

1

Acknowledgments

Grant Support<br>This work was supported by grants from the National Brain Tumor Society, the Pittsburgh Foundation and the National Institutes of Health (NIH) [CA132385; GM087798; CA148629] to RWS. Support for the UPCI Lentiviral Facility was provided to RWS by the Cancer Center Support Grant from the National Institutes of Health [P30 CA047904]. Support was also provided by the University of Pittsburgh Department of Pharmacology & Chemical Biology and a John S. Lazo Cancer Pharmacology Fellowship to EMG. LM was a recipient of a Hampton University / University of Pittsburgh Cancer Institute joint cancer education program fellowship, provided by a grant from NIH to RWS (CA132385).

Footnotes

Note: Supplementary data for this article is available at Cancer Research Online.

1

Tang et al., manuscript in revision.

Disclosure of Potential Conflicts of Interest

No potential conflicts of interest were disclosed.

References

  • 1.Sathornsumetee S, Rich JN. New treatment strategies for malignant gliomas. Expert Rev Anticancer Ther. 2006;6:1087–104. doi: 10.1586/14737140.6.7.1087. [DOI] [PubMed] [Google Scholar]
  • 2.Friedman HS, Kerby T, Calvert H. Temozolomide and treatment of malignant glioma. Clinical Cancer Research. 2000;6:2585–97. [PubMed] [Google Scholar]
  • 3.Stupp R, Mason WP, van den Bent MJ, et al. Radiotherapy plus concomitant and adjuvant temozolomide for glioblastoma. New England Journal of Medicine. 2005;352:987–96. doi: 10.1056/NEJMoa043330. [DOI] [PubMed] [Google Scholar]
  • 4.Stupp R, Hegi ME, Mason WP, et al. Effects of radiotherapy with concomitant and adjuvant temozolomide versus radiotherapy alone on survival in glioblastoma in a randomised phase III study: 5-year analysis of the EORTC-NCIC trial. Lancet Oncol. 2009;10:459–66. doi: 10.1016/S1470-2045(09)70025-7. [DOI] [PubMed] [Google Scholar]
  • 5.Sobol RW. Temozolomide. In: Schwab M, editor. Encyclopedia of Cancer. 2. Berlin, Heidelberg, New York: Springer; 2009. [Google Scholar]
  • 6.Pegg AE, Dolan ME, Moschel RC. Structure, function, and inhibition of O6-alkylguanine-DNA alkyltransferase. Prog Nucleic Acid Res Mol Biol. 1995;51:167–223. doi: 10.1016/s0079-6603(08)60879-x. [DOI] [PubMed] [Google Scholar]
  • 7.Wang JY, Edelmann W. Mismatch repair proteins as sensors of alkylation DNA damage. Cancer Cell. 2006;9:417–8. doi: 10.1016/j.ccr.2006.05.013. [DOI] [PubMed] [Google Scholar]
  • 8.Friedman HS, Johnson SP, Dong Q, et al. Methylator resistance mediated by mismatch repair deficiency in a glioblastoma multiforme xenograft. Cancer Res. 1997;57:2933–6. [PubMed] [Google Scholar]
  • 9.Roos WP, Batista LF, Naumann SC, et al. Apoptosis in malignant glioma cells triggered by the temozolomide-induced DNA lesion O6-methylguanine. Oncogene. 2007;26:186–97. doi: 10.1038/sj.onc.1209785. [DOI] [PubMed] [Google Scholar]
  • 10.Caporali S, Falcinelli S, Starace G, et al. DNA damage induced by temozolomide signals to both ATM and ATR: role of the mismatch repair system. Molecular Pharmacology. 2004;66:478–91. doi: 10.1124/mol.66.3.. [DOI] [PubMed] [Google Scholar]
  • 11.Hegi ME, Liu L, Herman JG, et al. Correlation of O6-methylguanine methyltransferase (MGMT) promoter methylation with clinical outcomes in glioblastoma and clinical strategies to modulate MGMT activity. J Clin Oncol. 2008;26:4189–99. doi: 10.1200/JCO.2007.11.5964. [DOI] [PubMed] [Google Scholar]
  • 12.Sarkaria JN, Kitange GJ, James CD, et al. Mechanisms of chemoresistance to alkylating agents in malignant glioma. Clin Cancer Res. 2008;14:2900–8. doi: 10.1158/1078-0432.CCR-07-1719. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Pollack IF, Hamilton RL, Sobol RW, et al. MGMT Expression Strongly Correlates with Outcome in Childhood Malignant Gliomas: Results from the CCG-945 Cohort. J Clin Oncol. 2006;24:3431–7. doi: 10.1200/JCO.2006.05.7265. [DOI] [PubMed] [Google Scholar]
  • 14.Cahill DP, Levine KK, Betensky RA, et al. Loss of the mismatch repair protein MSH6 in human glioblastomas is associated with tumor progression during temozolomide treatment. Clin Cancer Res. 2007;13:2038–45. doi: 10.1158/1078-0432.CCR-06-2149. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Yip S, Miao J, Cahill DP, et al. MSH6 mutations arise in glioblastomas during temozolomide therapy and mediate temozolomide resistance. Clin Cancer Res. 2009;15:4622–9. doi: 10.1158/1078-0432.CCR-08-3012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Trivedi RN, Almeida KH, Fornsaglio JL, Schamus S, Sobol RW. The Role of Base Excision Repair in the Sensitivity and Resistance to Temozolomide Mediated Cell Death. Cancer Res. 2005;65:6394–400. doi: 10.1158/0008-5472.CAN-05-0715. [DOI] [PubMed] [Google Scholar]
  • 17.Liu L, Gerson SL. Targeted modulation of MGMT: clinical implications. Clin Cancer Res. 2006;12:328–31. doi: 10.1158/1078-0432.CCR-05-2543. [DOI] [PubMed] [Google Scholar]
  • 18.Almeida KH, Sobol RW. A unified view of base excision repair: lesion-dependent protein complexes regulated by post-translational modification. DNA Repair. 2007;6:695–711. doi: 10.1016/j.dnarep.2007.01.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Trivedi RN, Wang XH, Jelezcova E, Goellner EM, Tang J, Sobol RW. Human methyl purine DNA glycosylase and DNA polymerase β expression collectively predict sensitivity to temozolomide. Molecular Pharmacology. 2008;74:505–16. doi: 10.1124/mol.108.045112. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Tang J, Goellner EM, Wang XW, et al. Bioenergetic Metabolites Regulate Base Excision Repair-Dependent Cell Death in Response to DNA Damage. Molecular Cancer Research. 2010;8:67–79. doi: 10.1158/1541-7786.MCR-09-0411. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Liu L, Taverna P, Whitacre CM, Chatterjee S, Gerson SL. Pharmacologic disruption of base excision repair sensitizes mismatch repair-deficient and -proficient colon cancer cells to methylating agents. Clinical Cancer Research. 1999;5:2908–17. [PubMed] [Google Scholar]
  • 22.Taverna P, Liu L, Hwang HS, Hanson AJ, Kinsella TJ, Gerson SL. Methoxyamine potentiates DNA single strand breaks and double strand breaks induced by temozolomide in colon cancer cells. Mutat Res. 2001;485:269–81. doi: 10.1016/s0921-8777(01)00076-3. [DOI] [PubMed] [Google Scholar]
  • 23.Fishel ML, He Y, Smith ML, Kelley MR. Manipulation of base excision repair to sensitize ovarian cancer cells to alkylating agent temozolomide. Clin Cancer Res. 2007;13:260–7. doi: 10.1158/1078-0432.CCR-06-1920. [DOI] [PubMed] [Google Scholar]
  • 24.Hottiger MO, Hassa PO, Luscher B, Schuler H, Koch-Nolte F. Toward a unified nomenclature for mammalian ADP-ribosyltransferases. Trends Biochem Sci. 2010;35:208–19. doi: 10.1016/j.tibs.2009.12.003. [DOI] [PubMed] [Google Scholar]
  • 25.Berger NA. Poly(ADP-ribose) in the cellular response to DNA damage. Radiat Res. 1985;101:4–15. [PubMed] [Google Scholar]
  • 26.Hassa PO, Haenni SS, Elser M, Hottiger MO. Nuclear ADP-ribosylation reactions in mammalian cells: where are we today and where are we going? Microbiol Mol Biol Rev. 2006;70:789–829. doi: 10.1128/MMBR.00040-05. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Yu SW, Wang H, Poitras MF, et al. Mediation of poly(ADP-ribose) polymerase-1-dependent cell death by apoptosis-inducing factor. Science. 2002;297:259–63. doi: 10.1126/science.1072221. [DOI] [PubMed] [Google Scholar]
  • 28.Andrabi SA, Kim NS, Yu SW, et al. Poly(ADP-ribose) (PAR) polymer is a death signal. Proc Natl Acad Sci U S A. 2006;103:18308–13. doi: 10.1073/pnas.0606526103. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Magni G, Orsomando G, Raffelli N, Ruggieri S. Enzymology of mammalian NAD metabolism in health and disease. Front Biosci. 2008;13:6135–54. doi: 10.2741/3143. [DOI] [PubMed] [Google Scholar]
  • 30.Revollo JR, Grimm AA, Imai S. The NAD biosynthesis pathway mediated by nicotinamide phosphoribosyltransferase regulates Sir2 activity in mammalian cells. J Biol Chem. 2004;279:50754–63. doi: 10.1074/jbc.M408388200. [DOI] [PubMed] [Google Scholar]
  • 31.Hasmann M, Schemainda I. FK866, a highly specific noncompetitive inhibitor of nicotinamide phosphoribosyltransferase, represents a novel mechanism for induction of tumor cell apoptosis. Cancer Res. 2003;63:7436–42. [PubMed] [Google Scholar]
  • 32.Cea M, Zoppoli G, Bruzzone S, et al. APO866 activity in hematologic malignancies: a preclinical in vitro study. Blood. 2009;113:6035–7. doi: 10.1182/blood-2009-03-209213. author reply 7–8. [DOI] [PubMed] [Google Scholar]
  • 33.Pogrebniak A, Schemainda I, Azzam K, Pelka-Fleischer R, Nussler V, Hasmann M. Chemopotentiating effects of a novel NAD biosynthesis inhibitor, FK866, in combination with antineoplastic agents. European journal of medical research. 2006;11:313–21. [PubMed] [Google Scholar]
  • 34.Park MJ, Kim MS, Park IC, et al. PTEN suppresses hyaluronic acid-induced matrix metalloproteinase-9 expression in U87MG glioblastoma cells through focal adhesion kinase dephosphorylation. Cancer Res. 2002;62:6318–22. [PubMed] [Google Scholar]
  • 35.Ishii N, Maier D, Merlo A, et al. Frequent co-alterations of TP53, p16/CDKN2A, p14ARF, PTEN tumor suppressor genes in human glioma cell lines. Brain Pathol. 1999;9:469–79. doi: 10.1111/j.1750-3639.1999.tb00536.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Stein GH. T98G: an anchorage-independent human tumor cell line that exhibits stationary phase G1 arrest in vitro. Journal of cellular physiology. 1979;99:43–54. doi: 10.1002/jcp.1040990107. [DOI] [PubMed] [Google Scholar]
  • 37.Begley U, Dyavaiah M, Patil A, et al. Trm9-catalyzed tRNA modifications link translation to the DNA damage response. Mol Cell. 2007;28:860–70. doi: 10.1016/j.molcel.2007.09.021. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Wyatt MD, Pittman DL. Methylating agents and DNA repair responses: Methylated bases and sources of strand breaks. Chem Res Toxicol. 2006;19:1580–94. doi: 10.1021/tx060164e. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Ame JC, Rolli V, Schreiber V, et al. PARP-2, A novel mammalian DNA damage-dependent poly(ADP-ribose) polymerase. J Biol Chem. 1999;274:17860–8. doi: 10.1074/jbc.274.25.17860. [DOI] [PubMed] [Google Scholar]
  • 40.Ueda S, Mineta T, Nakahara Y, Okamoto H, Shiraishi T, Tabuchi K. Induction of the DNA repair gene O6-methylguanine-DNA methyltransferase by dexamethasone in glioblastomas. J Neurosurg. 2004;101:659–63. doi: 10.3171/jns.2004.101.4.0659. [DOI] [PubMed] [Google Scholar]
  • 41.Marchesi F, Turriziani M, Tortorelli G, Avvisati G, Torino F, De Vecchis L. Triazene compounds: mechanism of action and related DNA repair systems. Pharmacol Res. 2007;56:275–87. doi: 10.1016/j.phrs.2007.08.003. [DOI] [PubMed] [Google Scholar]
  • 42.Palma JP, Wang YC, Rodriguez LE, et al. ABT-888 Confers Broad In vivo Activity in Combination with Temozolomide in Diverse Tumors. Clin Cancer Res. 2009 doi: 10.1158/1078-0432.CCR-09-1245. [DOI] [PubMed] [Google Scholar]
  • 43.Sawides P, Xu Y, Liu L, et al. Pharmacokinetic profile of the base-excision repair inhibitor methoxyamine-HCl (TRC102; MX) given as an one-hour intravenous infusion with temozolomide (TMZ) in the first-in-human phase I clinical trial. Oncology ASoC; ASCO Annual Meeting; 2010.2010. [Google Scholar]; Journal of Clinical Oncology. 2010:e13662. [Google Scholar]
  • 44.Krakstad C, Chekenya M. Survival signalling and apoptosis resistance in glioblastomas: opportunities for targeted therapeutics. Mol Cancer. 2010;9:135. doi: 10.1186/1476-4598-9-135. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Rovere-Querini P, Capobianco A, Scaffidi P, et al. HMGB1 is an endogenous immune adjuvant released by necrotic cells. EMBO reports. 2004;5:825–30. doi: 10.1038/sj.embor.7400205. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Ulloa L, Messmer D. High-mobility group box 1 (HMGB1) protein: friend and foe. Cytokine & growth factor reviews. 2006;17:189–201. doi: 10.1016/j.cytogfr.2006.01.003. [DOI] [PubMed] [Google Scholar]
  • 47.Watson M, Roulston A, Belec L, et al. The Small Molecule GMX1778 is a Potent Inhibitor of NAD+ Biosynthesis: Strategy for Enhanced Therapy in NAPRT1-Deficient Tumors. Mol Cell Biol. 2009 doi: 10.1128/MCB.00112-09. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Zong WX, Ditsworth D, Bauer DE, Wang ZQ, Thompson CB. Alkylating DNA damage stimulates a regulated form of necrotic cell death. Genes & Development. 2004;18:1272–82. doi: 10.1101/gad.1199904. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Alano CC, Garnier P, Ying W, Higashi Y, Kauppinen TM, Swanson RA. NAD+ depletion is necessary and sufficient for poly(ADP-ribose) polymerase-1-mediated neuronal death. J Neurosci. 2010;30:2967–78. doi: 10.1523/JNEUROSCI.5552-09.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Michelakis ED, Sutendra G, Dromparis P, et al. Metabolic modulation of glioblastoma with dichloroacetate. Science translational medicine. 2010;2:31ra4. doi: 10.1126/scitranslmed.3000677. [DOI] [PubMed] [Google Scholar]
  • 51.Olesen UH, Thougaard AV, Jensen PB, Sehested M. A preclinical study on the rescue of normal tissue by nicotinic acid in high-dose treatment with APO866, a specific nicotinamide phosphoribosyltransferase inhibitor. Mol Cancer Ther. 2010;9:1609–17. doi: 10.1158/1535-7163.MCT-09-1130. [DOI] [PubMed] [Google Scholar]
  • 52.Daponte A, Ascierto PA, Gravina A, et al. Temozolomide and cisplatin in avdanced malignant melanoma. Anticancer Res. 2005;25:1441–7. [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1

RESOURCES