Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Jan 1.
Published in final edited form as: Clin Cancer Res. 2016 Jun 29;23(1):311–319. doi: 10.1158/1078-0432.CCR-16-0549

Epigenome-wide Tumor DNA Methylation Profiling Identifies Novel Prognostic Biomarkers of Metastatic-lethal Progression in Men with Clinically Localized Prostate Cancer

Shanshan Zhao 1,*, Milan S Geybels 2,3,*, Amy Leonardson 2, Rohina Rubicz 2, Suzanne Kolb 2, Qingxiang Yan 4, Brandy Klotzle 5, Marina Bibikova 5, Antonio Hurtado-Coll 6, Dean Troyer 7, Raymond Lance 8, Daniel W Lin 2,9, Jonathan L Wright 2,9, Elaine A Ostrander 10, Jian-Bing Fan 5,¶,, Ziding Feng 4,, Janet L Stanford 2,11,†,
PMCID: PMC5199634  NIHMSID: NIHMS800344  PMID: 27358489

Abstract

Purpose

Aside from Gleason sum few factors accurately identify the subset of prostate cancer (PCa) patients at high risk for metastatic progression. We hypothesized that epigenetic alterations could distinguish prostate tumors with life-threatening potential.

Experimental Design

Epigenome-wide DNA methylation profiling was performed in surgically resected primary tumor tissues from a population-based (n = 430) and a replication (n = 80) cohort of PCa patients followed prospectively for at least five years. Metastasis was confirmed by positive bone scan, MRI, CT or biopsy, and death certificates confirmed cause of death. AUC, partial AUC (pAUC, 95% specificity), and P-value criteria were used to select differentially methylated CpG sites that robustly stratify patients with metastatic-lethal from non-recurrent tumors, and which were complementary to Gleason sum.

Results

Forty-two biomarkers stratified patients with metastatic-lethal versus non-recurrent PCa in the discovery cohort, and eight of these CpGs replicated in the validation cohort based on a significant (P <0.05) AUC (range: 0.66-0.75) or pAUC (range: 0.007-0.009). The biomarkers that improved discrimination of patients with metastatic-lethal PCa include CpGs in five genes (ALKBH5, ATP11A, FHAD1, KLHL8, and PI15) and three intergenic regions. In the validation dataset the AUC for Gleason sum alone (0.82) significantly increased with the addition of four individual CpGs (range: 0.86-0.89; all P <0.05).

Conclusion

Eight differentially methylated CpGs that distinguish patients with metastatic-lethal from non-recurrent tumors were validated. These novel epigenetic biomarkers warrant further investigation as they may improve prognostic classification of patients with clinically localized PCa and provide new insights on tumor aggressiveness.

Introduction

Prostate cancer (PCa) is a biologically and clinically heterogeneous disease with 180,890 new cases and 26,120 cancer-specific deaths expected in the U.S. for 2016 and over 300,000 deaths worldwide each year (1, 2). Prostate cancer most often has an indolent course, but a subset of patients progress to metastasis and eventually die from PCa (3, 4). The single most important predictor of PCa prognosis is Gleason sum,however, Gleason grading is frequently inaccurate (5). By comparing the Gleason sum of the diagnostic biopsy to subsequent prostatectomy, upgrading and downgrading occurs in 14% to 51% and 9%, respectively (5-7). Furthermore, while a tumor with Gleason sum ≤ 6 is low-risk and Gleason sum ≥8 is high-risk, tumors that are Gleason sum = 7 (ie, grades 3+4 or 4+3) are heterogeneous and comprise the majority of tumors (8). Thus, better prognostic biomarkers that can improve upon Gleason sum for stratification of higher risk patients most likely to benefit from targeted therapies are needed (4, 9-11).

Several recent biomarker studies of altered gene expression have led to development of mRNA signatures of tumor aggressiveness (9, 11, 12). Epigenetic alterations in tumor DNA may also provide valuable prognostic information (13, 14), and because DNA is 100-fold more stable than RNA it may be a more reliable biological material to use for tissue-based biomarkers. The most widely studied epigenetic alteration is DNA methylation, which occurs at CpG sites across the genome and regulates gene expression (15, 16). To date, most studies of DNA methylation and PCa progression have been limited to small sets of candidate genes in relation to biochemical (ie, PSA) recurrence (14, 17-19). Although patients with biochemical recurrence are at higher risk of cancer-specific death, most will not die from their PCa. Studies of patients with biochemical recurrence after radical prostatectomy found that only 17% to 22% died of PCa after a median follow-up of 10 years (20, 21). Therefore rather than focusing on PSA recurrence alone, biomarker studies of more serious clinical endpoints indicating metastatic progression and lethal PCa are needed.

We investigated epigenome-wide DNA methylation profiles in primary prostate cancers. The study includes patients derived from a population-based (multi-institution) radical prostatectomy cohort with long-term follow-up for metastatic progression and cancer-specific survival. The goal of this study was to identify differentially methylated biomarkers that could distinguish patients with metastatic-lethal PCa from those with less aggressive tumors. The most robust methylation biomarkers identified were then tested in an independent validation cohort.

Materials and Methods

Study populations

Fred Hutchinson (FH) Cancer Research Center cohort

The FH cohort includes 430 European-American PCa patients who underwent radical prostatectomy as primary therapy for clinically localized adenocarcinoma of the prostate. These patients were previously enrolled in population-based (multi-institutional) studies (22, 23), and their clinical characteristics (eg, age at diagnosis, Gleason sum, stage, PSA level) are similar to the larger group of European-American patients interviewed for the prior studies and who were treated surgically. The first study included men ages 40-64 years who were diagnosed between January 1993 and December 1996, and in the second study, men were ages 35-74 years and were diagnosed between January 2002 and December 2005. Gleason grade (primary and secondary patterns) and sum, diagnostic PSA, and pathological tumor stage were collected and centrally coded by the Seattle-Puget Sound Surveillance, Epidemiology, and End Results Program cancer registry. Vital status and underlying cause of death were also obtained from the cancer registry, and cause of death was confirmed by centralized review of death certificates. Prostate cancer-specific deaths included those with underlying cause of death attributed to ICD-9 code 180.0 or ICD-10 code C61.9. Prostate cancer recurrence status was determined from prospectively collected information from follow-up surveys that were completed by patients in 2004-2005 and in 2010-2011, review of medical records, and/or physician follow-up as needed. Metastatic progression was confirmed by positive bone scan, MRI, CT or biopsy. Patients who developed metastases or died from PCa were combined in a metastatic-lethal phenotype category. Over the follow-up period, 317 patients had no evidence of recurrence and 113 had recurred, including 86 PSA recurrences and 27 metastatic-lethal events. For the present analysis, patients with the metastatic-lethal phenotype were compared to patients who had not recurred. The FH Institutional Review Board approved the study and all participants signed informed consent statements.

Eastern Virginia (EV) Medical School cohort

The validation dataset includes 80 patients diagnosed with localized stage PCa who underwent radical prostatectomy at EV Medical School. The study population includes men who experienced disease progression to metastatic or lethal PCa (n = 31) and a similar number of patients (n = 49) selected on the basis of having no evidence of recurrence during five or more years after diagnosis (nested case-control design). Metastatic-lethal events were identified as described in the FH cohort, and all patients were European-Americans. The patients in the EV cohort were diagnosed in 1992-2009.

Tumor tissue sample preparation and DNA methylation profiling

Formalin-fixed paraffin-embedded prostate tumor tissue blocks were obtained from radical prostatectomy specimens for both cohorts and used to make hematoxalin and eosin stained slides, which were reviewed by pathologists to confirm the presence and location of adenocarcinoma. For each patient two 1-mm tumor tissue cores from the dominant lesion that were enriched with ≥75% tumor cells were taken for DNA and two cores for RNA purification. The RecoverAll Total Nucleic Acid Isolation Kit (Ambion/Applied Biosciences, Austin, TX) was used to extract DNA, which was then quantified (PicoGreen), aliquoted onto 96-well plates and shipped to Illumina (Illumina, Inc., San Diego, CA) for DNA methylation profiling.

The EZ DNA Methylation Kit (Zymo Research, Irvine, CA) was used to bisulfite convert tumor DNA samples. Controls on the array were used to track the bisulfite conversion efficiency. The Infinium® HumanMethylation450 BeadChip (Illumina) was used to measure genome-wide methylation using beads with target-specific probes designed to interrogate individual CpG sites (>485,000) (24). Samples from the FH cohort were assayed as one batch (7 plates) and the EV samples were assayed as a second batch (2 plates). Across the 96-well plates, we incorporated blind duplicate (FH, n = 16; EV, n = 7) and replicate (FH, n = 2; EV, n = 3) samples for each cohort. All plates also contained Illumina controls and two negative controls. PCa outcome events were randomly distributed across plates, and laboratory personnel were blinded to the location of duplicate and replicate samples.

Failed samples were identified by using the detection P-value metric (probability of a CpG being detected above the background level defined by negative control probes) according to Illumina protocols. A sample was excluded if less than 95% of the CpG sites for that sample on the array were detected with a detection P-value <0.05, resulting in removal of 17 FH and 15 EV samples. The final number of patients in the FH cohort and EV cohort was 327 (303 non-recurrent, 24 metastatic-lethal) and 65 (41 non-recurrent, 24 metastatic-lethal), respectively. Further, CpG sites with a detection P-value of >0.01 were excluded. After data filtering, 478,998 CpGs were available in the FH cohort and 479,103 in the EV cohort (477,460 overlapped). Correlation coefficients for duplicate samples in the FH and EV cohorts were 0.96-0.99 and 0.99, respectively. The correlation coefficients for replicate samples in FH and EV were 0.99 and 0.98.

The same FH and EV patients’ tumor samples used for DNA methylation profiling were also utilized for mRNA expression profiling using the Whole-Genome DASL® HT Assay (Illumina). Transcript correlations between duplicated samples (19 pairs) ranged from 0.96-0.99. In addition, replicate tumor RNA samples (6 pairs) were included, and the transcript correlations across plates were 0.95-0.99. There were 353 patients (FH: n = 288; EV: n = 65) with both tumor DNA methylation and mRNA expression data.

Statistical analysis

The methylation data were normalized using subset-quantile within array normalization (25) and batch effects were removed using Combat (26). Methylation β- and M-values were calculated, where β-values represent the percentage of DNA methylation at a CpG site. Methylation M-values are the logit transformed β-values that are approximately normally distributed. M-values were used for statistical testing and β-values to represent methylation differences between patient groups. Genome annotation of the CpGs was based on the Illumina protocol (27).

DNA methylation biomarkers for prognosis were identified using the FH cohort, following an a priori decision to select the top-ranked 5% of the CpGs based on their classification performance. First, for all individual CpG sites, the AUC and partial AUC (pAUC) for predicating metastatic-lethal versus non-recurrent PCa outcomes were calculated. The pAUC evaluates performance at a fixed high (95%) specificity as we aimed to select biomarkers with a low false-positive rate, providing more confidence that patients classified as high-risk by the biomarker indeed have high-risk tumors. Accordingly, we selected the top-ranked 4% of biomarkers based on pAUC and the top-ranked 1% based on AUC, yielding 22,290 CpGs for further analysis.

Next, we identified the subset of the 22,290 biomarkers that showed the greatest improvement in predicting metastatic-lethal PCa compared to Gleason sum alone. Because Gleason sum is the most widely used measure of tumor aggressiveness, we aimed to identify CpGs that could improve the prognostic discrimination of patients beyond that provided by Gleason sum alone. Other potential prognostic classifiers were also considered in models, including age at diagnosis, diagnostic PSA level, and pathologic tumor stage (local = pT2, N0/NX, M0; regional = pT3/T4 and/or N1, M0), but these factors did not improve the prediction of metastatic-lethal PCa compared to models with Gleason sum only (P >0.05), and were therefore not considered in further analyses.

A logistic regression model for discriminating patients with metastatic-lethal versus non-recurrent PCa was fit containing Gleason sum as the only predictor. Based on that model, forward model selection was done using three selection criteria: AUC, pAUC (95% specificity), and P-value (Wald test). For each criterion, we identified the CpG that showed the greatest improvement, ie, was the most significant in predicting metastatic-lethal PCa, compared to the base model with Gleason sum alone; the identified biomarker was then added to the model with Gleason. Forward selection was continued, each time selecting one additional CpG to be included in the model, until a pre-specified stopping criterion was met: for AUC this was an increment of AUC <0.005; for pAUC this was an increment of pAUC <0.0005; and for P-value this was >0.05. This entire process was repeated 100 times with bootstrap samples. The biomarkers that were selected multiple times in the different bootstrap cohorts (≥3 when considering AUC; ≥3 when considering pAUC; ≥4 when considering P-value) were chosen for further evaluation.

The methylation biomarkers that were most predictive for metastatic-lethal disease in the FH cohort were then tested in the EV cohort for validation. For each biomarker we calculated the AUC and pAUC (95% specificity) for distinguishing patients with metastatic-lethal versus non-recurrent PCa. P-values for AUC and pAUC were computed using 10,000 permutations, and 95% confidence intervals for AUC and pAUC were calculated using 2,000 stratified bootstrap replicates. Likelihood ratio tests were also computed to compare models fit with Gleason sum and a CpG biomarker compared to a model with Gleason sum only. All statistical analyses were conducted using R.

Results

There was no difference in mean age between patients with metastatic-lethal PCa compared to those who did not recur in either cohort (Table 1). In both cohorts, Gleason sum, pathological stage, and PSA level at diagnosis were higher in men with the metastatic-lethal phenotype relative to men with no evidence of recurrence (all P-values <0.01). The FH cohort had a mean follow-up time of 8.1 years for recurrence and 12.2 years for survival. The EV cohort was followed for outcomes on average for 9.0 years.

Table 1.

Characteristics of the prostate cancer patient populations

FH patients (Discovery cohort) EV patients (Validation dataset)
Non-recurrence (n = 303) Metastatic-lethal PCa (n = 24) Non-recurrence (n = 41) Metastatic-lethal PCa (n = 24)
Characteristic No. Mean (SD) No. % Mean (SD) P-valuea No. % Mean (SD) No. % Mean (SD) P-valuea
Age at diagnosis (years) 58.2 (7.1) 58.6 (7.3) 0.76 60.2 (6.0) 60.3 (5.7) 0.97
Gleason sum <0.01 <0.01
    ≤ 6 173 57.1 5 20.8 13 31.7 2 8.3
    7 (3+4) 101 33.3 8 33.3 24 58.5 5 20.8
    7 (4+3) 16 5.3 5 20.8 1 2.4 5 20.8
    8–10 13 4.3 6 25.0 3 7.3 12 50.0
Pathological stageb <0.01 <0.01
    Local 235 77.6 11 45.8 22 53.7 0 0.0
    Regional 68 22.4 13 54.2 19 46.3 24 100.0
PSA (ng/mL) at diagnosis <0.01 <0.01
    < 4.0 54 17.8 2 8.3 10 24.4 3 12.5
    4.0–4.9 189 62.4 7 29.2 27 65.9 14 58.3
    10.0–19.9 28 9.2 5 20.8 2 4.9 5 20.8
    ≥ 20 14 4.6 7 29.2 1 2.4 2 8.3

Note: PCa = prostate cancer; FH = Fred Hutchinson; EV = Eastern Virginia

a

A t-test (age) or chi-square test was used (all categorical variables).

b

Local stage = pT2, N0/NX, M0; Regional stage = pT3/T4 and/or N1, M0.

Table 2 shows the 42 DNA methylation biomarkers that were most predictive for metastatic-lethal PCa in the FH cohort. These CpGs were identified based on their ability to improve the prognostic discrimination beyond Gleason sum alone (Supplemental Table S1). Half of the 42 biomarkers showed higher methylation in patients with metastatic-lethal PCa compared to those with non-recurrent disease (Table 2). The 42 biomarkers had a mean methylation difference between patient groups (metastatic-lethal vs. non-recurrent) that ranged from 1% to 22% (average = 6.1%), and pAUC and AUC values for metastatic-lethal PCa ranged from 0.006 to 0.018 and 0.54 to 0.84, respectively. DNA methylation levels of the 42 biomarkers were not strongly correlated (all pairwise r2 <0.5).

Table 2.

Top-ranked 42 DNA methylation biomarkers for distinguishing patients with metastatic-lethal versus non-recurrent prostate cancer in the FH discovery cohort

CpG ID Gene Chr. Genetic location Epigenetic location Mean β non-recurrence Mean β metastatic-lethal Mean β difference AUC pAUC P-valueb
Higher DNA methylation levela
cg00022858 PDGFRA 4 5′UTR Island 0.09 0.11 0.02 0.58 0.008 8.51E-02
cg00107241 1 Intergenic Island 0.85 0.89 0.05 0.76 0.010 1.82E-04
cg00750074 SPG7 16 Body Island 0.91 0.93 0.02 0.75 0.007 8.98E-05
cg01135464 17 Intergenic OpenSea 0.41 0.63 0.22 0.84 0.018 1.25E-06
cg02223001 16 Intergenic Island 0.08 0.10 0.02 0.63 0.010 3.62E-02
cg04670359 NXT1 20 TSS200 Island 0.08 0.10 0.02 0.77 0.009 7.61E-04
cg07065941 KLF10 8 Body Island 0.09 0.11 0.02 0.75 0.006 9.18E-04
cg08092830 SOHLH1 9 TSS200 Island 0.69 0.74 0.04 0.74 0.012 4.90E-03
cg09734394 7 Intergenic S_Shore 0.69 0.78 0.09 0.71 0.011 1.00E-02
cg11084729 15 Intergenic OpenSea 0.94 0.95 0.01 0.56 0.008 1.85E-01
cg12300288 MUC4 3 Body S_Shore 0.81 0.85 0.05 0.70 0.013 1.70E-03
cg13371199 NDUFAF4 6 Body Island 0.06 0.08 0.01 0.66 0.012 1.47E-02
cg15850155 IGF2R 6 Body Island 0.90 0.92 0.02 0.63 0.008 3.37E-02
cg15965055 ILDR2 1 TSS1500 Island 0.11 0.14 0.03 0.69 0.010 8.70E-03
cg15996882 SREBF1 17 3′UTR Island 0.19 0.24 0.05 0.73 0.014 8.17E-04
cg16696648 SDHB 1 TSS1500 S_Shore 0.67 0.73 0.06 0.73 0.008 1.40E-03
cg18771570 2 Intergenic OpenSea 0.51 0.61 0.10 0.69 0.010 1.23E-02
cg19104976 PHF15 5 Body OpenSea 0.93 0.94 0.01 0.60 0.009 1.18E-01
cg22501793 1 Intergenic S_Shore 0.12 0.20 0.09 0.71 0.010 2.80E-03
cg24349665 PI15 8 TSS200 OpenSea 0.23 0.41 0.18 0.81 0.012 9.03E-06
cg24867247 13 Intergenic S_Shelf 0.24 0.33 0.10 0.71 0.011 2.20E-03
Lower DNA methylation levela
cg00837987 8 Intergenic OpenSea 0.72 0.56 0.16 0.74 0.008 9.69E-04
cg01166180 POLR2I;TBCB 19 TSS1500;Exon 1 Island 0.08 0.07 0.01 0.69 0.008 4.90E-03
cg02067030 EDIL3 5 Body OpenSea 0.66 0.49 0.17 0.84 0.009 7.09E-07
cg02394978 FHAD1 1 TSS1500 N_Shore 0.82 0.73 0.09 0.71 0.011 1.70E-03
cg03960699 USP34 2 Body OpenSea 0.04 0.04 0.01 0.64 0.009 1.18E-02
cg04086197 PPP1R13B 14 Body Island 0.86 0.80 0.06 0.69 0.012 4.80E-03
cg07166550 ALKBH5 17 Body S_Shore 0.80 0.74 0.06 0.60 0.012 1.44E-01
cg07466320 SRRM2 16 Body Island 0.80 0.75 0.05 0.72 0.009 2.00E-03
cg10462356 FBXO21 12 Body Island 0.87 0.80 0.07 0.69 0.009 1.52E-02
cg12629515 HIST1H3J;HIST1H2BO 6 TSS1500 N_Shore 0.77 0.70 0.07 0.60 0.012 1.21E-01
cg12817908 NMNAT3 3 TSS1500 S_Shore 0.46 0.38 0.09 0.64 0.012 2.52E-02
cg14162120 VSX2 14 Body Island 0.09 0.08 0.01 0.67 0.009 9.50E-03
cg14419310 RASEF 9 Body Island 0.06 0.05 0.01 0.54 0.008 1.82E-01
cg14769589 17 Intergenic N_Shore 0.32 0.24 0.08 0.83 0.015 1.06E-05
cg16713292 KLHL8 4 Body OpenSea 0.86 0.76 0.11 0.72 0.015 1.40E-03
cg17603271 1 Intergenic OpenSea 0.89 0.87 0.02 0.72 0.012 9.43E-04
cg20411049 RSF1 11 Body N_Shore 0.08 0.07 0.01 0.66 0.010 1.58E-02
cg21513610 ATP11A 13 Body S_Shore 0.90 0.86 0.04 0.73 0.010 9.90E-04
cg22282498 7 Intergenic N_Shelf 0.66 0.54 0.12 0.77 0.010 1.20E-03
cg25541259 DNER 2 Body N_Shore 0.47 0.38 0.09 0.76 0.007 5.31E-04
cg26756208 10 Intergenic Island 0.14 0.10 0.04 0.73 0.013 8.80E-03

Note: FH = Fred Hutchinson; TSS = transcription start site; UTR = untranslated region; CpG biomarkers highlighted in boldface were validated in the Eastern Virginia replication cohort.

a

Higher or lower DNA methylation level in tumor tissue of patients with metastatic-lethal vs. non-recurrent prostate cancer.

b

Based on a t-test comparing mean methylation level between patients with metastatic-lethal vs. non-recurrent prostate cancer.

We next evaluated the 42 top-ranked biomarkers in the EV replication cohort. For 30 of the CpGs, the difference in methylation level between patients with metastatic-lethal vs. non-recurrent PCa was in the same direction in the EV as in the FH cohort. Eight of these biomarkers demonstrated a significant AUC or pAUC in the EV cohort (all P-values <0.05; Table 3). One of the biomarkers had both a significant AUC and pAUC (ATP11A cg21513610). The CpG with the largest mean methylation difference was cg01135464 (P = 0.008). The biomarker with the highest AUC was KLHL8 cg16713292 (0.75), and the largest pAUC was for ATP11A cg21513610 (0.009). We next investigated whether methylation levels of these CpGs were correlated with methylation levels of adjacent CpGs in the same gene or intergenic region. For five of the CpGs the methylation levels were correlated (pairwise r2 >0.5) with methylation levels of nearby CpG sites (79 of 347 CpGs in ATP11A; 1 of 33 CpGs in FHAD1; 3 of 6 CpGs in PI15; 2 of 2 CpGs near cg01135464 [Chr. 17, OpenSea]; and 1 of 2 CpGs near cg22501793 [Chr. 1, S_Shore]).

Table 3.

Eight validated DNA methylation biomarkers for distinguishing patients with metastatic-lethal versus non-recurrent prostate cancer in the EV replication dataseta

CpG ID Gene or region Mean β differenceb AUC 95% CI AUC P-value AUC pAUC 95% CI pAUC P-value pAUC P-value t-testc
cg01135464 Intergenic (chr 17) 0.12 0.69 (0.56, 0.82) 0.008 0.006 (0, 0.018) 0.058 0.008
cg02223001 Intergenic (chr 16) 0.01 0.58 (0.44, 0.73) 0.279 0.008 (0.002, 0.019) 0.024 0.070
cg02394978 FHAD1 −0.06 0.71 (0.58, 0.83) 0.003 0.004 (0, 0.020) 0.159 0.007
cg07166550 ALKBH5 −0.05 0.66 (0.51, 0.79) 0.035 0.001 (0, 0.015) 0.566 0.037
cg16713292 KLHL8 −0.10 0.75 (0.63, 0.87) 0.0004 0.002 (0, 0.017) 0.359 0.002
cg21513610 ATP11A −0.06 0.66 (0.51, 0.78) 0.030 0.009 (0.0004, 0.025) 0.022 0.049
cg22501793 Intergenic (chr 1) 0.03 0.58 (0.42, 0.73) 0.319 0.007 (0.002, 0.017) 0.046 0.151
cg24349665 PI15 0.07 0.68 (0.54, 0.81) 0.014 0.006 (0.0003, 0.015) 0.074 0.029

Note: EV = Eastern Virginia; chr = chromosome

a

Biomarkers were considered validated when the AUC or pAUC (at 95% specificity) was significant (P-value <0.05). Significant P-values are highlighted in boldface.

b

A positive value indicates a higher DNA methylation level (a negative value indicates a lower DNA methylation level) in patients with metastatic-lethal vs. non-recurrent prostate cancer.

c

Based on a t-test comparing mean methylation levels in patients with metastatic-lethal vs. non-recurrent prostate cancer.

We then evaluated the performance of the eight validated biomarkers for classifying patients with metastatic-lethal PCa when combined with Gleason sum (Table 4). Figure 1 shows the ROC curves for Gleason sum alone, the eight individual CpGs, and each CpG plus Gleason sum. The AUC for Gleason sum alone in the EV cohort was 0.82. This is higher than what has been reported in other studies and likely reflects our nested case-control study design, which involved selecting patients with metastatic-lethal PCa and a similar number of patients without evidence of recurrence. For comparison, in the FH cohort that is unselected for patient outcomes, Gleason sum alone has an AUC of 0.75 for metastatic-lethal PCa. Gleason sum had a pAUC for predicting metastatic-lethal PCa of 0.010 in the EV dataset. Likelihood ratio tests were then performed comparing the model with Gleason sum only to a model that included both Gleason sum and one of the eight CpGs. This test was significant for four of the CpGs (P <0.05): ALKBH5 cg07166550, FHAD1 cg02394978, KLHL8 cg16713292, and PI15 cg24349665, providing further evidence that these biomarkers are complementary to Gleason sum for the prognostic discrimination of high-risk patients. Further adjustment for pathological stage (in addition to Gleason sum) increased the level of significance based on the likelihood ratio test for the three CpGs with the highest AUC values (AUC = 0.89), including the intergenic CpG on chr. 17 (P = 0.023) and the CpGs in KLHL8 (P = 0.0015) and PI15 (P = 0.004).

Table 4.

Performance of the eight validated DNA methylation biomarkers combined with Gleason sum for predicting metastatic-lethal prostate cancer in the EV replication dataset

CpG ID Gene or region CpG + Gleason P-valuea
AUC pAUC
cg01135464 Intergenic (chr 17) 0.89 0.004 0.073
cg02223001 Intergenic (chr 16) 0.85 0.004 0.219
cg02394978 FHAD1 0.86 0.013 0.038
cg07166550 ALKBH5 0.87 0.024 0.030
cg16713292 KLHL8 0.89 0.008 0.014
cg21513610 ATP11A 0.84 0.013 0.155
cg22501793 Intergenic (chr 1) 0.82 0.011 0.959
cg24349665 PI15 0.89 0.006 0.026

Note: EV = Eastern Virginia; chr = chromosome

a

P-value for the likelihood ratio test comparing a model with Gleason sum alone (AUC = 0.816, pAUC = 0.010) to a model with Gleason sum and the CpG biomarker; significant P-values are highlighted in boldface.

Fig 1.

Fig 1

ROC curves for predicting metastatic-lethal vs. non-recurrent prostate cancer for eight validated DNA methylation biomarkers. Curves are shown for each CpG biomarker alone, Gleason sum alone, and the biomarker plus Gleason sum.

In a final analysis we evaluated tumor mRNA expression in the same FH and EV patients’ tumor tissues that were used for methylation profiling. For two of the five genes that encompassed a validated CpG biomarker, DNA methylation levels were significantly correlated with transcript levels: ATP11A (Pearson r2 = −0.29, P = 2.78E-18) and PI15 (Pearson r2 = −0.28, P = 5.77E-08). ATP11A cg21513610 is located in the gene body, whereas cg24349665 is in the promoter region of PI15.

Discussion

Our results demonstrate that DNA methylation biomarkers measured in primary tumor tissue can distinguish patients with metastatic-lethal PCa from those men at least five years post-radical prostatectomy without disease recurrence. Of the 42 top-ranked differentially methylated CpG sites that stratified patients with aggressive tumors in our discovery cohort, and improved the prognostic discrimination beyond that provided by Gleason sum alone, eight were subsequently validated to predict metastatic-lethal outcomes in an independent patient cohort.

Prior studies of tumor DNA methylation in PCa mainly used biochemical recurrence as the outcome event (14, 18, 28, 29). Hypermethylation of CpGs in the promoter region of two genes, PITX2 and GSTP1, was previously associated with PSA recurrence (14, 29, 30). However, most men who have a rising PSA after surgery will not develop life-threatening disease (31), making biochemical recurrence less relevant than metastatic-lethal PCa outcomes as a study endpoint. A few prior candidate gene studies did assess lethal PCa (32-37), although the analyses were limited by both small sample sizes and short durations of follow-up for survival. Those studies highlighted a few differentially methylated genes (eg, APC, PITX2), but our results do not provide further support for aberrant methylation of these genes being biomarkers for progression to metastatic-lethal PCa.

The eight novel differentially methylated CpG sites validated in our study for the metastatic-lethal phenotype are located in five genes (ALKBH5, ATP11A, FHAD1, KLHL8, and PI15) and three intergenic regions (Chr. 1, 16, and 17). The five genes are involved in regulatory functions, response to hypoxia, protein-binding, developmental processes, and ion transport (38-42). The oxidative DNA demethylase ALKBH5, which is upregulated under hypoxia and also plays a role in spermatogenesis, belongs to the same gene family as ALKBH3 (Prostate Cancer Antigen 1), which is highly expressed in prostate tumors and is a potential therapeutic target for PCa (43). In a previous study expression of ATP11A, which belongs to an extended family of adenosine triphosphate-binding cassette transporters, was associated with colorectal cancer mortality (41). A small study found that PI15 (peptidase inhibitor 15) was amplified and overexpressed in 11% of advanced prostate tumors (44). The PI15 gene was also identified as a candidate oncogene in colorectal cancer (45), and has been implicated in regulating drug resistance in ovarian cancer (46). Interestingly, we also found that the DNA methylation alterations in PI15 and ATP11A were significantly correlated with mRNA expression of these genes in the same patients’ tumors. For PI15, the correlation was in the expected direction (ie, promoter hypermethylation and reduced expression). Specific molecular mechanisms whereby differential methylation of CpG sites in these five genes and three intergenic regions may enhance metastatic progression are unclear. There is biological plausibility for several of these genes contributing to more aggressive tumor biology, however further studies are needed to elucidate potential mechanisms.

Strengths of the current study include its relatively large sample size, the genome-wide approach for biomarker discovery, the population-based nature of the discovery cohort, with long-term prospective follow-up of patients diagnosed with clinically localized PCa, and a focus on the most serious clinical endpoint of metastatic-lethal disease. Validation of the DNA methylation biomarkers in an independent patient cohort is also critical, and confirms that these CpGs have added value to Gleason sum for predicting adverse patient outcomes. A potential weakness of our study is the limited number of patients with metastatic-lethal events, but these are not frequent outcomes in men diagnosed with clinically localized tumors that are treated surgically. Use of adjuvant therapy or salvage therapy may improve prognosis. In the FH cohort, adjuvant therapy use after RP was not frequent in non-recurrent (7.9%) or metastatic-lethal (16.7%) patients, making it unlikely that such therapies had a major impact on outcomes. As expected, most patients (94%) with metastatic progression received salvage therapy. Neither use of adjuvant nor salvage therapy, however, would affect methylation profiles in primary tumor tissue obtained at the time of surgery.

PCa is a heterogeneous disease and a combination of biomarkers may perform better than individual CpGs for prognostic classification. However, we did not intend to validate the combination of CpG biomarkers in our replication dataset due to the desire to avoid over fitting the data. Additional independent patient cohorts will be needed to build and test whether the combination of all or a subset of the eight CpG sites can further improve the prognostication for patients with more aggressive tumors. Further, additional investigation is needed to see if these biomarkers are predictive of patient outcomes in men not choosing radical prostatectomy as primary therapy.

In conclusion, we identified and then validated a novel panel of DNA methylation biomarkers in primary prostate tumor tissue that provide prognostic information, which improves upon Gleason sum for predicting metastatic-lethal patient outcomes. The methylation biomarkers replicated in this study have potential for improving clinical decision making by identifying patients likely to have a more aggressive cancer and who thereby are good candidates for adjuvant therapy or novel therapeutic clinical trials and who should be monitored more closely for metastatic progression. Future studies are needed to further evaluate the performance of our panel of prognostic DNA methylation biomarkers and to investigate if combining these epigenetic biomarkers may further improve their prognostic discrimination in early stage PCa patients. Investigations to elucidate the underlying molecular mechanisms through which these alterations in DNA methylation may enhance tumor aggressiveness are also needed.

Supplementary Material

1

Translational Relevance.

Prostate cancer (PCa) is a clinically heterogeneous disease and it is challenging to accurately predict which patients with localized stage disease harbor tumors with life-threatening potential. DNA methylation alterations may mediate tumor aggressiveness and could be informative for prognostication. We comprehensively profiled primary tumor DNA methylation (>485K CpGs) in two independent PCa cohorts followed prospectively for >5 years after radical prostatectomy to assess outcomes. An initial panel of 42 differentially methylated CpGs robustly distinguished patients with metastatic-lethal compared to non-recurrent tumors in the discovery cohort, and eight of these biomarkers were subsequently confirmed to predict metastatic-lethal events in the validation cohort, including CpGs in five genes (ALKBH5, ATP11A, FHAD1, KLHL8, and PI15) and three intergenic regions. These eight differentially methylated CpG sites warrant further investigation as novel prognostic biomarkers for distinguishing PCa patients who need closer monitoring for metastatic progression and who may benefit most from adjuvant therapy.

Acknowledgements

The authors thank Drs. Beatrice Knudson and Xiaotun Zhou for their assistance with the pathology, and the GU Tumor Biorepository, Eastern Virginia Medical School. We also thank all the men who participated in these studies.

Funding: This work was supported by grants from the National Cancer Institute (R01 CA056678, R01 CA092579, K05 CA175147, and P50 CA097186), with additional support provided by the Fred Hutchinson Cancer Research Center, the Intramural Program of the National Human Genome Research Institute, and the Prostate Cancer Foundation. Illumina, Inc. provided and performed the methylation arrays. M.G. is the recipient of a Dutch Cancer Society Fellowship (BUIT 2014–6645).

Footnotes

Competing interests: The authors declare that they have no conflict of interest.

References

  • 1.Siegel RL, Miller KD, Jemal A. Cancer statistics, 2016. CA Cancer J Clin. 2016;66:7–30. doi: 10.3322/caac.21332. [DOI] [PubMed] [Google Scholar]
  • 2.Ferlay J, Soerjomataram I, Ervik M, Dikshit R, Eser S, Mathers C, et al. [2015];GLOBOCAN 2012 v1.0, Cancer incidence and mortality worldwide: IARC CancerBase No. 11 [Interne]} 2012 doi: 10.1002/ijc.29210. [cited Available from: http://globocan.iarc.fr accessed 2015.; Available from: Available from: http://globocan.iarc.fr. [DOI] [PubMed]
  • 3.Attard G, Parker C, Eeles RA, Schroder F, Tomlins SA, Tannock I, et al. Prostate cancer. Lancet. 2015;387:70–82. doi: 10.1016/S0140-6736(14)61947-4. [DOI] [PubMed] [Google Scholar]
  • 4.Fraser M, Berlin A, Bristow RG, van der Kwast T. Genomic, pathological, and clinical heterogeneity as drivers of personalized medicine in prostate cancer. Urol Oncol. 2015;33:85–94. doi: 10.1016/j.urolonc.2013.10.020. [DOI] [PubMed] [Google Scholar]
  • 5.Porten SP, Whitson JM, Cowan JE, Cooperberg MR, Shinohara K, Perez N, et al. Changes in prostate cancer grade on serial biopsy in men undergoing active surveillance. J Clin Oncol. 2011;29:2795–800. doi: 10.1200/JCO.2010.33.0134. [DOI] [PubMed] [Google Scholar]
  • 6.Epstein JI, Feng Z, Trock BJ, Pierorazio PM. Upgrading and downgrading of prostate cancer from biopsy to radical prostatectomy: incidence and predictive factors using the modified Gleason grading system and factoring in tertiary grades. Eur Urol. 2012;61:1019–24. doi: 10.1016/j.eururo.2012.01.050. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Treurniet KM, Trudel D, Sykes J, Evans AJ, Finelli A, Van der Kwast TH. Downgrading of biopsy based Gleason score in prostatectomy specimens. J Clin Pathol. 2014;67:313–8. doi: 10.1136/jclinpath-2012-201323. [DOI] [PubMed] [Google Scholar]
  • 8.Wright JL, Salinas CA, Lin DW, Kolb S, Koopmeiners J, Feng Z, et al. Prostate cancer specific mortality and Gleason 7 disease differences in prostate cancer outcomes between cases with Gleason 4 + 3 and Gleason 3 + 4 tumors in a population based cohort. J Urol. 2009;182:2702–7. doi: 10.1016/j.juro.2009.08.026. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Choudhury AD, Eeles R, Freedland SJ, Isaacs WB, Pomerantz MM, Schalken JA, et al. The role of genetic markers in the management of prostate cancer. Eur Urol. 2012;62:577–87. doi: 10.1016/j.eururo.2012.05.054. [DOI] [PubMed] [Google Scholar]
  • 10.Loeb S, Bjurlin MA, Nicholson J. Overdiagnosis and overtreatment of prostate cancer. Eur Urol. 2014;65:1046–55. doi: 10.1016/j.eururo.2013.12.062. al. E. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Ross AE, D'Amico AV, Freedland SJ. Which, when and why? Rational use of tissue-based molecular testing in localized prostate cancer. Prostate Cancer and Prostatic Diseases. 2016;19:1–6. doi: 10.1038/pcan.2015.31. [DOI] [PubMed] [Google Scholar]
  • 12.Bostrom PJ, Bjartell AS, Catto JWF, Eggener SE, Lilja H, Loeb S, et al. Genomic predictors of outcome in prostate cancer. Eur Urol. 2015;68:1033–44. doi: 10.1016/j.eururo.2015.04.008. [DOI] [PubMed] [Google Scholar]
  • 13.Issa JP. DNA methylation as a clinical marker in oncology. J Clin Oncol. 2012;30:2566–68. doi: 10.1200/JCO.2012.42.1016. [DOI] [PubMed] [Google Scholar]
  • 14.Chao C, Chi M, Preciado M, Black MH. Methylation markers for prostate cancer prognosis: a systematic review. Cancer causes & control : CCC. 2013;24:1615–41. doi: 10.1007/s10552-013-0249-2. [DOI] [PubMed] [Google Scholar]
  • 15.Herman JG, Baylin SB. Gene silencing in cancer in association with promoter hypermethylation. N Engl J Med. 2003;349:2042–54. doi: 10.1056/NEJMra023075. [DOI] [PubMed] [Google Scholar]
  • 16.Jeronimo C, Bastian PJ, Bjartell A, Carbone GM, Catto JW, Clark SJ, et al. Epigenetics in prostate cancer: biologic and clinical relevance. Eur Urol. 2011;60:753–66. doi: 10.1016/j.eururo.2011.06.035. [DOI] [PubMed] [Google Scholar]
  • 17.Ashour N, Angulo JC, Andres G, Alelu R, Gonzalez-Corpas A, Toledo MV, et al. A DNA hypermethylation profile reveals new potential biomarkers for prostate cancer diagnosis and prognosis. Prostate. 2014;74:1171–82. doi: 10.1002/pros.22833. [DOI] [PubMed] [Google Scholar]
  • 18.Haldrup C, Mundbjerg K, Vestergaard EM, Lamy P, Wild P, Schulz WA, et al. DNA methylation signatures for prediction of biochemical recurrence after radical prostatectomy of clinically localized prostate cancer. J Clin Oncol. 2013;31:3250–8. doi: 10.1200/JCO.2012.47.1847. [DOI] [PubMed] [Google Scholar]
  • 19.Horning AM, Awe JA, Wang CM, Liu J, Lai Z, Wang VY, et al. DNA methylation screening of primary prostate tumors identifies SRD5A2 and CYP11A1 as candidate markers for assessing risk of biochemical recurrence. Prostate. 2015;75:1790–801. doi: 10.1002/pros.23052. [DOI] [PubMed] [Google Scholar]
  • 20.Freedland SJ, Humphreys EB, Mangold LA, Eisenberger M, Dorey FJ, Walsh PC, et al. Risk of prostate cancer-specific mortality following biochemical recurrence after radical prostatectomy. JAMA. 2005;294:433–9. doi: 10.1001/jama.294.4.433. [DOI] [PubMed] [Google Scholar]
  • 21.Abdollah F, Boorjian SA, Cozzarini C, Suardi N, Sun M, Fiorino C, et al. Survival following biochemical recurrence after radical prostatectomy and adjuvant radiotherapy in patients with prostate cancer: the impact of competing causes of mortality and patient stratification. Eur Urol. 2013;64:557–64. doi: 10.1016/j.eururo.2013.03.006. [DOI] [PubMed] [Google Scholar]
  • 22.Agalliu I, Salinas CA, Hansten PD, Ostrander EA, Stanford JL. Statin use and risk of prostate cancer: results from a population-based epidemiologic study. Am J Epidemiol. 2008;168:250–60. doi: 10.1093/aje/kwn141. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Stanford JL, Wicklund KG, McKnight B, Daling JR, Brawer MK. Vasectomy and risk of prostate cancer. Cancer Epidemiol Biomarkers Prev. 1999;8:881–6. [PubMed] [Google Scholar]
  • 24.Bibikova M, Barnes B, Tsan C, Ho V, Klotzle B, Le J, et al. High density DNA methylation array with single CpG site resolution. Genomics. 2011;98:288–95. doi: 10.1016/j.ygeno.2011.07.007. [DOI] [PubMed] [Google Scholar]
  • 25.Maksimovic J, Gordon L, Oshlack A. SWAN: Subset-quantile within array normalization for Illumina infinium HumanMethylation450 BeadChips. Genome biology. 2012;13:R44. doi: 10.1186/gb-2012-13-6-r44. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Johnson WE, Li C, Rabinovic A. Adjusting batch effects in microarray expression data using empirical Bayes methods. Biostatistics. 2007;8:118–27. doi: 10.1093/biostatistics/kxj037. [DOI] [PubMed] [Google Scholar]
  • 27.Hansen K. IlluminaHumanMethylation450kanno.ilmn12.hg19: Annotation for Illumina's 450k methylation arrays. R package version 0.2.1 [Google Scholar]
  • 28.Chiam K, Ricciardelli C, Bianco-Miotto T. Epigenetic biomarkers in prostate cancer: Current and future uses. Cancer letters. 2014;342:248–56. doi: 10.1016/j.canlet.2012.02.011. [DOI] [PubMed] [Google Scholar]
  • 29.Valdes-Mora F, Clark SJ. Prostate cancer epigenetic biomarkers: next-generation technologies. Oncogene. 2015;34:1609–18. doi: 10.1038/onc.2014.111. [DOI] [PubMed] [Google Scholar]
  • 30.Banez LL, Sun L, van Leenders GJ, Wheeler TM, Bangma CH, Freedland SJ, et al. Multicenter clinical validation of PITX2 methylation as a prostate specific antigen recurrence predictor in patients with post-radical prostatectomy prostate cancer. J Urol. 2010;184:149–56. doi: 10.1016/j.juro.2010.03.012. [DOI] [PubMed] [Google Scholar]
  • 31.Uchio EM, Aslan M, Wells CK, Calderone J, Concato J. Impact of biochemical recurrence in prostate cancer among US veterans. Archives of internal medicine. 2010;170:1390–5. doi: 10.1001/archinternmed.2010.262. [DOI] [PubMed] [Google Scholar]
  • 32.Henrique R, Ribeiro FR, Fonseca D, Hoque MO, Carvalho AL, Costa VL, et al. High promoter methylation levels of APC predict poor prognosis in sextant biopsies from prostate cancer patients. Clin Cancer Res. 2007;13:6122–9. doi: 10.1158/1078-0432.CCR-07-1042. [DOI] [PubMed] [Google Scholar]
  • 33.Richiardi L, Fiano V, Vizzini L, De Marco L, Delsedime L, Akre O, et al. Promoter methylation in APC, RUNX3, and GSTP1 and mortality in prostate cancer patients. J Clin Oncol. 2009;27:3161–8. doi: 10.1200/JCO.2008.18.2485. [DOI] [PubMed] [Google Scholar]
  • 34.Rosenbaum E, Hoque MO, Cohen Y, Zahurak M, Eisenberger MA, Epstein JI, et al. Promoter hypermethylation as an independent prognostic factor for relapse in patients with prostate cancer following radical prostatectomy. Clin Cancer Res. 2005;11:8321–5. doi: 10.1158/1078-0432.CCR-05-1183. [DOI] [PubMed] [Google Scholar]
  • 35.Vasiljevic N, Ahmad AS, Beesley C, Thorat MA, Fisher G, Berney DM, et al. Association between DNA methylation of HSPB1 and death in low Gleason score prostate cancer. Prostate cancer and prostatic diseases. 2013;16:35–40. doi: 10.1038/pcan.2012.47. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Vasiljevic N, Ahmad AS, Carter PD, Fisher G, Berney DM, Foster CS, et al. DNA methylation of PITX2 predicts poor survival in men with prostate cancer. Biomarkers in medicine. 2014;8:1143–50. doi: 10.2217/bmm.14.41. [DOI] [PubMed] [Google Scholar]
  • 37.Vasiljevic N, Ahmad AS, Thorat MA, Fisher G, Berney DM, Moller H, et al. DNA methylation gene-based models indicating independent poor outcome in prostate cancer. BMC cancer. 2014;14:655. doi: 10.1186/1471-2407-14-655. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Dhanoa BS, Cogliati T, Satish AG, Bruford EA, Friedman JS. Update on the Kelch-like (KLHL) gene family. Hum Genomics. 2013;7:13. doi: 10.1186/1479-7364-7-13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Durocher D, Jackson SP. The FHA domain. FEBS letters. 2002;513:58–66. doi: 10.1016/s0014-5793(01)03294-x. [DOI] [PubMed] [Google Scholar]
  • 40.Falak S, Schafer S, Baud A, Hummel O, Schulz H, Gauguier D, et al. Protease inhibitor 15, a candidate gene for abdominal aortic internal elastic lamina ruptures in the rat. Physiol Genomics. 2014;46:418–28. doi: 10.1152/physiolgenomics.00004.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Miyoshi N, Ishii H, Mimori K, Tanaka F, Nagai K, Uemura M, et al. ATP11A is a novel predictive marker for metachronous metastasis of colorectal cancer. Oncology reports. 2010;23:505–10. [PubMed] [Google Scholar]
  • 42.Thalhammer A, Bencokova Z, Poole R, Loenarz C, Adam J, O'Flaherty L, et al. Human AlkB homologue 5 is a nuclear 2-oxoglutarate dependent oxygenase and a direct target of hypoxia-inducible factor 1alpha (HIF-1alpha). PloS one. 2011;6:e16210. doi: 10.1371/journal.pone.0016210. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Koike K, Ueda Y, Hase H, Kitae K, Fusamae Y, Masai S, et al. anti-tumor effect of AlkB homolog 3 knockdown in hormone-independent prostate cancer cells. Curr Cancer Drug Targets. 2012;12:847–56. doi: 10.2174/156800912802429283. [DOI] [PubMed] [Google Scholar]
  • 44.Vainio P, Wolf M, Edgren H, He T, Kohonen P, Mpindi JP, et al. Integrative genomic, transcriptomic, and RNAi analysis indicates a potential oncogenic role for FAM110B in castration-resistant prostate cancer. The Prostate. 2012;72:789–802. doi: 10.1002/pros.21487. [DOI] [PubMed] [Google Scholar]
  • 45.Tuupanen S, Hanninen UA, Kondelin J, von Nandelstadh P, Cajuso T, Gylfe AE, et al. Identification of 33 candidate oncogenes by screening for base-specific mutations. Br J Cancer. 2014;111:1657–62. doi: 10.1038/bjc.2014.429. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Zou J, Yin F, Wang Q, Zhang W, Li L. Analysis of microarray-identified genes and microRNAs associated with drug resistance in ovarian cancer. Int J Clin Exp Pathol. 2015;8:6847–58. [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1

RESOURCES