Molecular Scores to Predict Ovarian Cancer Outcomes: A Worthy Goal, but Not Ready for Prime Time

Ovarian carcinoma has the highest mortality of all gynecological cancers. The American Cancer Society estimates that in 2012, about 22 280 new cases of ovarian cancer will be diagnosed, and 15 500 women will die of ovarian cancer in the United States (1). Despite achieving high rates of remission following radical surgery and platinum-based chemotherapy, most women relapse and ultimately die of chemoresistant disease. Advances in chemotherapy lengthen survival for women with advanced-stage (stages III and IV) disease but have not changed the likelihood of cure. Biomarkers, such as the rate of decline of serum cancer antigen 125 (CA-125, also known as mucin-16) level or the absolute CA-125 nadir, can be predictors of progression-free and overall survivals (2–4); however, when faced with a slowly declining level of CA-125 during primary treatment, the oncologist has few effective alternatives. With almost 80% of primary ovarian cancers initially responding to platinum-based therapy, a prospective biomarker would need to be highly predictive of treatment failure because alternative treatments would be purely experimental.

The advent of high-throughput molecular profiling technologies has led to numerous studies that defined ovarian carcinoma phenotypes by various profiles, including mRNA, protein, microRNA, DNA copy number, etc. In unsupervised analyses, the molecular profiles are used to sort the carcinomas into different categories by statistical algorithms without assumptions about the type or number categories present. In contrast, in supervised analyses, the investigator defines the characteristics of interest (such as response to chemotherapy) in a test set and determines which subset components of the molecular profile are differentially expressed, thereby defining a signature for the phenotype of interest. Because of the large number of comparisons made, statistically significant associations can occur by chance, and the “novel” signature will invariably perform well within the test set from which it was derived. Therefore, to assess reproducibility and clinical performance, the signature must be assessed in one or more “validation sets” of unrelated specimens.

The Cancer Genome Atlas (TCGA) network recently compiled detailed molecular datasets on more than 500 serous ovarian carcinomas and identified an expression profile signature that was associated with survival (5). This signature was then validated in several publicly available datasets. In this issue of the Journal, Kang et al. (6) have taken a different approach to the publicly available TCGA data. They applied a hypothesis-driven approach to test whether expression of specific DNA repair genes can predict clinical outcomes after platinum-based therapy. A score was developed based on the expression of 23 genes in DNA repair pathways that were thought to be responsible for repair of platinum-induced DNA damage and had shown a moderate association with survival (P < .15) in the TCGA set. They defined the score as a simple sum (up to 23) for each sample based on whether expression of each of the 23 genes deviated from the median expression in the direction favoring survival. The score was then categorized as “low” (ie, scores 1–10) or “high” (ie, scores 11–20), and clinical performance was assessed in two independent publicly available validation sets. The authors conclude that their DNA repair score can be used to predict outcomes and response to platinum therapy in ovarian cancer patients and is ready for application in a prospective clinical trial. Strengths of their analyses include the use of a large well-annotated set of serous ovarian carcinomas, use of comprehensive profiling with multiple platforms, and the development of a hypothesis-driven predictor.

The underlying hypothesis was that malignant cells deficient in DNA repair (such as homologous recombination and the Fanconi anemia pathway) will demonstrate a more durable response to platinum-based chemotherapy. This is a reasonable hypothesis, given that carcinomas from women with germline BRCA1/2 mutations are deficient in homologous recombination and have better overall survival compared with women with sporadic ovarian carcinoma (7). This phenotype has been somewhat elusively referred to as “BRCAness” (8); however, no one knows how to best define such a homologous recombination–deficient or platinum-sensitive phenotype. Konstantinopoulos et al. (9) tried to define a BRCA1/2-deficient (BRCA-like) phenotype with expression profiling, but interestingly, none of the 60 genes defined in this signature overlap with any of the 23 genes used in the score developed by Kang et al. (6).

Contrary to the initial hypothesis of Kang et al. (6), high expression of the DNA repair genes, rather than low expression, was associated with improved overall survival [table 2 in (6)]. This finding is counterintuitive and without an obvious biological basis. The authors suggest that high expression of DNA repair genes may reflect an attempt to compensate for a defective pathway, a worthwhile and testable hypothesis. Alternatively, increased expression of many DNA repair genes may actually lead to reduced DNA repair efficiency by disrupting the tight regulation normally present (10). Finally, alterations in one or more transcription factors could affect expression of multiple genes in the score (11).

The score was intended to predict platinum response and thus help direct therapy. But does their score reliably predict outcome and should we test it in a clinical arena? We would expect the score to perform well in the “test set,” that is, the TCGA samples from which it was derived. An alternative score derived from 17 DNA repair genes that were “not of interest” (which means the DNA repair genes that have not been implicated in cellular resistance to platinum) was also predictive in the TCGA samples [P < .001, supplementary Figure 2, A, available online, in (6)] demonstrating that evaluating the biomarker within the test set is not a valid assessment of performance. The authors’ conclusion that the 23-gene score outperformed known prognostic factors in TCGA samples is not meaningful, given that none of the known prognostic factors were actually predictive in the TCGA dataset. Clinical performance can only be meaningfully assessed in independent sample sets (the validation sets). Two validation sets, Tothill dataset (12) and Berchuck dataset (13), were used to establish clinical validity, but both were small. The score was not statistically significantly associated with survival in the larger (ie, Tothill) dataset until the authors eliminated one-third of the cases treated with platinum but not taxane, an analysis counterintuitive to the underlying hypothesis. And even in that selected subset, the score was not statistically significantly associated with survival in multivariable analysis (P = .055). In the second (ie, Berchuck) dataset, the score was statistically significantly associated with survival in multivariable analysis (P = .021), but that dataset included only 54 advanced-stage cancers, too few to rely on exclusively for validation. The receiver operating characteristic curves could be determined only for the smaller validation set, generating a limited area under the curve of 0.65, not sufficiently predictive for clinical decision making. In fact, the DNA repair score (6), turned out to be a less powerful prognosticator than the predictive expression profile already reported by TCGA using an unsupervised discovery approach and using same validation sets (5).

So what lessons have we learned from this study? First, the vast amount of TCGA data is challenging to access and analyze. TCGA used multiple platforms including three for expression profiling, two for exome sequencing, and five for copy number analyses, making integration of data difficult. Kang et al. (6) do not provide a detailed explanation of how they integrated data from the three expression platforms, but their methods appear to differ substantially from those used in the published TCGA analyses (5). This could explain discrepancies in data from the two analyses. For example, in evaluating the association of BRCA1 expression with survival (5), BRCA1 expression met the TCGA cut point (P < .01) and was included in the TCGA prognostic predictor. However, in the analysis by Kang et al. (6), BRCA1 expression was not associated with survival despite using a much lower cut point (P < .15) and despite overlap of the vast majority of samples between the two studies. We conclude that the application of different analytical methods to TCGA data will generate different findings. The lack of impact of known prognostic factors (stage, grade, age, and surgical cytoreduction) in TCGA is troublesome and will further complicate attempts to develop biomarkers from this dataset. TCGA samples were collected for molecular annotation, not chosen to be representative or to answer specific outcomes questions, likely explaining why the TCGA set behaves differently than most other collections. Therefore, the utility of TCGA in the identification of generalizable biomarkers remains uncertain.

Another important lesson is the need to better understand the molecular phenotype to successfully apply a hypothesis-driven approach. Our understanding of the regulatory mechanisms (somatic mutation, copy number alteration, changes in epigenetic or microRNA regulation, etc.) impacting the various DNA repair pathways and their interactions with each other remains incomplete. Ultimately, it may be more accurate to directly measure functional markers of DNA repair in neoplastic cells (such as DNA damage–induced formation of RAD51 nuclear foci), which might require fresh tissue samples and exposure to DNA-damaging agents (14,15). The molecular heterogeneity of high-grade serous ovarian carcinoma may result in one phenotype dominating in the early platinum responsive malignancy and another in the resistant recurrence, further complicating the picture. It may be important to define a homologous recombination-deficient phenotype in “real-time,” perhaps, even during the course of treatment. To date, the only clinically validated biomarker of homologous recombination function in ovarian carcinomas is BRCA1/2 germline mutation status, but even that can be circumvented in malignant cells by the development of secondary mutations that restore functional BRCA1/2 in the tumor (16–21). Therefore, the plasticity of response to chemotherapy may be as important as the initial phenotype.

The analysis of the TCGA dataset by Kang et al. (6) was unable to prove their hypothesis or validate their counterintuitive DNA repair score in a large independent dataset. Although their score is not yet ready for clinical application, it has raised some interesting questions: 1) what is the best way to assess DNA repair function in solid malignancies, and 2) with such redundancy in the DNA repair pathways, how do we define the driver vs passenger events and classify the important vs unimportant pathways and genes? The development of predictive biomarkers is critical to the successful application of individualized therapy. The study by Kang et al. (6) is an important effort in that direction but demonstrates the challenges we face as we attempt to utilize large datasets to develop personalized genomic medicines. The premature application of inadequately validated biomarkers may adversely impact the successful implementation of individualized therapies.

Funding

This work is supported, in part, by the National Institutes of Health (R01CA125636 to EMS and TT; R01CA131965 to EMS; P50CA83636 to EMS, TT, and BYK) and Howard Hughes Medical Institute (to TT). American Cancer Society (SIOP-06-258-06-COUN to BYK).