Multiple biomarker algorithms to predict epithelial ovarian cancer in women with a pelvic mass: Can additional makers improve performance?

Abstract

Introduction:

Management of a woman with a pelvic mass is complicated by difficulty in discriminating malignant from benign disease. Many serum biomarkers have been examined to determine their sensitivity for detecting malignancy. This study was designed to evaluate if the addition of biomarkers to HE4 and CA125, as used in the Risk of Malignancy Algorithm (ROMA), can improve the detection of EOC.

Methods:

This was an IRB approved, prospective clinical trial examining serum obtained from women diagnosed with a pelvic mass who subsequently underwent surgery. Serum biomarker levels for CA-125, HE4, YKL-40, transthyretin, ApoA1, Beta-2-microglobulin, transferrin, and LPA were measured. Logistic regression analysis was performed for various marker combinations, ROC curves were generated, and the area under the curves (AUCs) were determined.

Results:

A total of 184 patients met inclusion criteria with a median age of 56 years (Range 20–91). Final pathology revealed there were 103 (56.0%) benign tumors, 4 (2.2%) LMP tumors, 61 EOC (33.1%), 2 (1.1%) non-EOC ovarian cancers, 6 (3.3%) gynecologic cancers with metastasis to the ovary and 8 (4.3%) non-gynecologic cancers with metastasis to the ovary. The combination of HE4 and CA125 (i.e. ROMA) achieved an AUC of 91.2% (95% CI: 86.0–96.4) for the detection of EOC vs benign disease. The combination of CA-125, HE4, YKL-40, transthyretin, ApoA1, Beta 2 microglobulin, transferrin, LPA and menopausal status achieved the highest AUC of 94.6% (95% CI: 90.1–99.2) but this combination was not significantly better than the HE4 and CA125 combination alone (p=0.078).

Conclusions:

The addition of select further serum biomarkers to HE4 and CA125 does not add to the performance of the dual marker combination for the detection of ovarian cancer.

Introduction:

Approximately 289,000 women in the United States are diagnosed with a pelvic mass each year of which 10–20% will ultimately be diagnosed with a malignancy (1). The American Cancer Society projects there will be 22,240 women in 2018 diagnosed with ovarian cancer (2). Most of these cancers will be epithelial malignancies arising from the fallopian tubes, ovaries or peritoneum, which have the highest fatality-to-case ratio. While length of survival following diagnosis has improved dramatically with current standard treatment, overall survival remains poor. It is estimated that approximately 14,070 deaths in 2018 will be attributed to fallopian tube and ovarian cancers (2). Overall survival rates remain low as 4 out of 5 women diagnosed with fallopian tube and ovarian cancer are diagnosed at an advanced stage (2). Despite only a small percentage of women with a pelvic mass being diagnosed with a malignancy, triaging these patients to centers and providers experienced in the treatment and management of this disease is of the utmost importance due to the high morbidity and mortality associated with these malignancies. Numerous studies have shown improved outcomes for those patients with ovarian cancer who undergo primary surgery with a gynecologic oncologist and receive their care at high volume institutions (3–7).

Several algorithms have been developed to assist physicians with risk assessment of women presenting with a pelvic mass. The most common risk-assessment tools use different combinations of biomarkers, imaging and patient characteristics to estimate a patient’s individual risk of ovarian cancer. The earliest algorithms focused solely on the cancer antigen 125 (CA125) biomarker to stratify a patient’s risk of epithelial ovarian cancer. Jacobs et al introduced the Risk of Malignancy Index (RMI), which combined a patient’s menopausal status, ultrasound score and serum CA125 levels to assess risk for malignancy (8). A threshold of 200 for the RMI score yields a sensitivity of 85.5% with a specificity of 96.9%. As the RMI score cut point is lowered, the sensitivity increases but the specificity decreases. A threshold of 50 for the RMI score yields a sensitivity of 95.1% with a specificity of 76.5%. Several trials have validated the RMI algorithm, however without strict controls of ultrasound protocols, the performance of the RMI decreases (9). More recently, multiple marker algorithms have been developed and cleared by the US FDA and EU marked for clinical use. The addition of the serum biomarker human epididymis protein 4 (HE4) has been shown to be highly specific and sensitive for detecting epithelial ovarian cancers. When combined with serum CA125, these two markers complement one another with improved sensitivity and specificity over that of either marker alone (10). The Risk of Ovarian Malignancy Algorithm (ROMA) is a logistic regression algorithm that uses serum HE4 and CA125 levels along with the patient’s menopausal status to categorize patients into high and low risk probabilities that a malignancy will be found in a patient with a pelvic mass. ROMA has been demonstrated to achieve a sensitivity of 94% and a negative predictive value of 99% at a set specificity of 75% for predicting the presence of epithelial ovarian cancer in women presenting with a pelvic mass (11–13). The multivariate index assay (OVA1) is an unpublished algorithm that employs menopausal status, serum levels of CA125, transferrin, transthyretin (prealbumin), apolipoproteinA1, and beta-2-microglobulin to stratify women into high and low risk categories. The multivariate index assay achieved a sensitivity of 92.4% with a specificity of 53.5% and a negative predictive value of 96.8% (14). Despite the high sensitivities, there is room for improvement, especially in specificity, of these multiple marker assays.

Active areas of ovarian cancer research focus on identification of new markers to improve the ability to detect cancers at earlier stages and more importantly for screening. High throughput technology has yielded multiple biomarkers associated with ovarian cancer. With the number of biomarkers being identified as candidate serum biomarkers for ovarian cancers, studies comparing clinical performance of these biomarkers alone or in combination are needed in order to identify the best biomarker combinations. These studies not only have to focus on the statistical performance of multiple marker algorithms, but also on the cost of combining multiple analytes in a specific test. The most economically efficient test would utilize a minimal number of analytes without affecting the algorithms performance. The current study examines the performance of eight serum biomarkers that have been well documented to be active ovarian cancer biomarker, including 6 biomarkers currently used in the two FDA cleared algorithms as well as YKL40 and lysophosphatidic acid (LPA). The study objective was to determine the optimal combination of biomarkers to predict malignancy in women presenting with a pelvic mass.

Methods:

This was a prospective cohort study approved by the Women and Infants Hospital of Rhode Island Institutional Review Board (IRB: WIHIR-09-0030) overseeing the Program in Women’s Oncology at Women and Infant’s Hospital / Brown University from 2009 to 2015. The clinical trial was registered with clinicaltrials.gov (NCT00986206). The clinical trial inclusion criteria required that study participants be greater than age 18, have imaging documenting the presence of an adnexal mass, be scheduled for surgery, and be willing and able to provide informed consent. Individuals were excluded from the study if they had a prior history of ovarian cancer, had a prior history of bilateral oophorectomy, were currently pregnant, or were unable to give informed consent. Both pre-menopausal and post-menopausal women were included in the study. Menopausal status was determined based on history and physical examination. Imaging documenting an ovarian cyst, adnexal mass or other pelvic mass was required for enrollment onto the study. All imaging was reviewed at a multidisciplinary gynecologic oncology tumor board. All pathology results were read by gynecologic oncology pathologist with surgical pathology results collected and reviewed for histologic diagnosis. Operative and pathology reports were reviewed to document surgical stage for patients diagnosed with an invasive ovarian cancer.

Women who met inclusion criteria underwent serum measurement of biomarkers CA-125, HE4, YKL-40, transthyretin, ApoA1, Beta 2 microglobulin, transferrin and LPA prior to undergoing surgery. Serum CA125 and HE4 levels were measured on the ARCHITECT i2000 (Abbot Diagnostics Inc., Chicago, IL), serum YKL40 levels were measured by MicroVue YKL40 EIA kit (Quidel Inc., San Diego, CA), serum transthyretin levels were measured by TTR ELISA kit (Cusabio Biotech Co., Houston, TX), serum ApoA1 levels were measured by Human apoA1 ELISA kit (Mabtech Inc., Cincinnati, OH), serum Beta 2 microglobulin levels were measured by Human beta 2-Microglobin ELISA kit (R&D Systems, Minneapolis, MN), and serum Transferrin levels were measured on the Advia Chemistry system (Siemens Diagnostics, Tarrytown, NY). All serum biomarker kits were validated in duplicate using independent clinical samples from the study samples and prior to analysis of the study samples in the Center for Biomarkers and Emerging Technology at Women and Infants Hospital. LPA levels were measured using a novel assay developed as part of this research study in collaboration with the University of Portland reported in a separate publication (15).

Study participants received the standard of care in surgical management and those diagnosed with ovarian cancer intra-operatively underwent full surgical staging. Patients, surgeons, and pathologists were blinded to the results of the serum testing and investigators responsible for biomarker analysis were blinded to the surgical pathology results. Logistic regression analyses were performed for forty-six unique biomarker combinations (i.e. each marker alone, CA125 in combination with each other marker, HE4 in combination with each other marker, CA125 and HE4 in combination with all other marker combinations, and the combination of the OVA1 biomarkers), and ROC curves, along the area under the curves (AUCs) and with corresponding p-values for comparison to the combination of HE4 and CA125 as used in the ROMA algorithm, were generated for each combination. Sensitivity, specificity, Positive Predictive Value (PPV) and Negative Predictive Value (NPV) were determined. The primary endpoint of the study was to determine the optimal biomarker combination to predict cancer. The comparison of ROMA versus OVA1 was not directly performed in this analysis as the OVA1 algorithm has never been published and as such is not publicly available for independent validation. ROMA, the Risk of Ovarian Malignancy Algorithm, has been published and is available for independent analysis and validation.

Results:

One hundred and eighty-four women with adnexal masses were enrolled in the study. We evaluated eight specific biomarkers alone as well as in 46 unique combinations to determine their performance and compared them to one of the commonly used risk-assessment tools, ROMA, to see if there were any significant performance improvements. Of the evaluable 184 patients, 63 were pre-menopausal and 121 were post-menopausal with a median age of 56 years (range of 20–91). Demographic and pathologic information is presented below in Table 1. Study subjects are further classified based on pathologic diagnosis, grade and stage in Tables 2 and 3, respectively. Final pathology revealed there were 103 (56.0%) benign tumors, 4 (2.2%) Low Malignant Potential (LMP) tumors, 61 EOC (33.1%), 2 (1.1%) non-EOC ovarian cancers, 6 (3.3%) gynecologic cancers with metastasis to the ovary and 8 (4.3%) non-gynecologic cancers with metastasis to the ovary.

Table 1.

Total number of members of each race, further grouped into pathologic disease diagnosis and menopausal state.

Race	All Subjects		EOC		Other Cancers		Benign		Pre- Menopausal		Post- Menopausal
	N	%	N	%	N	%	N	%	N	%	N	%
White	168	91.3	60	98.4	15	75.0	93	90.3	54	85.7	114	94.2
Black	3	1.6	1	1.6	1	5.0	1	1.0	1	1.6	2	1.7
Hispanic	3	1.6	0	0.0	2	10.0	1	1.0	2	3.2	1	0.8
Other	8	4.3	0	0.0	1	5.0	7	6.8	6	9.5	2	1.7
Unknown	2	1.1	0	0.0	1	5.0	1	1.0	0	0.0	2	1.7
Total	184	100	61	33.2	20	10.9	103	56.0	63	34.2	121	65.8

Table 2.

Total number of premenopausal and postmenopausal subjects diagnosed with each ovarian cancer subtype.

	All Subjects		Pre- Menopausal		Post- Menopausal
Cancer	N	%	N	%	N	%
Epithelial Ovarian Cancer Total	61	75.3	12	54.5	49	83.1
Grade 1	13	21.3	4	33.3	9	18.4
Grade 2	5	8.2	2	16.7	3	6.1
Grade 3	43	70.5	6	50.0	37	75.5
Borderline / LMP Tumor	4	4.9	3	13.6	1	1.7
Non-Epithelial Ovarian Cancer	2	2.5	2	9.1	0	0.0
Other Gynecological Cancer	6	7.4	1	4.5	5	8.5
Metastatic Cancer	8	9.9	4	18.2	4	6.8
Total	81	100.0	22	27.2	59	72.8

Table 3.

Total number of premenopausal and postmenopausal subjects diagnosed with each stage of epithelial ovarian cancer (EOC).

	All Subjects		Pre- Menopausal		Post- Menopausal
EOC Stage	N	%	N	%	N	%
Stage I	19	31.7	4	33.3	15	31.3
Stage II	7	11.7	2	16.7	5	10.4
Stage III	33	55.0	6	50.0	27	56.3
Stage IV	0	0.0	0	0.0	0	0.0
Unstaged	1	1.7	0	0.0	1	2.1
Total	60	100.0	12	20	48	80.0

Forty-six unique combinations of possible biomarkers were examined with calculation of ROC-AUC and corresponding confidence intervals as well as p-values for comparison to ROMA ROC-AUC. The calculation of ROMA and each biomarker combination analysis also took menopausal status into consideration. Two different outcomes, EOC (n=61) vs. benign disease (n=103) and all cancers and LMP tumors (n=81) vs. benign tumors (n=103), were evaluated for each combination. Of the 46 unique biomarker combinations examined, only the four with highest ROC-AUCs are presented in this report. The individual performance of both CA125 and HE4 are also reported for comparison. The combination of HE4 and CA125 (i.e. ROMA) achieved an AUC of 91.2% (95% CI: 86.0–96.4%) for the detection of EOC vs benign disease. The combination of CA-125, HE4, YKL-40, transthyretin, ApoA1, Beta 2 microglobulin, transferrin, LPA and menopausal status achieved the highest AUC of 94.6% (95% CI: 90.1–99.2%), but this combination was not significantly better than the HE4 and CA125 combination (p=0.078). Table 4 displays ROC-AUC results for CA-125 alone, HE4 alone, ROMA, and the three additional biomarker combinations that achieved the highest predictive value for patients with EOC versus those with benign disease. The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and accuracy for ROMA as well as for the combination of CA-125, HE4, YKL-40, transthyretin, ApoA1, Beta 2 microglobulin, transferrin, LPA and menopausal status are provided for EOC versus benign disease in Table 5.

Table 4.

Diagnostic performance of biomarker combinations for differentiation of benign (N=103) vs EOC (N=61) compared to ROMA.

Marker combinations	ROC-AUC (95%CI)	p-value vs. ROMA
CA-125	86.6% (80.6–92.6%)	0.039
HE4	90.8% (85.7–95.8%)	0.671
ROMA (CA-125 + HE4 + Menopausal Status)	91.2% (86.0–96.4%)	–––
CA125 + Transthyretin + ApoA1 + Beta2 Microglobulin + Transferrin + Menopausal Status	89.7% (84.4–95.0%)	0.502
CA125 + HE4 + ApoA1 + Transferrin + Menopausal Status	93.2% (88.7–97.6%)	0.153
CA125 + HE4 + YKL-40 + Transthyretin + ApoA1 + Beta2 Microglobulin + Transferrin + LPA 14:0+ Menopausal Status	94.6% (90.1–99.2%)	0.078

Table 5.

Predictive statistics for ROMA and the combination of CA-125, HE4, YKL-40, transthyretin, ApoA1, Beta 2 microglobulin, transferrin, LPA and menopausal status (8 Marker Assay) for EOC versus benign disease at a specificity level of ~75%.

	ROMA (95% CI)	8 Marker Assay (95% CI)
Sensitivity	90.0% (79.5–96.2%)	94.0% (83.5–98.7%)
Specificity	76.7% (67.3–84.5%)	76.3% (66.4–84.5%)
PPV	69.2% (57.8–79.2%)	68.1% (55.8–78.8%)
NPV	92.9% (85.3–97.4%)	95.9% (88.6–99.2%)
Accuracy	81.6% (74.8–87.2%)	82.5% (75.3–88.4%)

The ROC-AUC values for the detection of all cancers and LMP vs benign tumors for these biomarker combinations are shown in Supplemental Table 1. ROMA achieved an AUC of 82.9% (95% CI: 76.3–89.6%). Again, the biomarker combination of CA-125, HE4, YKL-40, transthyretin, ApoA1, Beta 2 microglobulin, transferrin, LPA and menopausal status achieved the highest AUC of 86.8 (95% CI: 80.5–93.2%), but this was not significantly better than ROMA (p=0.076). Figure 1 shows the ROC curves for the four combinations of biomarkers and compares their ability to differentiate benign versus EOC (Curve A) and benign versus LMP tumors + all cancers (Curve B). The predictive statistics for ROMA as well as for the combination of CA-125, HE4, YKL-40, transthyretin, ApoA1, Beta 2 microglobulin, transferrin, LPA and menopausal status for all cancers including LMP tumors versus benign disease are provided in Supplemental Table 2.

Figure 1.

ROC-AUC curves showing performance of four biomarker combinations ability to distinguish benign disease versus EOC and benign versus LMP tumors + all cancers.

Discussion:

In the last decade, there have been major advances in biomarker research which have allowed for identification of a multitude of new serum-based tumor markers. One of the potential functions of these tumor markers is as a possible triage tool in women presenting with an adnexal mass. CA125 has been used for this purpose for many years and has also been used to monitor patients for the detection of progressive disease or for the detection of recurrence of disease in women that have been treated for ovarian cancer. CA125 as a single-marker assay to detect ovarian cancer has limitations due to the decreased sensitivity in early stage disease and decreased specificity due to the elevation of serum CA125 levels found in many benign gynecologic and non-gynecologic medical conditions. It has been demonstrated that the addition of HE4 to CA125 and the development of a dual-marker algorithm increases the sensitivity and specificity for detecting epithelial ovarian cancer in women with an ovarian cyst or pelvic mass including for early stage disease (10). Further studies have looked at the utility of adding additional biomarkers in hopes of creating an even more sensitive and specific clinical triage tool (16). Efforts over the last several years have focused on identifying new tumor-associated markers with aims to create a multi-marker test with superior performance. What is often missing is prospective data evaluating how these tests perform in the clinical setting. Our study used area under the receiver operator characteristic curves (AUC ROC) to examine the performance of several novel biomarkers that are currently being utilized clinically and that have been shown to have activity in differentiating a benign pelvic mass from a malignancy. While adding serum biomarkers to algorithms to detect malignancy often appears to increase sensitivity, this often comes at the expense of specificity for the test. With the addition of increasing numbers of biomarkers, there seems to be diminishing returns to predictive algorithms and the performance of these tests.

In the current study, we measured serum levels of eight different biomarkers in a large cohort of women with an ovarian cyst or pelvic mass prior to surgery. All patients underwent surgery and had a final pathologic diagnosis. We examined many different combinations of biomarkers and created ROC curves to determine the optimal multiple marker combination to predict whether an ovarian cyst or pelvic mass represented a malignancy. As in our previous studies and others, the current study found the dual marker combination of HE4 and CA125 (ROMA) to predict malignancy performed equally as well as any other biomarker combination (10, 16, 17). Although an eight-biomarker combination achieved a slightly higher AUC, the increase was not statistically significant. A similar study by Grenache et al comparing the biomarkers combination of HE4 and CA125 to the five-marker combination of CA125, transferrin, transthyretin, apolipoproteinA1 and beta-2-microglobulin also failed to find a significant increase in diagnostic sensitivity when HE4 was replaced by transferrin, transthyretin, beta 2 microglobulin, and ApoA1 (18). In addition, in this study, the dual marker combination (ROMA) was equally effective in predicting invasive EOC, LMP tumors and metastatic malignancy from benign disease when compared with all other biomarker combinations. At a specificity level of approximately 75%, the actual sensitivity and specificity of ROMA for detecting EOC versus benign disease were found to be 90.0% and 76.7%, respectively. When ROMA was used to discriminate between all cancers plus LMP tumors versus benign disease, the sensitivity and specificity decreased to 78.2% and 76.7%, respectively. Similar results were found when looking at the eight-biomarker combination plus menopausal status, where the sensitivity and specificity were found to be 94.0% and 76.3%, respectively, when looking at EOC versus benign disease. The performance of the eight-biomarker combination plus menopausal status decreased when used to discriminate between all cancers plus LMP versus benign disease (80.3% and 75.3%, respectively).

When examining the utility of a triage tool to differentiate benign pelvic masses from invasive EOC, LMP tumors or non-gynecologic malignancy with metastasis to the pelvis, we must consider the clinical impact of knowing this information. Identifying a pelvic mass as benign allows a patient to appropriately stay with her gynecologist for management. Therefore, a test with a high negative predictive value would be desired, resulting in a very low probability that a patient with a malignancy will be managed by a team not prepared for the finding of a malignancy at the time of surgery. On the other hand, the identification of patients with invasive EOC allows these women to be triaged and managed by gynecologic oncologists and teams familiar with the care and management of women with ovarian cancer, thus improving outcomes and survival (6, 19). The value of identifying LMP tumors preoperatively is less clear. Women ultimately diagnosed with LMP tumors do not need to undergo surgical staging or adjuvant chemotherapy. Therefore, the benefit of triage for these women to a gynecologic oncologist is less clear. For women diagnosed with non-gynecologic metastatic disease to the pelvis, outcomes and survival for these patients is not dependent on their management by a gynecologic oncologist as they already have stage 4 disease at the time of their presentation. It would seem the most critical function of a triage test would be to identify invasive EOC above all, as these patients have the most to gain from referral to a gynecologic oncologist, thus improving morbidity and survival for women with EOC.

Additional factors to consider include the cost of additional lab tests. Increasing the number of biomarkers used in an assay can significantly increase the cost of the test without creating a clinically meaningful increase in sensitivity or specificity. Caution must be exercised when adding tumor-associated, but not tumor-specific, markers to these panels. As an example, serum haptoglobin levels have been shown to be a marker that is significantly elevated in ovarian cancers (20, 21). However, haptoglobin is an acute phase reactant that is produced in the liver and is also elevated in conditions associated with inflammation, infection and other malignancies such as breast cancer (22). Another marker, transferrin, is a glycoprotein most well-known for a role in iron homeostasis where it delivers iron throughout the body. Test specificity is inevitably affected negatively by the addition of tumor-associated, but not specific biomarkers. The addition of increasing numbers of biomarkers in an algorithm to differentiate benign tumors from malignant tumors generally only serves to minimally increase sensitivity of the test while at the same time negatively affecting specificity greatly.

Limitations to the generalizability of this study include the relatively small number of minority women evaluated. Our study population was composed of 91.3% white, 1.6% black, and 1.6% Hispanic patients. This is consistent with white females being the most at-risk population for the development of EOC. The number of minority patients represented is somewhat less than reported in recent epidemiological data (2). Therefore, examining performance among more diverse populations would be helpful in increasing the generalizability of the study findings As well, this study may not reflect findings in the general population as the study cohort was from a tertiary care hospital.

While new biomarker identification continues to be an active area of research, additional areas of investigation may offer improved detection of malignant disease. The response of the immune system to malignant disease and our ability to detect genetic material from tumor cells in the bloodstream hold promise for future research (23–25). This study provides data showing that the six additional biomarkers studied are no more effective at clinically predicting patients with EOC than the dual marker combination of HE4 and CA125.

The present study did not find any value in adding additional biomarkers to CA125 and HE4 as used in the ROMA test. ROMA has been independently validated in multiple national and international multicenter trials to date and therefore further validation trials of ROMA should focus on the addition of novel biomarkers. The addition of immune markers and gene expression in combination with serum biomarkers show promise and are currently being investigated.

Highlights:

1. The combination of HE4 and CA125 is sensitive for predicting ovarian cancer.

2. Additional markers to HE4 and CA125 do not significantly improve the detection of ovarian cancer.

3. HE4 and CA125 are complimentary biomarkers and perform best when used in combination.