Assessing the risk of ovarian malignancy algorithm for the conservative management of women with a pelvic mass

Abstract

Objective

To evaluate the use of as an aid in the identification of women who can safely undergo conservative, non-surgical management.

Methods

All patients referred to the Program in Women’s Oncology for surgery with a pelvic mass are evaluated at a prospective multidisciplinary tumor board (TB) where ROMA and imaging are used for management recommendations. This study evaluated women presented to TB with a pelvic mass between 2009 and 2013 who had either surgical or conservative management.

Results

Of the 498 patients assessed, 392 (79%) had benign disease, 22 (4%) had LMP tumors, 28 (6%) had stage I-II epithelial ovarian cancer (EOC), 36 (7%) had stage III-IV EOC and 20 (4%) had non-EOC. Using clinical assessment in conjunction with ROMA, the TB recommended observation in 188 (37.8%) women. All patients diagnosed with an invasive malignancy were recommended for surgery by the TB. In the 315 patients managed surgically, 212 were found to have benign disease and 84 women were diagnosed with an invasive malignancy. The sensitivity for the initial TB recommendations using ROMA in conjunction with clinical judgment for detecting malignancy was 100% with a specificity of 47.7% and a NPV of 100%. When including low malignant potential tumors the sensitivity was 99.1%. For stage I-IV EOC ROMA alone had a sensitivity of 95.3%.

Conclusions

ROMA in conjunction with clinical assessment can safely identify women for conservative management.

1. Introduction

In the United States, approximately 289,000 women are hospitalized each year with an ovarian cyst or pelvic mass. Although the majority of these women will be diagnosed with benign disease, 5–10% will be diagnosed with an ovarian cancer [1,2]. Obtaining a biopsy is technically challenging and discouraged in patients with presumed early stage disease, as the risk for rupture and dissemination of potentially malignant cells is a concern. The most reliable way to rule out a malignancy is with surgery and pathologic examination, which carries significant risk for morbidity, costs and potentially unnecessary major surgery. It is estimated that, in the United States, women have a 5–10% lifetime risk of undergoing surgery for a suspected ovarian neoplasm [3]. Many of the procedures performed to evaluate a pelvic mass ultimately identify a benign process. In a large ovarian cancer screening trial, 20 cases (3.5%) of malignancy were found among 570 women who underwent surgical evaluation for suspected ovarian cancer [4]. Therefore, improving methods for distinguishing benign from malignant masses would avoid unnecessary surgeries.

Management guidelines put forth by the American College of Obstetricians and Gynecologists (ACOG) recommend that surveillance is appropriate for adnexal masses in asymptomatic premenopausal women with specific sonographic characteristics, and that in postmenopausal women, the combination of ultrasound and CA125 measurements should be used to guide decision making [5]. ACOG suggests that in postmenopausal women, most pelvic masses, with the exception of sonographically simple appearing cysts, will require surgical intervention, and that any elevation of CA125 is suspicious for malignancy [6, 7]. For the detection of malignancy, the use of imaging alone, including ultrasonography, CT and MRI provides a sensitivity in the range of 82–91% and the use of CA125 measurement alone has a sensitivity ranging from 78% [8,9].

Biomarkers or panels of biomarkers are generally used in combination with each other or with imaging and clinical findings to aid in the diagnosis of epithelial ovarian cancer (EOC). The Risk of Ovarian Malignancy Algorithm (ROMA) is a logistic regression algorithm that utilizes the serum biomarkers HE4 and CA125 along with menopausal status to assess the risk that an ovarian cyst or pelvic mass is benign or malignant [10,11]. ROMA has been validated to have a high sensitivity, specificity and negative predictive value in multiple multicenter prospective trials for predicting the presence of ovarian cancer in women with a pelvic mass. The combined HE4 and CA125 algorithm was found to be highly accurate in assigning patients to risk groups, with 95% of epithelial ovarian cancers correctly classified as high risk[11-13]. These trials led to USFDA clearance for ROMA as a test to assist in triaging high risk patients to a gynecologic oncologist and is becoming a tool that is increasingly being utilized by gynecologists and other physician specialists for pelvic mass risk assessment. However, ROMA has not yet been studied or approved as a guide for the conservative management of women with a pelvic mass.

The purpose of this study was to evaluate the use of ROMA in women with an adnexal mass to assist in identifying women at low risk for malignancy who can safely undergo conservative, non-surgical, management.

2. Materials and methods

This was a retrospective cohort study designed to evaluate the use of the ROMA along with clinical evaluation through a prospective tumor board process in women diagnosed with an ovarian cyst or pelvic mass. A secondary objective was to evaluate the performance characteristics of the ROMA in this patient population. This study was approved by the Women and Infants Hospital institutional review board.

At our institution, all patients with a pelvic mass referred to the Program in Women’s Oncology for surgery are presented prospectively to our multidisciplinary gynecologic oncology tumor board for determination of management recommendations. Women referred to the program between 2009 and 2013 were included in this evaluation. All imaging was prospectively reviewed at each TB by a radiologist specializing in women’s reproductive imaging. To be included in the study, each patient must have had the diagnosis of a pelvic mass and had their data reviewed prospectively at the TB including imaging (ultrasound and/or computerized tomography and/or magnetic resonance imaging), biomarkers (HE4 and CA125), have a known menopausal status and have either at least 3 months of follow-up and a minimum of two evaluations or have undergone surgery. For patients who underwent follow-up, the time interval for re-imaging was determined by the examining radiologist and in accordance with the society of radiologists’ consensus guidelines. A ROMA score was calculated for each patient. In premenopausal women, a ROMA score of <6% was considered low risk for ovarian malignancy. In post-menopausal women, a ROMA score of <10% was considered low risk for ovarian malignancy.

Patients who underwent conservative management as well as patients who underwent surgical management were evaluated. Patients who underwent conservative management with longitudinal follow-up of at least 3 months were evaluated for stability of their pelvic mass or for the subsequent diagnosis of malignancy. All patients who underwent conservative management as part of their initial TB recommendation were reviewed at TB on multiple occasions during the follow-up period. All patients who underwent surgical management were presented to TB prior to surgery and subsequently had a pathologic evaluation. Women with incomplete data and women who were pregnant at the time of diagnosis of their pelvic mass were excluded from analysis.

A list of women who met the inclusion criteria was obtained from the tumor board records. A chart review was performed using the patient’s medical records, the tumor board records and the pathology database to obtain demographic data (age, race, ethnicity, menopausal status) and initial HE4 and CA125 levels, imaging results including ultrasound, MRI and/or CT as well as the initial tumor board recommendation (surgery or conservative follow-up). For patients who underwent conservative management, follow-up data was collected, including subsequent tumor markers and imaging results as well as subsequent tumor board recommendations. For patients who underwent surgical management, pathologic data was collected, including histology and, if malignant, FIGO and TNM stage.

Serum CA125 levels were measured on the Immulite 2000 platform (Siemens, Los Angeles, CA, USA). Serum HE4 levels were measured using the Fujirebio HE4 EIA kit (Fujirebio Diagnostics Inc., Malvern, PA, USA). ROMA calculations were performed using the previously published algorithm with cut points determined in our laboratory for the Immulite CA125 and HE4 EIA combinations. The performance characteristics of ROMA were evaluated by determining the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and accuracy in both the conservative management and surgical management groups.

3. Results

A total of 507 patients diagnosed with a pelvic mass and presented to the tumor board were identified. Nine patients did not meet the inclusion criteria for the study and were excluded from the analyses, leaving 498 evaluable patients. In the evaluable patient set, there were 173 pre-menopausal (35%) and 325 post-menopausal (65%) women with a median age of 56 years (range = 15–90). There were 392 women with benign disease (78.7%) and 106 cancers, including 22 (4.4%) low malignant potential (LMP) tumors, 28 (5.6%) stage I–II EOC, 36 (7.2%) stage III–IV EOC, 9 (1.8%) non-epithelial ovarian cancers and 11 (2.3%) metastatic cancers to the ovaries. In this cohort, the incidence of all invasive cancer was 16.9%, with an incidence of EOC of 12.8% (Table 1).

Table 1.

Patient demographics.

	Premenopausal (N = 173)	Postmenopausal (N = 325)	All (N = 498)
Median age (years)	39.0	64.6	55.7
TB recommendation
Observe	62 (36%)	126 (39%)	188 (38%)
Surgery	111 (64%)	199 (61%)	310 (62%)
Actual management
Observe	55 (32%)	128 (39%)	183 (37%)
Surgery	118 (68%)	197 (61%)	315 (63%)
Final diagnosis
Benign	145 (84%)	247 (76%)	392 (79%)
LMP tumor	10 (6%)	12 (4%)	22 (4%)
EOC stage I–II	10 (6%)	18 (6%)	28 (6%)
EOC stage III–IV	2 (1%)	34 (10%)	36 (7%)
Non-EOC	5 (3%)	4 (1%)	9 (2%)
Metastatic cancer	1 (< 1%)	10 (3%)	11 (2%)

The initial tumor board recommendations took into account both ROMA and clinical assessment with imaging. There were a total of 392 women diagnosed with benign masses. Of the women presenting with a benign pelvic mass, 205 (52.3%) were recommended to undergo surgery and 187 (47.7%) were initially recommended to undergo observation. Of the 106 women diagnosed with a malignancy or LMP tumor, only 1 patient with an LMP tumor was recommended for observation while the remainder were recommended for surgery (Table 2). Overall, the sensitivity for the initial TB recommendations using ROMA in conjunction with clinical judgment for detecting malignancy including low malignant potential (LMP) tumors was 99.1% (95% CI: 94.9–100%), specificity was 47.7% (95% CI: 42.7–52.8%) and the NPV was 99.5% (95% CI: 97.1–100%) (Table 3). When LMP tumors were excluded as malignancies, the overall sensitivity was 100% (95% CI: 95.7–100%), specificity was 47.7% (95% CI: 42.7–52.8%) and the NPV was 100% (95% CI: 98.0–100%). Of the patients who underwent conservative management with follow-up imaging, 69.4% had >1 year of follow-up with 39.9% having >2 years of follow-up, 18.0% had 6–12 months of follow-up and 12.0% had <6 months of follow-up with only 4 patients having their only follow-up at 3 months.

Table 2.

Tumor board (TB) recommendation for patient management.

Final diagnosis all patients	Initial recommendation		Total
	Observe	Surgery
Benign	187	205	392
Borderline/LMP	1	21	22
Epithelial ovarian ca (I–II)	0	28	28
Epithelial ovarian ca (III–IV)	0	36	36
Non-epithelial ovarian ca	0	9	9
Metastatic cancer	0	11	11
Total	188	310	498

Table 3.

Statistical parameters for tumor board (TB) recommendation for patient management (all malignancies and LMP tumors).

	Pre-menopausal	Post-menopausal	All
Sensitivity	100.0%	98.7%	99.1%
Specificity	42.8%	50.6%	47.7%
PPV	25.2%	38.7%	33.9%
NPV	100.0%	99.2%	99.5%
FPR	57.2%	49.4%	52.3%
FNR	0.0%	1.3%	0.9%

Actual patient management differed slightly from initial tumor board recommendations because other factors not considered by the TB were taken into account by the patient and physician. These factors included physician and or patient individual choice for management, symptoms that may not have been known by the tumor board, and comorbid medical conditions that required a delay for surgery. A total of 183 (36.7%) women underwent conservative management with observation and 315 (63.3%) underwent surgery. Of the 183 patients who underwent conservative management, 180 (98.4%) had benign conditions. The remaining 3 patients ultimately underwent surgical management, of which 2 (1.1%) were diagnosed with LMP tumors and 1 (0.5%) with a stage IA, grade 1 mucinous ovarian cancer (Table 4). Actual patient management, when examining all patients (benign, malignant and LMP tumors), yielded a sensitivity of 97.2% (95% CI: 92.0–99.4%), specificity of 45.9% (95% CI: 40.9-51.0%) and NPV of 98.4% (95% CI: 95.3-99.7). Examination of actual patient management in those with invasive malignancies alone (i.e. excluding LMP tumors) yielded a sensitivity of 98.8% (95% CI: 93.5–100%), specificity of 45.9% (95% CI: 40.9–51.0%) and NPV of 99.4% (95% CI: 97.0–100%).

Table 4.

Actual patient management as determined by patient and physician.

Final diagnosis all patients	Actual patient Management		Total
	Observe	Surgery
Benign	180	212	392
Borderline/LMP	2	20	22
Epithelial Ovarian ca (I–II)	1	27	28
Epithelial ovarian ca (III–IV)	0	36	36
Non-epithelial ovarian ca	0	9	9
Metastatic cancer	0	11	11
Total	183	315	498

A secondary analysis of the performance of ROMA as a stand-alone test without the input of clinical assessment was also performed. Overall, ROMA classified 274 (55.0%) of the patients as low risk and 224 (45.0%) as high risk. ROMA classified 10 LMP tumors, 3 Stage I-II EOC, 3 non-epithelial ovarian cancer and 1 metastatic cancer to the ovary as low risk and 135 benign conditions as high risk. Considering all patients, ROMA alone resulted in a sensitivity of 84.0 (95% CI: 75.6–90.4%), specificity of 65.6% (95% CI: 60.6–70.3%) and NPV of 93.8% (95% CI: 90.3–96.3%). When considering the ability of ROMA to detect epithelial ovarian cancer only (i.e. excluding LMP tumors and non-EOC cancers), ROMA achieved a sensitivity of 95.3% (95% CI: 86.9–99.0%), specificity of 65.6% (95% CI: 60.6–70.3%) and NPV of 98.8% (95% CI: 96.7–99.8%) (Table 5). For stage I–II EOC, ROMA had a sensitivity of 89.3% (95% CI: 71.8–97.7%), a specificity of 65.6% (95% CI: 60.6–70.3%) and a NPV of 98.8% (95% CI: 96.7–99.8%).

Table 5.

Statistical parameters for the detection of EOC.

	ROMA alone			ROMA + tumor board
	Pre-menopausal	Post-menopausal	All	Pre-menopausal	Post-menopausal	All
Sensitivity	83.3%	98.1%	95.3%	100.0%	100.0%	100.0%
Specificity	77.9%	58.3%	65.6%	42.8%	50.6%	47.7%
PPV	23.8%	33.1%	31.1%	17.8%	35.1%	29.1%
NPV	98.3%	99.3%	98.8%	100.0%	100.0%	100.0%
FPR	22.1%	35.5%	34.4%	57.2%	49.4%	52.3%
FNR	16.7%	1.9%	4.7%	0.0%	0.0%	0.0%

Overall, based on TB recommendation, all 84 malignancies, including 28 early stage EOC patients, had recommendations for surgery. Only 1 of 22 patients with a LMP tumor was assigned to observation. Clinical assessment in conjunction with ROMA was able to recommend conservative management in 187 (37.4%) women who otherwise would have undergone surgery.

4. Discussion

In this study we identified a cohort of 498 women who were referred for surgical management of their pelvic mass or ovarian cyst. Through a prospective, multidisciplinary, gynecologic oncology tumor board utilizing ROMA and clinical assessment, these patients were re-evaluated as to the need for surgery, with their risk for having a malignancy (ROMA score) being taken into account. Over one third of these women were determined to be low risk and considered appropriate for conservative, non-surgical management and safely had their adnexal masses followed with imaging and biomarkers, avoiding a surgery.

Recently, two independent tests, ROMA and the multivariate index assay (MVI assay/Ova-1), have been developed to evaluate the risk of malignancy in women who are planning to undergo surgery for a pelvic mass. Both of these assays have been cleared by the US FDA for use in this patient population. Ueland et al. reported on the MVI to assess the likelihood of malignancy in women who were planning to undergo surgery for an adnexal mass. The MVI assay incorporates five proteins that are variably expressed in ovarian cancer, where CA125 and beta 2 microglobulin are upregulated and transferrin, transthyretin and apolipoprotein A1 are down regulated [14]. A computerized algorithm uses these values in conjunction with menopausal status to generate an ovarian malignancy score. In a prospective study, the MVI assay had an overall sensitivity for detection of ovarian cancer of 92–94%, ranging from 91 to 96% in premenopausal women and 100% in postmenopausal women. Specificity was 35–43% overall and ranged from 43 to 52% in premenopausal women and 33% in postmenopausal women [14,15].

ROMA has been evaluated in multiple prospective trials. In a prospective study of 531 women with a pelvic mass who were scheduled to have surgery, ROMA achieved a sensitivity of 92.3% in postmenopausal women and 76.5% in premenopausal women, successfully classifying 93.8% of EOC as high risk [11]. A second prospective trial of 472 patients showed that ROMA had a sensitivity for detecting ovarian cancer of 92.3% in post-menopausal women and 100% in premenopausal women, with an overall sensitivity of 93.8% and NPV of 99.0% [12]. There have also been multiple clinical trials worldwide validating ROMA as a diagnostic tool in differentiating benign and malignant adnexal masses [16-20].

More recently, the data submitted to the US FDA was reported in a publication by Moore et al., where ROMA was used in conjunction with the initial clinical risk assessment (ICRA) performed by generalist physicians in women who were already planning to undergo surgery for their pelvic mass. The ICRA included a combination of physical examination, serum CA125 measurement and imaging. HE4 and ROMA were not available to the clinicians for use in the ICRA. In this prospective, multi-institutional study, ICRA had a sensitivity of 85.4% and NPV of 97.8%. The addition of ROMA improved the sensitivity to 93.8% with a NPV to 98.8%. In this study, the addition of ROMA allowed for the detection of an additional 13 malignancies initially missed by ICRA alone [13]. Although the current study is composed of a somewhat different population of women, in that over a third of patients were allocated to conservative management and did not undergo surgery, the results are still in line with the previous trials. Compared to the results observed in the Moore et al. study, ROMA alone achieved a similar specificity and sensitivity of 65% and 95%, respectively.

The Risk of Malignancy Index (RMI) combines serum CA125, pelvic ultrasound and menopausal status into a score to predict the risk of ovarian cancer in women with an adnexal mass [21]. There have been multiple iterations of RMI (RMI I–IV) which have been evaluated for their ability to predict ovarian cancer, with resulting sensitivities as high as 78–79% and specificities of 81–89% [22,23]. RMI was compared to ROMA for prediction of epithelial ovarian cancer in patients with a pelvic mass in a prospective multicenter trial [24]. In that study, ROMA had a sensitivity of 94.3% compared to a sensitivity of 84.6% for the RMI to distinguish a benign mass from EOC. Additionally, ROMA had a sensitivity of 85.3% in patients with early stage disease compared with 64.7% for the RMI. The RMI algorithm demonstrates that the addition of biomarkers (CA125) to imaging improves the sensitivity for the detection of EOC. Likewise, in the current study, the use of ROMA along with imaging and clinical assessment provided a high sensitivity, resulting in no invasive EOC being assigned to the low risk group.

Previously, ROMA and the MVI assay have been studied only in women who have already been evaluated and for whom surgery is planned, and thus neither test is approved to decide whether to proceed with surgery. This study is the first to use biomarkers in an attempt to define a population of women for whom surgical intervention for their adnexal mass is unnecessary.

The use of ROMA, particularly in premenopausal women, where the incidence of EOC is low and benign disease is high, is valuable. In premenopausal women, the weight assigned to HE4 in the ROMA calculation is much greater than that given to CA125, as HE4 has greater specificity than CA125 in this population. Serum CA125 levels have been shown to be elevated in a number of benign gynecologic tumors and conditions, such as serous cyst adenomas, endometriosis and pelvic inflammatory disease. In 159 pre-menopausal women diagnosed with endometriosis, HE4 was elevated in only 3% of the cases compared with 72% for CA125. Similarly, when considering all benign tumors, serum HE4 levels were elevated in 6% of cases compared to 37% for CA125. In post-menopausal women with benign disease, similar results are seen, with serum HE4 levels being elevated in 12% of women compared to 20% for CA125 [25]. HE4 has a similar sensitivity for EOC as does CA125 but is more often elevated in early stage EOC cases. The increased specificity of HE4 over that of CA125 along with the complimentary sensitivity of HE4 to CA125 makes the dual biomarker combination more effective than either biomarker alone [10,26]. However, caution should be taken when using ROMA and/or HE4 in patients over the age of 80 years of age or in those with renal impairment, as serum HE4 levels tend to be higher in these patients [27].

The current study utilized a large cohort of patients referred for surgical management of an adnexal mass. The strengths of this study are that all patients were evaluated through a multidisciplinary prospective TB, with imaging being reviewed by radiologists and all biomarker results obtained prior to any intervention. All pathology was reviewed by a gynecologic pathologist and TB recommendations were obtained through a consensus opinion. The weaknesses of this study that place some limitations on the findings include the collection of the TB recommendations in a retrospective fashion. Additionally, no standard imaging modality or protocol was utilized, and many patients had a combination of ultrasound, CT scan and MRI. Also, comorbidity data is lacking and there were certainly patients who did or did not undergo surgery due to presence of comorbidities. Women were also initially referred to the gynecologic oncology clinic and thus may have already been at slightly higher risk of malignancy, however, the incidence of EOC in this study is similar to other low risk studies. However, this was a large database of over 500 patients with long term follow-up. It is also a real world snap shot of women with adnexal masses as they would present and be triaged outside of a randomized controlled trial population, which adds validity to the study. Bringing further validation to this trial is the fact that the performance of ROMA as a stand-alone test achieved a similar performance when compared with the original ROMA validation trials. Equally important is the fact that when comparing the TB recommendations using ROMA and clinical evaluation in the current trial to the performance characteristics of the Initial Clinical Risk Assessment (ICRA) reported in the ROMA FDA submission and the recently reported ICRA (using ROMA and clinical assessment) by Moore et al., the performance characteristics of these trials were similar[10-13]. These similarities support the validity of the findings of the current study given that this cohort study truly represents a patient population of women presenting with a pelvic mass in real world clinical setting.

While these findings are compelling, this study was performed in a large academic institution where the tumor board includes multiple gynecologic oncologists, gynecologic pathologists and radiologists specializing in women’s imaging. Additionally, as this was a retrospective cohort, not all women were compared against the gold standard of a pathologic diagnosis. Therefore, prior to using this algorithm in a broader community setting, further evaluation in a prospective trial would be warranted.

Overall, 37% of the women who would have undergone surgery for a benign mass were spared a surgical procedure and the potential morbidities resulting from an unnecessary surgery. Importantly, no invasive malignancies were assigned to the observation group through the TB recommendations, resulting in all women with a cancer undergoing surgery. From a patient perspective, this data provides reassurancethat non-surgical management with observation of a low risk pelvic mass is safe.

HIGHLIGHTS.

• ROMA with clinical assessment identifies women with a who can be managed conservatively 1/3 of women initially referred for surgery were able to undergo conservative management.

• ROMA helps triage women with a pelvic mass for non-sugical follow up.