Abstract
Aim
This study compared the diagnostic performance of the Risk of Ovarian Malignancy Algorithm (ROMA) and HE4 and CA125 for the presurgical differentiation of adnexal tumors.
Material and Methods
This prospective study included 302 patients admitted for surgical treatment due to adnexal tumors. The ROMA was calculated depending on CA125, HE4, and menopausal status.
Results
Fifty patients were diagnosed with malignant disease. In the differentiation of malignant from nonmalignant adnexal tumors, the area under curve (AUC) was higher for ROMA and HE4 than that for CA125 in both the premenopausal and postmenopausal subgroups. In the differentiation of stage I FIGO malignancies and epithelial ovarian cancer from nonmalignant pathologies, the AUC of HE4 and ROMA was higher than that of CA125. The ROMA performed significantly better than CA125 in the differentiation of all malignancies and differentiation of stage I FIGO malignancies from nonmalignant pathologies (p = 0.043 and p = 0.025, resp.). There were no significant differences between the ROMA and the tumor markers for any other variants.
Conclusions
The ROMA is more useful than CA125 for the differentiation of malignant (including stage I FIGO) from nonmalignant adnexal tumors. It is also as useful as HE4 and CA125 for the differentiation of epithelial ovarian cancer from nonmalignant adnexal tumors.
1. Introduction
Adnexal tumors represent a wide variety of diseases that may affect the ovaries and/or fallopian tubes. Tumors of the adjacent structures, such as uterine fibroids, can mimic adnexal tumors. Ovarian tumors can be functional, benign, or malignant. Ovarian malignancies can be primary or secondary, with primary tumors originating from epithelial cells, sex cords, or germinal cells [1, 2]. The heterogeneous nature of adnexal masses is one of the causes of preoperative difficulties in these tumors [3, 4]. Ovarian cancer (OC) is the fifth most common malignancy among women (5% of all cancers) and the fourth most common cause of mortality related to malignancy in Poland [5].
The tumor marker CA125, initially described by Bast et al., is widely used for the routine diagnosis of adnexal masses [6]. It is also used for monitoring the response to treatment, follow-up of the disease, and detection of disease recurrence [7]. This tumor marker can be increased in several gynecological and nongynecological diseases, and this reduces the diagnostic accuracy for the detection of ovarian cancer [8–11].
Endometriosis is a prominent cause of increased CA125 [12]. In 1991, Kirchoff et al. identified a major human epididymis-specific cDNA that encodes a protein with sequence homology to extracellular proteinase inhibitors. Northern blot and in situ transcript hybridization specifically localized the HE4 (human epididymis gene product) mRNA to the distal section of epithelial cells in the epididymal duct [13]. Subsequent studies have shown that HE4 is elevated in 90% of serous ovarian cancer cases and in most cases of endometrioid and clear cell cancer, whereas mucinous and germ cell tumors rarely express HE4 [14]. The marker HE4 is significantly increased in ovarian and endometrial cancer, but not in cases of endometriosis [12]; furthermore, it is less frequently elevated compared to CA125 in patients with benign disease, especially in premenopausal patients [15]. HE4 can be increased in nongynecological malignancies [16].
Several different mathematical models and scoring systems have been created, based on clinical features, ultrasound findings, and/or serum level of tumor markers, aimed at increasing the diagnostic performance of each individual parameter [3]. One such model is the Risk of Ovarian Malignancy Algorithm (ROMA) created by Moore et al. The ROMA combines the tumor markers CA125 and HE4 using two formulas, taking into account the menopausal status of each patient. The ROMA can classify patients as being at low and high risks for epithelial ovarian cancer (EOC), and 93.8% of cases in Moore et al.'s study were correctly classified under the high-risk category [17]. In 2010, Moore et al. concluded that ROMA achieved higher sensitivity than the risk of malignancy index (RMI) for identifying EOC in a prospective multicenter trial in 457 patients. The authors suggested that radiological imaging studies without central review may more accurately reflect actual clinical practice. By contrast, the serum levels of tumor markers provide objective results that showed more utility and more consistency and reproducibility between centers and between regions [18]. The analysis by Nolen et al. reaffirmed the superiority of assessing a combination of HE4/CA125 for the diagnosis of OC [19]. In 2011, the use of ROMA was validated in a low-risk population of women with adnexal masses who presented to a general practitioner. Despite the low incidence of malignancies in this trial (15% of all cases and 10% for EOC), the ROMA stratified patients into high- and low-risk groups, with 93.8% sensitivity and 74.9% specificity for predicting OC [20].
A meta-analysis by Li et al. in 2012 analyzed the performance of HE4, CA125, and ROMA in 11 studies and data from 7792 tests. The authors concluded that HE4 was no better than CA125 for either EOC or OC prediction, whereas ROMA was a promising predictor of EOC that could replace CA125. The overall estimates of ROMA for EOC prediction were as follows: a sensitivity of 89% (95% CI: 84%–93%), specificity of 83% (95% CI: 77%–88%), and AUC of 0.93 (95% CI: 0.90–0.95). However, the authors concluded that ROMA utilization requires further evaluation [21].
A meta-analysis by Dayyani et al. in 2016 analyzed five studies incorporating 1975 patients with adnexal masses. On the basis of the AUC (95% confidence interval) data for all patients, the authors concluded that the ROMA (0.921 [0.855–0.960]) showed a numerically greater diagnostic performance than CA125 (0.883 [0.771–0.950]) and HE4 (0.899 [0.835–0.943]). Similar results were shown in each of the subgroup populations, in particular, postmenopausal patients and patients with early OC. The meta-analysis had strict selection criteria for inclusion [22]. Several studies compared ROMA with CA125 and HE4 with contradicting results, which can be attributed to the subgroups analyzed, populations studied, oncology profile of the investigating center, cut-off levels of diagnostic tests, and choice of certain ovarian pathologies [23–26].
The optimum diagnosis of the malignant status of masses is important as it facilitates the selection of patients with malignant masses who need urgent referral to gynecological oncology centers and consequently improves the overall survival rate for patients with ovarian cancer [27].
The aim of the study was to compare the diagnostic performance of ROMA with tumor markers HE4 and CA125 in a selected Polish population. The comparison was performed for all patients, as well as for premenopausal and postmenopausal subgroups. In addition, the diagnostic value of ROMA was compared with that of tumor markers for differentiating between malignant adnexal tumor stage I according to FIGO and nonmalignant adnexal tumors and the differentiation of EOC from other nonmalignant adnexal tumors.
2. Material and Methods
This was a prospective study of 302 patients with adnexal masses referred to our clinic for surgery between October 2012 and April 2015. Patients were referred from physicians with different experience levels. The referral of patients was considered at outpatient units according to the local recommendations of these units depending on history, clinical examination, tumor markers, and ultrasound examination. The following inclusion criteria were used: age older than 18 years, measurement of serum concentration of tumor markers CA125 and HE4 less than five days before surgical intervention, histopathological results for the adnexal lesion, and obtainment of consent. Exclusion criteria included pregnancy, renal diseases, history of malignancy, chemotherapy and/or radiotherapy, fibroids > 5 cm, and a lack of histological assessment of the adnexal tumor. Serum HE4 and CA125 levels were measured for each patient at the same time with the same apparatus (Cobas 8000-e602), using an electrochemiluminescence immunoassay. The two logistic regression formulas described by Moore et al. were used to calculate ROMA. These formulas include a natural logarithm (ln) of CA125 and HE4 values. The predictive index (PI) was calculated for the premenopausal and postmenopausal subgroups. For the premenopausal subgroup, the following formula was used: predictive index (PI) = −12.0 + 2.38∗ln(HE4) + 0.0626∗ln(CA125). The formula for the postmenopausal subgroup was as follows: predictive index (PI) = −8.09 + 1.04∗ln(HE4) + 0.732∗ln(CA125). The following formula was applied to calculate the risk of malignancy based on the ROMA (%): (%) = exp(PI)/[1/exp(PI)]∗100 [17]. The cut-off level recommended by the manufacturer for CA125 was 35 U/mL, while the cut-off level for HE4 was 70 and 140 pmol/L for premenopausal and postmenopausal patients, respectively. The ROMA cut-off levels for high-risk patients were 11.4% and 29.9% for premenopausal and postmenopausal patients, respectively. The final decision for surgery was made individually by at least two gynecologists depending on the classical risk of malignancy index, tumor markers levels, and subjective assessment of adnexal tumors considering the patient's preference. The definitive diagnosis of the adnexal mass was established by the histopathological examination of the adnexal mass. Borderline tumors were considered as malignant in the statistical analysis. Malignant masses were staged according to the International Federation of Gynecology and Obstetrics (FIGO) guidelines. Menopause was defined as at least one year of absence of menstruation [17]. Descriptive analysis was used for patients with different adnexal pathologies. The Mann–Whitney U test was used to assess the statistical difference between mean serum levels of HE4 and CA125. The sensitivity, specificity, positive and negative predictive values, and accuracy of tumor markers and ROMA were calculated to distinguish different adnexal pathologies among different groups of patients. The receiver operating characteristics area under the curve (ROC-AUC) was constructed for each diagnostic test. The AUC of these tests were compared to each other using Hanley and McNeil methods. A p value less than 0.05 was assumed to be statistically significant. The study protocol was approved by the local ethical committee (Nr KB/192/2012).
3. Results
A total of 302 patients were included in the study. The patients were aged 18–85 years with a mean of 48.7 years and a standard deviation (SD) of 16.79 years. Premenopausal patients comprised the majority of patients (n = 188 [62.3%]) and only 114 (37.7%) patients were postmenopausal. Final histopathological examinations revealed 252 (83.4%) cases of nonmalignant and 50 (16.6%) cases of malignant adnexal pathologies. The vast majority (n = 48 [96%]) of malignant tumors were of ovarian origin; two were fallopian tube malignancies. The number of patients with each FIGO stage of malignant ovarian pathologies was as follows: stage IA, 9; IC, 6; IIA, 4; IIC, 1; IIIA, 1; IIIB, 2; IIIC, 24; and IVB, 1. Most malignant ovarian cases were of epithelial origin (n = 46). Tubal malignancies included one case of stage IC and one of IIIA. The distribution of final histological diagnoses of adnexal masses is presented in Table 1.
Table 1.
The distribution of final histological diagnoses of adnexal masses.
| Main adnexal type | Histological subtype | N (%) |
|---|---|---|
| Nonmalignant n = 252 | Endometriotic cyst | 56 (22.2%) |
| Dermoid cyst | 54 (21.4%) | |
| Simple cyst | 52 (20.6%) | |
| Serous cystadenoma | 41 (16.3%) | |
| Mucinous cystadenoma | 19 (7.5%) | |
| Tubo-ovarian abscess/salpingitis | 16 (6.3%) | |
| Paraductal cyst | 8 (3.2%) | |
| Ovarian fibroma | 6 (2.4%) | |
|
| ||
| Malignant n = 50 | Ovarian serous tumor | 22 (44%) |
| Ovarian endometrioid tumor | 11 (22%) | |
| Ovarian serous borderline | 5 (10%) | |
| Ovarian mucinous tumors | 3 (6%) | |
| Ovarian clear cell tumor | 3 (6%) | |
| Ovarian mucinous borderline tumors | 2 (4%) | |
| Fallopian tube malignancy | 2 (4%) | |
| Ovarian folliculoma | 1 (2%) | |
| Ovarian sarcoma | 1 (2%) | |
The serum level of HE4 and CA125 among the whole group, the premenopausal subgroup, and postmenopausal subgroup is presented in Table 2. The Mann–Whitney U test showed that both tumor markers showed significantly higher serum levels in patients with malignant adnexal masses.
Table 2.
Difference in serum tumor markers among groups by the Mann–Whitney U test.
| Group studied | Tumor marker | Mean serum tumor marker levels among nonmalignant adnexal masses | Mean serum tumor marker levels among malignant adnexal masses | p value |
|---|---|---|---|---|
| Whole group (n = 302) | HE4 (pmol/L) | 53.3 | 1138.8 | <0.001 |
| CA125 (U/mL) | 41.8 | 588.3 | <0.001 | |
| Premenopausal subgroup (n = 188) | HE4 (pmol/L) | 46.6 | 673.6 | <0.001 |
| CA125 (U/mL) | 48.1 | 1138.8 | 0.008 | |
| Postmenopausal subgroup (n = 114) | HE4 (pmol/L) | 69.3 | 1255.1 | <0.001 |
| CA125 (U/mL) | 26.5 | 450.7 | <0.001 |
The sensitivity, specificity, positive and negative predictive values, and the accuracy of HE4, CA125, and ROMA considering menopausal status are presented in Table 3. The diagnostic performance of these tests for differentiation of stage I FIGO malignant adnexal tumors and EOC from nonmalignant adnexal tumors is displayed in Table 4.
Table 3.
The sensitivity, specificity, positive predictive value, negative predictive value, and diagnostic accuracy of H4, CA125, and ROMA according to menopausal status.
| Group | Diagnostic test | Sensitivity (%) (95% CI) | Specificity (%) (95% CI) | Positive predictive value (%) (95% CI) | Negative predictive value (%) (95% CI) | Diagnostic accuracy (%) (95% CI) |
|---|---|---|---|---|---|---|
| Whole group (n = 302) | HE4 | 70% (61%–79%) | 92.5% (89.2%–95.7%) | 78.7% (70.1%–87.2%) | 88.6% (84.8%–92.4%) | 86.1% (82.5%–89.7%) |
| CA125 | 82% (71.4%–92.6%) | 68.3% (62.5%–74%) | 33.9% (25.5%–42.3%) | 95% (91.9%–98.2%) | 70.5% (65.4%–75.7%) | |
| ROMA | 80% (68.9%–91.1%) | 82.5% (77.9%–87.2%) | 47.6% (36.9%–58.3%) | 95.4% (92.6%–98.2%) | 82.1% (77.8%–86.4%) | |
| Premenopausal subgroup (n = 188) | HE4 | 70% (41.6%–98.4%) | 92.7% (88.9%–96.5%) | 35% (14.1%–55.9%) | 98.2% (96.2%–100%) | 91.5% (87.5%–95.5%) |
| CA125 | 80% (55.2%–100%) | 62.4% (55.2%–69.5%) | 10.7% (3.7%–17.7%) | 98.2% (95.8%–100%) | 63.3% (56.4%–70.2%) | |
| ROMA | 70% (41.6%–98.4%) | 82% (76.4%–87.7%) | 17.9% (5.9%–30%) | 98% (95.7%–100%) | 81.4% (75.8%–86.9%) | |
| Postmenopausal subgroup (n = 114) | HE4 | 70% (55.8%–84.2%) | 91.9% (85.7%–98.1%) | 82.4% (69.5%–95.2%) | 85% (77.2%–92.8%) | 84.2% (77.5%–90.9%) |
| CA125 | 82.5% (70.7%–94.3%) | 82.4% (73.8%–91.1%) | 71.7% (58.7%–84.8%) | 89.7% (82.5%–96.9%) | 82.5% (75.5%–89.4%) | |
| ROMA | 82.5% (70.7%–94.3%) | 83.8% (75.4%–92.2%) | 73.3% (60.4%–86.3%) | 89.9% (82.7%–97%) | 83.3% (76.5%–90.2%) |
CI: confidence interval; ROMA: Risk of Ovarian Malignancy Algorithm.
Table 4.
Diagnostic performance of HE4, CA125, and ROMA for discriminating stage I FIGO malignant adnexal tumors and epithelial ovarian cancer from nonmalignant adnexal tumors.
| Groups assessed by diagnostic tests | Diagnostic test | Sensitivity (%) (95% CI) | Specificity (%) (95% CI) | Positive predictive value (%) (95% CI) | Negative predictive value (%) (95% CI) | Diagnostic accuracy (%) (95% CI) |
|---|---|---|---|---|---|---|
| Stage I FIGO malignant adnexal tumor (n = 16) versus nonmalignant adnexal tumors (n = 252) | HE4 | 31.3% (8.5%–54%) | 92.5% (89.2%–95.7%) | 20.8% (4.6%–37.1%) | 95.5% (92.9%–98.1%) | 88.8% (85%–92.6%) |
| CA125 | 43.8% (19.4%–68.1%) | 68.3% (62.5%–74%) | 8% (2.3%–13.8%) | 95% (91.9%–98.2%) | 66.8% (61.2%–72.4%) | |
| ROMA | 43.8% (19.4%–68.1%) | 82.5% (77.9%–87.2%) | 13.7% (4.3%–23.2%) | 95.9% (93.2%–98.5%) | 80.2% (75.5%–85%) | |
| Epithelial ovarian cancer (n = 46) versus nonmalignant adnexal tumors (n = 252) | HE4 | 76.1% (63.8%–88.4%) | 92.5% (89.2%–95.7%) | 64.8% (52%–77.6%) | 95.5% (92.9%–98.1%) | 89.9% (86.5%–93.3%) |
| CA125 | 84.8% (74.4%–95.2%) | 68.3% (62.5%–74%) | 32.8% (24.3%–41.2%) | 96.1% (93.3%–98.9%) | 70.8% (65.6%–76%) | |
| ROMA | 82.6% (71.7%–93.6%) | 82.5% (77.9%–87.2%) | 46.3% (35.5%–57.1%) | 96.3% (93.8%–98.8%) | 82.6% (78.2%–86.9%) |
FIGO: Fédération Internationale de Gynécologie et d'Obstétrique; CI: confidence interval; ROMA: Risk of Ovarian Malignancy Algorithm.
A ROC-AUC was computed for tumor markers and ROMA for the whole group, as well as for the premenopausal and postmenopausal subgroups. All diagnostic tests significantly differentiated malignant adnexal tumors from nonmalignant adnexal tumors in the analysis of the whole group, as well as analysis of the postmenopausal and premenopausal subgroups. A ROC-AUC was also constructed for the tumor markers and ROMA to assess their performance for differentiation between stage I FIGO malignant adnexal tumors and EOC from nonmalignant adnexal tumors. Both HE4 and ROMA were significantly better than CA125 in differentiating stage I FIGO malignant tumors from nonmalignant adnexal tumors. Both tumor markers and ROMA were able to differentiate epithelial ovarian cancer from nonmalignant adnexal tumors.
The HE4, CA125, and ROMA AUCs, as well as the statistical differences and optimal cut-offs for the whole group and the premenopausal and postmenopausal subgroups are presented in Table 5. The tumor markers and ROMA AUCs for the differentiation of stage I FIGO malignant adnexal tumors and EOC from nonmalignant adnexal tumors are shown in Table 6. The ROC-AUCs for HE4, CA125, and ROMA for the whole group, the premenopausal subgroup, and the postmenopausal subgroup are presented in Figures 1(a)–1(c), respectively. The ROC-AUCs for HE4, CA125, and ROMA for the differentiation of stage I FIGO malignant tumors from nonmalignant adnexal tumors are shown in Figure 2(a). The ROC-AUCs of these tests for the differentiation of EOC from nonmalignant adnexal tumors are shown in Figure 2(b).
Table 5.
The AUCs with statistical differences and optimal cut-offs for tumor markers and ROMA in the presurgical differentiation of adnexal tumors in the whole group and premenopausal and postmenopausal subgroups.
| Diagnostic test | Whole group | Premenopausal subgroup | Postmenopausal subgroup | ||||||
|---|---|---|---|---|---|---|---|---|---|
| AUC (95% CI) | p value | Optimal cut-off | AUC (95% CI) | p value | Optimal cut-off | AUC (95% CI) | p value | Optimal cut-off | |
| HE4 | 0.928 (0.885–0.971) | <0.001 | 72.4 | 0.867 (0.739–0.996) | <0.001 | 54 | 0.915 (0.857–0.972) | <0.001 | 92 |
| CA125 | 0.838 (0.76–0.917) | <0.001 | 51 | 0.749 (0.548–0.950) | 0.008 | 42.7 | 0.891 (0.816–0.966) | <0.001 | 35 |
| ROMA | 0.928 (0.883–0.972) | <0.001 | 18.2 | 0.865 (0.729–1.000) | <0.001 | 8.8 | 0.917 (0.857–0.977) | <0.001 | 29.5 |
AUC: area under the curve; CI: confidence interval; ROMA: Risk of Ovarian Malignancy Algorithm.
Table 6.
AUCs with statistical differences and optimal cut-offs of tumor markers and ROMA for the differentiation of stage I FIGO malignant adnexal tumors and epithelial ovarian cancer from nonmalignant adnexal tumors.
| Diagnostic test | Stage I FIGO malignant adnexal tumors versus nonmalignant adnexal tumors | Epithelial ovarian cancer versus nonmalignant adnexal tumors | ||||
|---|---|---|---|---|---|---|
| AUC (95% CI) | p value | Optimal cut-off | AUC (95% CI) | p value | Optimal cut-off | |
| HE4 | 0.802 (0.695–0.910) | <0.001 | 56 | 0.928 (0.882–0.974) | <0.001 | 72.1 |
| CA125 | 0.559 (0.388–0.731) | 0.427 | — | 0.858 (0.781–0.935) | <0.001 | 54.4 |
| ROMA | 0.789 (0.679–0.898) | <0.001 | 10.5 | 0.929 (0.882–0.976) | <0.001 | 18.3 |
AUC: area under the curve; FIGO: Fédération Internationale de Gynécologie et d'Obstétrique; CI: confidence interval; ROMA: Risk of Ovarian Malignancy Algorithm.
Figure 1.
The ROC-AUC for HE4, CA125, and ROMA for the differentiation between malignant and nonmalignant adnexal masses in the whole group (a), premenopausal subgroup (b), and postmenopausal subgroup (c).
Figure 2.
(a) The ROC-AUC of HE4, CA125, and ROMA for the differentiation of stage I FIGO malignant tumors from nonmalignant adnexal tumors. FIGO: Fédération Internationale de Gynécologie et d'Obstétrique. (b) The ROC-AUC of HE4, CA125, and ROMA for the differentiation of epithelial ovarian cancer from nonmalignant adnexal tumors.
The AUCs of the diagnostic tests were compared using the Hanley and McNeil test. The results are presented in Table 7. The ROMA and HE4 were significantly (although marginally) better than CA125 for the differentiation of malignant tumors from nonmalignant tumors in the whole group (p = 0.043 and p = 0.043, resp.). Similarly, HE4 and ROMA were significantly better than CA125 for the differentiation of stage I FIGO malignant tumors from nonmalignant adnexal masses. The ROMA was not significantly better than HE4 or CA125 for the differentiation of EOC from nonmalignant adnexal masses in the whole group.
Table 7.
Results of the Hanley and McNeil test comparing the AUC of the diagnostic tests HE4, CA125, and ROMA for the differentiation of malignant from nonmalignant adnexal tumors.
| Compared tests | Malignant versus nonmalignant among the whole group | Malignant versus non-malignant among the premenopausal subgroup | Malignant versus nonmalignant among the postmenopausal subgroup | Stage I FIGO malignant versus nonmalignant adnexal tumors | Epithelial ovarian cancer versus nonmalignant adnexal masses |
|---|---|---|---|---|---|
| HE4 versus CA125 (p value) | 0.043 | 0.314 | 0.618 | 0.017 | 0.118 |
| HE4 versus ROMA (p value) | 0.999 | 0.985 | 0.965 | 0.893 | 0.979 |
| CA125 versus ROMA (p value) | 0.043 | 0.324 | 0.587 | 0.025 | 0.112 |
AUC: area under the curve; FIGO: Fédération Internationale de Gynécologie et d'Obstétrique; ROMA: Risk of Ovarian Malignancy Algorithm.
4. Discussion
Our study demonstrated that ROMA had the best ROC-AUC for the differentiation of EOC from nonmalignant adnexal masses, while CA125 had the worst ROC-AUC. However, there were no significant statistical differences in the performance of HE4, CA125, and ROMA. Our results are supported by those of Terlikowska et al., who also found no statistically significant difference between the ROC-AUC of these diagnostic tests [28]. The reported diagnostic performance of tumor markers and ROMA varies widely in the literature. Cho et al. reported that ROMA and HE4 showed significantly better performance than CA125 [29]. Romagnolo et al. reported that for the differentiation of EOC from benign adnexal diseases, the ROMA had the highest ROC-AUC in both premenopausal and postmenopausal patients [30]. Shen et al. revealed that the ROC-AUC of ROMA was significantly higher than that of HE4 and CA125 for the differentiation of all malignant diseases (including EOC, borderline tumors, and metastatic tumors) from other benign diseases. The high specificity and positive predictive value of HE4 may decrease the usefulness of adding CA125 into the diagnostic protocol, as patients with elevated HE4 are already considered to be at high risk [31]. By contrast, Van Gorp et al. revealed an insignificant difference in the diagnostic performance of HE4 and ROMA compared to CA125 in the differentiation of all malignant from nonmalignant pelvic masses in a prospective study of 389 patients [32]. Jacob et al. showed that the combination of both markers does not improve the diagnostic performance compared to HE4 alone and does not overcome the inability of both markers to adequately detect early-stage epithelial ovarian cancers. The authors suggested that the combination of HE4 and CA125 is beneficial in patients with a high score on the RMI due to elevated CA125. In that case, a normal level of HE4 will infer endometriosis rather than OC [33]. Fujiwara et al. analyzed the role of tumor markers and ROMA as diagnostic tools for type I and II epithelial ovarian cancers. For type I, HE4 and ROMA showed better sensitivity than CA125. At 75% specificity, the sensitivities of CA125 and HE4 were 92.1% for both markers for type II and 51.5% and 78.8% for type I, respectively. The sensitivity of the ROMA was better than the sensitivities of CA125 and HE4 and reached 84.8% and 97.4% for type I and type II, respectively [34]. In our study, when considering the total patient population and the premenopausal subgroup, CA125 had the highest sensitivity. In the postmenopausal subgroup, CA125 and ROMA had similar sensitivities. Compared to that of CA125 and that of ROMA, HE4 had the highest specificity for the whole group and for the premenopausal and postmenopausal subgroups. All diagnostic tests successfully differentiated adnexal masses, irrespective of menopausal status. For the whole group, HE4 and ROMA had the highest ROC-AUC, while HE4 had the highest ROC-AUC for the premenopausal subgroup; additionally, ROMA had the highest ROC-AUC for the postmenopausal subgroup. In the whole group, HE4 and ROMA were significantly (although marginally) superior to CA125 for the presurgical differentiation of adnexal masses (p = 0.043).
Age has a strong effect on serum levels of HE4. Urban et al. concluded that thresholds for HE4 are best defined for women of specific ages. Age-specific population thresholds for HE4 at 95% specificity ranged from 41.4 pmol/L for women aged 30 to 82.1 pmol/L for women aged 80 years [35]. Chudecka-Glaz et al. suggested a modified ROMA algorithm using a specific age range instead of the dichotomization of patients according to pre- and postmenopausal status. The authors concluded that the modified ROMA had higher specificity and positive predictive value than the original ROMA and suggested that a single cut-off level may be obtained for the entire population, regardless of menopausal status [36].
The positive predictive value of tumor markers and ROMA was much lower in the premenopausal subgroup compared to the postmenopausal subgroup. This difference may be attributed to a higher proportion (80%) of malignant cases among postmenopausal patients and the presence of endometriosis in premenopausal patients. Endometriosis is a main factor that may falsely increase serum levels of CA125 [37].
The consideration of tumor markers and ROMA for clinic-surgical assessment was beyond the scope of our study. Bandiera et al. revealed that in patients with EOC, elevated tumor marker levels and ROMA were associated with advanced FIGO stage, suboptimal debulking, ascites, positive cytology, lymph node involvement, and advanced age. The authors' multivariable analysis showed that HE4 and ROMA were independent prognostic factors for shorter overall survival rate, disease-free survival rate, and progression-free survival rate [9]. Li et al. demonstrated that high ROMA scores correlated with advanced ovarian cancer and ROMA were the strongest predictor of FIGO stage, with the highest specificity, accuracy, and positive predictive value (84.4%, 82.5%, and 87.0% for postmenopausal patients, resp., and 89.3%, 85.6%, and 74.3% for premenopausal patients, resp.) [38].
The BRCA1 mutation is a risk factor for ovarian cancer. In patients with BRCA1 mutation, the role of ROMA seems important. Chudecka-Glaz et al. investigated the diagnostic performance of tumor markers and ROMA in differentiation of pelvic masses, taking into consideration the BRCA1 mutation. In comparing ovarian cancer with benign ovarian disease in patients with BRCA1 mutation, ROMA had the best ROC-AUC, followed by CA125 and then by HE4. The authors showed that ROMA significantly differed from HE4 for the diagnosis. Similar results were revealed in postmenopausal patients. In premenopausal patients, the results were different in that CA125 had the best ROC-AUC followed by ROMA and then by HE4. However, there was no significant statistical difference in the diagnostic performance of these tests in this group of patients [39]. Chudecka-Glaz et al., in another study, concluded that patients with BRCA1 gene mutations have relatively low serum HE4 levels. Even the slightest elevation in HE4 or CA125 levels in female BRCA1 carriers undergoing prophylactic surgery should significantly increase oncological alertness [40].
The strength of our study is its prospective nature defined by a strict protocol. The tumor markers were measured within five days before surgical intervention and measured in the same way throughout the study. However, our study was not without limitations. It may have been affected by certain factors which should be considered in the interpretation of the results. The prevalence of malignancy was much higher than the prevalence in the community due to the referral of adnexal tumors suspicious of malignancy to our mixed gynecology-oncology referral center. However, the overall number of malignant cases in our study was lower than the number of nonmalignant cases. In consequence, our results cannot be applied to the primary healthcare setting. Simultaneously, because of the lower number of malignant adnexal tumors compared to that of nonmalignant tumor, our results cannot be applied to purely oncological centers where the prevalence of malignant cases is higher. This study enrolled patients who were referred for surgical management. The referral was dependent on ultrasound scans, clinical features, and tumor marker levels. We were unable to predict the number of cases that could have been referred for treatment earlier. Similarly, we were unable to predict the number of patients who were missed at the primary diagnostic level before referral because they were instead diagnosed with ovarian functional changes. Missed diagnosis and/or delayed referral to oncological centers increase the number of cases with advanced malignancy. We adopted exclusion criteria similar to those found in the literature for comparing the results. However, these criteria excluded the most difficult cases of adnexal masses. Renal diseases elevate the HE4 level while the level among pregnant women is lower than that among premenopausal patients [41, 42]. The incorporation of borderline ovarian tumors into the malignant group may have had an effect on results, although the overall number of these tumors in our study was small. Anton et al. showed higher sensitivity of tumor markers and ROMA when borderline ovarian tumors were classified as low-risk tumors [25]. Braicu et al. concluded that both CA125 and HE4 were not reliable biomarkers for the diagnosis of borderline ovarian tumors or for predicting the presence of invasive implants [43].
One benefit of ROMA that may distinguish it from other diagnostic modalities for adnexal masses is the elimination of ultrasound, which is highly dependent on examiner experience. For this reason, the use of serum markers in the diagnostic approach can be more objective and comparable [44].
There is still considerable debate on whether an ultrasound-based model or a tumor marker-based model should be used. In one study, a multicenter external validation using decision-curve analysis was performed to assess the clinical utility of risk models to refer patients with adnexal masses to specialized oncology care and concluded that three International Ovarian Tumor Analysis (IOTA) group models, including the ADNEX model, logistic regression model (LR2), and simple rules are clinically more useful than RMI and ROMA to select patients with adnexal masses for specialized oncology care [45]. In our study, ROMA is more useful than CA125 for the differentiation of malignant (including stage I FIGO) from nonmalignant adnexal tumors indicating the feasible use of ROMA in the presurgical differentiation of adnexal tumors.
Further modification of the ROMA might be needed given the lack of highly specific and sensitive diagnostic tests for the diagnosis of malignant adnexal tumors. Jeong et al. proposed a new reporting strategy for interpreting ROMA values based on the analytical measurement range and qualified-intervals of the HE4 and CA125 results. Reporting algorithms for the ROMA value were classified into three main categories. In the first category, the numerical ROMA value can be reported when quantitative HE4 and CA125 levels are reliable. In the second and third categories, patients were considered as low risk or undetermined depending on the levels of HE4 or CA125. The authors concluded that the new reporting strategy will provide more information on the utility of ROMA values in clinical practice [46]. Molina et al. suggested that the best algorithm to predict ovarian cancer was to classify all patients with increased HE4 as high-risk patients and to use ROMA for patients with normal HE4 and increased CA125 serum levels [47]. Despite variations between diagnostic tools assessed in our study, all of them significantly differentiated malignant from nonmalignant tumors preoperatively. Results showed the superiority of the ROMA compared to CA125 to differentiate malignancies, including stage I FIGO, from nonmalignant tumors. By contrast, the results showed no significant statistical difference between the overall performance of ROMA and HE4 in the differentiation of malignant and nonmalignant cases.
Our results may have economic implications for daily practice where only HE4 can be ordered for patients with adnexal masses, as ROMA requires the measurement of both tumor markers. Currently, most cases of ovarian cancer are detected in the advanced stages before referral to oncological centers; therefore, there is a need to use diagnostic tools to identify ovarian cancer as early as possible. In our study, both HE4 and ROMA were more helpful for detecting cases of stage I FIGO among patients referred to the oncology center. The appropriate diagnosis of early-stage malignancies at the oncology center enables the choice of radical surgery. Simultaneously, the diagnosis of nonmalignant adnexal tumor enables physicians to choose more conservative surgical interventions.
5. Conclusions
In the whole group, ROMA and HE4 were more useful than CA125 for the differentiation of malignant from nonmalignant adnexal tumors. This is also true for the differentiation of stage I FIGO malignant tumors from nonmalignant adnexal tumors. The ROMA is not significantly superior to tumor markers for the differentiation of EOC from nonmalignant adnexal tumors. Further studies on the diagnostic performance of the ROMA are needed to confirm our results.
Acknowledgments
This study was supported by a project grant from the Medical University of Warsaw for research and scientific work aimed at the scientific development of young doctors and PhD students (no. 2WA/PM21D/13).
Conflicts of Interest
The authors declare that there is no conflict of interest regarding the publication of this article.
References
- 1.Al Musalhi K., Al Kindi M., Al Aisary F., et al. Evaluation of HE4, CA-125, risk of ovarian malignancy algorithm (ROMA) and risk of malignancy index (RMI) in the preoperative assessment of patients with adnexal mass. Oman Medical Journal. 2016;31(5):336–344. doi: 10.5001/omj.2016.68. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Amor F., Alcázar J. L., Vaccaro H., León M., Iturra A. GI-RADS reporting system for ultrasound evaluation of adnexal masses in clinical practice: a prospective multicenter study. Ultrasound in Obstetrics & Gynecology. 2011;38(4):450–455. doi: 10.1002/uog.9012. [DOI] [PubMed] [Google Scholar]
- 3.Abdalla N., Winiarek J., Bachanek M., Cendrowski K., Sawicki W. Clinical, ultrasound parameters and tumor marker-based mathematical models and scoring systems in pre-surgical diagnosis of adnexal tumors. Ginekologia Polska. 2016;87(12):824–829. doi: 10.5603/GP.2016.0096. [DOI] [PubMed] [Google Scholar]
- 4.Karst A. M., Drapkin R. Ovarian cancer pathogenesis: a model in evolution. Journal of Oncology. 2010;2010:13. doi: 10.1155/2010/932371.932371 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Didkowska J., Wojciechowska U. Zachorowania i zgony na nowotwory złośliwe w Polsce. Krajowy Rejestr Nowotworów, Centrum Onkologii – Instytut im. Marii Skłodowskiej – Curie. 2017, http://onkologia.org.pl/k/epidemiologia/
- 6.Bast R. C., Jr., Feeney M., Lazarus H., Nadler L. M., Colvin R. B., Knapp R. C. Reactivity of a monoclonal antibody with human ovarian carcinoma. Journal of Clinical Investigation. 1981;68(5):1331–1337. doi: 10.1172/JCI110380. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Diamandis E. P., Bast R. C., Gold P., Chu T. M., Magnani J. L. Reflection on the discovery of carcinoembryonic antigen, prostate-specific antigen, and cancer antigens CA125 and CA19-9. Clinical Chemistry. 2013;59(1):22–31. doi: 10.1373/clinchem.2012.187047. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Hogendorf P., Skulimowski A., Durczyński A., et al. A panel of CA19-9, Ca125, and Ca15-3 as the enhanced test for the differential diagnosis of the pancreatic lesion. Disease Markers. 2017;2017:9. doi: 10.1155/2017/8629712.8629712 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Bandiera E., Romani C., Specchia C., et al. Serum human epididymis protein 4 and risk for ovarian malignancy algorithm as new diagnostic and prognostic tools for epithelial ovarian cancer management. Cancer Epidemiology, Biomarkers & Prevention. 2011;20(12):2496–2506. doi: 10.1158/1055-9965.EPI-11-0635. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Cohen J. G., White M., Cruz A., Farias-Eisner R. In 2014, can we do better than CA125 in the early detection of ovarian cancer? World Journal of Biological Chemistry. 2014;5(3):286–300. doi: 10.4331/wjbc.v5.i3.286. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Abdalla N., Pazura M., Słomka A., Piórkowski R., Sawicki W., Cendrowski K. The role of HE4 and CA125 in differentiation between malignant and non-malignant endometrial pathologies. Ginekologia Polska. 2016;87(12):781–786. doi: 10.5603/GP.2016.0088. [DOI] [PubMed] [Google Scholar]
- 12.Huhtinen K., Suvitie P., Hiissa J., et al. Serum HE4 concentration differentiates malignant ovarian tumours from ovarian endometriotic cysts. British Journal of Cancer. 2009;100(8):1315–1319. doi: 10.1038/sj.bjc.6605011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Kirchhoff C., Habben I., Ivell R., Krull N. A major human epididymis-specific cDNA encodes a protein with sequence homology to extracellular proteinase inhibitors. Biology of Reproduction. 1991;45(2):350–357. doi: 10.1095/biolreprod45.2.350. [DOI] [PubMed] [Google Scholar]
- 14.Hellstrom I., Hellstrom K. E. Two new biomarkers, mesothelin and HE4, for diagnosis of ovarian carcinoma. Expert Opinion on Medical Diagnostics. 2011;5(3):227–240. doi: 10.1517/17530059.2011.559459. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Moore R. G., Miller M. C., Steinhoff M. M., et al. Serum HE4 levels are less frequently elevated than CA125 in women with benign gynecologic disorders. American Journal of Obstetrics and Gynecology. 2012;206(4):351.e1–351.e8. doi: 10.1016/j.ajog.2011.12.029. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Cheng D., Sun Y., He H. The diagnostic accuracy of HE4 in lung cancer: a meta-analysis. Disease Markers. 2015;2015:7. doi: 10.1155/2015/352670.352670 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Moore R. G., McMeekin D. S., Brown A. K., et al. A novel multiple marker bioassay utilizing HE4 and CA125 for the prediction of ovarian cancer in patients with a pelvic mass. Gynecologic Oncology. 2009;112(1):40–46. doi: 10.1016/j.ygyno.2008.08.031. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Moore R. G., Jabre-Raughley M., Brown A. K., et al. Comparison of a novel multiple marker assay vs the risk of malignancy index for the prediction of epithelial ovarian cancer in patients with a pelvic mass. American Journal of Obstetrics & Gynecology. 2010;203(3):228.e1–228.e6. doi: 10.1016/j.ajog.2010.03.043. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Nolen B., Velikokhatnaya L., Marrangoni A., et al. Serum biomarker panels for the discrimination of benign from malignant cases in patients with an adnexal mass. Gynecologic Oncology. 2010;117(3):440–445. doi: 10.1016/j.ygyno.2010.02.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Moore R. G., Miller M. C., Disilvestro P., et al. Evaluation of the diagnostic accuracy of the risk of ovarian malignancy algorithm in women with a pelvic mass. Obstetrics & Gynecology. 2011;118(2, Part 1):280–288. doi: 10.1097/AOG.0b013e318224fce2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Li F., Tie R., Chang K., et al. Does risk for ovarian malignancy algorithm excel human epididymis protein 4 and CA125 in predicting epithelial ovarian cancer: a meta-analysis. BMC Cancer. 2012;12(1):p. 258. doi: 10.1186/1471-2407-12-258. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Dayyani F., Uhlig S., Colson B., et al. Diagnostic performance of risk of ovarian malignancy algorithm against CA125 and HE4 in connection with ovarian cancer. International Journal of Gynecological Cancer. 2016;26(9):1586–1593. doi: 10.1097/IGC.0000000000000804. [DOI] [PubMed] [Google Scholar]
- 23.Farzaneh F., Honarvar Z., Yaraghi M., et al. Preoperative evaluation of risk of ovarian malignancy algorithm index in prediction of malignancy of adnexal masses. Iranian Red Crescent Medical Journal. 2014;16(6, article e17185) doi: 10.5812/ircmj.17185. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Langmár Z., Németh M., Székely B., Borgulya G. The performance of the risk of ovarian malignancy algorithm. British Journal of Cancer. 2011;105(1):185–186. doi: 10.1038/bjc.2011.224. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Anton C., Carvalho F. M., Oliveira E. I., Maciel G. A., Baracat E. C., Carvalho J. P. A comparison of CA125, HE4, risk ovarian malignancy algorithm (ROMA), and risk malignancy index (RMI) for the classification of ovarian masses. Clinics. 2012;67(5):437–441. doi: 10.6061/clinics/2012(05)06. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Nolen B. M., Lokshin A. E. Biomarker testing for ovarian cancer: clinical utility of multiplex assays. Molecular Diagnosis & Therapy. 2013;17(3):139–146. doi: 10.1007/s40291-013-0027-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Bast R. C., Jr., Skates S., Lokshin A., Moore R. G. Differential diagnosis of a pelvic mass: improved algorithms and novel biomarkers. International Journal of Gynecological Cancer. 2012;22(Supplement 1):S5–S8. doi: 10.1097/IGC.0b013e318251c97d. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Terlikowska K. M., Dobrzycka B., Witkowska A. M., et al. Preoperative HE4, CA125 and ROMA in the differential diagnosis of benign and malignant adnexal masses. Journal of Ovarian Research. 2016;9(1):p. 43. doi: 10.1186/s13048-016-0254-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Cho H. Y., Park S. H., Park Y. H., et al. Comparison of HE4, CA125, and risk of ovarian malignancy algorithm in the prediction of ovarian Cancer in Korean women. Journal of Korean Medical Science. 2015;30(12):1777–1783. doi: 10.3346/jkms.2015.30.12.1777. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Romagnolo C., Leon A. E., Fabricio A. S. C., et al. HE4, CA125 and risk of ovarian malignancy algorithm (ROMA) as diagnostic tools for ovarian cancer in patients with a pelvic mass: an Italian multicenter study. Gynecologic Oncology. 2016;141(2):303–311. doi: 10.1016/j.ygyno.2016.01.016. [DOI] [PubMed] [Google Scholar]
- 31.Shen F., Lu S., Peng Y., et al. Performance of ROMA based on architect CA 125 II and HE4 values in Chinese women presenting with a pelvic mass: a multicenter prospective study. Clinica Chimica Acta. 2017;471:119–125. doi: 10.1016/j.cca.2017.05.029. [DOI] [PubMed] [Google Scholar]
- 32.Van Gorp T., Cadron I., Despierre E., et al. HE4 and CA125 as a diagnostic test in ovarian cancer: prospective validation of the risk of ovarian malignancy algorithm. British Journal of Cancer. 2011;104(5):863–870. doi: 10.1038/sj.bjc.6606092. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Jacob F., Meier M., Caduff R., et al. No benefit from combining HE4 and CA125 as ovarian tumor markers in a clinical setting. Gynecologic Oncology. 2011;121(3):487–491. doi: 10.1016/j.ygyno.2011.02.022. [DOI] [PubMed] [Google Scholar]
- 34.Fujiwara H., Suzuki M., Takeshima N., et al. Evaluation of human epididymis protein 4 (HE4) and risk of ovarian malignancy algorithm (ROMA) as diagnostic tools of type I and type II epithelial ovarian cancer in Japanese women. Tumour Biology. 2015;36(2):1045–1053. doi: 10.1007/s13277-014-2738-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Urban N., Thorpe J., Karlan B. Y., et al. Interpretation of single and serial measures of HE4 and CA125 in asymptomatic women at high risk for ovarian cancer. Cancer Epidemiology, Biomarkers & Prevention. 2012;21(11):2087–2094. doi: 10.1158/1055-9965.EPI-12-0616. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Chudecka-Głaz A., Cymbaluk-Płoska A., Jastrzębska J., Menkiszak J. Can ROMA algorithm stratify ovarian tumor patients better when being based on specific age ranges instead of the premenopausal and postmenopausal status? Tumour Biology. 2016;37(7):8879–8887. doi: 10.1007/s13277-015-4733-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Sarojini S., Tamir A., Lim H., et al. Early detection biomarkers for ovarian cancer. Journal of Oncology. 2012;2012:15. doi: 10.1155/2012/709049.709049 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Li Q. L., Wang C. J., Qi P., Zhang Y. X. Correlation of preoperative ROMA scores with clinical stage in epithelial ovarian cancer patients. Clinical and Translational Oncology. 2017;19(10):1260–1267. doi: 10.1007/s12094-017-1664-8. [DOI] [PubMed] [Google Scholar]
- 39.Chudecka-Głaz A., Cymbaluk-Płoska A., Luterek-Puszyńska K., Menkiszak J. Diagnostic usefulness of the risk of ovarian malignancy algorithm using the electrochemiluminescence immunoassay for HE4 and the chemiluminescence microparticle immunoassay for CA125. Oncology Letters. 2016;12(5):3101–3114. doi: 10.3892/ol.2016.5058. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Chudecka-Głaz A., Cymbaluk-Płoska A., Strojna A., Menkiszak J. HE4 serum levels in patients with BRCA1 gene mutation undergoing prophylactic surgery as well as in other benign and malignant gynecological diseases. Disease Markers. 2017;2017:13. doi: 10.1155/2017/9792756.9792756 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Nagy B., Krasznai Z. T., Balla H., et al. Elevated human epididymis protein 4 concentrations in chronic kidney disease. Annals of Clinical Biochemistry. 2012;49(4):377–380. doi: 10.1258/acb.2011.011258. [DOI] [PubMed] [Google Scholar]
- 42.Moore R. G., Miller M. C., Eklund E. E., Lu K. H., Bast R. C., Jr., Lambert-Messerlian G. Serum levels of the ovarian cancer biomarker HE4 are decreased in pregnancy and increase with age. American Journal of Obstetrics & Gynecology. 2012;206(4):349.e1–349.e7. doi: 10.1016/j.ajog.2011.12.028. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Braicu E., van Gorp T., Nassir M., et al. Preoperative HE4 and ROMA values do not improve the CA125 diagnostic value for borderline tumors of the ovary (BOT) – a study of the TOC consortium. Journal of Ovarian Research. 2014;7(1):p. 49. doi: 10.1186/1757-2215-7-49. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Nowak M., Janas Ł., Stachowiak G., Stetkiewicz T., R. Wilczyński J. Current clinical application of serum biomarkers to detect ovarian cancer. Menopausal Review. 2015;4(4):254–259. doi: 10.5114/pm.2015.55887. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Wynants L., Timmerman D., Verbakel J. Y., et al. Clinical utility of risk models to refer patients with adnexal masses to specialized oncology care: multicenter external validation using decision curve analysis. Clinical Cancer Research. 2017;23(17):5082–5090. doi: 10.1158/1078-0432.CCR-16-3248. [DOI] [PubMed] [Google Scholar]
- 46.Jeong T.-D., Cho E.-J., Ko D.-H., et al. A new strategy for calculating the risk of ovarian malignancy algorithm (ROMA) Clinical Chemistry and Laboratory Medicine. 2017;55(8):1209–1214. doi: 10.1515/cclm-2016-0582. [DOI] [PubMed] [Google Scholar]
- 47.Molina R., Escudero J. M., Augé J. M., et al. HE4 a novel tumour marker for ovarian cancer: comparison with CA 125 and ROMA algorithm in patients with gynaecological diseases. Tumour Biology. 2011;32(6):1087–1095. doi: 10.1007/s13277-011-0204-3. [DOI] [PMC free article] [PubMed] [Google Scholar]