Skip to main content
Journal of Southern Medical University logoLink to Journal of Southern Medical University
. 2019 Dec 20;39(12):1393–1401. doi: 10.12122/j.issn.1673-4254.2019.12.02

Comparison of serum cancer antigen 125, human epididymis protein 4, ROMA, and CPH-I for diagnosis of ovarian cancer in Chinese patients with ovarian mass

血清CA125和HE4水平及ROMA、CPH-I模型在鉴别卵巢良恶性肿瘤中的价值对比

Shipeng GONG 1,*, Yongning CHEN 1,2,*, Yadi ZHANG 1, Chanyuan LI 1, Qifan JIANG 1
PMCID: PMC6942985  PMID: 31907150

Abstract

Objective

To compare the performance of serum cancer antigen 125 (CA125), human epididymis protein 4 (HE4), Risk of Ovarian Malignancy Algorithm (ROMA) and Copenhagen index (CPH-I) for differential diagnosis of benign and malignant diseases in patients with ovarian mass.

Methods

We retrospectively analyzed the data of 719 women with pelvic mass, and the performance of preoperative serum levels of CA125 and HE4, ROMA and CPH-I for differential diagnosis of the masses was compared.

Results

Of the 710 women analyzed, 531 were diagnosed with benign ovarian lesions, 44 with borderline ovarian tumors (BOTs), 119 with epithelial ovarian cancers (EOCs), and 25 with non-EOCs. In differentiating ovarian cancer (OC) and BOT from benign lesions, the area under the receiver-operator characteristic (ROC) curve (AUC) was 0.854 for HE4, 0.856 for ROMA, 0.854 for CPH-I, and 0.792 for CA125, demonstrating better diagnostic performance of HE4, ROMA, and CPH-I than CA125 alone; the diagnostic sensitivity was 56.9% for HE4, 70.2% for CA125, 69.1% for ROMA, and 63.8% for CPH-I; the specificity was the best with HE4 (94.4%) and CPH-I (94.7%). In sub-analysis of EOC vs benign lesions, the AUCs of HE4, ROMA, and CPH-I increased to 0.946, 0.947, and 0.943, respectively, all greater than that of CA125 (0.888). In other sub-analyses, HE4, ROMA, and CPH-I all showed greater AUCs than CA125 alone.

Conclusion

This study confirms the accuracy of HE4, ROMA, and CPH-I for differentiating malignant from benign ovarian mass, and all these 3 tests show better performance than CA125. Furthermore, HE4 and CPH-I is superior to ROMA and CA125 in terms of specificity, while CA125 and ROMA have better diagnostic sensitivities.

Keywords: cancer antigen 125, human epididymis protein 4, Risk of Ovarian Malignancy Algorithm, Copenhagen index, ovarian neoplasms, differential diagnosis

INTRODUCTION

Ovarian cancer (OC) is one of the 3 major gynecologic malignancies, and its high fatality rate is mainly attributed to the fact that 70% of women with ovarian cancer are diagnosed at an advanced stage [1]. Various diagnostic tests including ultrasound, biomarkers, and multimodal diagnostic tests have been developed to improve the diagnosis of ovarian cancer.

Serum cancer antigen 125 (CA125) is the most commonly used tumor biomarker for diagnosis of ovarian cancer, but it has a limited diagnostic specificity [2]. Nearly 20% of women diagnosed with ovarian cancer have normal or marginally elevated levels of CA125, and its sensitivity for early-stage ovarian cancer remains low[3]. Human epididymis protein 4 (HE4), also known as WAP-type four disulfide core 2 (WFDC2), has shown potentials in differential diagnosis of ovarian tumors and has been approved for diagnosis of ovarian malignancies due to its similar sensitivity but a higher specificity compared to CA125[4, 5]. However, it is suggested that HE4 could be affected by menopausal status and age [6, 7]. The Risk of Ovarian Malignancy Algorithm (ROMA), a model that incorporates CA125, HE4, and menopausal status, was first introduced in 2009[8]. ROMA enhances the sensitivity, specificity, and accuracy for ovarian cancer prediction and shows a high sensitivity particularly in postmenopausal women[9, 10]

Nevertheless, menopause status is subjected to influences by such variables as age, duration of absence of menstrual bleed, serum level of follicle-stimulating hormone (FSH), and race[11]. Compared to menopausal status, age seems to be easier to determine and is more objective. A recent study by Karlsen et al [12]introduced a novel evaluation system, the Copenhagen Index (CPH-I), which is based on CA125, HE4, and age of the patients and showed encouraging results for differentiating EOC from benign ovarian tumors with a receiver operator characteristic (ROC)-area under curve (AUC) value as high as 0.951. But this study did not include ovarian cancer of other histological types. A later study included more subtypes, but the sample size was limited [13]. Moreover, the effect of women's menopausal status on the diagnostic performance of CPH-I has not been evaluated.

The primary aim of this study was to compare the clinical performance of CA125, HE4, ROMA, and CPH-I in differentiating ovarian cancer from benign ovarian disease. The secondary objective was to validate the application value of CPH-I in Chinese women for differential diagnosis of ovarian mass.

PATIENTS AND METHODS

Study design, setting and patients

This retrospective, single-center cross-sectional study was performed at Nanfang Hospital, Southern Medical University (Guangzhou, China) under approval by the Ethics Committee of Nanfang Hospital. We initially enrolled a total of 1018 consecutive women with an ovarian mass (including 239 malignant and 705 benign cases) between September, 2014 and November, 2016. We excluded the patients with ongoing pregnancy (15 patients); failure to obtain laboratory results of HE4 and CA125 biomarkers before surgery (245 patients); a history of adnexal surgery (23 patients); other malignant tumors in the past 5 years (7 patients); a serum creatinine level >133 μmol/L (5 patients); absence of descriptions of menopausal status in medical records (6 patients); and age below 18 years or over 90 years (8 patients). A total of 719 patients were included in the final analysis. Written informed consent was obtained from each participant prior to surgical treatment. In all the patients the ovarian mass was removed surgically, and all the diagnoses were histologically confirmed. The diagnostic results of CA125, HE4, ROMA, and CPH-I tests were compared against the pathological results, which served as the gold standard for differential diagnosis of ovarian mass. The staging of cancer was determined according to the revised FIGO staging system published in 2014[14]. Postmenopausal status was defined as the absence of a menstrual period for more than 12 months or after complete hysterectomy.

Serum CA125 and HE4 measurement

Blood samples were collected from the patients at least 7 days prior to surgery. Serum CA125 and HE4 were measured using the fully automatic chemiluminescent analyzer Cobs601 and the corresponding kit according to manufacturer's protocol (Roche Diagnostics, Indianapolis, IN, USA). The cut-off values were set as follows: 35 U/ mL for CA125[15]in all the patients; for HE4, the cut-off value was 68.79 U/mL in premenopausal women and 114.43 U/mL in postmenopausal women [16]; for ROMA, 11.4% in premenopausal women and 29.9% in postmenopausal women [15]; and for CPH-I, 7% in all the patients[12].

Calculation of ROMA index and CPH-I

ROMA predictive index was used to determine whether a patient was at high or low risk of ovarian cancer. It was calculated based on serum CA125 and HE4 levels stratified by menopausal status using the equations as previously described [8]. CPH-I was calculated based on the patients' serum CA125, HE4, and age as previously described[12].

Evaluation of diagnostic effectiveness

The diagnostic accuracy of the tests was measured by the AUC calculated from the ROC curve. The diagnostic sensitivities (SN), specificities (SP), positive predictive values (PPV), and negative predictive values (NPV) of CA125, HE4, ROMA, and CPH-I in differential diagnosis of ovarian cancer from benign diseases were estimated also using ROC analysis.

Statistical analysis

The count data with a non-normal distribution shown by normality test are presented as the median (5 to 75 percentile range), and the categorical data are presented as numerals or percentages. Comparisons of categorical data between groups were made using Pearson Chi-square test or Fisher's exact test. A statistical significance is defined for a P value < 0.05 except that an adjusted significance level was used for pairwise comparison of the count data: α'=0.05[/ No. of groups × (No. of groups-1)/2+1]. All the statistical analyses were performed using SPSS (Version 20.0, Chicago, IL, USA).

RESULTS

General clinical characteristics of the patients

Of all the 719 patients evaluated, 531 women were diagnosed with benign ovarian lesions, 44 with borderline ovarian tumors (BOTs), 119 with epithelial ovarian cancers (EOCs), 25 with non-EOCs, 10 with sex cord-stromal tumors, 11 with germ cell tumors, and 4 with metastatic ovarian cancer. The demographic and clinical characteristics of the patients including age, menopausal status, histopathology, and FIGO staging are listed in Tab. 1.

1.

Basic characteristics of the study participants

Characteristics Benign BOT Non-EOC EOC
All the data are presented as number of patients with percentage in parenthesis[n(%)]except for age, which is shown as median with 5 to 75 percentile range. BOT: Borderline ovarian tumor; Non-EOC: Nonepithelial ovarian cancer; EOC: Epithelial ovarian cancer.
n (%) 531 (73.9) 44(6.1) 25 (3.5) 119 (16.6)
Age (year) 33.0 (26.0-44.0) 42.5(29.5-58.5) 40.0(27.0-46.0) 54.0 (47.0-63.0)
Menopausal status [n (%)]
  Premenopausal 473 (89.1) 29 (65.9) 21 (84.0) 45(37.8)
  Postmenopausal 58 (10.9) 15(34.1) 4(16.0) 74 (62.2)
Histopathology [n (%)]
  Epitelial 308 (58.0) 44 (100.0) 119 (100.0)
  Serous papillary 8 (18.2) 8(18.2) 25(21.0)
  Serous 69 (13.0) 9 (20.5) 61 (51.3)
  Mucous 50 (9.4) 27 (61.4) 14(11.8)
  Endometriotic 183 (34.5) 11 (9.2)
  Clear cell 8(6.7)
  Seromucous 6(1.1)
  Germ cells 201 (37.9) 11 (44.0)
  Dysgerminoma 3(12.0)
  Yolk sac tumor 5 (20.0)
  Teratoma 201 (37.9) 3(12.0)
  Sex cord stroma 22(4.1) 10 (40.0)
  Granulosa cell tumor 10 (40.0)
  Theca fibroma 22(4.1)
  Metastatic tumor 4(16.0)
FIGO stage
  Early stage 42 (95.5) 17 (68.0) 37 (31.1)
  Stage Ⅰ 39 (88.6) 16 (64.4) 21(17.6)
  Stage Ⅱ 3(6.8) 1 (4.0) 16(13.4)
  Advanced stage 2(4.5) 8 (32.0) 82 (68.9)
  Stage Ⅲ 2(4.5) 4(16.0) 60 (50.4)
  Stage Ⅳ 4(16.0) 22(18.5)

Benign ovarian tumors were more common in premenopausal women than in postmenopausal women (89.1%), whereas EOC was more frequent in postmenopausal women (62.2%), and the difference was statistically significant (χ2=159.1, P < 0.001). Teratoma was the most common benign ovarian mass (37.9%). The most common subtype of BOT was mucous epithelial ovarian tumors (61.4%), and almost half of the malignant epithelial ovarian tumors were serous (51.3%). Twenty-five patients were found to have non-EOC, including 3 with asexual cell tumors, 5 with yolk sac tumors, 3 with immature teratomas, 10 with granulosa cell tumors, and 4 with metastatic tumors. Most of the EOC patients were diagnosed at advanced stages (early vs advanced stages: 31.1% vs 68.9%; χ2=56.3, P < 0.001).

Performance of CA125, HE4, ROMA and CPH-I for predicting ovarian cancers

The ranking of AUC (from the highest to the lowest) in differentiating ovarian cancer and BOT from benign lesions was ROMA (0.856), CPH-I (0.854), HE4 (0.854), and CA125 (0.792) (Tab. 2). The corresponding ROC curve is shown in Fig. 1. Similar ranking patterns were found for comparisons of other subtypes including EOC vs benign lesions, EOC vs non-EOC and BOT, and finally non-EOC and BOT vs benign lesions, as illustrated in Fig. 2-4. In differentiating OC and BOT from benign lesions, HE4 had the lowest sensitivity (56.9%); CA125 had the highest sensitivity (70.2%) but the lowest specificity (72.9%); CPH-I had the highest specificity (94.7%). For other sub-analyses, ROMA showed the highest sensitivity (87.4%) followed by CPH-I (84.9%), CA125 (84.0%), and HE4 (79.8%), while for EOC vs benign lesions and EOC vs non-EOC and BOT, CA125 had the highest sensitivity (46.4%) in differentiating non-EOC and BOT from benign lesions.

2.

Diagnostic performances of CA125, HE4, ROMA and CPH-I for discrimination of ovarian cancer from other subtypes

Group Tests AUC(95%CI) SN(%) SP(%) PPV(%) NPV(%)
OC: Ovarian cancer; BOT: Borderline ovarian tumor; Non-EOC: Non-epithelial ovarian cancer; EOC: Epithelial ovarian cancer; AUC: Area under curve; SN: Sensitivity; SP: Specificity; PPV: Positive predictive value; NPV: Negative predictive value.
OC+BOT (n=188) vs Benign (n=531)
CA125 0.792 (0.751, 0.833) 70.2 72.9 47.8 874
HE4 0.854(0.817, 0.891) 56.9 94.4 78.1 86.1
ROMA 0.856 (0.819, 0.893) 69.1 87.6 66.3 88.9
CPH-I 0.854 (0.818, 0.890) 63.8 94.7 81.1 88.1
EOC (n=119)vs Benign (n=531)
CA125 0.888 (0.849, 0.928) 84.0 72.9 41.0 95.3
HE4 0.946(0.915, 0.977) 79.8 94.4 76.0 95.4
ROMA 0.947 (0.916, 0.978) 87.4 87.6 61.2 96.9
CPH-I 0.943 (0.913, 0.974) 84.9 94.7 78.3 96.5
EOC (n=119) vs Non-EOC+BOT (n=69)
CA125 0.836(0.778, 0.893) 84.0 53.6 75.8 66.1
HE4 0.886 (0.839, 0.934) 79.8 82.6 88.8 70.4
ROMA 0.889 (0.842, 0.937) 87.4 62.3 80.0 74.1
CPH-I 0.888 (0.841, 0.934) 84.9 72.5 84.2 73.5
Non-EOC+BOT (n=69) vs Benign (n=531)
CA125 0.625 (0.557, 0.694) 46.4 72.9 18.2 91.3
HE4 0.696 (0.625, 0.767) 17.4 94.4 28.6 89.8
ROMA 0.700(0.629, 0.770) 37.7 87.6 28.3 91.5
CPH-I 0.700(0.633, 0.767) 27.5 94.7 40.4 91.0

1.

1

ROC curves for CA125, HE4, ROMA and CPH-I for differentiating ovarian cancer and borderline ovarian tumors from benign ovarian mass.

2.

2

ROC curves for CA125, HE4, ROMA and CPH-I for differentiating epithelial ovarian cancer from benign ovarian mass.

4.

4

Diagnostic performance of the 4 tests for discrimination of different subtypes of ovarian mass stratified by histological stage

3.

3

ROC curves for CA125, HE4, ROMA and CPH-I for differentiating epithelial ovarian cancer from non-epithelial ovarian cancers and borderline ovarian tumors.

Performance of the 4 tests for predicting ovarian lesions stratified by menopausal status

For differentiating ovarian cancer and BOT from benign disease in premenopausal women, the AUC was similar among ROMA (0.771), CPH-I (0.778), and HE4 (0.766), all higher than that of CA125 (0.729) (Tab. 3). In premenopausal women, CA125 test had the highest sensitivity (60.0%) but also the lowest specificity (72.3%); in postmenopausal women, ROMA and CPH-I had the highest sensitivity (both 81.7%).

3.

Diagnostic performance of the 4 tests for discrimination of different subtypes of ovarian mass stratified by menopausal status

Group Tests AUC(95%CI) SN(% SP(%) PPV(%) NPV(%)
OC+BOT (n=188) vs Benign (n=531)
CA125 0.729 (0.670, 0.789) 60 72.3 30.3 90.0
  Premenopausal HE4 0.766 (0.705, 0.828) 46.3 93.9 60.3 89.7
ROMA 0.771 (0.710, 0.832) 56.8 88.2 49.1 91.0
CPH-I 0.778 (0.721, 0.835) 46.3 96.0 69.8 89.9
CA125 0.870 (0.813, 0.928) 80.6 77.6 85.2 71.4
  Postmenopausal HE4 0.912 (0.867, 0.958) 67.7 98.3 98.4 65.5
ROMA 0.903 (0.855, 0.950) 81.7 82.8 88.4 73.8
CPH-I 0.904 (0.856, 0.951) 81.7 84.5 89.4 74.2
EOC (n=119) vs Benign (n=531)
CA125 0.836 (0.757, 0.915) 75.6 72.3 20.6 96.9
  Premenopausal HE4 0.895 (0.825, 0.965) 75.6 93.9 54.0 97.6
ROMA 0.898 (0.829, 0.968) 80.0 88.2 39.1 97.9
CPH-I 0.898 (0.835, 0.962) 73.3 96.0 63.5 97.4
CA125 0.924 (0.877, 0.972) 89.2 77.6 83.5 84.9
  Postmenopausal HE4 0.956 (0.922, 0.990) 82.4 98.3 98.4 81.4
ROMA 0.955 (0.919, 0.991) 91.9 82.8 87.2 88.9
CPH-I 0.953 (0.917, 0.989) 91.9 84.5 88.3 89.1
EOC (n=119) vs Non-EOC+BOT (n=69)
CA125 0.768 (0.666, 0.871) 75.6 54.0 59.6 71.1
  Premenopausal HE4 0.826 (0.737, 0.915) 75.6 80.0 77.3 78.4
ROMA 0.827 (0.738, 0.916) 80.0 64.0 66.7 78
CPH-I 0.828 (0.740, 0.916) 73.3 78.0 75.0 76.5
CA125 0.886 (0.816, 0.956) 89.2 52.6 88.0 55.6
  Postmenopausal HE4 0.915 (0.858, 0.972) 82.4 89.5 96.8 56.7
ROMA 0.922 (0.869, 0.976) 91.9 57.9 89.5 64.7
CPH-I 0.917 (0.861, 0.973) 91.9 57.9 89.5 64.7

In sub-analyses of EOC vs benign lesions, the AUCs of ROMA, CPH-I, and HE4 were also higher than that of CA125 regardless of the menopausal status. In premenopausal women, ROMA had the highest sensitivity (80.0%) and CPH-I had the highest specificity (96.0%), but in postmenopausal women, ROMA and CPH-I had the same sensitivity (91.9%), and HE4 had a better specificity (98.3%) than the other tests.

In sub-analyses of EOC vs non-EOC and BOT, the trends in AUC and sensitivity of the 4 tests were similar to those in differential diagnosis of EOC vs benign lesions, whereas HE4 had the best specificity in both pre- and postmenopausal women. All the tests showed lower AUC and sensitivities in premenopausal than in postmenopausal patients with ovarian cancer and BOT.

Performance of the 4 tests for predicting ovarian lesions stratified by histological stages

For predicting OC and BOT from benign ovarian mass in early stages, ROMA, HE4, and CPH-I all showed better AUC than CA125 (0.758, 0.756, 0.751, and 0.678, respectively; Tab. 4). All the tests had low sensitivities, with the highest being merely 55.2% with CA125. In patients with advanced stages of OC and BOT, CPH-I (0.986), ROMA (0.959), and HE4 (0.957) all showed high AUC above 0.95, and were all better than CA125 (0.911). ROMA and CPH-I showed higher sensitivities (89.1% and 88.0%, respectively) than both CA125 and HE4 in this group.

4.

ROC curves for CA125, HE4, ROMA and CPH-I for differentiating non-epithelial ovarian cancers and borderline ovarian tumors from benign ovarian mass.

Group Tests AUC(95%CI) SN(%) SP(%) PPV(%) NPV(%)
OC+BOT (n=188) vs Benign (n=531)
  Early stage CA125 0.678 (0.615, 0.740) 55.2 72.9 26.9 90.0
HE4 0.756 (0.695, 0.816) 33.3 94.4 51.6 88.7
ROMA 0.758 (0.698, 0.818) 50.0 87.6 42.1 90.6
CPH-I 0.751 (0.693, 0.810) 40.6 94.7 58.2 89.8
  Advanced stage CA125 0.911 (0.875, 0.947) 85.9 72.9 35.4 96.8
HE4 0.957 (0.928, 0.986) 81.5 94.4 71.4 96.7
ROMA 0.959 (0.930, 0.988) 89.1 87.6 55.4 97.9
CPH-I 0.961 (0.937, 0.985) 88.0 94.7 74.3 97.9
EOC (n=119) vs Benign (n=531)
  Early stage CA125 0.790 (0.693, 0.888) 70.3 72.9 15.3 97.2
HE4 0.869 (0.783, 0.954) 67.6 94.4 45.5 97.7
ROMA 0.866 (0.780, 0.953) 73.0 87.6 29.0 97.9
CPH-I 0.855 (0.769, 0.940) 67.6 94.7 47.2 97.7
  Advanced stage CA125 0.933 (0.901, 0.965) 90.2 72.9 33.9 98.0
HE4 0.981 (0.962, 0.999) 85.4 94.4 70.0 97.7
ROMA 0.983 (0.965, 1.000) 93.9 87.6 53.8 98.9
CPH-I 0.983 (0.970, 0.996) 92.7 94.7 73.1 98.8
EOC (n=119) vs Non-EOC+BOT (n=69)
  Early stage CA125 0.741 (0.624, 0.857) 70.3 54.2 49.1 74.4
HE4 0.791 (0.685, 0.896) 67.6 88.1 78.1 81.2
ROMA 0.781 (0.674, 0.889) 73.0 64.4 56.2 79.2
CPH-I 0.783 (0.673, 0.893) 67.6 76.3 64.1 78.9
  Advanced stage CA125 0.779 (0.617, 0.941) 90.2 50.0 93.7 38.5
HE4 0.856 (0.736, 0.976) 85.4 50.0 93.3 29.4
ROMA 0.876 (0.754, 0.997) 93.9 50.0 93.9 50.0
CPH-I 0.868 (0.761, 0.976) 92.7 50.0 93.8 45.5

For differentiating early-stage EOC from benign tumors, HE4 (0.869), ROMA (0.866), and CPH-I (0.855) showed higher AUC than CA125 (0.790). The AUC of all the tests were high in differentiating advanced EOC from benign ones, and those of ROMA and CPH-I reached as high as 0.983, while the sensitivities of ROMA (73.0% and 93.9%, respectively) were the highest regardless of EOC stage.

To distinguish EOC from non-EOC and BOT in early stages, HE4 (AUC of 0.791), CPH-I (AUC of 0.783), and ROMA (AUC of 0.781) had slightly better accuracies than CA125 (AUC of 0.741); all the tests showed higher accuracies in advanced stages. All the tests showed lower AUC and sensitivities for ovarian cancer and BOT in early stages than for those in advanced stages.

DISCUSSION

We found that ROMA, CPH-I, and HE4 had similar performance in differentiating various subtypes of ovarian mass, all slightly better than CA125 alone. The diagnostic performance of ROMA, CPH-I, and HE4 were better in postmenopausal women and in women with advanced ovarian cancer.

Early detection of ovarian cancer is essential for improving the patients' survival. Biomarkers including CA125 and HE4 were developed for general practitioners to facilitate the initial triage of patients with suspicious ovarian mass. Our results confirm the flaws associated with CA125, which showed the lowest accuracies among the 4 tests in differentiating malignant ovarian cancer from a benign mass, or EOC from non-EOC and BOT. However, CA125 had a higher sensitivity than HE4 and in both the overall population and the premenopausal and early-stage patients, CA125 showed the highest sensitivity for differentiating OC and BOT from benign lesions. Our results regarding the sensitivity of CA125 were consistent with those in a previous study, where increased levels of CA125 were found in approximately 80% of all EOC and 50% of stage I EOC patients[17]. The high sensitivity of CA125 might be explained by the low cut-off value used in the present study (35 U/mL). CA125 level was prone to increase in response to peritoneal irritation, especially in some benign cases such as endometriosis, where CA125 level could exceed 35 U/mL[18]. Xu et al [10] suggested the optimal cut-off values for CA125 of 60 U/mL in premenopausal women and 35 U/mL in postmenopausal women to improve the detection of epithelial ovarian cancer. American College of Obstetrics and Gynecologists also suggest that patients with CA125 >200 U/mL should consult a gynecologist for malignant ovarian lesions[19]. Although the sensitivities of CA125 seemed high due to the low cut-off value used in this study, its positive predictive value was poor for ovarian cancer screening, as was consistent with many other studies[20, 21].

Several studies have demonstrated that HE4 alone seems to be more specific than CA125 for detecting ovarian cancers, which is also confirmed in our study[4, 5]. Unlike CA125, HE4 expression can hardly be affected by benign ovarian lesions or other medical conditions [22]. Moore et al [4] found that among a series of tumor markers, HE4 demonstrated the highest sensitivity and specificity in distinguishing between benign and malignant adnexal masses, both alone and in combination with CA125. The excellent performance of HE4 might be attributed to its stability across different ethnicities[23] as well as its resistance to the handling and storage conditions[24]. HE4 levels elevated steadily along with age regardless of the menopausal status[7, 23, 25]. Notably, HE4 can increase in about half of ovarian cancer patients having normal CA125 levels [26]. However, we found that HE4 had the lowest sensitivity in differentiating malignant from benign lesions, which could be partly explained by its similar expression pattern in BOT and benign patients[27]. In addition, the overall expression level of HE4 is low in EOC patients, especially in those with low-grade serous papillary carcinoma and clear cell carcinoma[28, 29].

ROMA, by incorporating CA125, HE4, and the menopausal status, combines the advantages of both CA125 and HE4 markers. Our results demonstrated that ROMA performed slightly better than HE4 and CA125 alone in distinguishing OC and BOT, EOC from non-EOC and BOT, and non-EOC and BOT from benign lesions, consistent with the findings by Molina et al [6], who suggested that the greater contribution of HE4 and ROMA performed better than CA125 for ovarian mass differentiation. Terlikowska et al [30] also reported that compared to CA125 and HE4 (with AUC of 0.895 and 0.879, respectively), ROMA (AUC of 0.918) appeared to have a better performance in differentiating EOC from benign ovarian mass [6, 22, 31].

Menopausal status can be a clinical challenge for HE4 and ROMA due to its varying definitions and the influence by the patient's age [32]. Bolstadt et al [25] performed a comprehensive study investigating the HE4 levels in women of various ages. They found that compared with women in their twenties, HE4 levels increased year by year and reached as high as 101% in their eighties. Considering this important factor—age, Karlsen and colleagues developed a novel evaluation model, CPH-I, for distinguishing between benign and malignant lesions[12]. But their validation dataset did not include non-EOC, BOT, and ovarian metastases lesions; in a later study, Yoshida et al [13] enrolled different subtypes of ovarian cancer, and reported that CPH-I had an AUC of 0.84 for differentiating benign lesions from OC and BOT and of 0.94 for differentiating benign lesions and EOC, but the small sample size and failure of measuring menopausal status limited the clinical significance of the results. Our results, in agreement with those by Yoshida et al [13], suggested that ROMA and CPH-I were similar in differentiating between malignant and benign ovarian mass, and the AUCs of CPH-I were 0.854 and 0.943 for differentiating benign lesions from OC and BOT and from EOC, respectively, which further confirms the stability of CPH-I in detecting malignant ovarian tumors in different study cohorts. Interestingly, the CPH-I development study by Karlsen and his colleagues was conducted in an Asian population, where CPH-I seemed to have a slightly better sensitivity but a lower specificity than ROMA[12], which was also seen in our results.

We also performed sub-analyses to compare the diagnostic performances of the 4 tests in pre- and postmenopausal women. All these 4 tests were found to perform better for differential diagnoses in postmenopausal women than in premenopausal women. Specifically, for distinguishing EOC from benign tumors, the accuracies of HE4, ROMA, and CPH-I in postmenopausal women could exceed 0.95; for differentiating EOC from non-EOC and BOT, the AUCs of HE4, ROMA, and CPH-I in postmenopausal women were also above 0.91. This diagnostic discrepancy between pre- and postmenopausal women was also seen in several other studies. For instance, Xu et al [10]recently reported that the AUC of ROMA for distinguishing EOC from benign diseases was 0.82 in premenopausal women and 0.88 in postmenopausal women. Moore et al[33]also reported AUCs of ROMA of 0.791 in premenopausal women and 0.840 in postmenopausal women. Whether menopausal status affects the diagnostic efficacy of CPH-I has not yet been reported. Our study suggests for the first time that in premenopausal women, HE4, ROMA, and CPH-I all perform better than CA125 alone, indicating the superiority of the former 3 tests in the premenopausal population.

We also found that the tests performed better for a differential diagnoses in advanced stages of ovarian cancer. CPH-I, ROMA, and HE4 alone showed comparable performance for differential diagnosis of pelvic mass in both early and advanced stages. Yoshida et al [13] also suggested that CPH-I and ROMA showed good performance for advanced ovarian cancer, but not for early ovarian cancer. Our data suggest that CPH-I performs as well as HE4 and ROMA for different subtypes of ovarian cancers in early stages. Given such inconsistency in current evidence, the search for new biomarkers or models for early detection of ovarian cancer should continue.

To our knowledge, this is the first study investigating the clinical value of CPH-I among a Chinese population. The major strength of this study is the large sample size and comprehensive data collection. However, the optimal cut-off value of CPH-I has not been determined yet for Chinese populations, and the inter-ethnicity and inter-lab discrepancies might affect the performance of CPH-I.

In summary, this study confirms the accuracy of HE4, ROMA, and CPH-I in differentiating malignant and benign ovarian lesions, and all the 3 tests perform better than CA125. Furthermore, CA125 and ROMA show better diagnostic sensitivity, while the specificities of HE4 and CPH-I are superior to those of ROMA and CA125.

Funding Statement

Supported by Guangdong Science and Technology Department (2016A020215115); Huadu District Science and Technology Bureau (HD15CXY006) and Tianhe District Science and Technology Bureau (201604KW010)

广东省科技计划(2016A020215115);广州市花都区科技计划(HD15CXY006);广州市天河区科技计划(201604KW010)

Contributor Information

Shipeng GONG (龚 时鹏), Email: gsp2103@163.com.

Yongning CHEN (陈 咏宁), Email: cyngynecology@163.com.

References

  • 1.Ozols RF, Bookman MA, Connolly DC, et al. Focus on epithelial ovarian cancer. http://d.old.wanfangdata.com.cn/OAPaper/oai_pubmedcentral.nih.gov_3143433. Cancer Cell. 2004;5(1):19–24. doi: 10.1016/s1535-6108(04)00002-9. [Ozols RF, Bookman MA, Connolly DC, et al.Focus on epithelial ovarian cancer[J].Cancer Cell, 2004, 5(1):19-24.] [DOI] [PubMed] [Google Scholar]
  • 2.Buamah P. Benign conditions associated with raised serum CA-125 concentration. J Surg Oncol. 2000;75(4):264–5. doi: 10.1002/1096-9098(200012)75:4&#x0003c;264::AID-JSO7&#x0003e;3.0.CO;2-Q. [Buamah P.Benign conditions associated with raised serum CA-125 concentration[J].J Surg Oncol, 2000, 75(4):264-5.] [DOI] [PubMed] [Google Scholar]
  • 3.Duffy MJ, Bonfrer JM, Kulpa J, et al. CA125 in ovarian cancer:European Group on Tumor Markers guidelines for clinical use. Int J Gynecol Cancer. 2005;15(5):679–91. doi: 10.1111/j.1525-1438.2005.00130.x. [Duffy MJ, Bonfrer JM, Kulpa J, et al.CA125 in ovarian cancer:European Group on Tumor Markers guidelines for clinical use[J].Int J Gynecol Cancer, 2005, 15(5):679-91.] [DOI] [PubMed] [Google Scholar]
  • 4.Moore RG, Brown AK, Miller MC, et al. The use of multiple novel tumor biomarkers for the detection of ovarian carcinoma in patients with a pelvic mass. Gynecol Oncol. 2008;108(2):402–8. doi: 10.1016/j.ygyno.2007.10.017. [Moore RG, Brown AK, Miller MC, et al.The use of multiple novel tumor biomarkers for the detection of ovarian carcinoma in patients with a pelvic mass[J].Gynecol Oncol, 2008, 108(2):402-8.] [DOI] [PubMed] [Google Scholar]
  • 5.Holcomb K, Vucetic Z, Miller MC, et al. Human epididymis protein 4 offers superior specificity in the differentiation of benign and malignant adnexal masses in premenopausal women. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=b2792d58085bd7aeda3f1d032e220626. Am J Obstet Gynecol. 2011;205(4):358. doi: 10.1016/j.ajog.2011.05.017. [Holcomb K, Vucetic Z, Miller MC, et al.Human epididymis protein 4 offers superior specificity in the differentiation of benign and malignant adnexal masses in premenopausal women[J].Am J Obstet Gynecol, 2011, 205(4):358.e1-6.] [DOI] [PubMed] [Google Scholar]
  • 6.Molina R, Escudero JM, Auge JM, et al. HE4 a novel tumour marker for ovarian cancer:comparison with CA 125 and ROMA algorithm in patients with gynaecological diseases. Tumour Biol. 2011;32(6):1087–95. doi: 10.1007/s13277-011-0204-3. [Molina R, Escudero JM, Auge JM, et al.HE4 a novel tumour marker for ovarian cancer:comparison with CA 125 and ROMA algorithm in patients with gynaecological diseases[J].Tumour Biol, 2011, 32(6):1087-95.] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Moore RG, Miller MC, Eklund EE, et al. Serum levels of the ovarian cancer biomarker HE4 are decreased in pregnancy and increase with age. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=8face511364a66e2ba549cc6fc8f6881. Am J Obstet Gynecol. 2012;206(4):349. doi: 10.1016/j.ajog.2011.12.028. [Moore RG, Miller MC, Eklund EE, et al.Serum levels of the ovarian cancer biomarker HE4 are decreased in pregnancy and increase with age[J].Am J Obstet Gynecol, 2012, 206(4):349.e341-7.] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Moore RG, McMeekin DS, Brown AK, et al. A novel multiple marker bioassay utilizing HE4 and CA125 for the prediction of ovarian cancer in patients with a pelvic mass. Gynecol Oncol. 2009;112(1):40–6. doi: 10.1016/j.ygyno.2008.08.031. [Moore RG, McMeekin DS, Brown AK, et al.A novel multiple marker bioassay utilizing HE4 and CA125 for the prediction of ovarian cancer in patients with a pelvic mass[J].Gynecol Oncol, 2009, 112(1):40-6.] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Li F, Tie R, Chang K, et al. Does risk for ovarian malignancy algorithm excel human epididymis protein 4 and CA125 in predicting epithelial ovarian cancer:a meta-analysis. BMC Cancer. 2012;12:258. doi: 10.1186/1471-2407-12-258. [Li F, Tie R, Chang K, et al.Does risk for ovarian malignancy algorithm excel human epididymis protein 4 and CA125 in predicting epithelial ovarian cancer:a meta-analysis.BMC Cancer, 2012, 12:258.] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Xu Y, Zhong R, He J, et al. Modification of cut-off values for HE4, CA125 and the ROMA algorithm for early-stage epithelial ovarian cancer detection:Results from 1021 cases in South China. Clin Biochem. 2016;49(1-2):32–40. doi: 10.1016/j.clinbiochem.2015.07.029. [Xu Y, Zhong R, He J, et al.Modification of cut-off values for HE4, CA125 and the ROMA algorithm for early-stage epithelial ovarian cancer detection:Results from 1021 cases in South China[J].Clin Biochem, 2016, 49(1-2):32-40.] [DOI] [PubMed] [Google Scholar]
  • 11.Gracia CR, Sammel MD, Freeman EW, et al. Defining menopause status:creation of a new definition to identify the early changes of the menopausal transition. Menopause. 2005;12(2):128–35. doi: 10.1097/00042192-200512020-00005. [Gracia CR, Sammel MD, Freeman EW, et al.Defining menopause status:creation of a new definition to identify the early changes of the menopausal transition[J].Menopause, 2005, 12(2):128-35.] [DOI] [PubMed] [Google Scholar]
  • 12.Karlsen MA, Hogdall EV, Christensen IJ, et al. A novel diagnostic index combining HE4, CA125 and age may improve triage of women with suspected ovarian cancer-an international multicenter study in women with an ovarian mass. Gynecol Oncol. 2015;138(3):640–6. doi: 10.1016/j.ygyno.2015.06.021. [Karlsen MA, Hogdall EV, Christensen IJ, et al.A novel diagnostic index combining HE4, CA125 and age may improve triage of women with suspected ovarian cancer-an international multicenter study in women with an ovarian mass[J].Gynecol Oncol, 2015, 138(3):640-6.] [DOI] [PubMed] [Google Scholar]
  • 13.Yoshida A, Derchain SF, Pitta DR, et al. Comparing the Copenhagen Index (CPH-I) and Risk of Ovarian Malignancy Algorithm (ROMA):two equivalent ways to differentiate malignant from benign ovarian tumors before surgery? Gynecol Oncol. 2016;140(3):481–5. doi: 10.1016/j.ygyno.2016.01.023. [Yoshida A, Derchain SF, Pitta DR, et al.Comparing the Copenhagen Index (CPH-I) and Risk of Ovarian Malignancy Algorithm (ROMA):two equivalent ways to differentiate malignant from benign ovarian tumors before surgery[J]?Gynecol Oncol, 2016, 140(3):481-5.] [DOI] [PubMed] [Google Scholar]
  • 14.Shepherd JH. Revised FIGO staging for gynaecological cancer. Br J Obstet Gynaecol. 1989;96(8):889–92. doi: 10.1111/j.1471-0528.1989.tb03341.x. [Shepherd JH.Revised FIGO staging for gynaecological cancer[J].Br J Obstet Gynaecol, 1989, 96(8):889-92.] [DOI] [PubMed] [Google Scholar]
  • 15.Zhang P, Wang C, Cheng L, et al. Comparison of HE4, CA125, and ROMA diagnostic accuracy:a prospective and multicenter study for Chinese women with epithelial ovarian cancer. Medicine (Baltimore) 2015;94(52):e2402. doi: 10.1097/MD.0000000000002402. [Zhang P, Wang C, Cheng L, et al.Comparison of HE4, CA125, and ROMA diagnostic accuracy:a prospective and multicenter study for Chinese women with epithelial ovarian cancer[J].Medicine (Baltimore), 2015, 94(52):e2402.] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Tian Y, Wang C, Cheng L, et al. Determination of reference intervals of serum levels of human epididymis protein 4(HE4) in Chinese women. J Ovarian Res. 2015;8:72. doi: 10.1186/s13048-015-0201-z. [Tian Y, Wang C, Cheng L, et al.Determination of reference intervals of serum levels of human epididymis protein 4(HE4) in Chinese women[J].J Ovarian Res, 2015, 8:72.] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Zurawski VR Jr, Knapp RC, Einhorn N, et al. An initial analysis of preoperative serum CA 125 levels in patients with early stage ovarian carcinoma. Gynecol Oncol. 1988;30(1):7–14. doi: 10.1016/0090-8258(88)90039-X. [Zurawski VR Jr, Knapp RC, Einhorn N, et al.An initial analysis of preoperative serum CA 125 levels in patients with early stage ovarian carcinoma[J].Gynecol Oncol.1988, 30(1):7-14.] [DOI] [PubMed] [Google Scholar]
  • 18.Fujiwara H, Suzuki M, Takeshima N, et al. Evaluation of human epididymis protein 4(HE4) and Risk of Ovarian Malignancy Algorithm (ROMA) as diagnostic tools of type Ⅰ and type Ⅱ epithelial ovarian cancer in Japanese women. Tumour Biol. 2015;36(2):1045–53. doi: 10.1007/s13277-014-2738-7. [Fujiwara H, Suzuki M, Takeshima N, et al.Evaluation of human epididymis protein 4(HE4) and Risk of Ovarian Malignancy Algorithm (ROMA) as diagnostic tools of type Ⅰ and type Ⅱ epithelial ovarian cancer in Japanese women[J].Tumour Biol, 2015, 36(2):1045-53.] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Sturgeon CM, Duffy MJ, Stenman UH, et al. National Academy of Clinical Biochemistry laboratory medicine practice guidelines for use of tumor markers in testicular, prostate, colorectal, breast, and ovarian cancers. Clin Chem. 2008;54(12):e11–79. doi: 10.1373/clinchem.2008.105601. [Sturgeon CM, Duffy MJ, Stenman UH, et al.National Academy of Clinical Biochemistry laboratory medicine practice guidelines for use of tumor markers in testicular, prostate, colorectal, breast, and ovarian cancers[J].Clin Chem, 2008, 54(12):e11-79.] [DOI] [PubMed] [Google Scholar]
  • 20.Buys SS, Partridge E, Black A, et al. Effect of screening on ovarian cancer mortality:the Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Randomized Controlled Trial. JAMA. 2011;305(22):2295–303. doi: 10.1001/jama.2011.766. [Buys SS, Partridge E, Black A, et al.Effect of screening on ovarian cancer mortality:the Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Randomized Controlled Trial[J].JAMA, 2011, 305(22):2295-303.] [DOI] [PubMed] [Google Scholar]
  • 21.Kobayashi H, Yamada Y, Sado T, et al. A randomized study of screening for ovarian cancer:a multicenter study in Japan. Int J Gynecol Cancer. 2008;18(3):414–20. doi: 10.1111/j.1525-1438.2007.01035.x. [Kobayashi H, Yamada Y, Sado T, et al.A randomized study of screening for ovarian cancer:a multicenter study in Japan[J].Int J Gynecol Cancer, 2008, 18(3):414-20.] [DOI] [PubMed] [Google Scholar]
  • 22.Hellstrom I, Raycraft J, Hayden-Ledbetter M, et al. The HE4(WFDC2) protein is a biomarker for ovarian carcinoma. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=10.5372/1905-7415.0506.103. Cancer Res. 2003;63(13):3695–700. [Hellstrom I, Raycraft J, Hayden-Ledbetter M, et al.The HE4(WFDC2) protein is a biomarker for ovarian carcinoma[J].Cancer Res, 2003, 63(13):3695-700.] [PubMed] [Google Scholar]
  • 23.Urban N, Thorpe J, Karlan BY, et al. Interpretation of single and serial measures of HE4 and CA125 in asymptomatic women at high risk for ovarian cancer. Cancer Epidemiol Biomarkers Prev. 2012;21(11):2087–94. doi: 10.1158/1055-9965.EPI-12-0616. [Urban N, Thorpe J, Karlan BY, et al.Interpretation of single and serial measures of HE4 and CA125 in asymptomatic women at high risk for ovarian cancer[J].Cancer Epidemiol Biomarkers Prev, 2012, 21(11):2087-94.] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Sandhu N, Karlsen MA, Hogdall C, et al. Stability of HE4 and CA125 in blood samples from patients diagnosed with ovarian cancer. Scand J Clin Lab Invest. 2014;74(6):477–84. doi: 10.3109/00365513.2014.903430. [Sandhu N, Karlsen MA, Hogdall C, et al.Stability of HE4 and CA125 in blood samples from patients diagnosed with ovarian cancer[J].Scand J Clin Lab Invest, 2014, 74(6):477-84.] [DOI] [PubMed] [Google Scholar]
  • 25.Bolstad N, Oijordsbakken M, Nustad K, et al. Human epididymis protein 4 reference limits and natural variation in a Nordic reference population. Tumour Biol. 2012;33(1):141–8. doi: 10.1007/s13277-011-0256-4. [Bolstad N, Oijordsbakken M, Nustad K, et al.Human epididymis protein 4 reference limits and natural variation in a Nordic reference population[J].Tumour Biol, 2012, 33(1):141-8.] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Shah CA, Lowe KA, Paley P, et al. Influence of ovarian cancer risk status on the diagnostic performance of the serum biomarkers mesothelin, HE4, and CA125. Cancer Epidemiol Biomarkers Prev. 2009;18(5):1365–72. doi: 10.1158/1055-9965.EPI-08-1034. [Shah CA, Lowe KA, Paley P, et al.Influence of ovarian cancer risk status on the diagnostic performance of the serum biomarkers mesothelin, HE4, and CA125[J].Cancer Epidemiol Biomarkers Prev, 2009, 18(5):1365-72.] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Braicu EI, Van Gorp T, Nassir M, et al. Preoperative HE4 and ROMA values do not improve the CA125 diagnostic value for borderline tumors of the ovary (BOT)-a study of the TOC Consortium. J Ovarian Res. 2014;7:49. doi: 10.1186/1757-2215-7-49. [Braicu EI, Van Gorp T, Nassir M, et al.Preoperative HE4 and ROMA values do not improve the CA125 diagnostic value for borderline tumors of the ovary (BOT)-a study of the TOC Consortium[J].J Ovarian Res, 2014, 7:49.] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Jiang J, Bo D, Chang X, et al. Discrepant clinicopathologic characteristics and HE4 performance in type Ⅰ and type Ⅱ epithelial ovarian cancer. http://www.ncbi.nlm.nih.gov/pubmed/26885070. Int J Clin Exp Med. 2015;8(11):21303–10. [Jiang J, Bo D, Chang X, et al.Discrepant clinicopathologic characteristics and HE4 performance in type Ⅰ and type Ⅱ epithelial ovarian cancer[J].Int J Clin Exp Med, 2015, 8(11):21303-10.] [PMC free article] [PubMed] [Google Scholar]
  • 29.Montagnana M, Lippi G, Ruzzenente O, et al. The utility of serum human epididymis protein 4(HE4) in patients with a pelvic mass. J Clin Lab Anal. 2009;23(5):331–5. doi: 10.1002/jcla.20340. [Montagnana M, Lippi G, Ruzzenente O, et al.The utility of serum human epididymis protein 4(HE4) in patients with a pelvic mass[J].J Clin Lab Anal, 2009, 23(5):331-5.] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Terlikowska KM, Dobrzycka B, Witkowska AM, et al. Preoperative HE4, CA125 and ROMA in the differential diagnosis of benign and malignant adnexal masses. J Ovarian Res. 2016;9(1):43. doi: 10.1186/s13048-016-0254-7. [Terlikowska KM, Dobrzycka B, Witkowska AM, et al.Preoperative HE4, CA125 and ROMA in the differential diagnosis of benign and malignant adnexal masses[J].J Ovarian Res, 2016, 9(1):43.] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Ferraro S, Braga F, Lanzoni M, et al. Serum human epididymis protein 4 vs carbohydrate antigen 125 for ovarian cancer diagnosis:a systematic review. J Clin Pathol. 2013;66(4):273–81. doi: 10.1136/jclinpath-2012-201031. [Ferraro S, Braga F, Lanzoni M, et al.Serum human epididymis protein 4 vs carbohydrate antigen 125 for ovarian cancer diagnosis:a systematic review[J].J Clin Pathol, 2013, 66(4):273-81.] [DOI] [PubMed] [Google Scholar]
  • 32.Chudecka-Glaz A, Cymbaluk-Ploska A, Jastrzebska J, et al. Can ROMA algorithm stratify ovarian tumor patients better when being based on specific age ranges instead of the premenopausal and postmenopausal status? Tumour Biol. 2016:8879–87. doi: 10.1007/s13277-015-4733-z. [Chudecka-Glaz A, Cymbaluk-Ploska A, Jastrzebska J, et al.Can ROMA algorithm stratify ovarian tumor patients better when being based on specific age ranges instead of the premenopausal and postmenopausal status[J]?Tumour Biol.2016:8879-87.] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Moore RG, Miller MC, Disilvestro P, et al. Evaluation of the diagnostic accuracy of the risk of ovarian malignancy algorithm in women with a pelvic mass. Obstet Gynecol. 2011;118(2 Pt 1):280–8. doi: 10.1097/aog.0b013e318224fce2. [Moore RG, Miller MC, Disilvestro P, et al.Evaluation of the diagnostic accuracy of the risk of ovarian malignancy algorithm in women with a pelvic mass[J].Obstet Gynecol, 2011, 118(2 Pt 1):280-8.] [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Journal of Southern Medical University are provided here courtesy of Editorial Department of Journal of Southern Medical University

RESOURCES