Abstract
Purpose:
This study aims to compare the accuracy of the ADNEX MR scoring system and pattern recognition system to evaluate adnexal lesions indeterminate on the US exam.
Methods:
In this cross-sectional retrospective study, pelvic DCE-MRI of 245 patients with 340 adnexal masses was studied based on the ADNEX MR scoring system and pattern recognition system.
Results:
ADNEX MR scoring system with a sensitivity of 96.6% and specificity of 91% has an accuracy of 92.9%. The pattern recognition system’s sensitivity, specificity, and accuracy are 95.8%, 93.3%, and 94.7%, respectively. PPV and NPV for the ADNEX MR scoring system were 85.1 and 98.1, respectively. PPV and NPV for the pattern recognition system were 89.7% and 97.7%, respectively. The area under the ROC curve for the ADNEX MR scoring system and pattern recognition system is 0.938 (95% CI, 0.909-0.967) and 0.950 (95% CI, 0.922-0.977). Pairwise comparison of these AUCs showed no significant difference (p = 0.052).
Conclusion:
The pattern recognition system is less sensitive than the ADNEX MR scoring system, yet more specific.
Key Words: Adnexal lesions, pattern recognition, MR scoring system
Introduction
With roughly 240,000 women diagnosed with ovarian cancer yearly, adnexal masses are common in women’s pelvic imaging studies [1]. Ovarian cancer is one of the most frequent malignancies in women, accounting for 3.6% of all cancers and resulting in a death rate of 4.3 percent. In Europe, ovarian cancer is the leading cause of death from gynecologic cancers, the fifth most common cancer (after breast, colorectal, lung, and uterine cancers), and the sixth most common cause of cancer death (after breast, colorectal, lung, pancreatic and stomach cancers) [2].
Preoperative examination and estimation of malignancy-risk are essential to determine the treatment plan. If the risk of malignancy is low, patients can undergo minor and less invasive surgeries by the general surgeon, but if the risk of malignancy is significant, surgery may be required in referral centers. The specialized team includes a gynecologist oncologist [3, 4]. An optimal ovarian mass assessment requires a multidisciplinary approach based on clinical findings, laboratory results, and imaging modalities. In the assessment of adnexal lesions, ultrasound is the first-line imaging modality. Most ovarian masses are benign and properly classified as benign and malignant on ultrasound. However, according to ultrasound findings, about 20% of ovarian masses remain indeterminate [5-7].
In these cases, pelvic MRI has a high potential for preoperative examination of ovarian masses, and accuracy showed higher (88.9%) than transvaginal ultrasound (63.9%). With the addition of PERFUSION and DIFFUSION WEIGHTED sequences, with an increase of 25% and 15% accuracy compared to conventional MRI, respectively, the overall accuracy has increased by more than 90% [8-11]. Various MRI protocols have been introduced to examine adnexal masses [12, 13]. The MRI scoring system for adnexal lesions (ADNEX MR scoring system) introduced by Thomassin-Naggara in 2013 had a sensitivity of 93.5% and a specificity of 96.6% in diagnosing malignant adnexal masses [14]. Adnexal lesions based on pattern recognition, including morphological characteristics in contrast-enhanced MRI and clinical and laboratory data, were studied before the development of this approach.
Based on the radiologist expertise, in the lesions which are placed in the malignant category as false positive according to ADNEX MR scoring system, pattern recognition system seems to have higher accuracy.
This study aims to compare the accuracy of these two protocols to evaluate adnexal lesions, which are indeterminate on the US exam.
Materials and Methods
In this cross-sectional retrospective study, pelvic MRI of all women over 18 years of age who underwent pelvic MRI by contrast injection due to the indeterminate ultrasound findings (lesions which could not be categorized as benign or malignant based on US exam findings) in two referral hospital centers during March 2016-August 2021, have been examined. All patients whose histopathological outcome was available following adnexal mass surgery or underwent follow-up for at least one year and available follow-up information were included in the study. The cases with Imaging artifacts or inadequate MRI protocol and the Patients whose subsequent surgical or follow-up data were unavailable were excluded. A total of 245 patients with 340 adnexal masses were studied, of which 87 patients had more than one adnexal mass.
Ethics statement
This study was approved by the local ethics committee and the need for written informed consent was waived by the ethical approval committee. The privacy rights of human subjects were observed throughout the research process.
MRI Protocol
Patients have been fasting for at least three hours before MRI. Imaging was performed with three Tesla MRI (GE 3Tdiscovery750Gem, phase array pelvic coil surface).
Axial, sagittal, and coronal T2-weighted fast spin echo, axial T2-Weighted sequences with fat suppression, and T1-sequences with and without fat suppressions were performed. Diffusion weighted images were captured in the axial plane with b-value = 50,500 and 1,000 s / mm2, which is routinely used in clinical practice.
Contrast material gadolinium chelate with a dose of 0.2ml / Kg body weight at a rate of 3ml / sec was injected followed by 20cc of normal saline by the injector. Beginning 15 seconds after the contrast injection, 20-second intervals (5 intervals) were taken to produce post-contrast images. This technique is based on Thomassin-MRI Naggara’s scoring method for adnexal lesions (ADNEX MR scoring system), which was developed in 2013.
Image Analysis:
Two radiologists with at least 10 years of experience in Female pelvic MRI reporting independently and without knowledge of clinical and laboratory findings, histopathology, and follow-up results studied MRI images and the calculation of ADNEX MR Score. Each radiologist evaluated about half of the cases. Then, after 3 months intervals, based on the pattern recognition system according to total clinical, laboratory, and MRI findings (solid or cystic nature of the lesion, size of the lesion, having enhancing solid portions, showing diffusion restriction, and other important findings based on the experience of a radiologist, not just based on published scoring systems), the masses were divided into two categories: benign and malignant. Each radiologist evaluated one another half of the cases that were assessed previously based on the ADNEX MR Score. Clinical date was based on the patient’s claimed history and consulted with clinicians. Borderline masses were considered malignant in statistical analysis [15].
The calculation of the score in the MR Scoring system for adnexal lesions (ADNEX MR scoring system) is as follows
1. Lack of adnexal mass (not included in the study)
2. Benign masses include: purely cystic, presence of fatty or endometrioid masses, lack of wall enhancement in masses without solid segment, low signal on T2 images or diffusion-weighted within the solid segment, masses with solid segment with type 2 curve (TIC) Or Non-feasible and lack of wall enhancement.
3. Probably benign masses: wall enhancement in masses without solid segment or type 1 curve enhancement (TIC) in the solid segment.
4. Indeterminate masses: solid tissue with type 2 curve (TIC) and wall enhancement
5. Probably malignant masses: Peritoneal implant or type 3 curve (TIC) in the solid segment
The standard indicator in this study is histopathological diagnosi18s [16].
The cases that were not candidates for surgery and whose histopathological results are unavailable will be evaluated based on clinical monitoring and imaging within at least one year. For cases categorized as Score 3 based on ADNEX MR Score, the follow-up US exam was considered the follow-up method, and further evaluation was according to the US exam result. In these cases, the origin of the mass considered adnexa if there was agreement by two experienced readers.
Statistical analysis
Data were analyzed using SPSS software version 22. The descriptive section reported quantitative variables with mean and standard deviation and qualitative variables with number and percentage. The relationship between patients’ baseline information and histology type with mass type was investigated using the Chi-Square test in the analytical ward. The relationship between the findings of scoring systems and the type of adnexal masses was assessed using Chi-Square and Fisher’s Exact tests. Diagnostic parameters of scoring systems, including sensitivity, specificity, positive predictive value, negative predictive value and accuracy were calculated for both indicators (ADNEX MR scoring system and Pattern recognition), and their validity was evaluated in ROC test. In all tests, P <0.05 was considered significant. Score >4 was considered as cut off for malignancy [14]. Interobserver agreement was assessed using unweighted and Fleiss kappa indices.
Results
A total of 245 patients with 340 adnexal masses were studied, of which 87 patients had more than one adnexal mass. The mean age of patients is 43+13 years. Patients with malignant masses were older than those with benign masses (mean age of 41 compared to 46 years / P= 0.011).Malignant masses were larger compared to benign types (mean largest mass diameter 88 mm in malignant types and 65 mm in benign masses with P <0.0001). 81.8 % of adnexal masses was originated from the ovary (Table 1).
Table 1.
Description and Evaluation of the Relationship between Patients' Basic Information and Adnexal Masses
| Variable | Benign | Malignant | P Value |
|---|---|---|---|
| Age (Year) | 41+13 | 46+14 | 0.011 |
| Largest Diameter (mm) | 65±44.5 | 88±50 | <0.0001 |
| Follow up | 142 (64) | 0 (0) | <0.0001 |
| Surgery | 80 (36) | 118 (100) | |
| premenopausal | 194 (87.4) | 83 (70.3) | <0.0001 |
| postmenopausal | 28 (12.6) | 35 (29.7) |
Among masses with malignant pathology, tumors of epithelial origin were the most common (80 masses / 67.8%). Metastases were the second most common type of malignant tumors (28 masses / 23.7%), and tumors of SEX CORD origin (7 masses / 6%) and GERM CELL (two masses / 1.7%) were the next most common types, respectively.
ADNEX MR scoring system with a sensitivity of 96.6% and specificity of 91% has an accuracy of 92.9%.
The pattern recognition system’s sensitivity, specificity, and accuracy are 95.8%, 93.3%, and 94.7%, respectively.
PPV and NPV for the ADNEX MR scoring system were 85.1 and 98.1, respectively.
PPV and NPV for pattern recognition systems were 89.7% and 97.7%, respectively (Table 2).
Table 2.
Diagnostic Parameters of ADNEX MR Scoring System and Pattern Recognition System of Patients with benign and Malignant Adnexal Masses
| Parameter | Scoring system | |
|---|---|---|
| ADNEX MR scoring system | pattern recognition |
|
| Sensitive (%) | 96.6 | 95.8 |
| Specificity (%) | 91 | 93.3 |
| PPV (%) | 85.1 | 89.7 |
| NPV (%) | 98.1 | 97.7 |
| Accuracy (%) | 92.9 | 94.7 |
The area under the ROC curve for the ADNEX MR scoring system and pattern recognition system is 0.938 (95% CI, 0.909-0.967) and 0.950 (95% CI, 0.922-0.977), respectively, with a significant difference (<0.0001). (Table 3 and Figure 1)
Table 3.
Area under the ROC Curve of ADNEX MR Scoring System and Pattern Recognition System
| Scoring system | Area | 95% Confidence Interval | P Value |
|---|---|---|---|
| ADNEX MR scoring system |
0.938 | 0.909-0.967 | <0.0001 |
| pattern recognition system | 0.95 | 0.922-0.977 | <0.0001 |
Figure 1.
ROC Curve Related to the ADNEX MR Scoring System and Pattern Recognition System
Among masses with benign pathology, 21 masses (9.5%) upgraded to malignant using the ADNEX MR scoring system. These cases have received> SCORE 4 regarding having a solid enhancing part with TIC of 2 or 3.
The histopathological result of these masses includes the following:
Infected endometrioma (1 case)/ (Figure 2), Serous cyst adenoma (1 case)/ (Figure 3), Mucinous cyst adenoma (1 case) Serous cyst adenofibroma (1 case), seromucinous cyst adenoma (1 case), Fibrothecoma (2 cases), spindle cell tumor/leiomyoma (1 case), Struma ovarii (1 case), Tubo-ovarian abscess (6 cases), Salpingitis (2 cases), Fibroma with liquified necrosis (1 cases), Benign mesenchymal tumor (hemangioma or adenomatoid tumor) (1 case) and corpus luteum (2 cases)
Figure 2.
Images from 38-Year-Old Woman with Left-Sided Cystic Adnexal Mass Indeterminate in Transvaginal Ultrasound. (a) and (b) sagittal T2-weighted magnetic resonance (MR) images show leftsided multiloculated adnexal mass, with largest diameter of 52mm, with intermediate signalintensity locules and intermediate-signal-intensity thickened and irregular internal septa. (c) and (d) dynamic contrast enhanced MRI sequence yielded intermediate time-intensity curve with type-2 plateau (green line) in comparison with adjacent external myometrium (purple line). Histology confirmed benign infected endometrioma. ADNEX MR score was upgraded to 4 (type-2 curve)
Figure 3.
Images from 49-Year-Old Woman with Left-Sided Cystic Adnexal Mass Indeterminate in Transvaginal Ultrasound. (a) axial T2-weighted magnetic resonance (MR) images show left-sided multiloculated adnexal mass, with largest diameter of 137mm, with high signal-intensity locules and intermediate-signal-intensity thickened and irregular internal septa. (b) and (c) dynamic contrast enhanced MRI yielded intermediate time-intensity curve with type-2 plateau (purple line) in comparison with adjacent external myometrium (green line). Histology confirmed benign serous cystadenoma
14 masses (4.11%) with benign pathology were considered malignant based on pattern recognition. The histopathological result of these masses includes the following:
Serous cyst adenofibroma (1 case), mucinous cyst adenoma (1 case), seromucinous cyst adenoma (1 case), Serous inclusion cyst (1 case), Fibrothecoma (4 cases), spindle cell tumor/leiomyoma (1 case), Struma ovarii (1 case), Benign mesenchymal tumor (hemangioma or adenomatoid tumor) (1 case) and corpus luteum (2 cases)
The level of interobserver agreement was high (κ = 0.91).
Discussion
In this study, the sensitivity and specificity of the ADNEX MR scoring system are 96.6% and 91%, respectively.
In a retrospective study, in 2013, Thomassin-Naggara et al, a sensitivity of 93.5% and specificity of 96.6% was reported for the ADNEX MR scoring system [14]. Another study in 2016 found a sensitivity of 91.7% and a specificity of 92.7% for the diagnosis of malignancy with an ADNEX MR score of 4 < [17]. In the study with HOTTAT et al. in 2020, the sensitivity of 95.5% and specificity of 86.6% were obtained for ADNEX MR scoring system [15].
The observed differences could be in terms of the distribution of pathologies among these populations.
The sensitivity and specificity of the pattern recognition system are 95.8%, 93.3%, and 94.7%, respectively, which is less sensitive than the ADNEX MR scoring system and more specific. This difference could be in terms of pathology proved benign masses (12 masses) that upgraded to malignant based on the ADNEX MR scoring system but considered benign using pattern recognition.
Out of 340 masses evaluated, 20 cases (5.88%) with benign pathology upgraded to malignant using the ADNEX MR scoring system mentioned above in the Results. Based on the pattern recognition system, 14 masses (4.11%) with benign pathology were considered malignant, as mentioned above.
The pattern recognition system considers clinical findings, including history and physical examinations (after consulting the relevant clinician), and laboratory findings (including inflammatory and tumor markers). In cases with acute clinical symptoms, consideration of clinical and laboratory factors may effectively increase MRI accuracy. Notably, it should be borne in mind that in the present study, the assessment of Adnexal masses using both the ADNEX MR scoring system and pattern recognition was performed by experienced radiologists with more than 10 years of experience in the field of female pelvic MRI reporting and the details of both systems are known. Then, it cannot be said that the criteria considered in each system are only used for the same system specifically, which can cause errors in assessing the accuracy of the systems independently and separately.
Moreover, if the radiologist tries to make an assessment by limiting his/her vision using only the criteria of each system, there is still a possibility of error and the study result may not be generalized to the real model because in practice the radiologist interprets based on his total knowledge and not necessarily include a system in its entirety and in detail. Out of 340 masses, 12 were pathology proven benign serous cyst adenoma. Of these cases, 11 masses (91.6%) were classified as a score <4 in the ADNEX MR System, but 1 multiloculated cystic mass with enhancing solid portion with TIC=2 indicates a score of 4. The pathology result of this mass was serous cyst adenoma with focal proliferation. This mass was also considered malignant as a false positive based on pattern recognition.
Out of 34o masses, 3 were pathologies proven benign mucinous cyst adenoma. One of which, with an ADNEX MR score 5 had an enhanced solid segment with TIC = 3. Using a pattern recognition system, this mass was also classified as malignant.
According to studies, benign cyst adenoma tumors (including serous and mucinous) may have small papillary projections, but the presence of an enhancing solid portion in MRI favors malignancy [18]. Further studies are needed to determine whether the accuracy of the ADNEX MR scoring system for cyst adenomas differs from that of masses with other pathologies.
Limitations of the study
One year of follow-up information is available in adnexal masses without surgery. This time may be short for some slow-growing borderline masses, and longer follow-ups may be needed. The retrospective nature of this study could be considered another limitation.
In conclusion, the pattern recognition system is more specific and less sensitive than the ADNEX MR scoring system. However, using MRI scoring systems alone is more applicable with a younger and less experienced radiologist and take less time, considering clinical findings and Lab Data will result in more accurate diagnosis.
Author Contribution Statement
All authors contributed equally in this study.
Acknowledgements
Conflict of interest
The authors of the article believe that there is no conflict of interest.
References
- 1.Webb PM, Jordan SJ. Epidemiology of epithelial ovarian cancer. Best Pract Res Clin Obstet Gynaecol. 2017;41:3–14. doi: 10.1016/j.bpobgyn.2016.08.006. [DOI] [PubMed] [Google Scholar]
- 2.Ferlay J, Soerjomataram I, Dikshit R, Eser S, Mathers C, Rebelo M, et al. Cancer incidence and mortality worldwide: Sources, methods and major patterns in globocan 2012. Int J Cancer. 2015;136(5):E359–86. doi: 10.1002/ijc.29210. [DOI] [PubMed] [Google Scholar]
- 3.Menon U, Griffin M, Gentry-Maharaj A. Ovarian cancer screening--current status, future directions. Gynecol Oncol. 2014;132(2):490–5. doi: 10.1016/j.ygyno.2013.11.030. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Forstner R, Sala E, Kinkel K, Spencer JA. Esur guidelines: Ovarian cancer staging and follow-up. European Radiology. 2010;20(12):2773–80. doi: 10.1007/s00330-010-1886-4. [DOI] [PubMed] [Google Scholar]
- 5.Timmerman D, Valentin L, Bourne TH, Collins WP, Verrelst H, Vergote I. Terms, definitions and measurements to describe the sonographic features of adnexal tumors: A consensus opinion from the international ovarian tumor analysis (iota) group. Ultrasound Obstet Gynecol. 2000;16(5):500–5. doi: 10.1046/j.1469-0705.2000.00287.x. [DOI] [PubMed] [Google Scholar]
- 6.Timmerman D, Ameye L, Fischerova D, Epstein E, Melis GB, Guerriero S, et al. Simple ultrasound rules to distinguish between benign and malignant adnexal masses before surgery: Prospective validation by iota group. BMJ. 2010;341:c6839. doi: 10.1136/bmj.c6839. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Timmerman D, Van Calster B, Testa A, Savelli L, Fischerova D, Froyman W, et al. Predicting the risk of malignancy in adnexal masses based on the simple rules from the international ovarian tumor analysis group. Am J Obstet Gynecol. 2016;214(4):424–37. doi: 10.1016/j.ajog.2016.01.007. [DOI] [PubMed] [Google Scholar]
- 8.Bernardin L, Dilks P, Liyanage S, Miquel ME, Sahdev A, Rockall A. Effectiveness of semi-quantitative multiphase dynamic contrast-enhanced mri as a predictor of malignancy in complex adnexal masses: Radiological and pathological correlation. Europ Radiol. 2012;22(4):880–90. doi: 10.1007/s00330-011-2331-z. [DOI] [PubMed] [Google Scholar]
- 9.Dilks P, Narayanan P, Reznek R, Sahdev A, Rockall A. Can quantitative dynamic contrast-enhanced mri independently characterize an ovarian mass? Eur Radiol. 2010;20(9):2176–83. doi: 10.1007/s00330-010-1795-6. [DOI] [PubMed] [Google Scholar]
- 10.Takeuchi M, Matsuzaki K, Nishitani H. Diffusion-weighted magnetic resonance imaging of ovarian tumors: Differentiation of benign and malignant solid components of ovarian masses. J Comput Assist Tomogr. 2010;34(2):173–6. doi: 10.1097/RCT.0b013e3181c2f0a2. [DOI] [PubMed] [Google Scholar]
- 11.Thomassin-Naggara I, Toussaint I, Perrot N, Rouzier R, Cuenod CA, Bazot M, et al. Characterization of complex adnexal masses: Value of adding perfusion- and diffusion-weighted mr imaging to conventional mr imaging. Radiology. 2011;258(3):793–803. doi: 10.1148/radiol.10100751. [DOI] [PubMed] [Google Scholar]
- 12.Forstner R, Thomassin-Naggara I, Cunha TM, Kinkel K, Masselli G, Kubik-Huch R, et al. Esur recommendations for mr imaging of the sonographically indeterminate adnexal mass: An update. Eur Radiol. 2017;27(6):2248–57. doi: 10.1007/s00330-016-4600-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Allen BC, Hosseinzadeh K, Qasem SA, Varner A, Leyendecker JR. Practical approach to mri of female pelvic masses. AJR Am J Roentgenol. 2014;202(6):1366–75. doi: 10.2214/AJR.13.12023. [DOI] [PubMed] [Google Scholar]
- 14.Thomassin-Naggara I, Aubert E, Rockall A, Jalaguier-Coudray A, Rouzier R, Daraï E, et al. Adnexal masses: Development and preliminary validation of an mr imaging scoring system. Radiology. 2013;267(2):432–43. doi: 10.1148/radiol.13121161. [DOI] [PubMed] [Google Scholar]
- 15.Hottat NA, Van Pachterbeke C, Vanden Houte K, Denolin V, Jani JC, Cannie MM. Magnetic resonance scoring system for assessment of adnexal masses: Added value of diffusion-weighted imaging including apparent diffusion coefficient map. Ultrasound Obstet Gynecol. 2021;57(3):478–87. doi: 10.1002/uog.22090. [DOI] [PubMed] [Google Scholar]
- 16.Carcangiu ML, Kurman RJ, Carcangiu ML, Herrington CS. WHO classification of tumours of female reproductive organs. 4th ed. International Agency for Research on Cancer; 2014. [Google Scholar]
- 17.Ruiz M, Labauge P, Louboutin A, Limot O, Fauconnier A, Huchon C. External validation of the mr imaging scoring system for the management of adnexal masses. Eur J Obstet Gynecol Reprod Biol. 2016;205:115–9. doi: 10.1016/j.ejogrb.2016.07.493. [DOI] [PubMed] [Google Scholar]
- 18.Valentini AL, Gui B, Miccò M, Mingote MC, De Gaetano AM, Ninivaggi V, et al. Benign and suspicious ovarian masses-mr imaging criteria for characterization: Pictorial review. J Oncol. 2012;2012:481806. doi: 10.1155/2012/481806. [DOI] [PMC free article] [PubMed] [Google Scholar]