Integrating Contrast-enhanced US to O-RADS US for Classification of Adnexal Lesions with Solid Components: Time-intensity Curve Analysis versus Visual Assessment

Abstract

Purpose

To compare the diagnostic performance of time-intensity curve (TIC) analysis and subjective visual assessment of contrast-enhanced US (CEUS) when integrated with the Ovarian-Adnexal Reporting and Data System (O-RADS) US risk stratification system for characterizing adnexal lesions with solid components.

Materials and Methods

In this prospective multicenter study conducted from September 2021 to December 2022, female individuals with suspected adnexal lesions containing solid components detected at routine US were enrolled. All participants underwent preoperative CEUS examinations. Histopathologic findings were used as the reference standard for diagnosis. Lesions were classified according to the O-RADS US system. Enhancement of solid tissue compared with the outer myometrium was evaluated using both TIC analysis and subjective visual assessment. The diagnostic performance of O-RADS alone and each CEUS assessment method when integrated with the O-RADS US system was assessed and compared using receiver operating characteristic curve analysis.

Results

A total of 180 lesions (80 malignant and 100 benign histopathologic outcomes) in 175 participants (median age, 47 years [IQR, 33–56]) were analyzed. Incorporating CEUS (assessed through both TIC analysis and subjective visual assessment) with O-RADS US showed significantly improved diagnostic performance over O-RADS US alone, with an area under the receiver operating characteristic curve (AUC) of 0.86 (95% CI: 0.80, 0.91) compared with 0.78 (95% CI: 0.71, 0.84). No evidence of a difference was observed between the AUCs of TIC analysis and subjective visual assessment in the enhancement evaluation of solid tissue with CEUS for adnexal malignancy categorization (P = .83).

Conclusion

Subjective visual assessment and TIC analysis of CEUS features when integrated with the O-RADS US scoring system showed comparable diagnostic performance in assigning adnexal malignancy risk.

Keywords: Adnexal Lesions, Contrast-enhanced US, O-RADS, Time-intensity Curve Analysis

Supplemental material is available for this article.

© RSNA, 2024

An earlier incorrect version appeared online. This article was corrected on October 23, 2024.

Summary

Subjective visual assessment of solid tissue enhancement demonstrated comparable performance to time-intensity curve analysis when integrating contrast-enhanced US to the O-RADS US score for assigning adnexal malignancy risk.

Key Points

• ■ Time-intensity curve analysis and subjective visual assessment of solid tissue enhancement in contrast-enhanced US demonstrated comparable diagnostic performance in classifying adnexal malignancy (area under the receiver operating characteristic curve [AUC], 0.86 for both; P = .83) when integrated with the Ovarian-Adnexal Reporting and Data System (O-RADS) US score.

• ■ Inclusion of both quantitative and qualitative contrast-enhanced US assessment significantly improved the diagnostic performance of O-RADS (AUC for contrast-enhanced US combined with O-RADS vs O-RADS alone: 0.86 vs 0.78; P < .001).

• ■ There was moderate to very good agreement between visual assessment and time-intensity curve analysis of contrast-enhanced US features, with κ values ranging from 0.58 to 1.00.

Introduction

Ovarian cancer ranks as the second leading cause of cancer-related deaths globally, characterized by a 5-year survival rate below 45% (1,2). The accurate distinction between benign and malignant ovarian lesions is critical for effective patient management. Prompt referral of patients diagnosed with malignant tumors to gynecologic oncologists is associated with improved outcomes. In contrast, benign tumors often allow for conservative management, thus avoiding unnecessary or extensive surgery (3–6).

US is the preferred first-line imaging technique for evaluating adnexal lesions (6,7). Since 2010, several US-based systematic classification schemes have been developed to predict adnexal malignancy, including the International Ovarian Tumor Analysis (IOTA) simple rules (8) and the Gynecologic Imaging Reporting and Data System (GI-RADS) (9). The American College of Radiology introduced the Ovarian-Adnexal Reporting and Data System (O-RADS) in their 2019 consensus guidelines (10). O-RADS, focusing on the US characteristics of ovarian lesions, seeks to standardize risk stratification and management approaches.

Contrast-enhanced US (CEUS) demonstrates high diagnostic accuracy in assessing adnexal lesions, outperforming conventional color Doppler sonography (11). Combining O-RADS with CEUS has shown improved diagnostic performance in differentiating benign from malignant adnexal masses compared with O-RADS alone (12,13). The integration of CEUS into O-RADS potentially enhances specificity in risk stratification of adnexal tumors. However, previous studies mainly relied on qualitative assessments of CEUS (12,13). Time-intensity curve (TIC) analysis, used for quantification of tumor vascularization, might provide greater accuracy than subjective assessments, particularly in evaluating tumor enhancement phases. Yet the process of TIC analysis is relatively laborious and requires advanced technical capabilities. The question that arises is whether the subjective assessment by experienced radiologists is as accurate as TIC analysis when CEUS is integrated within the O-RADS framework.

This study aimed to evaluate whether the diagnostic performance of subjective visual assessment of CEUS features is as good as TIC analysis when integrated with the O-RADS US risk stratification system in characterizing adnexal lesions with solid components.

Materials and Methods

Participants

This prospective multicenter study was conducted at 12 tertiary hospitals in China, with approvals from the institutional review boards of each participating hospital. Written informed consent was obtained from all participants prior to enrollment. From September 2021 to December 2022, a total of 200 individuals with 207 adnexal masses with solid components detected at routine US and scheduled for surgical intervention were screened for the study. All participants underwent preoperative CEUS examinations. Exclusion criteria included pregnancy and a history of bilateral oophorectomy. Pathologic diagnosis following surgery (reference standard) confirmed all adnexal lesions. For the purpose of this study, borderline lesions were classified as malignant.

US Protocol

Radiologists from each participating center, having at least 5 years of experience in gynecologic US, received standardized training before the study commenced. Each center had two qualified radiologists to conduct US and CEUS examinations, adhering to established guidelines and regulations. The examinations were performed using two US systems: the Epiq 7 with a C10-3v transvaginal transducer and a C5-1 transabdominal transducer (Philips Healthcare) and the Nuewa R9 with a V11-3HU transvaginal and a SC6-1U transabdominal transducer (Mindray).

A standardized protocol was followed for all CEUS examinations to ensure consistency among study participants. CEUS was performed, primarily transvaginally, immediately after B-mode US. For larger lesions that could not include a portion of the outer myometrium as well as the solid component of an adnexal mass in the same field of view using transvaginal CEUS, transabdominal CEUS was performed. As dictated by our study protocol, the area examined with CEUS included the adjacent myometrium of the uterus and the most vascularized solid part of the adnexal mass at power Doppler imaging. The contrast agent, SonoVue (Bracco Imaging), was administered intravenously as a 2.4-mL bolus (both for transabdominal and transvaginal protocols), followed by a 5-mL flush of 0.9% sodium chloride solution through the cubital vein. A 120-second CEUS video clip was recorded with the probe held motionless for each adnexal lesion. The imaging and video data of the lesion were saved as a Digital Imaging and Communications in Medicine file and uploaded to the principal investigator’s unit via the cloud storage. The principal investigator (X.Z., with over 15 years of experience in gynecologic US) conducted quality control of the imaging data, and cases where the lesions and the adjacent uterine myometrium were clearly visible were considered eligible for the study.

Image Review

For quality assurance, images and video clips from all centers were independently interpreted by two experienced senior radiologists from the Third Affiliated Hospital of Sun Yat-sen University, each with over 5 years of experience in gynecologic US. Both radiologists were blinded to the patients’ clinical and pathologic data and independently analyzed the contrast-enhanced videos quantitatively and qualitatively. Each lesion was assigned an O-RADS category based on the standardized lexicon by each radiologist independently.

The solid components of adnexal lesions at CEUS were reviewed independently by the same radiologists using cine clips for both qualitative and quantitative analysis, with the normal myometrium serving as a reference. Visual assessment of enhancement in solid or solid-appearing components relative to the outer myometrium was initially conducted, categorizing the enhancement timing of the lesion as earlier, synchronous, or later than the myometrium. The enhancement degree was classified as no enhancement, hypoenhancement, or iso- or hyperenhancement, compared with the outer myometrium at maximum contrast intensity of the lesion. Visual assessment also included recording the contrast material washout, defined as any intensity loss from peak enhancement.

Quantitative analysis involved placing two regions of interest (each 5 × 5 mm) on the solid tissue at maximum contrast intensity (subjectively selected) and on the outer myometrium, respectively. The TIC analysis was analyzed at the peak intensity of the adnexal lesion. CEUS quantitative parameters were recorded using TIC analysis with QLAB offline analysis software (release 10; Philips Ultrasound) or Ultra Office offline analysis software (release 1; Mindray Ultrasound).

In cases with bilateral lesions, each was reviewed and categorized independently. Any disagreements in evaluations were resolved through discussion until a consensus was reached.

O-RADS Integrated with CEUS

The scoring criteria for CEUS, as outlined in Table 1, follow those established in a prior study (13). The O-RADS category was adjusted based on the CEUS scores: upgraded by one level for scores of 4 or higher and downgraded by one level for scores of 2 or lower. A score of 3 resulted in no change to the O-RADS category.

Table 1:

Description of CEUS Scoring Criteria

Statistical Analysis

Sample size calculation was conducted using the NCSS-PASS software (version 15.0). Sample size calculation was based on expected sensitivity and specificity of 95% and 75%, respectively, as determined from previous research (12). To achieve a statistical power of 90% at a two-sided significance level of 5%, a total of 120 lesions were required for the sample analysis.

Continuous variables are presented as medians with IQRs in parentheses, and categorical variables are represented as numbers and percentages in parentheses. The Cohen κ statistic was employed to evaluate the agreement of CEUS features between qualitative and quantitative analyses. The interobserver agreement of O-RADS US and modified O-RADS US score based on CEUS of the two senior radiologists were assessed with the weighted κ statistic. The level of agreement for the κ statistic was interpreted as follows: 0.01–0.20 indicated poor agreement, 0.21–0.40 signified fair agreement, 0.41–0.60 denoted moderate agreement, 0.61–0.80 indicated good agreement, and 0.81–1.00 represented very good agreement. Accuracy was evaluated to assess the ratio of the number of correct predictions. Receiver operating characteristic curve analysis was employed to assess the discrimination ability of various parameters. The DeLong test was utilized to compare the areas under the receiver operating characteristic curve (AUCs) among different parameters. A score of O-RADS US 4 was used as cutoff for adnexal malignancy. Sensitivity, specificity, positive predictive value, and negative predictive value, along with their corresponding 95% CIs, were calculated.

All statistical analyses were performed using IBM SPSS Statistics software for Windows (version 22) and MedCalc software (version 11.2). A two-tailed P value less than .05 was considered statistically significant.

Results

Participant and Lesion Characteristics

Of the initially assessed 207 lesions from 200 participants, 10 adnexal lesions from nine participants were excluded due to the absence of surgical intervention, and 17 adnexal lesions from 16 participants were excluded due to the absence of a uterus, as shown in Figure 1. The final study cohort comprised 180 adnexal lesions from 175 female participants, with a median age of 47 years (IQR, 33–56). Pathologic analysis revealed 100 benign and 80 malignant lesions, indicating a malignancy prevalence of 44.4% (80 of 180) within the study population. Detailed demographics of participants and lesions are summarized in Table 2.

Figure 1:

Study inclusion flowchart. This flowchart details the process of participant selection and inclusion criteria for the study. There are five participants with bilateral lesions. Three participants had malignant tumors in both adnexal regions, and one participant had benign tumors in both adnexal regions. One participant had a benign tumor on one side and a malignant tumor on the other side of the adnexal region.

Table 2:

Participant and Lesion Characteristics

Table 3 details the histopathologic outcomes of the evaluated adnexal lesions categorized according to the O-RADS system alongside their corresponding CEUS assessments using both visual assessment and TIC analysis. Using O-RADS 4 and 5 as markers of malignancy, the O-RADS US score accurately classified lesions as benign or malignant in 57.8% (104 of 180) and misclassified 42.2% (76 of 180) of lesions. When combining O-RADS with features observed at CEUS through visual assessment, the accuracy improved to 77.8% (140 of 180) for correct categorization, with a misclassification rate of 22.2% (40 of 180) in differentiating between benign and malignant (including borderline and invasive) lesions. Utilizing TIC analysis, the combination of O-RADS and CEUS features accurately categorized 76.1% (137 of 180) of lesions, while 23.9% (43 of 180) were misclassified. In our study, there were no substantial technical failures during the CEUS examinations.

Table 3:

Descriptive Results of Assessed Adnexal Lesions Expressed as Histopathologic Results with CEUS Scores Using TIC Analysis (Objective) and Visual Assessment (Subjective)

Agreement of CEUS Features between Qualitative and Quantitative Analysis

Table S1 shows the agreement of CEUS features between qualitative and quantitative analysis. Among the 37 adnexal masses showing no internal enhancement, visual assessment and TIC analysis demonstrated complete agreement, with a 100% concordance rate. Among the 143 adnexal masses showing internal enhancement, there was good agreement between visual assessment and TIC analysis in terms of enhancement timing and intensity relative to the myometrium, with κ values of 0.65 (95% CI: 0.52, 0.77) and 0.63 (95% CI: 0.51, 0.76), respectively. During the washout phase, the agreement between these methods was moderate, evidenced by a κ value of 0.58 (95% CI: 0.45, 0.72).

Interobserver Agreement of the O-RADS US and Modified O-RADS US Scores of the Two Senior Radiologists

As shown in Table S2, the two senior radiologists demonstrated very good agreement in O-RADS US category assignments, with a weighted κ value of 0.80 (95% CI: 0.73, 0.86). The two senior radiologists also demonstrated very good interobserver agreement in the modified O-RADS US category assignments based on CEUS examinations, both for subjective analysis and TIC analysis, evidenced by a weighted κ value of 0.81 (95% CI: 0.74, 0.88) and 0.82 (95% CI: 0.76, 0.89), respectively.

The two reviewers did not agree and had to reach a consensus for 40 and 34 cases during subjective and TIC analysis, respectively. During subjective analysis of CEUS examinations, radiologist classifications differed by more than one category when examining one lesion in a 38-year-old female participant with confirmed diagnosis of a borderline tumor which manifested as a multilocular cyst with small papillary projections at US. There were no cases in which radiologist classifications differed by more than one category during TIC analysis of CEUS examinations.

Comparison of Performance between Visual Assessment and TIC

Diagnostic performance results when combining visual assessment and TIC analysis of CEUS examinations with O-RADS are summarized in Table 4. Compared with using the O-RADS US score alone (AUC, 0.78 [95% CI: 0.71, 0.84]), integrating CEUS with O-RADS improved diagnostic performance significantly (AUC, 0.86 [95% CI: 0.80, 0.91] for both visual assessment and TIC analysis; P < .001 for both) (Fig 2).

Table 4:

Comparison of Performance of O-RADS Combined with CEUS between TIC Analysis and Visual Assessment

Figure 2:

Receiver operating characteristic curves for contrast-enhanced US with Ovarian-Adnexal Reporting and Data System (O-RADS). The receiver operating characteristic curves compare the diagnostic performance of combining both quantitative and qualitative contrast-enhanced US assessments with O-RADS against O-RADS US alone. Category 4 is used as the threshold for malignancy detection.

Using a score of O-RADS US 4 as cutoff for malignancy, the O-RADS US score alone showed a high sensitivity of 98.8% (95% CI: 93.2%, 100.0%) but a lower specificity of 25.0% (95% CI: 16.9%, 34.7%). When including CEUS via visual assessment and TIC analysis, accuracy in distinguishing benign from malignant (including borderline and invasive) lesions increased 20.0% (36 of 180) and 18.3% (33 of 180), respectively. Both methods exhibited high sensitivity for malignancy detection: 97.5% (95% CI: 91.3%, 99.7%) for visual assessment and 98.8% (95% CI: 93.2%, 100.0%) for TIC analysis, with only one cancer case missed by both. Visual assessment showed slightly higher specificity (59.0% [95% CI: 48.7%, 68.7%]) compared with TIC analysis (57.0% [95% CI: 46.7%, 66.9%]). TIC analysis misclassified three additional benign lesions as malignant compared with visual assessment. No evidence of a difference was observed between the AUCs of these two methods when integrated with O-RADS for malignant tumor diagnosis (P = .83). Figures 3 and 4 illustrate examples of combining CEUS with O-RADS to modify O-RADS US scores using both subjective analysis and TIC analysis. As previously mentioned, both subjective analysis and TIC analysis of CEUS missed one malignant case in the study (refer to Fig 5).

Figure 3:

Representative case of a 50-year-old female participant with pathologically confirmed serous cystadenocarcinoma. (A) Transvaginal US image displays a 5.7-cm unilocular cyst with solid components. The color Doppler US image shows abundant flow within the solid component (color score = 3). The lesion was assigned as Ovarian-Adnexal Reporting and Data System (O-RADS) category 4. The subjective analysis of contrast-enhanced US assessments revealed that the solid components of the unilocular cyst (red solid arrows) showed synchronous enhancement (yellow dotted arrows) and hyperenhancement compared with the myometrium. The dynamic change of the enhancement of the lesion was hyperenhancement to hyperenhancement. Thus, the lesion was assigned with a contrast-enhanced US score of 4 and upgraded to O-RADS category 5. (B) Time-intensity curve (TIC) analysis also demonstrated early and hyperenhancement of solid tissue of the adnexal lesion relative to the myometrium. The dynamic change of the enhancement of the lesion was also hyperenhancement to hyperenhancement, and the lesion was thus assigned with a contrast-enhanced US score of 4 and upgraded to O-RADS category 5.

Figure 4:

Representative case of a 38-year-old female participant with pathologically confirmed serous cystadenoma. (A) Transvaginal US image displays a 1.7-cm unilocular cyst with one papillary projection. Color Doppler US image shows no flow within the papillary projection (color score = 1). The lesion was assigned as O-RADS category 4. The subjective analysis of contrast-enhanced US assessments revealed that the papillary projection of the lesion (red solid arrows) showed late enhancement (yellow dotted arrows) and hypoenhancement compared with the myometrium. The dynamic change of the enhancement of the lesion was hypoenhancement to hypoenhancement. Thus, the lesion was assigned a contrast-enhanced US score of 2 and downgraded to O-RADS category 3. (B) Time-intensity curve (TIC) analysis revealed slow enhancement and hypoenhancement of the papillary projection of the adnexal lesion relative to the myometrium. The dynamic change of the enhancement of the lesion was hypoenhancement to hypoenhancement, and the lesion was assigned a contrast-enhanced US score of 2 and downgraded to O-RADS category 3.

Figure 5:

Representative case of a 54-year-old female participant with pathologically confirmed serous borderline tumor. (A) Transvaginal US image displays a 6.2-cm unilocular cyst with solid components. Color Doppler US image shows no flow within the solid components (color score = 1). The lesion was assigned as O-RADS category 4. The subjective analysis of contrast-enhanced US assessments revealed that the solid components of the lesion (red solid arrows) showed late enhancement (yellow dotted arrows) and hypoenhancement compared with the myometrium. The dynamic change of the enhancement of the lesion was hypoenhancement to hypoenhancement. The lesion was assigned with a contrast-enhanced US score of 2 and was downgraded to O-RADS category 3. (B) Time-intensity curve (TIC) analysis also revealed slow and hypoenhancement of solid tissue of the adnexal lesion relative to the myometrium. The dynamic change of the enhancement of the lesion was hypoenhancement to hypoenhancement. The lesion was assigned with a contrast-enhanced US score of 2 and the O-RADS category was downgraded from 4 to 3.

Discussion

The application of CEUS in evaluating adnexal lesions has garnered increasing attention (11,14). Our previous findings indicated that adding CEUS data to O-RADS US categories 4 and 5 improves the distinction between benign and malignant lesions. A key question has been whether visual assessment is as effective as TIC analysis in analyzing CEUS acquisitions. Our results showed that both TIC analysis and visual assessment of solid tissue enhancement with CEUS provided similar diagnostic performance for classifying adnexal malignancy, with AUC values of 0.86 each (P = .83). Moreover, there was a good agreement between TIC analysis and visual assessment regarding the presence of solid tissue enhancement (κ value, 1.00 [95% CI: 1.00, 1.00]), initial enhancement time (κ value, 0.65 [95% CI: 0.52, 0.77]), and intensity relative to the myometrium (κ value, 0.63 [95% CI: 0.51, 0.76]).

The O-RADS system, validated across various regions, exhibits high sensitivity and negative predictive value in differentiating benign from malignant adnexal masses (15–20). It is designed with a preference for sensitivity over specificity. Recent studies comparing CEUS combined with O-RADS US to O-RADS US alone have shown an improved specificity in O-RADS US for assigning adnexal malignancy risk when CEUS is included (12,13). In our dataset, the inclusion of CEUS with O-RADS resulted in an AUC of 0.86 for detecting malignant adnexal lesions, slightly lower than the AUC of 0.93 reported by Yuan et al (13). This discrepancy might be due to differences in patient selection, as our study excluded lesions lacking solid components. Such variations could reflect differences in O-RADS categories, malignancy rates, and contrast-enhanced features across studies.

In this study, all adnexal lesions with solid components that showed no internal enhancement at CEUS were confirmed benign. This finding is consistent with European Federation of Societies for Ultrasound in Medicine and Biology guidelines which propose that the absence of enhancement in adnexal masses suggests benignity (21). However, this diagnostic criterion was not included in the CEUS scoring criteria in either our study or by Yuan et al (13). Future prospective studies should focus on optimizing and validating CEUS scoring criteria for enhanced diagnostic accuracy.

In contrast to the study by Wengert et al (22), which demonstrated superiority of TIC analysis over visual assessment with O-RADS MRI score, our findings suggest that TIC analysis and visual assessment with CEUS are comparable in diagnostic accuracy. This discrepancy can be attributed to two factors. First, the use of different imaging modalities (CEUS vs MRI) may influence the characteristics of adnexal lesions. Second, in our study, interpretation was exclusively done by senior radiologists, suggesting the possibility that junior radiologists might find TIC analysis more accurate than visual assessment due to lesser experience.

The interobserver variability of TIC analysis and visual assessment, especially among radiologists with different experience levels, is a critical factor to consider (23). Previous research indicates that quantitative analysis using CEUS is highly consistent and accurate among experienced observers, supporting the use of TIC analysis in such contexts. Nevertheless, the perceived objectivity of TIC analysis relies on radiologists’ experience in selecting regions of interest, and even slight movements during imaging can impact data quality. Additionally, the limited availability of TIC analysis in routine clinical practice due to the need for advanced technical capabilities is a notable barrier.

One of the primary benefits of CEUS compared with MRI in clinical practice is its cost-effectiveness. CEUS is generally less expensive than MRI, making it a more economical option, especially in settings with limited resources. Additionally, CEUS is widely accessible and can be performed in various clinical settings, including outpatient departments and rural health care centers where MRI machines may not be available. In terms of patient comfort, patients may be more comfortable undergoing CEUS, as it is a noninvasive procedure and does not require close proximity to a strong magnetic field, as is the case with MRI. This can be particularly beneficial for patients who are claustrophobic or who have implanted medical devices that are incompatible with MRI. Furthermore, the duration of a CEUS examination is relatively short, typically taking between 10 and 30 minutes depending on the complexity of the case. CEUS is less time-consuming compared with MRI, which can take upwards of an hour or more. The additional time required for CEUS over a standard US acquisition is minimal, often less than 15 minutes. However, CEUS also has some limitations. The quality of CEUS imaging is highly operator dependent, requiring skilled radiologists to obtain and interpret the images accurately. In contrast, MRI provides more detailed soft tissue contrast and is less dependent on user expertise. Additionally, CEUS may not be suitable for pregnant patients and those with severe allergic reactions to the US contrast agents or severe pulmonary disease.

This study had several limitations. First, our study sample was skewed toward a high percentage (44.4%) of malignant lesions given the need for a solid component and surgical proof. The generalizability of our conclusions must be validated in future prospective studies in different study samples. Second, despite differences in transvaginal and transabdominal CEUS protocols, we did not perform both transvaginal and transabdominal CEUS in our study for the same adnexal lesion. Future studies are needed to compare and evaluate the performance of these two different CEUS protocols. Finally, the relatively small and unevenly distributed sample size across lesion categories calls for larger prospective studies to validate these findings.

In conclusion, this study demonstrates that visual assessment of CEUS offers diagnostic accuracy comparable with TIC analysis in characterizing adnexal lesions with solid components. Visual assessment is more available and convenient compared with TIC analysis, but the potential trade-off between accessibility and diagnostic precision should be considered when choosing between TIC analysis and visual analysis. This is particularly pertinent given the limited accessibility of TIC analysis in routine clinical settings. Therefore, future research should focus on validating these findings across broader patient cohorts and determining the optimal integration of TIC analysis and visual assessment of CEUS within routine clinical practice.