MRI accuracy and interobserver agreement in locally advanced cervix carcinoma

Abstract

Objectives:

The main standard of care for locally advanced cervix carcinoma (LACC) is radiochemotherapy (RCT) followed by brachytherapy. A surgical approach may still be discussed based on pelvic MRI-derived residual tumour evaluation. As no interobserver agreement study has ever been conducted to our knowledge, the aim of the present study was to report on pelvic MRI accuracy and interobserver agreement in LACC.

Methods:

We carried out a retrospective study in a French university hospital. Medical records of all consecutive patients treated with curative intent for LACC by RCT followed by brachytherapy and completion pelvic surgery between January 2014 and January 2020 were reviewed. Local response was assessed through pelvis MRI and histological analysis after completion surgery. MRI data were independently evaluated by two radiologists with varying experience. The two main interobserving criteria we used were complete response and residual tumour.

Results:

23 patients fulfilled the inclusion criteria. Agreement between the junior and senior radiologist was moderate to strong. Indeed, regarding main criteria, κ was 0.65 for complete response and 0.57 for residual tumour. Interestingly, the present study shows a lower sensitivity whatever the radiologists than in the international literature.

Conclusion:

The present study highlights a low interobserver variability regarding pelvic MRI in the assessment of RCT followed by brachytherapy in LACC. Yet, sensitivity was lower than in literature.

Advances in knowledge:

Radiology is part of treatment decision-making, the issue of heterogeneity regarding radiologists’ training and experience to cancer (sensitivity and specificity) turns essential, so does MRI accuracy.

Introduction

Cervix carcinoma is the fourth leading cause of cancer-related death in females worldwide.¹ Despite the increasing use of vaccines, the recurrence rate remains rather high, especially in transition countries. The main standard of care remains radiochemotherapy (RCT) upfront followed by brachytherapy. A surgical approach may still be discussed for the completion of the treatment. In the end, the final decision of surgery is based on pelvic MRI-derived residual tumour evaluation. The FIGO classification is used to stage cervix carcinomas. Locally advanced cervical cancers range from IB2 to IVA. At initial workflow, once the diagnosis is confirmed by histology, MRI plays a key role in primary staging as well as clinical exam, PET CT and ± lomboaortic dissection.²

After completion of RCT followed by brachytherapy, pelvic MRI was performed between 6 and 12 weeks as recommended. Indeed, if performed before, MRI specificity becomes less reliable (25% of false positive) as radiation-related oedema and inflammation turn into a high-signal intensity on T₂ weighted images several weeks after radiotherapy treatment completion. It is well accepted that after 12 weeks all these inflammatory components disappear.³ Pelvic MRI sensitivity ranges from 75 to 92% however, specificity remains low (range: 0–55%).^4–6

Radiology is key to evaluation in oncology, imaging is part of initial workflow for both staging and therapeutic assessment, so it is essential in treatment decision making. That’s why the female pelvis subcommittee of the European Society of Urogenital Radiology (ESUR) formed a working group to establish technical guidelines for uterine cervical cancer.⁷ Of note, one major issue is the heterogeneity regarding training, experience to cancer so sensitivity and specificity may result in part from the radiologist. Actually, many studies should be dedicated not only to the technical procedures of image acquisition but also to the quality training of radiologists themselves. Many interobserver agreement studies were conducted about pelvic MRI. They are mainly about initial staging in prostate cancer and post-treatment evaluation in rectal cancer. Regarding cervix carcinomas, to our knowledge, no interobserver agreement study has ever been conducted.

The aim of the present study was to report on pelvic MRI accuracy and interobserver agreement in locally advanced cervix carcinoma (LACC).

Methods and materials

A retrospective study was carried out in a French university hospital. The institutional ethics committee approved the study that was conducted in compliance with the Helsinki Declaration. The French National Agency in charge of regulating Data Protection agreed with our database. Patients were informed their medical data would be used to conduct retrospective studies.

Patients’ characteristics and outcomes

Medical records of all consecutive patients treated with curative intent, for LACC by RCT followed by brachytherapy and completion pelvic surgery between January 2014 and January 2020 were reviewed. LACC included Stage IB2 to IVA, based on the 2018 classification of the International Federation of Gynecology and Obstetrics (FIGO). Local response to RCT followed by brachytherapy was assessed through pelvic MRI and histological analysis after completion surgery.

Pelvic MRI evaluation

All pelvic MRI images were acquired on Siemens machines with ≥1.5 T field strength with surface coil and without endorectal coil. Patients fasted for 3 h before the examination; semi-bladder repletion was expected as well as intra- or subcutaneous antispasmodic drug intake. Pelvic MRI protocol was composed of axial and sagittal spin-echo T₂ weighted imaging (slice thickness 4 mm, field of view (FOV) 220 × 220 mm, matrix 320 × 320, time to repetition (TR) 6400 ms, time to echo (TE) 121 ms, acquisition time was 3 min and 19 s), a diffusion-weighted image sequence (slice thickness 4 mm, FOV 400 × 400 mm, matrix 141 × 176, TR 6100 ms, TE 72 ms, acquisition time was 3 min and 40 s) with calculation of apparent diffusion coefficients (ADCs) using two b values (0 and 1000 sec/mm²). A sagittal or axial, dynamic or not, contrast-enhanced (gadolinium) T₁ weighted sequence was performed (slice thickness 4 mm, FOV 250 × 250 mm, matrix 256 × 320, TR 706 ms, TE 10 ms, acquisition time was 3 min and 50 s). A large axial T2 turbo spin-echo of the abdomen and pelvic area was optional (slice thickness 5 mm, FOV 400 × 400 mm, matrix 320 × 320, TR 1600 ms, TE 121 ms, acquisition time was 1 min and 32 s). So was axial T₁ weighted sequence with and without fat saturation (slice thickness 4 mm, FOV 250 × 250 mm, matrix 256 × 320, TR 706 ms, TE 10 ms, acquisition time was 3 min and 50 s).

MRI data were independently evaluated by two radiologists with varying experience (one highly specialized in gynecological imaging -called “senior” - and one in training - called “junior”. They were blinded to the interpretation to the other reader as well as clinical and pathological information in compliance with the 2011 ESUR guidelines. Post-treatment MRI was compared with initial staging MRI. The following imaging parameters were studied: post-treatment tumour volume (ellipsoidal formula) and downsizing of initial tumour volume illustrated in Figure 1, shortened cervix illustrated in Figure 2, cervix low-signal intensity in T₂ weighted imaging, pericervical fibrosis ring reappearance illustrated in Figure 3, cervix contrast-enhancement illustrated in Figure 4, restricted diffusion and ADC at 1.7 × 10–3 mm²/sec. illustrated in Figure 5, parametrial involvement illustrated in Figure 6, vagina involvement, pelvic lymph node, para-aortic lymph node, bladder and rectum involvement, distant metastasis. All the patients (Figures 1–6) had an initial stage IIB cervical cancer according to the 2018 FIGO classification. The two main criteria we used were complete response and residual tumour.

Figure 1.

Calculation of the tumour volume using the largest diameters of the tumour on sagittal (A, C) and axial (B,D) spin echo T₂ weighted images: A and B are pre-treatment images whereas C and D are post-treatment. Patient with an initial stage IIB cervical cancer (2018 FIGO classification).

Figure 2.

Sagittal spin echo T₂ weighted images showing a tissue mass inside the cervix before treatment (A) and then a shrinking of the cervix after treatment (B, arrow). Patient with an initial stage IIB cervical cancer (2018 FIGO classification).

Figure 3.

Axial spin-echo T₂ weighted imaging showing pericervical fibrosis ring reappearance (single arrow) and a homogenous hyposignal T2 imaging of the cervix (double arrow) after treatment. Patient with an initial stage IIB cervical cancer (2018 FIGO classification).

Figure 4.

Axial contrast-enhanced (gadolinium) T₁ weighted sequences with fat saturation showed: a homogenous enhancement of the cervix after complete local response (A) and heterogeneous with the presence of tumour deposit. Patient with an initial stage IIB cervical cancer (2018 FIGO classification).

Figure 5.

Axial diffusion-weighted sequences with b = 1000 s/mm²showing hypersignal of the cervix tumour before treatment (A) and its complete shrinking after treatment (C). ADC calculations showing a restricted ADC before treatment (B: mean value = 0.709×10⁻³ s/mm²) and after treatment, D value was higher (D: mean value = 1.6×10⁻³ s/mm²). Patient with an initial stage IIB cervical cancer (2018 FIGO classification). ADC, apparent diffusion coefficient.

Figure 6.

Axial spin-echo T2 (A) and axial T₁ weighted imaging after fat saturation and contrast-enhancement (gadolinium) showing parametrial involvement: T2 SE fat parametrial involvement (A, arrow) and parametrial fat increase (B, arrow). Patient with an initial stage IIB cervical cancer (2018 FIGO classification).

Statistical analyses

Variables were described by percentage, mean (given with standard deviation (SD) or with the range) and median (given with the first and third quantiles or with the range) values. Interclass correlation coefficient (ICC) was used to compare quantitative variables and qualitative variables were compared using κ coefficient (k). Low ICC (=0) means similarity is low, whereas high ICC (=1) means similarity is high. Regarding κ, agreement between the two radiologists was excellent (0.81–1), strong (0.61–0.80), moderate (0.41–0.60), low (0.21–0.40), very low (0–0.20); disagreement (<0). Statistical analyses were conducted using R software.

Results

Patients and characteristics

Between January 2014 and January 2020, 23 patients fulfilled the inclusion criteria. All patients had squamous cell carcinomas. Median age at diagnosis was 56 years-old (range: 34–79). No life-threatening comorbidity was evidenced. The main FIGO stage was IIB due to unilateral or bilateral parametrial involvement (60% of cases); IB3, IIA, IIIB and IIIC stages represented 9% each so 36% overall and, only one patient had a IB2 stage carcinoma (4%). No IIA and IVA stages were observed. All details were depicted in Table 1. After pelvic surgery, 7 out of the 23 patients (30.4%) presented with histological residual tumour. Histological complete pathologic response was found in 16 out of the 23 patients (69.6%). Median time between the end of CRT and brachytherapy procedures was 3.2 weeks (range: 1.7–5.7). Median time between brachytherapy and restaging MRI was 6.4 weeks (range: 2.3–10.1). Median time between MRI and final surgery was 3.5 weeks (range: 0.3–9.6). Pelvic surgery was performed in a mean time of 10.1 weeks after brachytherapy (range: 6.1–14.9).

Table 1.

Patients characteristics

Characteristics (N = 23)	Values
Median age at diagnosis:	56 (60, 34–79)
Years (median, range)
Age :	5 (22%)
<39	2 (9%)
40–49	13 (56%)
50–69	3 (13%)
>70
FIGO Stage
IB2	1 (4%)
IB3	2 (9%)
IIA	2 (9%)
IIB	14 (60%)
IIIA	0
IIIB	2 (9%)
IIIC	2 (9%)
IVA	0
Pelvic lymph node status
N0	22 (96%)
N+	1 (4%)
Lomboaortic lymph node status
N0	19 (83%)
N+	1 (4%)
Nx	3 (13%)

MRI findings

The junior observer found a residual tumour in 12 patients out of 23 and no residual tumour in 11 patients. The junior’s predictions were: four true positive, eight true negative, eight false positive and three false negative, thus a sensitivity of 57.14, a specificity of 50%, a positive predictive value (PPV) of 33.33% and a negative predictive value (NPV) of 72.74%.

The senior observer found a residual tumour in 11 patients out of 23 and no residual tumour in 12 patients. The senior predictions were: four true positive, nine true negative, seven false positive and three false negative thus a sensitivity of 57.14, a specificity of 56.25%, a positive predictive value of 36.36% and a negative predictive value of 75%. Information is available in Table 2.

Table 2.

Post-treatment MRI accuracy in cervix and rectal locally advanced cancer

References	Date of publication	Radiologists	Interobserver agreement	Accuracy (%)	Se (%)	Sp (%)	VPP (%)	VPN (%)	FP (%)	FN (%)
Cervix carcinoma
(4)	2011	Two senior radiologists with experience	NA	NA	80	55	50	83	50	17
(5)	2018	NA	NA	NA	75	36.4	30	70	NA	NA
(6)	2019	NA	NA	NA	92.3	0	48	0	NA	NA
Present study	2020	Junior radiologist	Moderate to strong (K = 0.57 à 0.65)	NA	57.14	50	33.33	72.74	34	13
		Senior radiologist		NA	57.14	56.26	36.36	75	30	13
Rectum (Restaging T score with MRI in locally advanced rectal cancer after pre-operative CT)
(7)	2005	NA	NA	52	NA	NA	NA	NA	38	10
(8)	2014	NA	NA	68	32	90	65	69	NA	NA
(9)	2014	NA	NA	NA	T0: 15.3	NC	NA	NA	NA	NA
					T3/T4: 82.1	53.5			NA	NA
(10)	2014	NA	Moderate (K = 0.33)	64.7	NA	NA	NA	NA	38	NA
(11)	2015	NA	NA	NA	41.4	98.6	58.3	97.3	NA	NA
(12)	2015	NA	NA	55.2	38.5	58.1	13.3	84.9	NA	NA
(13)	2015	One reinterpretation	Poor (K = 0.156)	49	NA	NA	NA	NA	40.4	10.6
(14)	2015	NA	NA	34 to 82 (mean 52)	NA	NA	NA	NA	NA	NA
(15)	2016	NA	NA	43	NA	NA	NA	NA	34	18
(16)	2017	Three readers independently	Fair to moderate (κ = 0.20–0.41)	NE	0 to 67	25 to 100	0 to 61	42 to 85	NA	NA
(17)	2018	NA	NA	68	NA	NA	NA	NA	NA	NA
(18)	2020	NA	NA	NA	81 (T3/T4)	67 (T3/T4)	NA	NA	NA	NA
(19)	2020	NA	NA	NA	62	89	NA	NA	NA	NA

FN, false negative; FP, false positive;NA, not applicable.

Regarding main criteria, κ was 0.65 for complete response and 0.57 for residual tumour.)

As to secondary criteria, κ was one for para-aortic lymph nodes, bladder and rectum involvement and distant metastases. κ was excellent (0.84) for cervix contrast-enhancement. κ was strong for cervix low signal intensity in T₂ weighted imaging (0.65), pericervical fibrosis ring reappearance (0.74), parametrial involvement (0.65), vagina involvement (0.65), ADC > 1.7 (0.63) and pelvic lymph nodes (0.78). κ was moderate for restricted diffusion (0.58). κ was low for shortened cervix (0.09).

Discussion

The present study highlights a low interobserver variability regarding pelvic MRI in the assessment of RCT followed by brachytherapy in LACC. Indeed, agreement between the junior and senior radiologists was moderate to strong. Moreover, no significant difference between them regarding sensitivity, specificity, PPV and NPV was evidenced. Interestingly, the present study shows a lower sensitivity (57.14 whatever the radiologist) than in the international literature (range: 75–92).^4–6 One could explain differences by several reasons. First, 4 patients out of 23 did not perform the total pelvic MRI procedures as described in material and methods, i.e. 2 patients without DWI sequence and 2 patients without gadolinium injection. Undoubtedly, MRI protocol procedures must be standardised and mandatory as in other cancers, especially rectal cancer.^8–10 Secondly, a technical reason is the cut-off of ADC that was chosen at 1.7 × 10–3 mm²/sec based on the study by Levy et al,¹¹ whereas most recently published studies advise a lower cut-off at 1.1 × 10–3 mm²/sec.^12,13 Accuracy might have been improved with the change of ADC cut-off. Thirdly, MRI was performed too early (<6 weeks) in 6 patients out of 23, thus the risk of false positive increased and specificity decreased.

Regarding rectal cancer, restaging T score in locally advanced rectal cancer with MRI show sensitivity ranging from 32 to 81% and specificity ranging from 25 to 100%.^14–26 This demonstrates that pelvic MRI accuracy may be compromised, in part due to the radiologist, but also due to MRI technical procedures.

In the near future, new pelvic MRI sequences could improve examination accuracy and thus, treatment decision-making. For example, functional MRI imaging and intravoxel incoherent motion (IVIM) MR imaging could be useful in predicting the therapeutic efficiency of CRT in patients with LACC.^27,28 The future potential of radiomics in treatment for patients with cervical cancer, and more generally in oncology, is under study.²⁹ In addition, other studies about variability and MRI accuracy in LACC are needed.