Diagnostic accuracy of b800 and b1500 DWI‑MRI of the pelvis to detect residual rectal adenocarcinoma: a multi‑reader study

Abstract

Purpose

To compare the sensitivity, specificity and intra-observer and inter-observer agreement of pelvic magnetic resonance imaging (MRI) b800 and b1500 s/mm² sequences in the detection of residual adenocarcinoma after neoadjuvant chemoradiation (CRT) for locally advanced rectal cancer (LARC).

Introduction

Detection of residual adenocarcinoma after neoadjuvant CRT for LARC has become increasingly important and relies on both MRI and endoscopic surveillance. Optimal MRI diffusion b values have yet to be established for this clinical purpose.

Methods

From our MRI database between 2018 and 2019, we identified a cohort of 28 patients after exclusions who underwent MRI of the rectum before and after neoadjuvant chemoradiation with a protocol that included both b800 and b1500 s/mm² diffusion sequences. Four radiologists experienced in rectal MRI interpreted the post-CRT MRI studies with either b800 DWI or b1500 DWI, and a minimum of 2 weeks later re-interpreted the same studies using the other b value sequence. Surgical pathology or endoscopic follow-up for 1 year without tumor re-growth was used as the reference standard. Descriptive statistics compared accuracy for each reader and for all readers combined between b values. Inter-observer agreement was assessed using kappa statistics. A p value of 0.05 or less was considered significant.

Results

Within the cohort, 19/28 (67.9%) had residual tumor, while 9/28 (32.1%) had a complete response. Among four readers, one reader had increased sensitivity for detection of residual tumor at b1500 s/mm² (0.737 vs. 0.526, p = 0.046). There was no significant difference between detection of residual tumor at b800 and at b1500 for the rest of the readers. With all readers combined, the pooled sensitivity was 0.724 at b1500 versus 0.605 at b800, but this was not significant (p = 0.119). There was no difference in agreement between readers at the two b value settings (67.8% at b800 vs. 72.0% at b1500), or for any combination of individual readers.

Conclusion

Aside from one reader demonstrating increased sensitivity, no significant difference in accuracy parameters or inter-observer agreement was found between MR using b800 and b1500 for the detection of residual tumor after neoadjuvant CRT for LARC. However, there was a suggestion of a trend towards increased sensitivity with b1500, and further studies using larger cohorts may be needed to further investigate this topic.

Introduction

Although rectal magnetic resonance imaging (MRI) has proven utility in evaluating patients with locally advanced rectal cancer (LARC) after neoadjuvant chemoradiation (CRT), the optimal choice for DWI b values for detection of residual tumor has not been established [1–6]. Advanced postprocessing techniques, such as intravoxel incoherent motion (IVIM) [7] and diffusion kurtosis imaging (DKI) [8] have been studied to assess response to CRT in LARC, but are complex techniques that are not presently ready for routine use in the clinical setting. Qualitative assessment of post-CRT rectal MRI is currently a more practical and clinically applicable approach [9], and optimizing imaging protocols is necessary. Many previous studies have sought to establish optimal b values for other organs, including the prostate [10–12], breast [13], lung [14], and liver [15], but to our knowledge this study is the first comparison of different b values in post-CRT rectal MRI. It is essential to note that following neoadjuvant therapy for rectal cancer, MRI findings should be interpreted in conjunction with endoscopic assessment, as suggested by a recent expert consensus from the European Society of Gastrointestinal and Abdominal Radiology (ESGAR) [16].

The current standard of care for patients with locally advanced rectal cancer (LARC) includes neoadjuvant chemoradiation, with subsequent surgical resection and adjuvant chemotherapy. A rectal cancer is considered locally advanced when it is T-stage is T3 or higher, or when there is metastatic involvement of locoregional lymph nodes [17]. However, because of the morbidity and quality of life issues associated with surgical resection, either via low-anterior resection (LAR) or abdominoperineal resection (APR), the concept of non-operative management for LARC is gaining traction, if a clinical complete response can be achieved with neoadjuvant CRT alone [18, 19]. Non-operative management depends on close observation using a combination of surveillance imaging with rectal MRI interpreted by experienced radiologists and endoscopic assessment by experienced colorectal surgeons or gastroenterologists [20].

In an effort to optimize the efficacy of MR imaging of post-CRT LARC, investigation of different MR techniques is necessary to help standardize imaging practices. An analogous scenario is prostate cancer, in which multiple studies have sought to establish optimal b values for the detection of clinically significant cancer [10–12, 21]. The purpose of this study was to conduct a preliminary comparison of the accuracy of multiplanar T2 images with either b800 or b1500 DWI sequences for the detection of viable tumor in LARC after neoadjuvant CRT. In addition, we compare the interobserver agreement and concordance among experienced readers using these two b value settings.

Methods

Patient selection

In 2017, our department began incorporating both b800 and b1500 s/mm² diffusion-weighted sequences into our routine protocol for MRI rectum, as we became interested in evaluating the utility of higher b value DWI and diffusion kurtosis imaging in rectal cancer. In August 2018, a review of MRI rectum studies performed between January 1, 2017 and March 31, 2018 found 1595 total exams. Exclusions were made for studies where the history was an indication other than restaging after neoadjuvant chemoradiation, duplicate studies on the same patient, if no baseline MR rectum was available for comparison, or if the patient did not have the appropriate reference standard (either surgical resection or 1 year of endoscopic follow-up from the time of index MRI). A summary of the patient selection process is included as Appendix 1.

The final cohort included 28 consecutive patients who were evaluated with 3.0 Tesla MRI with both b800 and b1500 sequences after receiving neoadjuvant CRT for LARC. A baseline MRI prior to neoadjuvant therapy and most recent prior MRI (typically after induction chemotherapy and before chemoradiation) was also required for reference, so that readers could optimally locate the site of the original tumor.

Patients were required to have undergone subsequent surgical resection or biopsy confirmed residual tumor via endoscopy, or they must have persistent complete clinical response confirmed via endoscopy for 1 year from the date of the index MRI.

Image acquisition

MR imaging was acquired with a 3-T MRI unit (Discovery MR750; GE Medical Systems, Waukesha, WI). A 32-channel phased array coil was employed for signal reception. Multiplanar T2-weighted images and diffusion-weighted images, including both b800 and 1500 s/mm², were acquired. Post-contrast images were not included for interpretation as part of the analysis. The standard MR rectum protocol used at our institution is included as Appendix 2.

Image anonymization

At our institution, it is common practice to give induction chemotherapy in lieu of adjuvant chemotherapy, a modern approach that is also known as total neoadjuvant therapy [22]. For this reason, patients often undergo baseline MRI, post-induction MRI and a final post-CRT MRI. Clinical decisions as to proceeding to surgery versus possible non-operative management are made after the final post CRT MRI combined with clinical assessment. Images from the index MRI (post-CRT), baseline MRI and most recent prior MRI (post-induction) were anonymized, and Digital Imaging and Communications (DICOM) files were uploaded into an open source software for interpretation (XNAT v1.7.4.1, Washington University School of Medicine). This step was taken to prevent the readers from inadvertent exposure to clinical information in the hospital picture archiving and communication system (PACS) that might unduly influence their image interpretation.

Image interpretation

Four fellowship-trained body imagers with varying degrees of experience reading rectal MRI were included (JGP, 4 years; VP, 4 years; LF, 11 years; and MJG, 20 years). Readers evaluated patients using multiplanar T2W images and DWI, either using only either b800 or only b1500 s/mm² for a given patient with corresponding apparent diffusion coefficient maps. After a minimum two-week period meant to reduce recall bias, the readers interpreted the same MR studies, but this time with the complimentary b value for each patient. Both the first and second sessions included an equal mixture of b800 and b1500 s/mm² cases.

Readers were asked to indicate whether there was residual tumor or a complete response by MR imaging, in a binary (Y/N) qualitative assessment. Our group made the decision to use a binary assessment rather than a semi-quantitative scale, because the binary model more closely reflects what referring clinicians want to know: whether or not the radiologist thinks there is residual tumor based on the MRI.

Reference standard

The reference standard for MR interpretations was residual tumor or complete response confirmed by pathology after surgical resection. Additionally, since not all patients underwent surgical resection after neoadjuvant CRT, and instead underwent expectant monitoring (“Watch and Wait”) with flexible sigmoidoscopy and MRI, residual tumor or complete response via endoscopy was also included as reference standard. For patients with complete response who did not undergo surgery, they must have had no residual tumor (or tumor re-growth) confirmed by endoscopy at 1 year after the index MRI to be considered a true complete response [23].

Statistical analysis

Measures of accuracy on detecting residual tumor were estimated for 4 readers under each b value (b800 and b1500), including the sensitivity, the specificity, the positive predictive value (PPV) and the negative predictive value (NPV). The McNemar test was used to compare the sensitivity or the specificity between different b values for each reader separately. We also combined all readings from 4 readers and compared the sensitivity or the specificity using the generalized linear regression with a logit link function, and fixed effects of b value and reader.

Kappa statistics were used to assess inter-reader agreement on residual tumor for every pair of readers (among 4 readers); Light’s kappa was estimated for all readers under each b value. Kappa (κ) values were interpreted as follows: 0.00–0.20, slight agreement; 0.21–0.40, fair agreement; 0.41–0.60, moderate agreement; 0.61–0.80, substantial agreement; and 0.81–1.00, almost perfect agreement. All statistical analyses were performed in software packages R version 3.6.0 (The R Foundation for Statistical Computing).

Results

A total of 28 patients were included, 13 female and 15 male with a mean age of 57 years old (range 30–83). Among subjects, 19/28 (67.9%) had residual tumor after CRT, while 9/28 (32.1%) had a complete response. Twenty patients underwent surgical resection as the reference standard, whereas eight had endoscopic follow-up as the reference standard (Table 1).

Table 1.

Patient demographics, n (%) or median (range)

Variable	N = 28
Age	57 (30, 83)
Gender
F	13 (46%)
M	15 (54%)
Reference standard
Residual tumor	19 (68%)
Variable	N = 28
Age	57 (30, 83)

The sensitivity and the specificity for each reader using either b800 or b1500 are summarized in Table 2. At b800, detection of residual tumor included a range of sensitivities from 0.526 to 0.789, and a range of specificities from 0.667 to 0.889. At b1500, detection of residual tumor showed a range of sensitivities from 0.632 to 0.789, and a range of specificities from 0.556 to 0.889. Reader 2 had increased sensitivity for the detection of residual tumor after CRT (0.737 vs. 0.526, p = 0.046).

Table 2.

Measures of accuracy on residual tumor

	Sensitivity (95% CI), n	Specificity (95% CI), n	PPV (95% CI), n	NPV (95% CI), n
b800
Reader 1	0.579 (0.335, 0.797), 11/19	0.667 (0.299, 0.925), 6/9	0.786 (0.492, 0.953), 11/14	0.429 (0.177, 0.711), 6/14
Reader 2	0.526 (0.289, 0.756), 10/19	0.889 (0.518, 0.997), 8/9	0.909 (0.587, 0.998), 10/11	0.471 (0.23, 0.722), 8/17
Reader 3	0.526 (0.289, 0.756), 10/19	0.778 (0.4, 0.972), 7/9	0.833 (0.516, 0.979), 10/12	0.438 (0.198, 0.701), 7/16
Reader 4	0.789 (0.544, 0.939), 15/19	0.778 (0.4, 0.972), 7/9	0.882 (0.636, 0.985), 15/17	0.636 (0.308, 0.891), 7/11
b1500
Reader 1	0.737 (0.488, 0.909), 14/19	0.556 (0.212, 0.863), 5/9	0.778 (0.524, 0.936), 14/18	0.5 (0.187, 0.813), 5/10
Reader 2	0.737 (0.488, 0.909), 14/19	0.778 (0.4, 0.972), 7/9	0.875 (0.617, 0.984), 14/16	0.583 (0.277, 0.848), 7/12
Reader 3	0.632 (0.384, 0.837), 12/19	0.889 (0.518, 0.997), 8/9	0.923 (0.64, 0.998), 12/13	0.533 (0.266, 0.787), 8/15
Reader 4	0.789 (0.544, 0.939), 15/19	0.889 (0.518, 0.997), 8/9	0.938 (0.698, 0.998), 15/16	0.667 (0.349, 0.901), 8/12

Although average sensitivity was higher under b1500 compared to b800 (0.724 vs. 0.605) when all reader interpretations were combined, the difference was not quite significant (p = 0.119). Average specificity was the same under b800 and b1500 (0.778), and the analysis did not show any difference (p = 0.999).

The concordance in readings for each individual reader between different b values showed no significant difference between b800 and b1500. P values from the McNemar test for sensitivity and specificity are summarized in Tables 3, 4, respectively.

Table 3.

Compare sensitivity under different b values for each reader

Reader	Discordant n (%)	p value
1	9 (47.4%)	0.317
2	4 (21.1%)	0.046
3	2 (10.5%)	0.157
4	2 (10.5%)	0.999

Bold value is statistically significant (p < 0.05)

Table 4.

Compare specificity under different b values for each reader

Reader	Discordant n (%)	p value
1	3 (33.3%)	0.564
2	3 (33.3%)	0.564
3	3 (33.3%)	0.564
4	1 (11.1%)	0.317

Discussion

In our cohort of patients with LARC who had undergone neoadjuvant CRT, only one of the four readers showed a significant increase in sensitivity using b1500 instead of b800. The other readers individually showed no significant difference in accuracy for detection of viable tumor between MRI studies using either b800 or b1500. When the results from all readers were combined and reads from b800 studies were compared with those from b1500 studies, the sensitivities for detecting residual tumor were 0.605 versus 0.724, respectively, but the difference was not quite significant (p = 0.119). There was also no difference in the interobserver agreement in all combinations of individual readers, and no difference in concordant reads among all readers combined (Fig. 1).

Fig. 1.

46 year-old female with locally advanced rectal cancer, treated with neoadjuvant induction chemotherapy followed by chemoradiation. Axial oblique T2-weighted MR image (a, arrow) shows scar and intermediate T2 signal thickening in the right lateral rectal wall. Diffusion-weighted images at (b, arrows) b800 s/mm² and (c, arrows) b1500 s/mm², with corresponding apparent coefficient diffusion maps below, show restricted diffusion in the tumor bed. The patient subsequently underwent low anterior resection, with pathology demonstrating residual carcinoma and treatment response involving 40% of the tumor

Based on these results, our investigation suggests b800 and b1500 s/mm² are equivalent for the detection of residual adenocarcinoma after CRT, or if there is a difference, it is very small and would require a much larger study to confirm. Interestingly, in Reader 3, both the sensitivity and specificity increased when reading scans with b800 and b1500, but with overlapping confidence intervals, thus not statistically significant. This is worth mentioning namely because the link between sensitivity and specificity is generally thought to be an inverse relationship [24], and they often do not increase together. Our data also suggests the two b values are equivalent in terms of concordance of interpretations between multiple readers. Thus, except for increased sensitivity in Reader 2 at b1500 s/mm², our study found no significant difference between MR protocols using these different b values (Table 5).

Table 5.

Agreement between 4 readers under each b value

Reader combination	b800 Kappa (95% CI)	b1500 Kappa (95% CI)	b800 percent concordance (%)	b1500 percent concordance (%)
R1 & R2	0.214 (− 0.178, 0.552)	0.255 (− 0.106, 0.592)	60.7	64.3
R1 & R3	0.286 (− 0.14, 0.579)	0.23 (− 0.099, 0.532)	64.3	60.7
R1 & R4	0.214 (− 0.172, 0.571)	0.255 (− 0.137, 0.602)	60.7	64.3
R2 & R3	0.632 (0.251, 0.855)	0.505 (0.189, 0.786)	82.1	75
R2 & R4	0.454 (0.172, 0.771)	0.708 (0.312, 0.92)	71.4	85.7
R3 & R4	0.376 (0.071, 0.689)	0.646 (0.357, 0.924)	67.9	82.1
4 Readers	0.363 (0.153, 0.581)	0.433 (0.229, 0.648)	67.8*	72*

^*Percent concordance for 4 readers was the average of that value on all possible pairs

To our knowledge, there are no other published studies comparing b values in rectal MRI for this indication. However, given the small size of our cohort, it would reasonable to further investigate this question with larger samples. When similar studies were being conducted in prostate cancer, some found no difference using conventional versus high b value DWI, whereas others established superiority with higher b values. This study should be considered an early investigation into an important question with clinical implications.

The clinical implications of the findings in our study are that both b800 and b1500 may be considered acceptable for use in rectal MRI after neoadjuvant CRT for LARC, since we observed no difference between the two groups. Our results should be taken in context, however, as there are several limitations.

Among the limitations that must be considered here, the relatively small sample size is pertinent. Although we combined the reads for b800 and b1500 across all readers, yielding 120 reads for each b value, respectively, we believe additional studies with larger sample sizes would be useful to further investigate this question. In addition, our cohort is derived from a single tertiary academic medical center using experienced high volume readers of rectal MRI, and our findings may not be replicated in other centers or with less experienced readers. We did not use a 5-point scale for readers to score their confidence for the presence of residual tumor, which would have enabled us to assess for relative confidence among readers between the two b values. Furthermore, we used a 12-month follow-up for establishing a clinical complete response in our study, whereas some suggest 24-month follow-up may be preferable in this setting, and our results may not be applicable to other models. At our institution, induction chemotherapy is used prior to neoadjuvant chemoradiation, which is not yet the standard of care across the United States at this time. Lastly, the retrospective nature of this study limits it inherently, and future prospective work on this question may be useful.

In summary, our study found no significant difference in sensitivity or specificity for detecting viable tumor after CRT for LARC, and no difference in concordant reads among multiple experienced readers. These results imply that either b800 or b1500 s/mm² diffusion sequences may be used in post-neoadjuvant rectal MRI.

Conclusion

Except for one reader demonstrating increased sensitivity for detection of tumor at the higher b value, our study did not detect significant difference in the sensitivity, specificity or inter-observer agreement using b800 and b1500 s/mm² sequences in MRI for detecting residual tumor after neoadjuvant CRT for LARC. However, there is suggestion of a trend towards increased sensitivity to detect residual tumor, and further studies with larger cohorts are warranted in the future to further investigate this question.

Appendix 1

See Fig. 2.

Fig. 2.

Method for selecting cohort for the study

Appendix 2

See Table 6.

Table 6.

Standard MR rectum protocol used in our cohort with b800 and b1500 s/mm² diffusion-weighted sequences

Sequences	1	2	3	4	4		5	7
Series descriptor	Axial T2	Sagittal T2	Oblique axial T2	Oblique coronal T2 thru tumor	Oblique coronal T2 sphincter	G	Axial DWI b 800	Focus diffusion
Generic sequence name	FSE T2	FSE T2	FSE T2	FSE T2	FSE T2	L	2D	Focus diffusion
Plane	Axial	Sagittal	Oblique axial	Oblique coronal	Oblique coronal	U	Axial	Axial
Options	Fast/NPW/ED	Fast/NPW/ED	Fast/NPW/ED	Fast/NPW/ED	Fast/NPW/ED	C	EPI, DIFF, ASSET	EPI, DIFF
Field of view (cm)	20–24	18	18	18	18	A	24	16
Slice thickness (mm)	5	4	3	3	3	G	5	5
Gap (mm)	1	1	1	1	1	O	1	1
b value							b800	1500
Saturation pulse	S/I/A	A	S/I/A	A	A	N	N/A	N/A
TE1/TE2	110	102	102	102	102		Min	Min
TR	4000–6000	4000–6000	4000–6000	4000–6000	4000–6000	I	6000	6000
Flip angle	90	90	90	90	90	V	N/A	N/A
Bandwidth (kHz)	32	32	32	32	32		N/A	N/A
ETL	24	24	24	24	24	G	na	N/A
NEX	3	4	4	4	4	I	16	16
Frequency steps	320	320	320	320	320	E	128	70
Phase encoding steps	224	224	224	224	224	V	128	140
Frequency direction	A/P	A/P	A/P	A/P	A/P	N	R/L	R/L