Deformable Mapping of Rectal Cancer Whole-Mount Histology with Restaging MRI at Voxel Scale: A Feasibility Study

Abstract

Purpose

To develop a radiology-pathology coregistration method for 1:1 automated spatial mapping between preoperative rectal MRI and ex vivo rectal whole-mount histology (WMH).

Materials and Methods

This retrospective study included consecutive patients with rectal adenocarcinoma who underwent total neoadjuvant therapy followed by total mesorectal excision with preoperative rectal MRI and WMH from January 2019 to January 2022. A gastrointestinal pathologist and a radiologist established three corresponding levels for each patient at rectal MRI and WMH, subsequently delineating external and internal rectal wall contours and the tumor bed at each level and defining eight point-based landmarks. An advanced deformable image coregistration model based on the linearized iterative boundary reconstruction (LIBR) approach was compared with rigid point-based registration (PBR) and state-of-the-art deformable intensity-based multiscale spectral embedding registration (MSERg). Dice similarity coefficient (DSC), modified Hausdorff distance (MHD), and target registration error (TRE) across patients were calculated to assess the coregistration accuracy of each method.

Results

Eighteen patients (mean age, 54 years ± 13 [SD]; nine female) were included. LIBR demonstrated higher DSC versus PBR for external and internal rectal wall contours and tumor bed (external: 0.95 ± 0.03 vs 0.86 ± 0.04, respectively, P < .001; internal: 0.71 ± 0.21 vs 0.61 ± 0.21, P < .001; tumor bed: 0.61 ± 0.17 vs 0.52 ± 0.17, P = .001) and versus MSERg for internal rectal wall contours (0.71 ± 0.21 vs 0.63 ± 0.18, respectively; P < .001). LIBR demonstrated lower MHD versus PBR for external and internal rectal wall contours and tumor bed (external: 0.56 ± 0.25 vs 1.68 ± 0.56, respectively, P < .001; internal: 1.00 ± 0.35 vs 1.62 ± 0.59, P < .001; tumor bed: 2.45 ± 0.99 vs 2.69 ± 1.05, P = .03) and versus MSERg for internal rectal wall contours (1.00 ± 0.35 vs 1.62 ± 0.59, respectively; P < .001). LIBR demonstrated lower TRE (1.54 ± 0.39) versus PBR (2.35 ± 1.19, P = .003) and MSERg (2.36 ± 1.43, P = .03). Computation time per WMH slice for LIBR was 35.1 seconds ± 12.1.

Conclusion

This study demonstrates feasibility of accurate MRI-WMH coregistration using the advanced LIBR method.

Keywords: MR Imaging, Abdomen/GI, Rectum, Oncology

Supplemental material is available for this article.

© RSNA, 2024

Summary

Accurate coregistration of preoperative MRI and whole-mount histology was feasible using an advanced deformable image registration technique in patients with rectal adenocarcinoma.

Key Points

• ■ Advanced regularized Kelvinlet linearized iterative boundary reconstruction (LIBR) demonstrated higher accuracy over traditional rigid point-based registration and state-of-the-art multiscale spectral embedding registration (MSERg) (eg, target registration error, 1.54 mm ± 0.39 [SD] vs 2.35 mm ± 1.19 and 2.36 mm ± 1.43, respectively; P < .05) in aligning MRI with whole-mount histology in patients with rectal adenocarcinoma.

• ■ LIBR coregistration also achieved a 40-fold improvement in computation time per whole-mount histology slice over MSERg (35.1 seconds ± 12.1 vs 1452.3 seconds ± 24.6, respectively; P < .001).

• ■ LIBR coregistration is promising to guide MRI-to-pathology reference standard mapping for supervised machine learning toward improving MRI assessment of treatment response in rectal cancer.

Introduction

Colorectal cancer is the third most common cancer and the second leading cause of cancer deaths worldwide (1). While there is an overall downward trend in colorectal cancer incidence and mortality, there is an alarming increase in the rate of early-onset colorectal cancer among young adults (1–3). Survivors of early-onset colorectal cancer bear a higher annual economic burden as well as increased employment disability and productivity losses (4), stemming from the long-term toxicity of therapy in current standard-of-care treatment.

Traditionally, neoadjuvant chemoradiotherapy followed by total mesorectal excision—and systemic adjuvant chemotherapy, if needed, for high-risk patients—has been the main treatment for locally advanced rectal cancer. Recently, total neoadjuvant therapy, in which systemic chemotherapy in combination with chemoradiotherapy are performed before surgery, has gained prominence due to better rates of pathologic complete response, enhanced patient outcomes (4), and higher rates of nonsurgical cure (sustained clinical complete response) observed in up to approximately 50% of patients enrolled in nonoperative management (5). Although encouraging, predicting which individuals can safely avoid surgery is challenging using current methods. Therefore, nonoperative management is performed in only a few specialized cancer centers. Digital rectal examination, endoscopy, and MRI are essential components of restaging assessment after neoadjuvant therapy; however, there is subjective interpretation and lack of uniform performance standards, leading to inconsistent interreader reliability (6–9). This variability further limits the widespread use of nonoperative management.

Currently, radiologists rely predominantly on T2-weighted imaging and, in some institutions, diffusion-weighted imaging for restaging MRI (10,11). However, there is varying diagnostic performance and interreader agreement with these imaging sequences (6–9,12). While several studies have investigated the added value of supplementary tools, including radiomics (13,14), artificial intelligence (15,16), and dynamic contrast-enhanced MRI (17), to increase MRI diagnostic performance in predicting treatment response (18), only a few included a reliable reference standard such as whole-mount histology (WMH).

WMH has the potential to provide a comprehensive evaluation of the entire rectum, preserving crucial spatial distribution of the structures. It serves as a fundamental reference point for validating and calibrating rectal MRI techniques and imaging assessment. Early in the application of rectal MRI, Brown et al (19,20) used WMH as the reference standard to define key landmarks, as well as imaging criteria, for preoperative baseline high-resolution MRI assessment. Since then, however, only a few studies have used WMH to assess restaging rectal MRI performance (21–23). Additionally, there is a lack of robust fusion strategies using WMH and rectal MRI, with only two reported pilot studies (24,25). Both used rigid transformations that cannot account for rectal tissue deformations between in vivo imaging and postoperative histologic processing.

To provide a foundation for accurate treatment response evaluation after neoadjuvant therapy, the current study aimed to develop a radiology-pathology coregistration method for 1:1 automated spatial mapping between preoperative rectal MRI and ex vivo rectal WMH.

Materials and Methods

Study Sample

This retrospective single-institution study was approved by the institutional review board with a waiver for written informed consent and complied with the United States Health Insurance Portability and Accountability Act.

The institutional database was searched for consecutive patients with rectal adenocarcinoma who underwent neoadjuvant therapy followed by total mesorectal excision at our institution from January 2019 to January 2022 and who had available WMH specimens. Of the initial 150 patients identified, 132 were excluded (99 because their pathology slices did not fit into a single whole-mount cassette, 30 because presurgical MRI data were unavailable, and three because of severe artifacts) (Fig 1). Some included patients (seven) were previously reported in a study evaluating the performance of diffusion-weighted MRI in detecting viable extramural venous invasion and tumor deposit in patients with locally advanced rectal cancer after neoadjuvant therapy, using WMH specimens as a reference (21).

Figure 1:

Flowchart of patient inclusion. This retrospective study included consecutive patients with rectal adenocarcinoma who underwent total mesorectal excision at our institution from January 2019 to January 2022 and who had available whole-mount histology (WMH) specimens. The exclusion criteria were as follows: WMH slices not fitting into a single whole-mount cassette, presurgical MRI data unavailable, and severe restaging MRI artifacts.

In Vivo Rectal MRI Protocol

Restaging rectal MRI examinations were performed using a 1.5- or 3.0-T scanner (Discovery MR750, Optima MR450w, Signa EXCITE, or Signa HDxt; GE HealthCare) equipped with a phased-array coil. Images were acquired using three orthogonal planes of the pelvis for anatomic orientation. Initially, T2-weighted sagittal images of the rectum were obtained to localize the tumor and enable the planning of axial section acquisitions. Subsequently, T2-weighted axial images of the pelvis were acquired in a plane perpendicular to the rectal wall (T2-weighted axial oblique plane), including the tumor and 5 cm above and below the tumor (26). The T2-weighted axial oblique plane had a field of view of 180 × 180 mm, a scan matrix of 512 × 512, an in-plane resolution of 0.35 × 0.35 mm, echo times from 79 to 125 msec, and repetition times from 2500 to 6562 msec. The section thickness was consistently 3 mm with section spacing of 4 mm, and the flip angle was fixed at 111° to ensure clear T2-weighted imaging. If the tumor was near or involved the anal canal, coronal imaging of the anal canal was also performed, using similar imaging parameters as those for T2-weighted axial imaging.

Postcontrast sequences were not used due to their limited acceptance and use in both baseline and restaging settings (18,27,28). Conversely, diffusion-weighted imaging has proven beneficial for identifying viable tumor within the tumor bed (29). In our study, we opted to segment the tumor bed using T2-weighted imaging, considering its superior spatial resolution for this specific purpose.

Whole-Mount Histology

The rationale of WMH is to display the tumor and its relationship to other histologic structures on the same horizontal plane. We retrospectively assessed the WMH slides, and the following methodologic workflow was executed. First, the rectal cancer resection specimen was kept intact and unopened in formalin for more than 48 hours. Then, circumferential margins were marked with different ink colors to enable proper orientation (Fig 2). Subsequently, starting from the distal point to the proximal end, the entire tumor region was sliced transversely to obtain 4-mm-thick WMH slices (21). Each WMH slice was photographed and placed into a megacassette (approximately five times larger than a regular tissue cassette). Slides were then stained with hematoxylin-eosin. Each stained slide was digitized to facilitate side-by-side comparison with rectal MR images.

Figure 2:

Demarcation of a standard total mesorectal excision specimen on (A, B) surgical and whole-mount histology (WMH) slices and (C) radiology-pathology workflow of segmentation and landmark definition. (A) Circumferential margins of the rectal cancer resection specimen were marked with different ink colors to distinguish anterior and posterior and right and left regions of the specimen and enable proper orientation. (B) WMH slice of the total mesorectal excision specimen demonstrates the color code that was used to guide the spatial localization of the rectum portions, as follows: black = anterior, red = posterior, blue = left, and green = right. (C) A gastrointestinal pathologist and a radiologist conducted a collaborative review of WMH and MR images to ensure precise correspondence between pathology and high-resolution T2-weighted imaging. Three corresponding levels were established for each patient at WMH and MRI: the midpoint of the tumor bed, one slice or section above, and one slice or section below. Subsequently, both experts manually delineated the external rectal contour (the outer edge of the muscularis propria), internal rectal contour (inner aspect of the mucosa), and tumor bed at each designated level (illustrated at the midpoint level). Additionally, the radiologist and pathologist annotated eight corresponding point-based landmarks in each modality along the internal and external borders of the rectal wall, encompassing the anterior, posterior, leftward, and rightward directions.

Image Segmentation and Landmarks

Digitized WMH slides served as the reference standard for the restaging rectal MR images (T2-weighted images) using the high-resolution oblique sequence, perpendicular to the tumor bed. A gastrointestinal pathologist (C.F., with 7 years of experience) and a radiologist (J.M., with 6 years of experience) performed side-by-side review of WMH and MRI to achieve precise point-by-point correlation between WMH and MRI, establishing three corresponding levels for each patient at WMH and MRI—midpoint of the tumor bed, one section above, and one section below—and then manually delineating the external rectal contour (the external edge of the muscularis propria), internal rectal contour (inner aspect of the mucosa), and tumor bed at each designated level. To delineate the tumor bed accurately, the radiologist analyzed the baseline MRI scan using the T2-weighted imaging sequence, which is known to enhance the delineation of the tumor bed at restaging MRI (30). Furthermore, the radiologist and pathologist annotated eight corresponding point-based landmarks in each modality along the internal and external borders of the rectal wall in the anterior, posterior, leftward, and rightward directions (Fig 3). In addition to the eight landmarks, three additional corresponding target points were identified to validate accuracy by selecting three corresponding validation points anatomically localized between the external and internal rectal contours. Target points were identified along well-discriminated common vein positions, texture patterns, and anatomic geometric relationships visible in both radiologic and pathologic images. These target points were selected within the rectal wall in the muscularis propria layer. The specific features selected as target points differed among patients depending on the best discriminatory point between radiology and pathology.

Figure 3:

Coregistration workflow for the proposed linearized iterative boundary reconstruction (LIBR) MRI-histopathology fusion method after segmentation and landmark characterization as demonstrated in Figure 2. Prior to coregistration, eight point-based landmarks are annotated on the external and internal rectal contours on each MR and whole-mount histopathology (WMH) image. A series of regularized Kelvinlet control points is then distributed across the external and internal rectal contours, from which a biomechanical deformation basis is constructed. After point-based registration of landmarks, deformation of the histopathology sample relative to MRI is computed from the rectal wall contours via the LIBR approach. A series of control points are distributed across the external and internal rectal contours to establish a regularized Kelvinlet deformation basis for the WMH image. The LIBR approach estimates the deformation between WMH and MRI by maximizing the agreement between rectal wall contours subject to this deformation basis. Finally, the registered WMH is fused with MRI to indicate spatial correspondence between reference standard histopathologic features and MRI.

An experienced gastrointestinal pathologist (J.S., with 23 years of experience) was consulted for equivocal pathology slides, and final determinations were based on consensus. Similarly, for equivocal MR images, an experienced gastrointestinal radiologist (N.H., with 10 years of experience) was consulted.

Radiology-Pathology Image Coregistration

Due to intermodality differences between radiology and pathology, we assumed that contour-based coregistration methods using continuum models of soft tissue deformation to constrain realistic tissue behavior would be more reliable than intensity-based coregistration methods. Thus, we developed an advanced multimodal radiology-pathology contour-based coregistration workflow to enable spatial mapping between WMH and MRI. An advanced deformable image registration method was developed using regularized Kelvlinet solutions to linear elastic biomechanics (31) to establish a deformation basis that is optimized within a linearized iterative boundary reconstruction (LIBR) framework (32) to optimize agreement between rectal wall contours. This proposed approach maintains biomechanical consistency when estimating deformation between WMH tissue samples and MR images with excellent computational efficiency. Details are included in the Biomechanical Model, Deformation Energy, and Deformable Registration Model sections in Appendix S1. Regularized Kelvinlets establish a highly efficient, closed-form decomposition of the underlying mathematical equations that govern soft tissue mechanics. Incorporating regularized Kelvinlets into the LIBR approach subsequently leverages this deformation basis to rapidly and accurately reconstruct tissue deformations at low computational expense.

Figure 3 outlines the workflow of the proposed coregistration method. The workflow begins with rigid point-based registration (PBR) with rescaling using a standard singular value decomposition method (33) to register the eight point-based landmarks that were manually annotated on the external and internal rectal contours in WMH and MR images and thus initialize the multimodal alignment. Then, the proposed regularized Kelvinlet LIBR method was used to refine differences in radiology-pathology agreement caused by soft tissue deformations occurring between the imaging time point and the fixation of the resected specimen in formalin.

Separately from our proposed contour-based LIBR coregistration workflow, a second intensity-based deformable image registration method, the state-of-the-art multiscale spectral embedding registration (MSERg) (34) method, was evaluated. This method uses a radiomic-based intensity pattern matching approach to detect common independent components of imaging signal between modalities and drive a B-spline deformation model.

Statistical Analysis

Coregistration accuracy for each coregistration method (PBR, LIBR, and MSERg) was evaluated using three measures. The Dice similarity coefficient (DSC) was used to assess overlap in the segmented tumor bed and external and internal rectal contours between radiologic and pathologic modalities. The modified Hausdorff distance (MHD) was used to assess the spatial distance to agreement of the segmented rectal contours and tumor bed between radiologic and pathologic modalities. Finally, target registration error (TRE) was assessed as the reference standard measure for spatial registration error according to the distance after coregistration between corresponding target points identified in the radiologic and pathologic images. Further details of these validation measures are provided in the Evaluation Metrics section of Appendix S1.

Mean ± SD values of DSC, MHD, and TRE were computed for each patient across each of the three axial levels, as well as averaged across the entire study sample. Statistical differences in average accuracy between our LIBR method and the PBR or MSERg methods were tested for significance via paired t tests, at a significance level of α = .05.

Computation time per pathology slide for each coregistration method was also measured.

Coregistration methods and statistical analyses were performed in MATLAB 2022b (MathWorks).

Results

Patient Characteristics

The final study sample (Table 1) comprised 18 patients (mean age, 54 years ± 13 [SD]; nine male patients [50%] and nine female patients [50%]). The majority underwent total neoadjuvant therapy (15 of 18, 83%) (six with induction chemotherapy and nine with consolidation chemotherapy), while the remaining (three of 18, 17%) underwent only chemotherapy. Chemotherapy regimens used in total neoadjuvant therapy included fluorouracil, leucovorin, and oxaliplatin (ie, mFOLFOX), capecitabine and oxaliplatin (ie, CAPOX), or fluorouracil, leucovorin, and oxaliplatin (ie, FLOX). Standard chemoradiation involved long-course radiation therapy (45 Gy in 1.8-Gy fractions to the pelvis, with a 5.4-Gy boost to the tumor) with concurrent fluorouracil or capecitabine.

Table 1:

Patient Characteristics

The mean duration between MRI and resection was 22.8 days ± 10.1, with a range from 4 to 38 days. Most patients (12 of 18, 67%) underwent low anterior resection. When assessing the pathologic tumor stage (pT), the majority (10 of 18, 56%) were classified as pT3. A substantial portion of the patient sample (14 of 18, 78%) showed no nodal involvement, no metastasis (17 of 18, 94%), no tumor deposits (14 of 18, 78%), and no extramural vascular invasion (17 of 18, 94%).

Coregistration Results

Contour overlap.— LIBR had a higher DSC compared with PBR for the external rectal contour, internal rectal contour, and tumor bed (external rectal contour: 0.95 ± 0.03 vs 0.86 ± 0.04, respectively, P < .001; internal rectal contour: 0.71 ± 0.21 vs 0.61 ± 0.21, P < .001; tumor bed: 0.61 ± 0.17 vs 0.52 ± 0.17, P = .001).

Comparing LIBR and MSERg, LIBR demonstrated a higher DSC for the internal rectal contour (0.71 ± 0.21 vs 0.63 ± 0.18, P < .001), and MSERg had a higher DSC for the external contour (0.97 ± 0.02 vs 0.95 ± 0.03, P = .04). No evidence of a difference was observed between LIBR and MSERg in the DSC for the tumor bed (P = .10).

Distance to agreement.— LIBR demonstrated a lower MHD compared with PBR for the external rectal contour, internal rectal contour, and tumor bed (external rectal contour: 0.56 ± 0.25 vs 1.68 ± 0.56, respectively, P < .001; internal rectal contour: 1.00 ± 0.35 vs 1.62 ± 0.59, P < .001; tumor bed: 2.45 ± 0.99 vs 2.69 ± 1.05, P = .03).

Additionally, LIBR had a lower MHD compared with MSERg for the internal rectal contour (1.00 ± 0.35 vs 1.62 ± 0.59, respectively; P < .001), with no evidence of a difference in MHD for the external rectal contour or tumor bed between LIBR and MSERg.

Target registration error.— Regarding TRE, LIBR demonstrated lower values (1.54 ± 0.39) compared with both PBR (2.35 ± 1.19, P = .003) and MSERg (2.36 ± 1.43, P = .03).

Table 2 provides quantitative details of the accuracy of each coregistration method.

Table 2:

Comparison of Different MRI–Whole-Mount Histology Coregistration Methods Using Three Measures

Figure 4 provides a visualization of the alignment quality achieved by each coregistration method, allowing for a straightforward visual comparison of each method’s effectiveness in different scenarios (best-, average-, and worst-performing cases within our study sample in terms of alignment). Figure 5 illustrates the distribution of accuracy values for each coregistration method across the study sample. Tables S1–S3 provide DSC, MHD, and TRE values, respectively, at the patient level.

Figure 4:

Visualization of the alignment quality achieved by each coregistration method. Restaging MR–whole-mount histology image fusion is shown in three representative cases. Case 10 (a 43-year-old female patient), case 15 (a 65-year-old male patient), and case 7 (a 47-year-old male patient) represent cases with the best Dice similarity coefficient, average Dice similarity coefficient, and worst Dice similarity coefficient, respectively. Between the investigated linearized iterative boundary reconstruction (LIBR), rigid point-based registration (PBR), and multiscale spectral embedding registration (MSERg) methods, LIBR produced better alignment. Note that in case 7, MSERg produced a misalignment of approximately 90° between the pathology slice and the MR image.

Figure 5:

Distribution of accuracy values for each coregistration method across the study sample. Coregistration accuracy was assessed for our linearized iterative boundary reconstruction (LIBR) method, the rigid point-based registration (PBR) method, and the multiscale spectral embedding registration (MSERg) method using the Dice similarity coefficient (DSC), the modified Hausdorff distance (MHD), and the target registration error (TRE). The LIBR method produced lower TRE, DSC, and MHD than the rigid PBR method across all features evaluated (external rectal contour, internal rectal contour, and tumor bed) and also produced lower TRE, DSC, and MHD than MSERg along the internal rectal wall.

Computation Time

Computation time per WMH slice was lower for PBR compared with LIBR (0.0003 seconds ± 0.0004 vs 35.1 seconds ± 12.1, respectively; P < .001) and for LIBR compared with MSERg (35.1 seconds ± 12.1 vs 1452.3 seconds ± 24.6, respectively; P < .001). Each coregistration method differed by more than one order of magnitude in computation speed. Table S4 provides computation times at the patient level.

Discussion

There is an urgent need to establish a precise reference standard to enable accurate imaging evaluation of treatment response after neoadjuvant therapy in patients with rectal cancer. WMH provides a comprehensive evaluation of the entire rectum, preserving crucial spatial distribution of the structures. Thus, we developed a radiology-pathology coregistration method for accurate 1:1 automated spatial mapping between preoperative restaging rectal MRI after neoadjuvant therapy and ex vivo rectal WMH. Our advanced biomechanically constrained contour-based LIBR deformable coregistration method outperformed rigid PBR, as well as state-of-the-art MSERg, an intensity-based deformable coregistration method. LIBR demonstrated higher DSC versus PBR for external and internal rectal contours and tumor bed (external: 0.95 ± 0.03 vs 0.86 ± 0.04, respectively, P < .001; internal: 0.71 ± 0.21 vs 0.61 ± 0.21, P < .001; tumor bed: 0.61 ± 0.17 vs 0.52 ± 0.17, P = .001) and versus MSERg for internal rectal contours (0.71 ± 0.21 vs 0.63 ± 0.18, P < .001). LIBR demonstrated lower MHD versus PBR for external and internal rectal contours and tumor bed (external: 0.56 ± 0.25 vs 1.68 ± 0.56, respectively, P < .001; internal: 1.00 ± 0.35 vs 1.62 ± 0.59, P < .001; tumor bed: 2.45 ± 0.99 vs 2.69 ± 1.05, P = .03) and versus MSERg for internal rectal contours (1.00 ± 0.35 vs 1.62 ± 0.59, respectively; P < .001). LIBR demonstrated lower TRE (1.54 ± 0.39) versus PBR (2.35 ± 1.19, P = .003) and MSERg (2.36 ± 1.43, P = .03). LIBR coregistration also achieved a 40-fold improvement in computation time per WHM slice over MSERg (35.1 seconds ± 12.1 vs 1452.3 seconds ± 24.6, respectively; P < .001).

In the realm of coregistering radiology images with pathology specimens, the field remains relatively specialized, with limited publications to date. The existing literature predominantly revolves around prostate cancer (35–38), with only two references to rectal cancer (24,25) and another to lung nodules (39). These studies explored a range of coregistration methods, including the use of injected fiducials, thin-plate spline, and a combination of rigid and nonrigid techniques.

Only two studies proposing a coregistration technique between MRI and WMH exist in the scientific literature to date (24,25). Both studies used a rigid PBR approach that depends on manually annotated landmark positions estimated along the inner and outer walls of the rectum in both MRI and WMH. We reproduced this PBR approach as a baseline comparator for the deformable registration method we propose in this work. Whereas several deformable coregistration methods have been proposed primarily in the context of prostate cancer, comprehensively reviewed by Pham et al (25), no existing methods have been applied to coregistration of rectal WMH.

Antunes et al (24) used a rigid landmark-based registration method equivalent to our PBR comparator. Their method yielded coregistration accuracy of 1.50 mm ± 0.63 between pathology and MRI in six patients with rectal cancer; however, this level of accuracy was not reproducible in our larger sample of 18 patients, likely due to larger tissue deformations, differences in treatment response, and differences in tissue processing, despite a similar time interval from MRI to surgery. Our findings indicate that PBR, despite its initial promise, exhibited limitations in coregistration accuracy according to all accuracy metrics (ie, DSC, MHD, and TRE). While Antunes et al suggested that the radiology-pathology fusion approach could aid interpretation of restaging MRI, they found that their method was limited in distinguishing T2-weighted imaging intensity ranges between fibrotic tissue and residual cancer. Machine learning models will likely be necessary to address clinical scenarios where distinguishing between viable and nonviable tissue is critical, and accurate mapping of reference standard information from pathology at the voxel scale will be essential toward training these models.

In comparison, both deformable image registration methods (LIBR and MSERg) displayed greater accuracy in terms of DSC and MHD metrics than did PBR, with LIBR emerging as the most effective method for coregistration, especially for tumor bed and internal border segmentations. Only LIBR restored a similar level of coregistration accuracy as that reported in Antunes et al (24). MSERg, which was originally developed for registration of prostate MRI with WMH, was less accurate than LIBR. This likely arises from the dependence of MSERg on correlating cross-modality texture information, whereas LIBR depends only on geometric alignment of the rectal walls while maintaining biomechanical consistency throughout the sample; this is done by modeling soft tissue deformations in a computationally efficient manner that also minimizes dependence on patient-specific tissue properties. Although MSERg had the best DSC for the external rectal wall, overfitting is likely, and LIBR generally showed higher accuracy across all other metrics while reducing computation time more than 40-fold compared with MSERg. The overfitting behavior of MSERg arises due to this method applying a binary mask of the external rectal wall to the intensity values of both the MR and WMH images. This subsequently forces high conformity of the external contours, which may negatively impact alignments of other internal structures.

The multimodal workflow proposed for MRI and WMH coregistration builds toward new capabilities to robustly map the tumor bed habitat from high-resolution histopathology images to MRI, allowing for the creation of highly accurate reference standard labels for training machine learning models to predict residual disease at the voxel scale. Future work will use the methods developed herein to correlate reference standard labels from pathology to MRI to train artificial intelligence and radiomics models to distinguish viable from nonviable rectal tumor tissue. These models aim to improve prediction of which patients may safely avoid surgery after neoadjuvant therapy or to more precisely localize and target disease for local-regional therapy within the treatment course of rectal cancer. However, we acknowledge challenges related to pathology angulation of the surgical specimen, which can potentially differ from MRI angulation, such as large rectal specimens that may not fit into a single whole-mount megacasette and the manual effort needed to establish voxel-scale correlations. Addressing these challenges will involve exploring automated methods for aligning pathology and MR images, optimizing hardware and software to accommodate large rectal specimens, and implementing efficient algorithms to establish voxel-scale correlations, thus paving the way for more streamlined and accurate clinical applications.

The primary limitations of our study included the relatively small retrospective sample size of 18 patients, which restricts the robustness and generalizability of our findings. Considering the feasibility phase of this study, we opted to include only patients whose entire rectum could be accommodated within one megacassette. This decision, while reducing the sample size, facilitated a more homogeneous method. Additionally, the retrospective nature of the study with a mean interval between MRI and surgery of 22 days presented challenges in the coregistration process. To address concerns about potential orientation discrepancies between tissue slices and MRI sections, our method focused on identifying and annotating specific anatomic landmarks and target points visible in both modalities. This approach allowed for accurate validation and alignment between WMH and MR images, accounting for any potential differences in orientation. While subjective interpretation may play a role to some extent, the extensive experience of the reviewers helped ensure consistency and reliability in the correlation process. This concern is further compounded by inherent methodologic challenges, such as the limited capacity for PBR to account for soft tissue deformation and the need for further refinement to improve automatability of coregistration methods. Technical and operational constraints, including imaging resolution, precision of segmentation techniques, and potential for human error in landmark annotation, likely had a substantial impact on the results. Furthermore, as this was a single-center study, our results may predominantly reflect local practices and patient demographics, potentially limiting generalizability.

In conclusion, we proposed a mechanics-based deformable registration algorithm for multimodal coregistration between MRI and WMH in rectal cancer and found that the proposed technique outperforms state-of-the-art techniques for rigid and deformable pathology image fusion. While a multicenter study with a larger patient sample and prospectively assessing MRI-WMH using controlled slice angulation directed by MRI are crucial for optimizing the workflow, improving alignment between the modalities and enhancing generalizability, this study demonstrates the feasibility of this approach, laying a solid foundation for future endeavors.