Registration methodology for histological sections and in-vivo imaging of human prostate

Introduction

Registration is a process that establishes a spatial correspondence between two scans so that both can be viewed in the same spatial frame. Often registration is a necessary pre-processing step before we can make a quantitative comparison between two scans as it is difficult to locate exactly corresponding spatial features in both scans free of personal bias. Here we present a method to register histology sections with various in-vivo images including anatomical MRI, diffusion MRI, and PET of human prostate. Histology slides carry the ground truth information regarding locations and extents of tumor determined by a pathologist. Once mapped onto the in-vivo imaging space, we can observe how different features of in-vivo imaging help to characterize tumor and normal tissues determined by histology. We can observe how well anatomical MRI, diffusion MRI, and PET differentiate between normal and tumor tissues for prostate cancer. This information will enable us to determine which modality or combination of modalities is suited best to stage prostate cancer. The current practice of staging prostate cancer uses anatomical MRI and/or transrectal ultrasound (TRUS), both of which have unsatisfactory accuracy (1).

Registration of histology sections and in-vivo imaging modalities is a challenging problem. The key issue is to account for the complex deformations that occur in the prostate as it is removed from the body and then sectioned for histological evaluation. First, the prostate potentially undergoes non-linear deformation as it is removed from the body, because there are no surrounding organs to constrain the prostate and vascular pressurization as well as intracellular fluids are lost. Second, the prostate is further deformed as it is fixed and then sliced for histological evaluation. The other challenge is the difference in available information between in-vivo imaging (e.g., resolution in millimeters) and histology (e.g., resolution in microns). Registration among in-vivo imaging modalities is relatively easy, as the deformation involved is less drastic than deformation involved with registering histology.

We use three in-vivo imaging acquisitions, anatomical T₂ weighted 3T MRI of pelvis, diffusion weighted 1.5T MRI of pelvis, and whole body ¹¹C-choline PET. Anatomical MRI is the standard imaging to stage prostate cancer. Diffusion MRI and PET provide functional tissue contrast information not visible in anatomical MRI. Diffusion MRI demonstrates water molecular diffusion (2) and ¹¹C-choline PET demonstrates increased tumor accumulation in slow growing malignancies such as prostate cancer (3). Because of the negligible excretion of ¹¹C-choline into the urinary system, ¹¹C-choline PET has been applied in particular to pelvic malignancies, for which low background radioactivity is critical for successful imaging (4).

Most of existing research on registering histology and in-vivo imaging focuses on brains of small animals, typically rats (5-9). One can still apply the same methodology used to register in-vivo rat scans and histology to human prostate case, but there are difficulties with the application to human prostate. The biggest problem is the inferior quality of in-vivo MRI for human prostate in obese patients; while the signal to noise ratio (SNR) of 3T MRI is approximately two times better than at 1.5T, penetration is limited in obese patients. Currently rats undergo high field MRI, such as 7T MRI, while humans typically only undergo 3T MRI. In addition to the difference in magnetic field strength (B0), the human prostate is located far inside the body, which further attenuates the radiofrequency (B1) coil field and degrades the SNR of human in-vivo prostate MRI.

In the following, we briefly cover existing literature on registering histology and in-vivo imaging. Kim et al. used a 3D automatic registration between neuro-autoradiograph and in-vivo MRI of rat brains via the intermediate step of registering both onto a stack of images obtained from tissue block face (5). Kim’s method is based on mutual information (MI) and thin plate splines (TPS). Jacobs et al. registered in 3D rigid fashion using a “head and hat” algorithm and then applied 2D TPS warping to force the previously rigidly aligned histology contours to match those of the in-vivo MRI of rat brains (6). Wilson’s group first registered rigidly in 3D in-vivo MRI with stacked tissue images using manually identified needle fiducials with an iterative closest point algorithm and then registered histology with stacked tissue images using manually identified landmarks with TPS (7). Wilson’s method was applied to sheep brains and rabbit thighs. Zarow et al. registered postmortem MRI of human brain with stacked tissue brain section using Pearson cross-correlation coefficient and MI (8). Zarow et al. identified one planar MRI slice that corresponded to one tissue section and corrected for warping using an n-th order polynomial. Li et al. registered in-vivo MRI of rabbit brains with brain tissue sections using manually identified landmarks with TPS, which also assumed one corresponding MRI slice for tissue section (9). Most existing methods either have laborious manual components typically associated with identifying landmarks, or make the assumption that one histological tissue section has only one corresponding in-vivo imaging slice except Kim’s paper. Most of the current literature comparing cross-sectional in-vivo prostate imaging with histology is rather limited due to a considerable uncertainty of the true location of histological findings in 3D space in respect to in-vivo imaging. Either “histological confirmation” of imaging results is assumed if a particular focus on imaging simply matched the correct side of the histologically confirmed malignancy, or imaging results are compared with multiple (up to 12) biopsies taken from several prostate segments. However, the exact location of a focus, particularly when small, is never truly spatially matched with histological findings. Recently Meyer et al. demonstrated a 3D automatic registration of histology slides with in-vivo 7T coronal MRI of rat brain using MI and TPS. This work neither assumes a one-to-one correspondence between MRI slices nor assumes that a planar histology section maps onto a single 2D plane of any orientation in the MRI volume necessitated by all the tissue deformations encountered during surgical extraction and histological preparation of the sample (10).

In this study, we build on this methodology, Meyer et al., to address the difficulties of co-registration related to the human prostate. To test the feasibility of this new approach, the methodology was applied to two patients.

Materials and Methods

In this section, we cover general registration framework, scan acquisition details, and overall schematic to register both in-vivo and ex-vivo imaging modalities. We obtained informed consents from the patients after fully explaining the nature of data collection protocol and this study.

In this paper we break the difficult, direct registration of histology and in-vivo imaging into more accurate sub-registration tasks involving intermediate modalities (i.e., specimen MRI and block face photographs of the prostate). We first register each histology section onto the block face photo taken before the histology section was microtombed; this 2D registration removes the deformations associated with microtoming. Then we register the collection of geometry-corrected block face photos onto the ex-vivo specimen MRI, and finally register ex-vivo specimen MRI onto in-vivo MRI all in 3D. These sub-registration tasks can be performed more accurately since the deformations being modeled are far less complex. In addition, the mutual information content of each step is higher than that involved in registering histology and in-vivo MRI directly. Block face photos are stacked to form a 3D volume before they are registered onto specimen MRI. This stacking process further increases the information content and thus improves the accuracy and stability in registering block face photos onto specimen MRI. Our method is automatic after the user’s initialization and thus unbiased. It also allows a histology section to be registered onto a curved 2D baffle potentially intersecting multiple in-vivo MRI image slices. Our work here is a proof-of-concept paper on data from two patients.

Registration Framework

Registration is a process of establishing a spatial correspondence between two scans so that both can be viewed in the same spatial frame. Registration methods have been well reviewed by Hill at al. (11). Briefly, two main components need to be addressed for any registration algorithm: the similarity measure which measures degree of alignment and the geometric interpolant which defines the geometric transform between two scans. We choose mutual information (MI) as the similarity measure and thin-plate splines (TPS) as the geometric interpolant (12). Computing the MI involves calculating probability density functions of grayscale value distributions. A simple histogram with fixed bin width is used to estimate the probability density function. The process of registration can be formulated as maximizing the chosen similarity measure (i.e., MI) under a hypothetical geometric transform,

$$\begin{array}{l}\widehat{T}=\underset{T\in F}{\mathrm{arg\; max}}\mathit{MI}(A(•),B(T(•))\\ \widehat{T};\mathrm{estimate\; of\; the\; transform}\\ F;\mathrm{family\; of\; feasible\; transforms}\end{array}$$

A simplex optimizer is used to maximize the objective function (13). One can also choose other combinations of similarity measures (e.g., normalized MI) and geometric transform (e.g., B-spline) (14). We use a software package developed in our lab based on MI and TPS (12).

The degrees of freedom (DOF) of TPS are determined by the number of control points. Control points are different from manually picked landmarks or markers as they can be optimized automatically through a registration algorithm. Complex geometric transform can be well modeled with a large number of control points at appropriate locations. Control points primarily affect local geometry though the effect is not strictly local. For example, control points in liver region primarily affect the geometric transform in the liver region. This is particularly true for high DOF TPS transforms. The number and location of control points differ from one registration task to another depending on the deformation it is trying to correct. For example, scan A and scan B are registered using N where (N > 4) control points for warping. The user needs to initialize (i.e., approximately specify locations of control points in both scans) the TPS transform. This initialization becomes laborious as the number of control points increase, thus we adopt a standard multi-level approach. In this case, N control points are manually defined in scan A and the first 4 roughly corresponding points are manually identified in scan B. First, the registration is performed using the first 4 control points in both scans resulting in a specific optimized geometric transform. We then use this optimized transform to initialize the N control point registration. The remaining N-4 control points defined in scan A are mapped onto scan B using the optimized transform. Thus, we have initialized an N-point TPS registration. In summary, the user needs to specify N control points on one scan and only 4 points on the other scan to start the registration for a N point registration task. Once the registration task starts, location of control points are optimized. The registration process is automatic after the initial placement of 4 control points.

Scan Acquisition

We acquired following in-vivo modalities: T₂ weighted anatomical 3T MRI of pelvis and diffusion weighted 1.5T MRI of pelvis were obtained. We use the regular body coil for anatomical 3T MRI and an endorectal coil for diffusion 1.5T MRI for better contrast. ¹¹C-choline PET/CT was obtained on a Siemens BioGraph hybrid PET/CT system which consists of a Siemens HR+ PET scanner and a 2 slice CT scanner. The intrinsic PET image resolution of the system varies between 4.1 mm FWHM in the center and 7.8 mm at r = 20 cm in axial direction. T₂ weighted MRI defines the prostate’s in-vivo geometry. Diffusion weighted MRI measures water molecule mobility and is useful in differentiating normal/tumor tissue in peripheral zone of the prostate (2). Diffusion MRI was not done on the 3T scanner due to the presence of exaggerated artifacts resulting from lack of adequate fat suppression and respiration. The scan acquisition parameters are as follows;

• T₂ weighted anatomical MRI: field strength = 3T, axial orientation, high resolution sequence, TR = 3208 ms, TE = 86 ms, 720×720×24 matrix, 0.31×0.31×4 mm³ voxel dimension.

• Diffusion MRI: field strength = 1.5T, axial orientation, fast spin echo sequence, TR = 4950 ms, TE = 122 ms, 256×256×25 matrix, 1.01×1.01×4 mm³ voxel dimension.

• Choline PET: half life = 6586 s, 128×128×210 matrix, 3.90×3.90×2.4 mm³ voxel dimension

• Abdominal CT; non-contrast, 512×512×210 matrix, 0.98×0.98×2.4 mm³ voxel dimension

The patient undergoes prostatectomy and the prostate specimen is removed from the body. The prostatectomy specimen is first fixed in paraformaldehyde. After 48 hour fixation time, the prostate specimen is then imaged with a 3T MRI scanner surrounded by perfluorcarbon solution. The specimen is then grossly sectioned into ~3mm whole mount slices for further processing (i.e., alcohol dehydrated, stained, paraffin-embedded), and microtomed into 5-15μm thick slide-mounted sections. The slides are stained using the standard hematoxylin and eosin stain. In addition to Pathology’s standard methodology of processing of slides, we acquired digital photographs of the stained and fixed whole mount slides (referred to in the text herein multiple times as “block face photo”) before each microscopic slide section is cut. We acquired the following ex-vivo image modalities, i.e., specimen MRI of the en-block excised prostate, block face photographs of the specimen, and photographs of their corresponding histology sections. Parameters for ex-vivo scan acquisition are as follows;

• Specimen MRI: field strength = 3T, T₂ weighted spin echo sequence, TR = 1500 ms, TE = 80 ms, 512×512×60 matrix, 0.195×0.195×0.7 mm³ voxel dimension.

• Block face photograph: obtained from digital camera, 3648×2736 matrix, 0.353² mm² pixel dimension, 6 or 7 slices in total, roughly 3 mm apart in apex-base direction

• Histology: obtained from digital camera, 4256×2848 matrix, 0.085² mm² pixel dimension, 6 or 7 slices in total, roughly 3 mm apart in apex-base direction

Registration Schematic

The overall goal is to register histology sections of prostate and in-vivo imaging including anatomical MRI, diffusion MRI, and PET. We choose in-vivo anatomical MRI as the reference space onto which all other scans will be registered. Registrations within in-vivo imaging modalities are relatively simple as the deformations involved are mild as the prostate is entirely contained in the abdomen before surgery. While our main goal is to register histology sections and in-vivo MRI, all other in-vivo modalities (i.e., PET and diffusion MRI) are also registered onto the same common reference. The overall registration schematic is given in Fig. 1 and details of individual sub-registration tasks are provided here in conjunction with Fig. 1.

Figure 1. Registration schematic.

All registration sub-tasks are 3D registrations except for histology onto block face registration. Solid arrows indicate each sub-registration task. Dotted arrow indicates the difficult direction registration between anatomical MRI and histology. Pictures of anatomical MRI (references space), ex-vivo specimen MRI, block face photo, and histology section are provided for better visualization. Note that block face photo and histology section is made into grayscale. Registration results still may be applied to the original color block face photo and histology.

• First, we register each histology photo onto the corresponding block face photo using 6 control points where 5 points are placed along the prostate’s outer boundary in a pentagon fashion and one point is placed at the center. This registration process is 2D and is repeated for all available histology sections. Pathology at our institution typically produces 6 or 7 histology sections depending on the size of the prostate. This registration is quite robust as histology slides are prepared from the sections and many common features can be found in both histology slides and block face photos. Since the act of cutting the sections via microtome introduces independent 2D deformations in adjacent sections, it is important to remove these artifacts independently in each section using a 2D warping.

• Second, we stack block face photos to form a sparsely sampled 3D volume. Successive block face photos are registered in rigid (i.e., rotate-translate) fashion and then stacked. We place rigidly registered photos at the same spacing used to section the prostate (i.e., 3 mm) and insert zero valued slices in between so that individual slices are thinner. Four zero valued slices are used between non-zero slices (i.e., photo slices), thus slice thickness is 3/5 = 0.6 mm. Not inserting these zero valued slices will make regular photo slices (i.e., non-zero slices) too thick, which is far from the reality, since histology slides are very thin (i.e., order of μm). Inserting more zero valued slices will make slices thinner but it will lead to partial volume effects (i.e., resolution mismatch) in the next registration step. This volumetric stacking process is illustrated in Fig. 2.

• Third, we perform a 3D registration between stacked block face photos and the ex-vivo specimen MRI using 18 control points where 5 points are placed in a pentagon plus one in the center, similar to the first registration step, for 3 slices. If one block face photo slice tends to be incorrectly registered in the specimen MR then the adjacent slices below and above will discourage that slice in question to be registered onto a wrong position. Basically, adjacent slices will help the slice in question to achieve better registration. Without stacking of block face photos, we need to perform a difficult and often unstable slice (i.e., block face photo) to volume (i.e., specimen MRI) registration. The stacking process increases the information content compared to having just a single slice so that the registration process is more accurate and stable. This stacking part is novel compared to what was proposed in our previous paper on rat brain (10). Registration is best achieved when both scans have similar voxel dimension, thus we cannot have the stacked photo volume be far thinner than ex-vivo MRI whose thickness is 0.7 mm. The second and third tasks described here account for 3D, non-linear deformation of the prostate specimen associated with excision, dehydration, paraffin embedding and gross sectioning.

• Fourth, we register ex-vivo specimen MRI and in-vivo anatomical MRI using 7 control points where 3 points are placed along the rectum/prostate boundary, 2 points are along the prostate/bladder boundary, and 2 points are at top/bottom of the prostate shown in the left picture of Fig. 3. This registration process accounts for the deformation of prostate as it is taken out of the body.

• Finally, we combine registration results from these four tasks and achieve registration between histology slides and in-vivo anatomical MRI. The stacked block face photo volume is the key bridge between histology and in-vivo MRI.

Figure 2.

Volumetric stacking to form a 3D volume of block face photos.

Figure 3. Control points used for some sub-registration tasks.

Control points are displayed in the in-vivo anatomical MRI space with blue spheres centered over control point locations. The left figure shows the set of 7 control points used to register the in-vivo anatomical MRI and the ex-vivo specimen MRI. The 7^th point is hidden as it is placed in the apex of the prostate. The right figure shows the set of 10 control points (only 5 points are visible) used to register the in-vivo anatomical MRI and the in-vivo diffusion MRI. The hidden 5 points are placed towards the apex of the prostate.

To register the other in-vivo modalities to the anatomical 3T MRI (all shown on the left side of Fig. 1) we apply standard registration methodology.

• First, we register the diffusion MRI with anatomical MRI. This registration process takes care of presence/absence of endorectal coil as the anatomical MRI was taken using the regular body coil on the 3T magnet while diffusion MRI was taken on the 1.5T magnet using an endorectal coil. We use 10 control points where two sets of 5 control point configurations, each consisting of a triad enclosing the prostate and 2 points in the rectum, are placed at 1/3 and 2/3 height of prostate in slice direction shown in the right picture of Fig. 3. Acquisition of the diffusion MRI in the 1.5T scanner involves collecting a 3-vector data set, i.e., high and low B-field data, as well as a T₂ weighted MRI, all in the same coordinate space. It is this T₂ weighted MRI that is actually registered with our reference, the 3T anatomical MRI. We then apply the same geometric transform to the diffusion MRI computed from the high and low B-field acquisitions to map it to the reference space.

• Second, we register PET and CT using an identity transform. Registration of PET and CT is trivial as both CT and PET come from a PET/CT scanner where both scans reside at the same spatial framework. Thus, we use an identify transform to register CT and PET. Then we register the CT and the anatomical MRI using 6 control points where two triads are placed at mid prostate and the boundary between top of prostate and bladder. Combing two registrations, we map PET onto the reference space. In summary, we can register PET onto the reference space via CT and diffusion MRI onto the reference space employing the standard registration approach.

Once all sub-registration tasks are performed, then spatial correspondence between any combination of modalities can be established by combining results from suitable sub-registration tasks. For example, if we want to map histology onto diffusion MRI, we combine registration of histology onto stacked block face, registration of stacked block face onto specimen MRI, registration of specimen MRI onto anatomical MRI, and registration of anatomical MRI onto diffusion MRI sequentially.

Results

In this section, we show registration results of histology, anatomical MRI, diffusion MRI, and PET for two patients. Intermediate registration between stacked block photos and ex-vivo MRI is covered as well.

Registration results: patient 1

We present registration results by mapping anatomical MRI, diffusion MRI, and PET onto the histology space. This is feasible as any modality can be mapped onto any other modality by combining results from suitable sub-registration tasks. Once mapped onto the histology space, we can observe how in-vivo imaging features are correlated with high resolution histological truth. Registration results could be shown in the anatomical MRI space (i.e., reference space), but in this case high resolution histology information will suffer from the partial volume effect as one anatomical MRI voxel corresponds to many histology pixels. Therefore, we map in-vivo imaging onto histology space to better visualize results with histology. Note that this is a presentation issue: we are still using the same set of geometric transforms from the computed sub-registration tasks, whether they are forward or inverse transform.

Registration results of patient 1 are shown in Fig. 4. Patient 1 has 6 available histology slices and we show results for slices 3 and 4. Results for other histology slides are of similar registration quality. Everything is viewed in the high resolution histology space as in-vivo modalities are mapped onto histology space. Slice 3 has a tumor located at both left and right lobes of peripheral zone confirmed by histology. Cancerous tissue is identified with lower gray values in anatomical MRI and diffusion MRI and higher gray values for PET compared to values in the central zone. There is no sign of tumor in the registered anatomical MRI and PET, while diffusion MRI shows tumor activity at some parts of the peripheral zone. PET has higher signal in the central zone. All the findings made on the in-vivo space are compared with histological truth. Slice 4 has similar results. Tumor is identified in the peripheral zone and it is only picked up by diffusion MRI. Registration of slice 4 shows a well aligned “fissure” like structure found both in anatomical MRI and histology as indicated by yellow circles in Fig. 4. For this patient, diffusion MRI seems to be able to predict cancerous tissue well. Inspecting registered diffusion MRI with respect to histology shows that rectum/peripheral zone boundary is not well aligned with the histology boundary. Registered PET is well aligned as a whole, but inspecting the registered boundary is difficult as PET has fuzzy boundaries due to its inherent lower spatial resolution. Overall, in-vivo modalities are well aligned with histology.

Figure 4. Registration of anatomical MRI, diffusion MRI, and PET onto histology for patient 1.

Registration results for histology slices 3 (top figure) and 4 (bottom figure) out of 6 slices are presented. Columns of images are histology (grayscale converted), registered anatomical MRI, registered diffusion MRI, and registered PET, respectively from left to right. Cancerous tissue is enclosed in dotted lines on histology. Middle row images are colored overlay of registered anatomical MRI, registered diffusion MRI, and registered PET all in grayscale with histology in a green hue. Bottom row images are alternating checkerboard fusion of registered anatomical MRI, registered diffusion MRI, and registered PET with histology. On histology slides, R, L, A, and P denotes right, left, anterior, and posterior respectively. Slice 4 has the “fissure” like structure marked with yellow circles.

Registration results: patient 2

Registration results of patient 2 are shown in Fig 5. Patient 2 has 7 available histology slices and we show results for slices 4 and 5. Registration results for other histology slides are of similar registration quality. Slice 4 has a large tumor located at the right lobes of the peripheral zone. Registered anatomical MRI and diffusion MRI show signs of cancer (i.e., lower gray value) in the same area. Registered PET shows increased tracer uptake (i.e., higher gray value) in the same area. Slice 5 has similar results. The tumor is identified in the right lobe of peripheral zone and it is picked up by all in-vivo modalities. For this patient, all in-vivo modalities seem to predict cancerous tissue well. Inspecting registered in-vivo modalities shows similar result as patient 1. The rectum/peripheral zone boundary for diffusion MRI is not well registered, PET is well aligned on a large scale, and overall all in-vivo modalities are well aligned with histology.

Figure 5. Registration of anatomical MRI, diffusion MRI, and PET onto histology for patient 2.

Registration results for histology slices 4 (top figure) and 5 (bottom figure) out of 7 slices are presented. Columns of images are histology (grayscale converted), registered anatomical MRI, registered diffusion MRI, and registered PET, respectively from left to right. Cancerous tissue is enclosed in dotted lines on histology. Middle row images are colored overlay of registered anatomical MRI, registered diffusion MRI, and registered PET all in grayscale with histology in a green hue. Bottom row images are alternating checkerboard fusion of registered anatomical MRI, registered diffusion MRI, and registered PET with histology. On histology slides, R, L, A, and P denotes right, left, anterior, and posterior respectively.

Intermediate registration between stacked block photos and ex-vivo MRI

Here we show registration results of one of the sub-registration tasks. Registration of stacked block photos and ex-vivo MRI is shown in Fig. 6. Fig. 6 shows that there is no single 2D plane of ex-vivo MRI in any orientation that corresponds to one block face photo section. A block face photo section is likely to be mapped onto a 3D manifold spanning many MRI slices due to the complex deformation encountered during surgical extraction and histological sample preparation. From Fig. 1 and descriptions of our registration methodology, we note that one histology section maps onto one block face photo (i.e., 2D registration) and registration between specimen MRI and in-vivo MRI is a warping mapping where one slice of ex-vivo MRI can be mapped over many in-vivo MRI slices. Considering these two observations along with the fact that the block face photo slice is mapped over many in-vivo MRI slices as shown in Fig. 6 as indicated by the curved block face slices, it is not reasonable to assume that there is one particular in-vivo MRI slice that corresponds to one particular histology section. This overly simplified assumption is found quite commonly in the existing literature (6-9). The work presented here need not make this assumption since our registration methodology allows one histology section to be mapped over many in-vivo MRI slices.

Figure 6. Intermediate registration results of stacked block face photos and ex-vivo MRI.

Registration result of patient 1 is shown in the left figure and that of patient 2 is shown in the right figure. Stacked block face photos is mapped onto the ex-vivo MRI. Stacked block face photos are shown in green hue with ex-vivo MRI overlay in grayscale. Note the curved appearance of mapped stacked block face photos.

Registration error from two patients

Registration between in-vivo imaging modalities is well established (11), thus the registration errors between in-vivo imaging modalities are not reported here. Instead, we focus on registration errors between in-vivo imaging and histology. Validation of a registration methodology for human scans almost always requires implanted markers or intrinsic landmarks that can be identified for all modalities. Unfortunately, those patients with extrinsic markers (i.e., brachytherapy seeds used for human prostate) do not have surgeries that result in histological section of the prostate with markers. Landmarks are difficult to identify especially in the periphery, and even though histology sections carry a wealth of information, it is very difficult to find corresponding landmarks on in-vivo imaging.

One feature common to all scans in our study is that all patients have primary prostate cancer. Post surgical resection histology contains tumor boundary marking by the pathologist. Cancerous tissue, especially diffusive tissue, is indicated by low Apparent Diffusion Coefficient (ADC) value areas in diffusion MRI. Thus we can validate our registration methodology by comparing surrogates for tumor boundary markings in histology and tumor boundary in diffusion MRI indicated by low ADC value areas. Tumor boundary in histology is defined by a pathologist and the tumor boundary in diffusion MRI is defined by an expert. Diffusion MRI is chosen as the space where the surrogate for in-vivo MRI is defined as diffusion MRI has better sensitivity to cancerous tissue than anatomical MRI. Our data gathering protocol is restricted to clinical stage T2 prostate cancer patients so it is very likely that tumors in these patients are large enough to be picked up by diffusion MRI.

Tumors identified in diffusion MRI may appear smaller than the tumors identified in histology as the resolution of diffusion MRI is many orders of magnitude worse than histology. We propose using an error measure which will not penalize a good registration if the boundary in diffusion MRI is an eroded (i.e., shrunken) version of the boundary found in histology. In this case, a spatial overlap measure (15) will be small even if tumor volumes are well aligned. Our error measure is based on the observation that the medial axis of a boundary is relatively insensitive to the erosion operation. We define the error as the difference between two medial axes over the range of the shorter medial axis. We propose a distance measure, Euclidean distance between overlapping two medial axes computed from two boundaries. This distance measure compares two surrogates in a way that it is insensitive to morphological erosion or dilation of the boundaries. The 3D medial axis of a boundary is defined to be the locus of the centers of spheres that are tangent to the boundary in two or more points, which roughly translates to a curve that runs along the middle of a boundary. From this definition, two boundaries, one of which is an eroded version of the other, have the same overlapping root medial axis. We compute the distance measure between the shorter medial axis and the overlapping portion of the longer medial axis. We represent the medial axis with discretized voxels, which is very similar to “skeletonization” in 3D (16). Skeletons are discretized connected components that are one voxel thick. Two skeletons (i.e., collection of voxels) of shorter and longer medial axes are computed in the diffusion MRI and histology space respectively. Distance between two axes can be computed only if they are mapped onto the same space, thus we map the longer medial axis (of histology) onto the diffusion MRI space. For a given voxel in the shorter skeleton, we find the closest voxel in the mapped longer skeleton and the Euclidean distance value is recorded. This process is repeated and the average is computed to be the distance measure. Thus, we take the average of the Euclidean distances between the voxels in the shorter medial axis and the voxels in the mapped longer medial axis. MATLAB was used to implement the 3D medial axis algorithm and compute the distance between the medial axes. Fig. 7 shows tumor boundaries and their associated medial axes for patient 1’s case.

Figure 7. Medial axes for tumor boundaries for patient 1.

The top left figure shows the longer medial axis of the tumor boundaries from the histology with medial axis shown in light blue dots. The medial axis exists in 3D but here we show histology slice 4 which is the closest slice to the most of the medial axis points. The top right figure is the shorter medial axis of low ADC areas from diffusion MRI with medial axis shown in light blue dots. The medial axis also exists in 3D but we just choose the closest slice to visualize results. The bottom left figure is the 3D view of the short medial axis and the mapped (onto diffusion MRI space) longer medial axis. The mapped longer medial axis is shown in white dots while the shorter medial axis is shown in light blue dots. Registration error can be thought as the distance between white dots and light blue dots in the bottom left figure.

The distance measures (i.e., registration errors between diffusion MRI and histology) for patient 1 and 2 are 3.74 mm and 2.26 mm respectively, whose mean and std are 3.00 mm and 1.05 mm respectively. Registration error is commonly assessed in the units of voxel dimension. As our registration methodology is a combination of sub-registration tasks, its accuracy is limited by the worst case voxel dimension, which comes from the 1.5T diffusion MRI whose dimension is 1.01×1.01×4 mm³ or 1.603 mm³ for an equivalent cubic voxel. The mean registration error, 3.00 mm, is 1.87 times the worst case equivalent cubic voxel dimension, a very respectable performance for a registration algorithm. The registration errors reported here have come from only 2 cases, lacking statistical power. Thus, our work here is a proof of concept paper.

Discussions

Patient 1 has a relatively small tumor while patient 2 has a larger tumor. Diffusion MRI is the only in-vivo modality that detects a sub-volume of the tumor for patient 1, while all in-vivo modalities detect the tumor for patient 2. One possible explanation for our results is that bigger tumors are larger than the resolutions of all in-vivo modalities, thus the tumors show up in all in-vivo modalities. Registered diffusion MRI shows more misalignment in the rectum/peripheral zone, while anatomical 3T MRI shows better alignment in the same region. The deformation between diffusion 1.5T MRI and anatomical 3T MRI is due to the incompletely corrected deformation caused by the inconsistent presence of endorectal coil in the 1.5T diffusion study and its absence in the 3T MRI anatomical study, which was registered using 10 control points. The potential reason why this deformation was not correctly modeled is the lack of grayscale information in the 1.5T anatomical MRI scan. Thus even when we use more control points, we do not obtain much improvement in the registration quality because the intrinsic information content of the images does not support the effective use of control points. Registration of PET scans is adequate considering the lower resolution of PET. Boundaries of the prostate found in PET are well registered with boundaries of histology. Our methodology could be easily adopted for other PET radionuclides potentially useful for molecular imaging of prostate cancer such as ¹⁸F-Fluorocholine, ¹⁸F-FDG, ¹⁸F-FAZA, and ¹¹C-acetate (17-20). We expect that in the near future the fat suppression and motion compensation techniques will improve on the 3T magnet such that all MRI data can be acquired on the same scanner.

Our overall registration schematic involves many sub-registration tasks presented here with different sets of control points used each time. We believe it is possible to have one registration approach applied to all sub-registration tasks, greatly simplifying the whole process. An adaptive registration approach is possible, where the DOF is gradually increased (i.e., density of control points is increased) from low to high until the underlying information from the scans is no longer able to support the increased DOF. The user may start the registration using a uniform grid of control points at a low density on one scan and the first 4 roughly corresponding points on the other scan as discussed in “Registration Framework” section. The adaptive registration algorithm will increase the DOF as far as the underlying information of scans supports. We leave this for future work.

Conclusions

We have shown how histology may be registered with in-vivo imaging including anatomical MRI, diffusion MRI, and PET for two patients without the use of extrinsic fiducial markers or unsubstantiated assumptions regarding geometric relationships between in and ex-vivo image sets. With this registration process we may be able to better quantify which in-vivo combination of modalities is better suited to stage prostate cancer.