A method of 2D/3D registration of a statistical mouse atlas with a planar X-ray projection and an optical photo

Abstract

The development of sophisticated and high throughput whole body small animal imaging technologies has created a need for improved image analysis and increased automation. The registration of a digital mouse atlas to individual images is a prerequisite for automated organ segmentation and uptake quantification. This paper presents a fully-automatic method for registering a statistical mouse atlas with individual subjects based on an anterior-posterior X-ray projection and a lateral optical photo of the mouse silhouette. The mouse atlas was trained as a statistical shape model based on 83 organ-segmented micro-CT images. For registration, a hierarchical approach is applied which first registers high contrast organs, and then estimates low contrast organs based on the registered high contrast organs. To register the high contrast organs, a 2D-registration-back-projection strategy is used that deforms the 3D atlas based on the 2D registrations of the atlas projections. For validation, this method was evaluated using 55 subjects of preclinical mouse studies. The results showed that this method can compensate for moderate variations of animal postures and organ anatomy. Two different metrics, the Dice coefficient and the average surface distance, were used to assess the registration accuracy of major organs. The Dice coefficients vary from 0.31±0.16 for the spleen to 0.88±0.03 for the whole body, and the average surface distance varies from 0.54±0.06 mm for the lungs to 0.85±0.10 mm for the skin. The method was compared with a direct 3D deformation optimization (without 2D-registration-back-projection) and a single-subject atlas registration (instead of using the statistical atlas). The comparison revealed that the 2D-registration-back-projection strategy significantly improved the registration accuracy, and the use of the statistical mouse atlas led to more plausible organ shapes than the single-subject atlas. This method was also tested with shoulder xenograft tumor-bearing mice, and the results showed that the registration accuracy of most organs was not significantly affected by the presence of shoulder tumors, except for the lungs and the spleen.

1. Introduction

Mice are widely used in preclinical studies because their biochemical interactions and pathways resemble human physiologic conditions. In the past two decades there have been many advances in small animal imaging techniques that facilitate or combine the non-invasive observation of anatomical and functional information from living mice. Along with the development of mouse imaging approaches, there is increasing demand of computerized image analysis to assist the definition of organ regions (Cheng-Liao and Qi, 2010; Khmelinskii et al., 2011a; Maroy et al., 2008), the quantification of molecular probe uptake (Maroy et al., 2010) and the construction of physiological models (Song et al., 2007; Zheng et al., 2011), so as to make data analysis less subjective, more accurate and faster. To serve these purposes, the registration of a mouse atlas to individual subjects is generally required to provide organ-level anatomical references. Various whole-body scale mouse atlases (Dogdas et al., 2007; Johnson et al., 2002; Khmelinskii et al., 2011a; Segars et al., 2004) have been developed and several approaches have been proposed to register these atlases with tomographic images like micro computed tomography (micro-CT) (Baiker et al., 2010; Baiker et al., 2011; Wang et al., 2012a), micro magnetic resonance imaging (micro-MR) (Khmelinskii et al., 2010), micro positron emission tomography (micro-PET) (Kesner et al., 2006) and micro single photon emission tomography (micro-SPECT) (Khmelinskii et al., 2011b).

In recent years, an emerging direction of small animal imaging uses limited-view projections instead of full tomography to guide the atlas registration. For example, Baiker et al. (Baiker et al., 2009) registered the mouse atlas with the body silhouettes captured by three optical cameras. Savinaud et al. and Zhang et al. (Savinaud et al., 2010; Zhang et al., 2009) used multiple-view optical photos or video sequences to fuse mouse atlases with optical molecular imaging. Li et al. (Li et al., 2009) developed a conical mirror to acquire laser scans of the entire animal surface and then registered the mouse atlas based on a surface-volume-combined elastic registration method (Joshi et al., 2010b). There are also efforts made to register 3D micro-CT images with multiple optical photographs of the same subject, based on 3D distance transform (Wildeman et al., 2009) or affine transformation (Xia et al., 2008). The benefits of using limited-view mouse atlas registrations are the reduced system complexity and cost, and the potential for high-throughput imaging and straight-forward organ region definitions. Due to simpler system design, the limited-view systems are also easier to integrate with molecular imaging modalities to provide combined functional/atlas information. In an overview of multiple potential limited-view mouse imaging systems (Wang et al., 2011), we simulated 11 combinations of three non-tomographic imaging devices (optical camera, planar X-ray and surface scanner) and compared the atlas registration accuracy of the 11 combinations. Based on the comparison, a mouse atlas registration system was designed composed of an anterior-posterior (AP) miniature X-ray projector and a lateral-view optical camera (Wang et al., 2012b). Fig. 1(a) illustrates the architecture of such a system. This system generates bi-planar projections where the X-ray presents the body region, the skeleton and the lungs, and the optical photo illustrates the lateral silhouette. Although the bi-planar projections do not provide full tomographic information, they already imply rich anatomical information because many low X-ray contrast organs (e.g. the brain, heart, liver, and kidneys) are anatomically correlated with the high X-ray contrast organs, (the body, skeleton and lungs), and the optical silhouette also reveals lateral spine curvature. Therefore, with a proper atlas registration method, a reasonable estimation of the whole-body anatomy can be achieved with the AP-view X-ray projection and lateral optical photo, as illustrated by Fig. 1(b).

Fig 1.

(a) The bi-planar mouse imaging system, including an AP-view X-ray projector and a lateral-view optical camera. (b) The purpose of this study is to register the mouse atlas with the bi-plane projections, and yield a 3D estimation of the mouse organ regions.

Beyond the preclinical field, 2D/3D atlas registration has been widely studied for clinical applications (Markelj et al., 2012), with the purpose of assisting image-guided interventions in bones and vascular structures (Groher et al., 2009). In recent years, statistical shape models (SSMs) became popular for 2D/3D atlas registration and have been applied to different bone parts like the femur (Baka et al., 2011; Hans et al., 2006; Zheng et al., 2009), the pelvis (Sadowsky et al., 2011; Yao and Taylor, 2003), the vertebrae (Benameur et al., 2003; Fleute et al., 1999) and the ribs (Dworzak et al., 2010). The SSM represents 3D organ shapes as a linear combination of the mean shape and the principal variations of the training set. By optimizing the combination coefficients of the SSM, the model can be fitted to the X-ray projections of the individual patient, leading to patient-specific reconstruction of anatomical models.

Compared with the clinical 2D/3D atlas registrations, the work presented here has unique challenges:

Most clinical applications target on local bony structures, while this study aims for the whole mouse body, which is a non-rigid multi-organ system with inter-subject variations of body postures and organ anatomies.

Not all the mouse organs have good contrast in the projection images. From the X-ray and optical images, only the body, the skeleton and the lungs can be distinguished, while other soft organs are visually imperceptible.

The number of projections involved in this study is limited (only two orthogonal projections), which poses challenges to accurate estimation of the atlas deformation.

To address the above challenges, the following strategies are used.

A statistical whole-body mouse atlas is constructed based on multiple training subjects. Statistical shape models (SSM) of the mouse organs are constructed to represent the inter-subject variations of postures and organ anatomies.

The organs in the atlas are divided into two groups according to their contrasts in the projection images, i.e. the high contrast group and the low contrast group. The anatomical correlation between the two groups is represented with a conditional Gaussian model (CGM) and thin-plate-spline (TPS) interpolation.

For atlas registration, the SSM of the high contrast group is first registered with the X-ray and optical images, and then the low contrast group is mapped using the CGM and the TPS interpolation.

To register the atlas with the limited number of projections, we use a 2D-registration-back-projection strategy which registers the atlas projections in 2D, and then back-projects the 2D registration results to 3D to guide the deformation of the atlas.

2. Methods

2.1. Statistical mouse atlas

Due to the articulated skeleton motion and the variations in strain, weight, sex and age, normal laboratory mice present significant inter-subject differences of body postures and organ shapes. To overcome these differences, existing mouse atlas registration methods generally use a global non-rigid deformation to match a single-subject atlas with the target subject. However, this global deformation is inadequate to compensate for local organ differences. Moreover, since the deformation is normally derived from the mapping of skin or skeleton, it tends to distort the internal organs which don’t always follow the skin/skeleton deformations. To address these problems, we constructed a statistical mouse atlas from multiple training subjects instead of only one subject. Using the statistical shape model, the modes of inter-subject posture differences and organ deformations are learned from the training population. In one of our previous studies (Wang et al., 2012a), a statistical mouse atlas was constructed for the trunk region. In this study, we extend the atlas from the trunk to the whole-body, and increase the training sample size (from 45 to 83) to improve the coverage of subject variability. In the following paragraphs, we will briefly describe the atlas construction method and highlight the differences from the previous work.

The training set includes 83 healthy subjects of different strains, age, and sex selected from the database of the Crump Institute for Molecular Imaging, UCLA (Stout et al., 2005). Three commonly used strains (Nude, C57Bl/6 and severe-combined immunodeficient (SCID)) were included and the body weights varied from 15 g to 40 g. The training subjects were imaged at prone positions inside standardized multimodality imaging chambers (Suckow et al., 2009). Although these chambers provide moderate constrains of subject posture, random flexures of the spine and limbs were included in the training set. The Fenestra™ (ART, Quebec, Canada) liver-contrast agent was used to enhance the contrast of soft organs. Human experts were invited to segment the major organs from the micro-CT images, including the skin, skeleton, heart, lungs, liver, spleen and kidneys. The segmented organs were converted into triangular surface meshes, and the vertex numbers for different organ meshes were: 3000 for each of the skin and skeleton, 1500 for each of the lungs and liver and 800 for each of the heart, spleen and kidney. To establish the vertex correspondence between the training subjects, a reference subject was selected for each organ (except for the skeleton) and registered to the rest of the training subjects using the point set registration method based on Gaussian mixture models (Bing and Vemuri, 2005). For the skeleton, the point set registration method doesn’t work well due to the skeleton’s articulated nature, therefore the articulated skeleton atlas (Khmelinskii et al., 2011a) was used and registered to each training subject to obtain the vertex correspondences.

To describe the inter-subject anatomical/postural variations, the statistical shape model (SSM) was constructed based on the corresponding mesh vertices. The SSM is based on a linear point distribution model described as

$$\mathbf{V}=\overline{\mathbf{V}}-\mathbf{\Phi }\mathbf{b},$$ (1)

where V ∈ R^3N is a shape instance of the model represented as a column vector concatenated with the coordinates of N mesh vertices, V̄ ∈ R^3N is the mean value of the shape instances in the training set, Φ∈ R^3N×k is the matrix of k shape variation modes, which are obtained using the principal component analysis (PCA) of the mesh vertices and b∈ R^k represents the shape coefficients that control the linear combination of the variation modes.

As explained in the introduction section, the organs are divided into two groups. The first group was composed by the skin, skeleton and lungs, which present high contrast in the X-ray or optical images. The second group was composed by the low contrast organs including the heart, liver, spleen and kidneys. For each of the two groups, all the organs belonging in that group are modeled as a single object by concatenating the vertex coordinates of multiple organs into a single vector. The purpose of doing this is to guarantee that multiple organs deform in accordance with each other (i.e. avoiding organ intersections). The shape vector of the high contrast group is represented as

$${\mathbf{V}}_i^{\mathbf{H}}={[{\mathbf{V}}_i^{\text{skin}},{\mathbf{V}}_i^{\text{skeleton}},{\mathbf{V}}_i^{\text{lungs}}]}^{\text{T}}\in {\text{R}}^{3N^{\text{H}}},$$ (2)

where N^H =N^skin+N^skeleton+N^lungs, and N^skin, N^skeleton, and N^lungs are the number of vertices of the skin, skeleton and lungs, respectively; ${\mathbf{V}}_i^{\text{skin}}=[x_{i,1}^{\text{skin}},y_{i,1}^{\text{skin}},z_{i,1}^{\text{skin}},x_{i,2}^{\text{skin}},y_{i,2}^{\text{skin}},z_{i,2}^{\text{skin}},\dots ,x_{i,N^{\text{skin}}}^{\text{skin}},y_{i,N^{\text{skin}}}^{\text{skin}},z_{i,N^{\text{skin}}}^{\text{skin}}]\in R^{3N^{\text{skin}}}$, with ( $x_{i,j}^{\text{skin}},y_{i,j}^{\text{skin}},z_{i,j}^{\text{skin}}$) being the 3D coordinates of the j^th vertex of training subject i. The same notations also apply to skeleton and lungs. Similarly, the shape vector of the low contrast group is defined as

$${\mathbf{V}}_i^{\mathbf{L}}={[{\mathbf{V}}_i^{\text{heart}},{\mathbf{V}}_i^{\text{liver}},{\mathbf{V}}_i^{\text{spleen}},{\mathbf{V}}_i^{\text{left}\text{kidney}},{\mathbf{V}}_i^{\text{right}\text{kidney}}]}^{\text{T}}\in {\text{R}}^{3N^{\text{L}}},$$ (3)

where N^L =N^heart+N^liver+N^spleen+N^{left kidney}+N^{right kidney}, and N^heart, N^liver, N^spleen, N^{left kidney}, and N^{right kidney} are the number of vertices of the heart, liver, spleen, left kidney and right kidney, respectively.

Two SSMs, namely SSM^H and SSM^L, are constructed for the high and low contrast groups based on the training sets of ${\mathbf{V}}_i^{\mathbf{H}}$ and ${\mathbf{V}}_i^{\mathbf{L}}$, respectively. To integrate into the atlas model, the anatomical correlation between SSM^H and SSM^L was described using the conditional Gaussian model (CGM),

$$\begin{array}{l}P({\mathbf{b}}^{\mathbf{L}}\mid {\mathbf{b}}^{\mathbf{H}})=N({\overline{\mathbf{b}}}^{\mathbf{L}\mid \mathbf{H}},{\mathrm{\sum }}^{\mathbf{L}\mid \mathbf{H}})\\ {\overline{\mathbf{b}}}^{\mathbf{L}\mid \mathbf{H}}={\overline{\mathbf{b}}}^{\mathbf{H}}+{\mathrm{\sum }}^{\mathbf{L},\mathbf{H}}{({\mathrm{\sum }}^{\mathbf{H}})}^{-1}({\mathbf{b}}^{\mathbf{H}}-{\overline{\mathbf{b}}}^{\mathbf{H}})\\ {\mathrm{\sum }}^{\mathbf{L}\mid \mathbf{H}}={\mathrm{\sum }}^{\mathbf{L}}+{\mathrm{\sum }}^{\mathbf{L},\mathbf{H}}{({\mathrm{\sum }}^{\mathbf{H}})}^{-1}{\mathrm{\sum }}^{\mathbf{H},\mathbf{L}},\end{array}$$ (4)

where b^H and b^L are the shape coefficients of SSM^H and SSM^L, respectively; b̄^L|H and Σ^L|H are the mean and covariance of the conditional distribution between b^H and b^L; b̄^H, Σ^H and b̄^L, Σ^L are the mean and covariance of b^H and b^L; Σ^L,H and Σ^H,L are the cross-covariance between b^H and b^L, respectively. The values of b̄^H, Σ^H, b̄^L, Σ^L, Σ^L,H and Σ^H,L are estimated from the training set.

The CGM provides a method to estimate the conditional distribution of low contrast organs according to the registered high contrast organs. In this study, the CGM was used for the heart, liver, spleen and kidneys, but not for the brain, which is highly correlated to the skull cavity. For the brain, simple TPS interpolation is sufficient to obtain reasonable brain mapping. The brain structures of the Digimouse atlas (Dogdas et al., 2007) were borrowed for this study, and the skull vertices of the articulated skeleton atlas were used as the TPS control points.

2.2. Registration method

The workflow of the proposed method is demonstrated in Fig. 2 and a supplementary video of the web version of this article. The atlas is registered in a hierarchical manner: SSM^H is registered first, then the low contrast organs are mapped based on the registered SSM^H. The registration of SSM^H is an iterative process. In each iteration, SSM^H is virtually projected as an AP-view X-ray and a side-view silhouette, and these virtual projections are registered with the measured subject projections in 2D, resulting in 2D deformations of each virtual projection. At last the 2D deformations are back-projected into 3D space, and the SSM^H is deformed via the back-projected 3D deformation. The details of each step are explained in the following subsections.

Fig 2.

Workflow of the registration method.

2.2.1. Preprocessing of the input images

Previous to the registration, camera calibrations are performed for both the AP- and lateral-views, and the X-ray and optical images are corrected for perspective and barrel distortions. As shown in Fig.1, the ventral part of the optical silhouette is blocked by the imaging chamber that supports the subject. To eliminate the influence of the imaging chamber, the optical photo is cropped, leaving only the dorsal silhouette (Fig.3(c)). Binary body regions are extracted from both the X-ray image (Fig.3(b)) and the optical photo (Fig.3(d)), using the ISODATA clustering method (Velasco, 1980). For the X-ray, 10 clusters are segmented and the darkest cluster is classified as the background; for the optical photo, two clusters are segmented and the brightest cluster is classified as the background. To facilitate subsequent registration, the back contour (Fig.3(e)) of the mouse is extracted from the lateral body region (Fig.3(d)). To compensate for the missing information of the ventral silhouette, the bottom line of the mouse bed (Fig.3(e)) is added underneath the back contour by projecting the known location of the mouse bed surface onto the lateral image. Since the position of the bed is fixed in the imaging system gantry, the bed projection only needs to be performed once for each imaging system.

Fig 3.

Preprocessing of the X-ray image and the optical photo. (a) Input X-ray image. (b) Segmentation of body region from (a). (c) Input optical photo. (d) Segmentation of body region from (c). (e). Contour of mouse back extracted from (d), with the bed bottom line obtained by projecting the known location of the mouse bed onto the lateral-view image.

2.2.2. Atlas initialization

At the very beginning of the registration, the mean shape of SSM^H is automatically placed at a fixed position and orientation in the imaging system gantry space. Then, initial adjustment of the atlas position, size and shape is calculated to give a rough alignment in whole-body scale. The initial adjustment is described by a transformation T_init = [l, b^H] that is applied to SSM^H, where l ∈ R⁹ is a 3D linear transformation, including three translations, three rotations and three anisotropic scaling factors; b^H ∈ R²⁴ is as described above, the shape coefficient of SSM^H, which contains 24 principal components that represent 95% of the training set variations. The purpose of the initialization step is to optimize the value of T_init to minimize the discrepancy between the virtual atlas projections and the measured subject projections in both projection views. We use an iterative optimization scheme to achieve this goal. For each iteration, a new value of T_init is evaluated; the evaluated T_init value is applied to SSM^H to calculate the 3D vertex coordinates of the high contrast organs:

$${\mathbf{V}}^{\mathbf{H}}=\mathbf{L}\left({\overline{\mathbf{V}}}^{\mathbf{H}}+{\mathbf{\Phi }}^{\mathbf{H}}{\mathbf{b}}^{\mathbf{H}}\right)$$ (6)

where L is the linear transform corresponds to l, V̄^H and Φ^H are the mean shape and shape variation matrix of SSM^H, respectively. Binary ray-tracing is performed to project the mesh of V^H into AP- and lateral-views by judging whether the projection ray is blocked by the skin mesh triangles. These projections produce two 2D binary images of the atlas body region, which are compared with the 2D body regions of the subject (as shown in Fig.3.(b) and (d)) based on the Kappa Statistics (KS) metric (Zijdenbos et al., 1994),

$$\text{KS}=\frac{2{\text{N}}_{\text{A},\text{S}}}{2{\text{N}}_{\text{A},\text{S}}+{\text{N}}_{\text{A}}+{\text{N}}_{\text{S}}},$$ (7)

where N_A,S is the number of pixels of value 1 in the projection images of both the atlas and the subject, N_A is the number of pixels of value 1 only in the atlas projection image, and N_S is the number of pixels of value 1 only in the subject projection image. KS has the range of [0, 1] where 0 means no overlap and 1 means complete overlap. The combined discrepancy metric of the two projection views between the atlas and the subject is defined as (1-KS_X)+(1-KS_C), where KS_X and KS_C indicates the KSs of the X-ray and optical camera images, respectively. To minimize this discrepancy metric with respect to T_init, the Powell method (Powell, 1964) which does not require explicit derivatives is used, because it is not straightforward to derive analytical functions between 3D mesh and its binary projections. The initial values of l and b^H are set to the identity transform and 0, respectively. To accelerate the convergence and avoid local minima, l is optimized first by keeping b^H as 0, and then followed by a joint optimization of l and b^H. Please note that during the Powell optimization, only the skin part of SSM^H is used for the binary projection. The skeleton and lungs are not projected because the binary projection rays are blocked by the skin thus cannot reach the internal organs. As a result, the Powell method will only optimize the skin deformation, but the skeleton and lungs will follow the skin deformation, because the skin, skeleton and lungs are all modeled together as a single object (see section 2.1, equation (2)). This is the first step necessary for initialization of the atlas registration.

2.2.3. 2D registration of the projections

After initialization, SSM^H enters the iteration loops of 2D-registration-back-projection. At the beginning of each loop, SSM^H is projected into both AP- and lateral-view, generating the virtual X-ray projection and virtual lateral silhouette of the atlas (as shown in Fig.2). The virtual X-ray projection is generated using the mesh-based ray-tracing approach (Wang et al., 2011) which computes the X-ray attenuation based on the traveling distance inside different tissues. The virtual lateral silhouette is generated using the binary projection approach described in section 2.2.2.

The virtual X-ray projection of SSM^H is registered with the subject X-ray projection using the mutual information-based 2D B-spline registration method (Thevenaz and Unser, 2000). The size of the B-spline control grid is 10×10 pixels. A multi-resolution registration scheme with five levels is used to accelerate the registration. The down-sampling ratios for the five resolutions are 16, 8, 4, 2 and 1 and an adaptive stochastic gradient descent algorithm is used as the optimizer. The registration is implemented using the elastix software (Klein et al., 2010). The output of the registration is the 2D deformation field D_X : R2→R² that maps the atlas X-ray to the subject X-ray, and D_X is represented as the 2D B-spline interpolation of the control grid point displacements. To obtain the atlas deformation from the AP-view, all the vertices (including skin, skeleton and lungs) of SSM^H are projected onto the X-ray image plane. Let V^H be the 3D vertex coordinates of SSM^H, and P_X : R3→R² be the projection from 3D space to the 2D X-ray image plane, the 2D projection of V^H on the X-ray image plane is ${\mathbf{V}}_{\mathbf{X}}^{\mathbf{H}}={\text{P}}_{\mathbf{X}}\left({\mathbf{V}}^{\mathbf{H}}\right)$, and then D_X is applied to ${\mathbf{V}}_{\mathbf{X}}^{\mathbf{H}}$ to obtain the deformed coordinates ${\text{D}}_{\text{X}}\left({\mathbf{V}}_{\mathbf{X}}^{\mathbf{H}}\right)$ of the projected vertices.

In a difference from the X-ray registration, we do not use mutual information-based 2D B-spline registration for the optical lateral silhouette registration. Instead, we use edge point searching and 2D thin-plate-spline (TPS) interpolation. This is because the lateral silhouette does not contain any anatomical structure inside the body region for the B-spline registration to match with. In this case, simple edge point searching is more efficient than the 2D B-spline registration. As demonstrated in Fig. 4, from each edge point of the atlas silhouette (gray line), line searching is performed along the edge normal to find the nearest subject silhouette edge (white line) (as shown in Fig.3(e)). The result of the line search is the vectors (red arrows) which point from the edge of the atlas silhouette to the edge of the subject silhouette. The 2D deformation field D_C : R²→R² of the lateral image is represented as the 2D TPS interpolation of the edge point vectors. Let P_C : R3→R² be the projection from 3D coordinate to 2D optical camera image coordinates, the 2D projection of V^H onto the camera image plane is calculated as ${\mathbf{V}}_{\mathbf{C}}^{\mathbf{H}}={\text{P}}_{\mathbf{C}}\left({\mathbf{V}}^{\mathbf{H}}\right)$, and then D_C is applied to ${\mathbf{V}}_{\mathbf{C}}^{\mathbf{H}}$ to obtain the deformed coordinates ${\text{D}}_{\mathbf{C}}\left({\mathbf{V}}_{\mathbf{C}}^{\mathbf{H}}\right)$ of the projected vertices.

Fig 4.

Edge point searching of the lateral-view silhouette. Line searching is performed for the edge points on the atlas silhouette (blue color) to find the corresponding edge points on the subject silhouette (white color). The red arrows indicate the deformation vectors of the atlas silhouette edge. Local parts (in dashed circles) with small arrows are magnified for clearer view. (For interpretation of the references to colors in this figure, the reader is referred to the web version of this article.)

2.2.4. Back-project 2D deformations into 3D

After the 2D registrations are accomplished, the 2D deformations of the two virtual projection views are back-projected into 3D to derive a 3D deformation of the atlas. The principle of the back-projection is illustrated by Fig.5, in which only one vertex (the red dot V^H) of SSM_H is demonstrated for clarity. Let O_X and O_C be the X-ray point source and the camera pinhole, respectively. Let $\overline{{\text{O}}_{\text{X}}{\text{D}}_{\text{X}}\left({\mathbf{V}}_{\mathbf{X}}^{\mathbf{H}}\right)}$ be the 3D line connecting O_X and ${\text{D}}_{\text{X}}\left({\mathbf{V}}_{\mathbf{X}}^{\mathbf{H}}\right)$, and $\overline{{\text{O}}_{\text{C}}{\text{D}}_{\text{C}}\left({\mathbf{V}}_{\mathbf{C}}^{\mathbf{H}}\right)}$ be the 3D line connecting O_C and ${\text{D}}_{\text{C}}\left({\mathbf{V}}_{\mathbf{C}}^{\mathbf{H}}\right)$. The back-projected 3D deformation is D_3D defined as

Fig 5.

Illustration of back-projecting the 2D deformations into 3D.

$${\text{D}}_{3\text{D}}\left({\mathbf{V}}^{\mathbf{H}}\right)=\text{CP}\left(\overline{{\text{O}}_{\text{X}}{\text{D}}_{\text{X}}\left({\mathbf{V}}_{\mathbf{X}}^{\mathbf{H}}\right)},\overline{{\text{O}}_{\text{C}}{\text{D}}_{\text{C}}\left({\text{V}}_{\text{C}}^{\text{H}}\right)}\right),$$ (8)

where CP ( )is the function that calculates the closest point to the two lines $\overline{{\text{O}}_{\text{X}}{\text{D}}_{\text{X}}\left({\mathbf{V}}_{\mathbf{X}}^{\mathbf{H}}\right)}$ and $\overline{{\text{O}}_{\text{C}}{\text{D}}_{\text{C}}\left({\mathbf{V}}_{\mathbf{C}}^{\mathbf{H}}\right)}$, i.e. D_3D(V^H) is the midpoint of the shortest connection between $\overline{{\text{O}}_{\text{X}}{\text{D}}_{\text{X}}\left({\mathbf{V}}_{\mathbf{X}}^{\mathbf{H}}\right)}$ and $\overline{{\text{O}}_{\text{C}}{\text{D}}_{\text{C}}\left({\mathbf{V}}_{\mathbf{C}}^{\mathbf{H}}\right)}$, where the shortest connection is shown as line section AB in Fig.5. The calculation of the closest point between two skew 3D lines is an analytical geometry problem which can be solved in various ways. We refer to (Ericson, 2005) for an efficient solution.

The back-projection provides straight-forward guidance of the 3D atlas deformation. However, the back-projected 3D deformation might be anatomically implausible, because there may be conflicts between the two 2D deformations, and each 2D deformation is also subject to the anatomical ambiguity caused by the 3D to 2D projections. To obtain anatomically plausible results, we project D_3D(V^H) into the shape space of the statistical atlas, using similar strategy to the active shape models (Heimann and Meinzer, 2009). As a result, the shape coefficient of SSM^H is updated as

$${\mathbf{b}}^{\mathbf{H}}\leftarrow {({\mathbf{\Phi }}^{\mathbf{H}})}^{+}(\text{Sim}\left({\text{D}}_{3\text{D}}\left({\mathbf{V}}^{\mathbf{H}}\right)\right)-{\overline{\mathbf{V}}}^{\mathbf{H}}),$$ (9)

where (Φ^H)⁺ is the pseudo-inverse of Φ^H, and Sim is a 3D similarity transformation (including three translations, three rotations and one scaling). Sim satisfies the minimization of the summed squared distance ||Sim (D_3D (V^H))– V̄^H|| and is calculated using the Generalized Procrustes Analysis (Bookstein, 1996). Finally, the vertices of SSM^H are updated as

$${\mathbf{V}}^{\mathbf{H}}\leftarrow {\text{Sim}}^{-1}\left({\overline{\mathbf{V}}}^{\mathbf{H}}+{\mathbf{\Phi }}^{\mathbf{H}}{\mathbf{b}}^{\mathbf{H}}\right),$$ (10)

where Sim⁻¹ is the inverse transform of Sim. Based on the updated V^H, the registration proceeds into the next iteration loop, and the 2D registrations and back-projections are repeated consecutively until convergence. The criteria of convergence is d_atlas<ε, where d_atlas is the mean vertex distance of the deformed atlas between two loops, and ε is set to 0.2 mm, which is the voxel size of micro-CT images used for the atlas construction.

2.2.5. Map low contrast organs

After SSM^H is registered, SSM^L is mapped using the CGM. The details of CGM-based organ mapping can be referred to (Wang et al., 2012a), and the basic principle has been described by equation (2). As a result of the CGM-based mapping, the low contrast organs closely follow the deformation of the high contrast organs without appreciable intersections. Finally, the brain is mapped using TPS-interpolation via the skull control points, as described in section 2.1.

3. Test data and validation metrics

The registration method was validated based on imaging studies performed by researchers at the Crump Institute for Molecular Imaging at UCLA. The AP-view, X-ray projection and lateral-view optical photos of the mice were acquired with the mouse atlas registration system (MARS) (Wang et al., 2012b) which is integrated into the Genisys4™ micro-PET scanner (Sofie Biosciences, Culver City, U.S.A.). The X-ray imaging protocol was 40 kVp, 100 μA, and 3 sec exposure, the output image was 1024×1000 pixels, each measuring 96x96μm. The optical photo was acquired with a 5mm focal length lens and 200 ms exposure, and the output image was 640×480 pixels, covering ~71×53 mm² field of view at the imaging plane.

In total, 55 subjects were involved in this validation, including 38 normal subjects and 17 tumor-bearing subjects with shoulder subcutaneous xenografts. The tested subjects were separated from the atlas training set. The test set included four commonly used strains (29 black C57Bl/6 mice, 15 Nude mice, nine SCID mice and two C3H agouti mice), and the body weights varied from 17 g to 45 g. All the subjects were imaged with prone positions inside multimodality imaging chambers. The normal subjects were used to test the method’s ability to handle different postures and body sizes, as well as the method’s convergence properties and registration accuracy. The tumor-bearing subjects were used to evaluate the influence of tumors on the registration accuracy.

To evaluate the registration accuracy of internal soft organs, the Fenestra™ liver-contrast agent was used for 14 of the normal subjects and 5 of the tumor-bearing subjects. The injection dose of the contrast agent was minimized to maintain reasonable CT contrast without undermining the spine contrast in the X-ray projections. Immediately after X-ray projection and photograph imaging in the MARS, tomographic micro-CT scans of the 19 contrast-enhanced subjects were acquired, and the animal motion between the MARS and CT imaging was minimized by the multimodality imaging chambers with anesthesia (Chow et al., 2006). The CT system was a MicroCAT II small animal CT scanner (Siemens Preclinical Solutions, Knoxville, TN, USA). The exposure settings were 70 kVp, 500 mAs, 500 ms and 360° rotation in 1° steps with 2.0 mm aluminum filtration. Images were reconstructed using a modified Feldkamp process to isotropic voxel size 0.20 mm, and a matrix size 256×256×496.

Human expert segmentations of the contrast enhanced micro-CT images were used as reference standard for organ regions. The micro-CT images were rigidly aligned with the MARS coordinate space by projecting the outlines of the segmented skin, skeleton and lungs onto the X-ray images and optical photos. Manual fine-tuning of the 3D rigid transformation (including three translations and three rotations) was applied to make sure the projected CT organ outlines matched the organ edges of the X-ray and optical images. Two accuracy metrics, the Dice coefficient and the averaged surface distance (ASD), were used to evaluate the registration accuracy. The Dice coefficient is defined as

$$\text{Dice}=2\frac{|{\text{R}}_{\text{A}}\cap {\text{R}}_{\text{S}}|}{\left|{\text{R}}_{\text{A}}\right|+\left|{\text{R}}_{\text{S}}\right|},$$ (11)

where R_A and R_S represent the organ region of the registered atlas and the expert segmentation, respectively (|·| denotes the number of voxels and ∩ means the overlap between two regions). The Dice coefficient has the value range of [0, 1], where 1 corresponds to complete overlap and 0 corresponds to no overlap. The ASD is defined as

$$\text{ASD}=\frac{1}{2}\left(\frac{1}{n_{\text{A}}}\sum _{i=0}^{n_{\text{A}}}{d}_i+\frac{1}{n_{\text{S}}}\sum _{j=0}^{n_{\text{S}}}{d}_j\right),$$ (12)

where n_A and n_S are the number of vertices in the surface mesh of the registered atlas and the expert segmentation, respectively. d_i is the minimum distance from the i^th vertex of the atlas surface to all the vertices of the expert segmentation surface, and d_j is the minimum distance from the j^th vertex of the expert segmentation surface to all the vertices of the atlas surface. Based on this definition, smaller ASD means better registration accuracy.

4. Results

4.1. Visual inspection of registration results

Although the investigators performing the experiments were asked to position the animals in prone positions with stretched limbs, the test subjects still presented moderate posture variations across the test dataset. The purpose of the normal subject evaluation was to validate the ability of the proposed method to match subjects with various postures, body size and internal anatomy. Fig. 6 and 7 demonstrate the results of twelve representative subjects from the total of 38 normal subjects. To visually compare the registered atlas with the target subjects, the organ outlines of the registered atlas are projected onto the X-ray and optical images. For clarity, the organs are divided into two groups for display. Fig. 6 shows the outlines of high contrast organs, i.e. the skin, skeleton and lungs, while Fig. 7 shows the outlines of internal soft organs, i.e. the brain, lungs, heart, liver, spleen and kidneys. Fig.6(a) shows two subjects with normal body size and standard posture; Fig.6(b) presents two subjects with irregular posture, with different limb orientations and spine curvatures; Fig.6(c) demonstrates two subjects with different body size, where the upper one is underweight and the lower one is overweight. Fig.7(a) also shows two subjects with different body sizes, where the upper one is overweight and the right one is underweight. Fig.7(b) displays two subjects with different spine curvature from the AP-view, where the upper one curves to the left and the lower one curves to the right; Fig.7(c) displays two subjects with different spine curvature from the lateral-view, where the upper one has a stretched spine and the lower one has a curved spine. Overall, the registration method is capable to provide close registration of the high contrast organs and reasonable estimation of the internal soft organs.

Fig 6.

Registration results of high contrast organs for the normal subjects. The organ outlines are projected onto the X-ray and optical images, respectively. Different organs are distinguished with different colors; White, skin; Yellow, skeleton; Blue, lungs. (For interpretation of the references to colors in this figure, the reader is referred to the web version of this article.) (a) Two subjects with normal body sizes and standard postures. (b) Two subjects with irregular postures, and different open angles of the fore and hind limbs. (c) Two subjects with different body sizes: the upper one is underweight and the lower one is overweight.

Fig 7.

Registration results of low contrast (soft tissue) organs of the normal subjects. The organ outlines are projected onto the X-ray and optical images, respectively. Different organs are distinguished with different colors; Yellow, brain; Red, heart; Dark blue, lungs; Green, liver; Purple, spleen; Light green, Kidneys. (For interpretation of the references to colors in this figure, the reader is referred to the web version of this article.) (a) Two subjects with different body sizes: the upper one is overweight and the lower one is underweight. (b) Two subjects with different spine curvatures from the AP-view. (c) Two subjects with different spine curvatures from the lateral-view.

4.2. Quantitative evaluation of atlas registration accuracy

Based on the 14 contrast-enhanced normal subjects, the Dice coefficients and ASD were calculated for the skin, skeleton, lungs, brain, heart, liver, spleen and kidneys. In the Dice evaluation, the skin is also called “whole body”, since even though the skin is a surface object, it coincides with the whole body as the volumetric region enclosed by the skin. Fig.8 demonstrates the box and whisker plots of the Dice coefficients and ASD for the major organs. It can be seen that most organs have median Dice coefficients over 0.5 (except for the skeleton and the spleen) and median ASD below 0.8 mm (except for the skin and the brain). The reason of the organ-dependant accuracy will be discussed in the discussion section.

Fig 8.

Dice coefficient and average surface distance (ASD) for the major organs of the 14 normal test subjects.

Since the skin and skeleton are whole-body scale organs, it is important to measure the regional registration accuracy for different parts of the skin and the skeleton. Fig. 9 illustrates the average local surface distance of the skin and skeleton across the 14 normal subjects. The local surface distance is measured as the minimum distance from each vertex of the atlas mesh to all the vertices of the CT segmentation surface mesh. It can be seen that for most parts of the skin and skeleton, the average local surface distance is less than 1.4 mm. Comparing the skin with the skeleton, the skin has larger surface distance than the skeleton at each corresponding location, because the 3D shape of the skin is more difficult to be determined from bi-planar projections. The hindlimbs have larger surface distance than the forelimbs because the hind-limbs in our implementation are always out of the X-ray field of view. The dorsal part of the head has larger surface distance for both skin and skeleton, because the presence of the ears slightly biases head registration. The dorsal part of the back skin and the spine have relative larger surface distance than their surroundings, because the atlas fitting tends to sacrifice this small and high-curvature part to optimize global fitting.

Fig 9.

Average local surface distances of the registered atlas skin and skeleton. Both the dorsal and ventral views are shown for the skin and skeleton.

4.3. Evaluation of convergence properties

The proposed registration method has three major steps: the starting point where the mean shape of SSM^H is placed at a fixed starting position, the atlas initialization step where the shape and size of SSM^H is initialized according to the binary body regions, and the final registration step where SSM^H is registered via an iterative process. To evaluate the accuracy of the three steps, the Dice coefficient and ASD were calculated at the end of each step, based on the 14 normal contrast enhanced subjects. To observe the influence of high-contrast organ accuracy on low-contrast organ accuracy, the low-contrast organs were mapped for the results of each step. Fig. 10 demonstrates a visual example of the registered atlas at each step (from the left to the right are the starting point, after atlas initialization and after registration, respectively), and the mean values and standard deviations of Dice coefficients and ASDs of each step are shown in Fig.11(a) and (b), respectively. It is clear that the accuracy improves and the standard deviation decreases as the registration progresses from the first to the last step. It can also be seen that the accuracy of low contrast organs improves from the first to the last step, meaning the accuracy of low contrast organs is directly affected by the high contrast organs.

Fig 10.

An example of the registered atlas at each step. From the left to the right: the starting point, after atlas initialization and after registration.

Fig 11.

Registration accuracy of the different steps of the proposed method. (a) Dice coefficients of different organs at each step. (b) ASDs of different organs at each step.

As a validation of the convergence speed, the entire registration took 176±45 (mean ± standard deviation) seconds on a PC with a 3.05 GHz CPU and 5.99 GB RAM, with the final registration step requiring an average of 6.4±2.2 iteration loops. The registration method was programmed with IDL 7.1 (ITT Visual Information Solutions, Boulder, CO, USA), except that the atlas initialization step was programmed with C++, and the 2D X-ray registration was realized by calling the external elastix software.

4.4. Comparison with other registration methods

The proposed method has two main features, i.e. the 2D-registration-back-projection strategy and the use of a statistical mouse atlas. To validate the effectiveness of the two features, a comparison test was performed by turning off each feature separately:

Turning off the back-projection strategy means that the atlas deformation is optimized directly in 3D. Direct 3D optimization has been used in several clinical studies (Benameur et al., 2003; Fleute et al., 1999; Yao and Taylor, 2003) to register a statistical bone model to X-ray projections. In a previous study (Wang et al., 2012b), we also used such a strategy to register the statistical mouse atlas with the AP-view X-ray projection and lateral-view optical photo. To realize this, the deformation of SSM^H (T=[l, b_H] similar as introduced in section 2.2.2) was optimized to maximize the similarity between the projections of the atlas and the subject. The initial values of l and b_H were set to identity transform and 0, respectively. The registration was executed iteratively, and at each iteration the similarity (f_X) between the atlas X-ray projection and the individual X-ray projection was measured with mutual information (Wells et al., 1996), and the similarity (f_S) between the atlas silhouette and the individual silhouette was measured with 1-KS (KS is the Kappa Statistics introduced in section 2.2.2). The Powell method was used to obtain the optimal T value that minimized the combined similarity f = f_X + f_S . A tri-resolution scheme was used to accelerate the registration and the down-sampling ratios of the three resolutions were 4, 2 and 1. After SSM^H was registered, the low contrast organs were mapped in the same way as described in section 2.2.5. In this work, the direct 3D optimization method is called no back-projection.

Turning off the statistical mouse atlas means using a single-subject atlas for the registration. This is the general strategy used by most existing 2D/3D mouse atlas registration methods (Baiker et al., 2009; Joshi et al., 2010b; Savinaud et al., 2010; Wang et al., 2011; Zhang et al., 2009). For our test, the single-subject atlas was created by combining the mean shapes of SSM^H and SSM^L, as well as the brain structures from the Digimouse atlas. The atlas initialization step was similar as in section 2.2.2, but only l was initialized because a single-subject atlas doesn’t have shape coefficients b. For registration, the skin, skeleton and lungs of the single-subject atlas were virtually projected and registered in 2D, and then the back-projection was applied to derive the 3D deformation (similar as section 2.2.4). However, the back-projected deformation was not smoothed as in section 2.2.4, because the single-subject atlas doesn’t have the atlas deformation space which must be learned from multiple training subjects. After the skin, skeleton and lungs were registered, the low-contrast organs were mapped via TPS interpolation of the back-projected deformation. In this work, the single-subject atlas registration strategy is called single atlas.

Fig.12 illustrates the comparison of Dice coefficients and ASD (mean values and standard deviations) between these two methods and the proposed method, based on the 14 contrast-enhanced normal subjects. Statistical tests were performed to evaluate the significance of differences between each of the two methods with the proposed method. For the results with normal distributions, paired student’s t-tests were used, otherwise the nonparametric Wilcoxon signed rank tests were performed. The star sign indicates p value less than 0.001, and the plus sign indicates p value between [0.001, 0.05). For p≥0.05, the results are considered not significant thus no sign is used. It can be seen that the proposed method is significantly more accurate than the “no back-projection” method for most organs, in terms of both Dice coefficient and ASD. This result confirms the effectiveness the 2D-registration-back-projection strategy. However, the proposed method is not significantly better than “no back-projection” for the brain, mainly because of the inaccurate registration of the dorsal head part as revealed by Fig.9. For the spleen, the advantage of the proposed method over the “no back-projection” method is weak for both the Dice coefficient and the ASD. The spleen is difficult to register because it is a small organ with banana-like curved shape, and its position is flexible due to the influence of surrounding organs. Comparing the proposed method with the “single atlas” method, the proposed method is more accurate for most organs (judging from the mean values of Dice and ASD), but the differences are not significant. This is because the proposed method has an additional step of atlas smoothing (equation (3) and (4)), which greatly improves the plausibility of organ shapes but only slightly improves the registration accuracy. To illustrate such a point, an example of the three methods is shown in Fig.13, where the “single atlas” method demonstrates reasonably accurate registration but implausible deformations of the skull, the limbs, the sternum and the lungs. On the contrary, the “no back-projection” method presents plausible but less accurate result.

Fig 12.

Comparison results between the “no back-projection” method, the “single atlas” method and the proposed method, based on the 14 contrast-enhanced normal subjects. For each major organ, the mean values and standard deviations of the three methods are plotted. Statistical tests are performed evaluate the significance of differences between the proposed method and each of the “no back-projection” method and the “single atlas” method. The star sign indicates p value less than 0.001, and the plus sign indicates 0.001≤p<0.05. For p≥0.05, no sign is plotted because the results are considered as not significantly different.

Fig 13.

Visual comparison of the “no back-projection” method, the “single atlas” method and the proposed method. For each method, from the left to the right, the registered atlases are presented as the projection onto the X-ray images and the surface rendering, respectively.

Regarding convergence speed, the “no back-projection” method took 226±54 seconds, and the “single atlas” method took 114±36 seconds. The “no back-projection” method was programmed using IDL, with the atlas projection and Powell optimization steps programmed in C++; The “single atlas” method was programmed using IDL, with the atlas initialization step programmed in C++ and the 2D X-ray registration realized by calling the external elastix software.

4.5. Results of tumor-bearing subjects

The proposed method was tested using 17 subjects with shoulder tumor xenografts. Fig.14 demonstrates six representative results from these 17 subjects. For each subject in Fig.14, the left part shows the X-ray image (with a red arrow pointing out the tumor location), the middle and right parts show the registered SSM^H overlaid on the X-ray and optical images, respectively. It can be seen that despite the presence of the tumor, the registration method still produces reasonable fitting of the organ outlines. Notably, the tumor outlines are not fitted because the atlas is trained from healthy subjects lacking this feature. For all the 17 test subjects, there is only one case demonstrating visually apparent misalignment, as shown by the lower right subject of Fig.14, whose head is inaccurately registered due to severe head twisting caused by the tumor.

Fig 14.

Representative registration results of tumor-bearing subjects. For each subject, the left part shows the X-ray projection image with a red arrow pointing out the tumor location, the middle and right parts show the registered SSM^H overlaid on the X-ray and optical images, respectively. (For interpretation of the references to colors in this figure, the reader is referred to the web version of this article.)

To quantitatively validate the registration accuracy of the tumor-bearing subjects, the Dice coefficient and ASD were computed for the five contrast-enhanced tumor-bearing subjects. Table 1 reports the comparison of the Dice coefficient and ASD between the 14 normal subjects and the five tumor-bearing subjects. It is interesting to see that the tumor-bearing group has larger mean Dice coefficients and smaller mean ASDs than the normal group for most organs, except for the lungs and the spleen. However, it is not sufficient to conclude that the proposed method performs better on tumor-bearing group than on normal group, because the two groups are composed of different subjects. The tumor-bearing group has better registration accuracy mainly because the body weights of the tumor-bearing subjects (22~24g) match well with the majority of the atlas training subjects. Nevertheless, the comparison results at least confirm that the proposed method performs reasonably well on shoulder tumor subjects. Moreover, although the tumor-bearing group shows better accuracy for most organs, it shows slightly worse accuracy for the lungs and much worse accuracy for the spleen. Possible reasons for this are that the presence of shoulder tumor slightly impaired the local accuracy of the thorax registration, and the tumor bearing subjects always have severely enlarged spleen due to immune response.

Table 1.

Comparison of the organ registration accuracy between the normal subjects and tumor-bearing subjects. In each entry the mean value ± standard deviation is reported

		Body	Skeleton	Lungs	Brain	Heart	Liver	Spleen	L-Kidney	R-Kidney
Dice	Normal	0.88±0.03	0.41±0.06	0.63±0.14	0.66±0.13	0.72±0.05	0.69±0.05	0.32±0.10	0.64±0.08	0.74±0.10
	Tumor	0.89±0.04	0.43±0.05	0.62±0.07	0.73±0.09	0.77±0.14	0.71±0.07	0.20±0.18	0.69±0.12	0.75±0.09
ASD (mm)	Normal	0.85±0.15	0.65±0.12	0.54±0.21	0.87±0.28	0.66±0.12	0.67±0.10	0.79±0.13	0.72±0.13	0.56±0.22
	Tumor	0.74±0.13	0.58±0.07	0.56±0.09	0.64±0.14	0.53±0.31	0.66±0.12	1.01±0.26	0.60±0.20	0.52±0.19

5. Discussion and conclusions

Based on the normal test subjects, the registration method demonstrated the capability of capturing posture differences and anatomical variations. As shown in Fig.6(a) and (b), most posture differences in imaging chambers occur due to changing limb orientations and spine curvatures, which are successfully captured by the registration method. Moreover, the body size differences, as shown by Fig.6(c) and Fig.7(a), are also compensated. Note that when the body size changes from thin to fat, the organ anatomy does not undergo simple scaling. The main difference between thin and fat subjects is the amount of subcutaneous and visceral fat, while the size of internal organs remains relatively stable. Such thin-fat differences are captured by the proposed method, due to the use of the statistical mouse atlas. Fig.7(b) shows two subjects with different AP-view spine curve directions. The results show that when the spine curvature changes between left and right, the positions of the abdominal organs (especially the kidneys) don’t change dramatically. As a result, it appears in the X-ray images that the left-curved spine overlaps with the left kidney, and the right-curved spine overlaps with the right kidney. This finding coincides with the principle that the animal body is designed to minimize the influence of external motion onto internal organs. Fig.7(c) illustrates the difference between the stretched and curved bodies. It can be seen that the internal organs move accordingly with the body (seen from the lateral photo), implying that the conditional Gaussian model has the ability to follow the SSM^H deformation.

For a quantitative evaluation, Fig.8 shows that most of the organs have a Dice coefficient over 0.5 and ASD less than 0.8mm. It is worth pointing out that the Dice coefficients in this study are comparable to the results in (Baiker et al., 2010) which registers single-subject atlas with full tomographic micro-CT images. This achievement is due to the use of the statistical mouse atlas, which uses prior anatomical knowledge to compensate for missing information of non-tomographic projections. From Fig.8, it can also be observed that the registration accuracy is organ-dependent. The accuracy of a specific organ is a combined effect of the organ size, shape, contrast and position. Larger organs (e.g. the whole body and the liver) tend to have higher Dice coefficients because larger targets are easier to hit. Sphere-shaped organs (e.g. the heart and the kidneys) have higher Dice coefficients than curve-shaped organs (e.g. the skeleton and the spleen), because a spherical shape is easier to get overlapped than a curved shape. However, a high Dice coefficient doesn’t necessarily mean low ASD, such as the highest Dice organ (the whole body) doesn’t have the lowest ASD, and the lowest ASD organ (the lungs) doesn’t have the highest Dice. A major factor that affects the ASD value is the organ contrast. High contrast organs present clearer edges in the X-ray and optical images, thus they have more accurate surface fitting. It can also be observed that the high contrast organs (the skin/whole body, skeleton and lungs) have more compact distributions of both Dice coefficient and ASD, because these organs are deterministically registered with SSM^H, while the low contrast organs are probabilistically estimated using the CGM. Moreover, if an organ is close to a good-accuracy neighbor, it also tends to have good accuracy. For example, the heart and the liver are close to the lungs, thus they have relatively larger Dice coefficient and lower ASD. Similarly, the right kidney has larger Dice coefficient and lower ASD than the left kidney, because the right kidney is close to the liver, while the left kidney is close to the spleen.

For the comparison with other registration methods, Fig.12 and Fig.13 show that the 2D-registration-back-projection strategy is significantly more accurate than the direct 3D optimization strategy. This is because the former is a bottom-up strategy which finds the 2D edge correspondences first and then back-projects them into 3D, while the latter is a top-down strategy which guesses the 3D correspondence first and then projects it into 2D. It is clear that the bottom-up approach is more effective, because the edge correspondences are easier to be found in 2D. Moreover, the direct 3D optimization strategy takes hundreds of iterations while the 2D- registration-back-projection strategy takes 6.4 iterations on average, therefore the 2D-registration-back-projection also converges faster than the direct 3D optimization. On the other hand, although the 2D-registration-back-projection is more effective, it is susceptible to the errors of 2D registration, making the back-projected deformation discontinuous and unrealistic, as shown by the results of single-atlas registration (Fig. 13). Thanks to the use of the statistical mouse atlas, the back-projected deformation is regulated within the plausible deformation space, and realistic organ shapes are preserved. It should be mentioned that other authors also developed 2D/3D registration methods based on single-subject mouse atlas, and they used different strategies to regulate the unrealistic deformations, such as using elastic energy minimization (Joshi et al., 2010a) or simplifying the TPS-interpolation with a small number of landmarks (Baiker et al., 2009). However, none of these methods are based on the organ deformation model that is learned from the training population.

It is worth noting that in the clinical field, there are also some methods that combine similar 2D-matching-back-projection strategies with statistical atlases for 2D/3D registration of human bones (Baka et al., 2011; Zheng et al., 2009). The difference is that the clinical methods mostly perform edge point searching for the X-ray images, while we use intensity-based registration, because the mouse body presents noisier edges than human bones. The mouse body includes multiple soft organs which produce faint edges in the X-ray image, and even bony structures produce rugged edges because of the complex 3D shapes (like for the skull and the spine). As a result, we find mutual information-based image registration more reliable than edge point searching. Furthermore, unlike clinical studies which only focus on high X-ray contrast organs, our application involves both high and low X-ray contrast organs. The use of CGM facilitates the estimation low contrast organs based on high contrast organs. From Fig.11, it can be seen that when the accuracy of high contrast organs improves along with the registration steps, the accuracy of low contrast organs also improves, implying that the accuracy of low contrast organs depends on the high contrast organs.

For the tumor-bearing subjects, Fig.14 shows that this method produces healthy body shapes as if there is no tumor. This finding allows us to entertain the possibility of tumor detection from X-ray images, i.e. by subtracting the registered body region from the subject body region, and the leftover pixels could become tumor candidates. While tumor detection is out of the scope of this work, this could become an interesting topic of future research. Based on table 1, it can be seen that the implant of a shoulder tumor negatively affected the registration accuracy of the lungs and the spleen, because the presence of the tumor distorted the 2D registration of the thorax, while the tumor also stimulated an immune response leading to abnormally large spleen. Moreover, in this study, we only tested with shoulder xenografts, but didn’t test with visceral or other orthotopic tumors. This is because our data sources come from ongoing preclinical research at our institute where shoulder xenografts are more popular. Overall, we consider this tumor-bearing test as preliminary. In the future, it might be necessary to conduct more comprehensive tumor subject tests which involve animal growing, tumor implanting and subject selection. We may also need to construct a specialized atlas for tumor-bearing anatomy (e.g. enlarged spleen), and use finite element modeling to simulate the tissue distortion caused by shoulder or visceral tumors.

In this paper, the shapes of the body skin and skeleton are modeled via linear point distribution models. However, the articulated motions of the limbs and spine are nonlinear. The proposed method is only effective for subject postures constrained in specialized animal chambers, because the training set covers most common postures inside the chambers like those used in this work. It is unclear how well the registration method performs without such chambers, or with other types of animal holding devices (e.g. test tubes), but it is obvious that this method cannot capture free animal motions. For future study, to broaden the capture range of animal posture, we will consider adding articulated motion into the atlas skeleton, and solve the problem of moving the skin along with the skeleton. The studies of (Gilles et al., 2010; Khmelinskii et al., 2011a; Savinaud et al., 2010) have explored similar problems, and the exiting human and animal animation techniques (Brett et al., 2003; Robert and Jovan, 2004) are also candidate solutions, but to adapt these approaches for 2D/3D registration is still challenging. There are also approaches that cluster the training subjects into sub-groups, and construct linear point distribution models for each sub-group. Since the nonlinearity of each sub-group is much reduced compared to the whole training set, this strategy works reasonably well for articulated motions while maintains the efficiency of linear model. Sub-grouping of the training set can be achieved with either fuzzy k-means (Bowden, 1999) or manifold learning (Kadoury and Paragios, 2010) methods. However, the prerequisite of the sub-grouping strategy is a large enough training size which covers most possible postures. Apparently the 83 training subjects of this paper is not sufficient to cover all free postures, but increasing the straining size will lead to nontrivial time and expense costs.

To describe the inter-subject deformations of multiple organs in the atlas, we constructed the SSMs by treating multiple organs as a single object. This strategy effectively avoids the intersection of nearby organs, but potentially limits the deformation ability of each single organ. In future studies, we may implement other existing multi-organ SSM approaches (Cerrolaza et al., 2012; Gorczowski et al., 2007; Okada et al., 2008; Zhang et al., 2011). The principal nested sphere method (Jung et al., 2010) could also be an alternative of the PCA to give better modeling of the organ deformations. Last but not least, we also ought to consider building different atlases for different weights, strains, and sex, to see if more dedicated atlases can future improve the registration accuracy.

Overall, in this paper, a 2D/3D mouse atlas registration method is presented for preclinical mouse studies. The registration is based on AP-view X-ray projection and lateral-view optical photo. The combined use of the 2D-registration-back-projection strategy and the statistical mouse atlas yield promising registration results for both normal subjects and shoulder tumor-bearing subjects. This method could compensate for moderate posture changes and inter-subject anatomical variations, and it registers both high and low X-ray contrast organs by combining the statistical shape model with the conditional Gaussian model. Using two projection views is a low cost, low radiation dose option and our automated processing means this method is suitable for routine use with everyday imaging study. Future study will focus on improving the atlas construction method and conducting more comprehensive tumor-bearing subject tests.

Research highlights.

• Registration of a statistical mouse atlas with an X-ray projection and a lateral optical photo.

• A statistical mouse atlas for posture and anatomical variations.

• A 2D-registration-back-projection strategy for efficient and accurate atlas deformation.

• Compares favorably to two existing methods, and is robust even with the presence of shoulder tumor.