Feasibility of automated pancreas segmentation based on dynamic MRI

Abstract

Objective:

MRI-guided radiotherapy is particularly attractive for abdominal targets with low CT contrast. To fully utilize this modality for pancreas tracking, automated segmentation tools are needed. A hybrid gradient, region growth and shape constraint (hGReS) method to segment two-dimensional (2D) upper abdominal dynamic MRI (dMRI) is developed for this purpose.

Methods:

2D coronal dynamic MR images of two healthy volunteers were acquired with a frame rate of 5 frames per second. The regions of interest (ROIs) included the liver, pancreas and stomach. The first frame was used as the source where the centres of the ROIs were manually annotated. These centre locations were propagated to the next dMRI frame. Four-neighborhood region transfer growth was performed from these initial seeds before refinement using shape constraints. Results from hGReS and two other automated segmentation methods using integrated edge detection and region growth (IER) and level set, respectively, were compared with manual contours using Dice's index (DI).

Results:

For the first patient, the hGReS resulted in the organ segmentation accuracy as a measure by the DI (0.77) for the pancreas, superior to the level set method (0.72) and IER (0.71). The hGReS was shown to be reproducible on the second subject, achieving a DI of 0.82, 0.92 and 0.93 for the pancreas, stomach and liver, respectively. Motion trajectories derived from the hGReS were highly correlated to respiratory motion.

Conclusion:

We have shown the feasibility of automated segmentation of the pancreas anatomy on dMRI.

Advances in knowledge:

Using the hybrid method improves segmentation robustness of low-contrast images.

Patients with pancreatic adenocarcinoma have a poor prognosis with cumulative 5-year survival <5%.^1,2 Many patients present with unresectable locally advanced lesion at the time of diagnosis. Although radiotherapy alone is unlikely to cure pancreatic cancer, with sufficiently high doses, it is possible to achieve local control or resectability conversion^3–8 that is correlated to significantly prolonged patient survival.⁹ However, radiation doses to the pancreas are limited by the surrounding radiosensitive serial organs. The goals to deliver sufficient tumour dose are further complicated by significant organ motion in this region.^10–12 Currently, large motion margins to sufficiently cover the tumour motion excursion in the planning target volume are used, resulting in increasing high dose spillage to surrounding critical structures. Consequently, the doses needed for local tumour control are not typically achievable.¹³

A pre-requisite for overcoming the challenge and developing a motion management method for pancreatic cancer treatment is quantitative imaging. Four-dimensional CT (4DCT), which has been successfully used in lung tumour motion management, is not commonly performed in the upper abdominal region owing to its intrinsic poor soft-tissue contrast and, if imaging contrast is administrated, the difficulty to synchronize pharmacokinetics of imaging contrast and 4DCT acquisition.¹⁴ Furthermore, 4DCT is either unavailable during treatment or available with further degraded qualities so is less useful for the upper abdominal region. Alternatively, dynamic MRI (dMRI) has been employed to describe the tumour motion and has shown complementary features to the CT-based images. In addition to superior soft-tissue contrast, dMRI has the flexibility to image in the orientations most relevant to motion and for a prolonged duration without ionizing radiation. These abilities have been utilized to provide a statistically robust characterization of the lung tumour.^15–18 The superior soft-tissue contrast in the upper abdomen, and its non-ionizing nature makes MR a better suited modality to provide continuous imaging guidance for intrafractional image-guided pancreas treatment. The feasibility of utilizing MR to guide radiotherapy has been enhanced by the advent of MR-guided linacs and Co-60 systems.^19–21 With MRI-guided radiotherapy, gated pancreas treatment can be performed at an anatomical position that maximizes critical organ sparing when there is a greater separation between the tumour and the dose-limiting critical organs. To do so, the locations of the pancreas and its nearby organs need to be known based on their respective contours but an automated tool for such purpose has not been developed. Without such a tool, tedious manual contouring is required on the large number of time-resolved dMRI images. This deficiency would severely limit the utilization of MRI guidance for pancreatic motion management.

In this study, we develop a hybrid gradient, region growth and shape-constraint (hGReS) imaging segmentation method based on seed transfer and organ cosegmentation for this challenging problem.

METHODS AND MATERIALS

MRI acquisition

Thoracic-abdomen dMRI images of two healthy volunteers were used for the study. For the first healthy volunteer, two-dimensional (2D) coronal dMRI images were performed on a Siemens 1.5 T Avanto scanner (Siemens Medical Solutions, Erlangen, Germany). The MRI was performed utilizing a fully sampled Cartesian TrueFISP (TFI) sequence (5 images per second, sagittal and coronal orientations) with an 18-channel body receiver coil. Data were acquired with the scan parameters: repetition time (TR)/echo time (TE), 2.90/1.04 ms; field of view (FOV), 300 × 360 mm²; flip angle, 52°; slice thickness, 7 mm; matrix dimension, 160 × 192 pixels. The position of the coronal slices was selected to show the pancreas head, stomach and liver. 300 continuous imaging frames were obtained and analysed.

To test if the segmentation method was image set dependent, a second healthy volunteer was imaged using a modified TFI sequence based on radial trajectory using the following scan parameters: TR/TE, 2.9/1.5 ms; FOV, 360 × 360 mm²; flip angle, 41°; slice thickness, 6 mm; matrix dimension, 192 × 192 pixels. Three coronal planes each with 300 continuous frames were obtained for this subject.

Segmentation methods

Figure 1 shows a flowchart of the automated segmentation routine. In the source task, imaging priors was extracted from the source frame (Frame 0), which was the first frame of the dMRI images but could be generalized to use images from a different sequence of the same subject to save time in the case of MRI-guided radiotherapy with a priori MRI. Seeds were manually placed near the centre of the regions of interest (ROIs), which included the stomach, left liver lobe and pancreas head. This step took <1 min. In the target task, the seed locations and their ROI association were transferred to the second dMRI frame. Automated segmentation was performed using hybrid gradient, region growth and shape constraints (hGReS). New seed locations, which were the centres-of-mass of the automated contours, were be transferred to the next imaging frame. Seed propagation and ROI segmentation were repeated until all imaging frames were segmented. The details of hGReS are as follows.

Figure 1.

Flowchart of the proposed hybrid gradient, region growth and shape constraint (hGReS) method. ROIs, regions of interest.

Source task: obtain a priori information

Initial seeds were placed near the centre of each ROI by a human operator. The co-ordinates of the three points are denoted by (p_i, q_i), i = 1, 2, 3 for the liver, stomach and pancreas, respectively, as shown in Figure 2. A highly conspicuous landmark in the lung was selected to establish a respiratory trace. The seed locations were automatically transferred to the target images by projection to initiate automated segmentation.

Figure 2.

Initial seed location as denoted by number 1 (left liver lobe), 2 (stomach) and 3 (pancreas) circles. A blood vessel in the lung with high conspicuity denoted by number 4 was selected to create a respiratory motion reference.

Target task

The target images were first pre-processed to calculate their associated gradient and edge matrices. The gradient matrix of the kth dMRI frame was defined by:

$$G_k\left(p,q\right)=\sqrt{{\left[I\left(p+1,q\right)-I\left(p,q\right)\right]}^2+{\left[I\left(p,q+1\right)-I\left(p,q\right)\right]}^2}$$ (1)

I(p, q) is the grey value of the pixel (p, q) of an image.

The Canny operator²² was used to obtain the edge matrix E_k.

Seed transfer

From the source domain, we obtained probable scopes of the three ROIs and then built three template matrices, denoted by Y₁(p_i, q_i), Y₂(p_i, q_i) and Y₃(p_i, q_i). Initial seeds I(p_i, q_i) were projected to the target image as the rough estimate of the new seed locations. For each seed, the average of its 3 × 3 neighbourhood was denoted by “ave”. Additionally, the variance of its 5 × 5 neighbourhood was denoted by “var”. The size of the neighbourhood was empirically determined after comparing the performance of smaller and larger patches.

Cosegmentation of regions of interests

Cosegmentation of ROI included the following three steps.

(i) Four-neighbourhood region transfer growth

The seed pool, initially only consisting the seeds projected from the source frame, grew by adding the four-neighbourhood pixels U(x_i, y_i) of the seeds I(p_i, q_i):

$$\text{if}\left|U\left(x_i,y_i\right)-\text{ave}\right|\le \text{var}\hspace{0.17em}\hspace{0.17em}$$ (2)

The average grey value of the seed pool was subsequently updated by:

$${\text{ave}}_{\text{new}}=\frac{1}{m+s}\left[\text{ave}\times m+{\displaystyle \sum }_{i=1}^{s}I(p_i,q_i)\right]$$ (3)

where m is the maximum pixel number template matrix Y_i, i = 1, 2, 3, and s is the number of seeds. The seed pool grew until all pixels in the template Y_i were evaluated.

(ii) Organ extraction

To properly assign the remaining regions after the four-neighbourhood region transfer growth, specific segmenting operations were performed on the three ROIs based on their individual morphological and imaging characteristics as follows.

Because of the relatively sharp liver boundary, gradient matrix condition was used to determine if a point (p_i, q_i) belonged to the liver seed pool. The gradient matrix condition was defined as:

$$\left|G\left(p_i,q_i\right)\right|\le T,\hspace{0.17em}T=52$$ (4)

(iii) Shape constraint

A shape constraint was imposed on the stomach owing to its elliptical representation in the MRI, for all pixels in the ambiguous area if:

$$\sqrt{{(p_i-p_j)}^2+{(q_i-q_j)}^2}\le r,\hspace{0.17em}r=13$$ (5)

T and r were empirically selected. Additionally, an edge matrix constraint was used for both the stomach and the pancreas that:

$$E\left(p_i,q_i\right)=0$$ (6)

The preliminary segmentation results after these operations are shown in Figure 3a,b. Figure 3b shows merging contours and ambiguous pixels that could belong to either the stomach or pancreas after region growth and gradient operation, but the ambiguity was substantially reduced by applying the stomach shape and size constraint.

Figure 3.

First-stage segmentation results. (a) Seed transfer growth result of three organs. (b) Transfer growth results with shared pixels of the liver (R1), stomach (R2) and pancreas (R3) from adjacent imaging frames.

Propagation of segmentation to subsequent imaging frames

The centre-of-mass locations of the three ROIs were projected to the next imaging frame as the initial seed locations and Step (ii) was repeated until all images are processed and segmented.

Comparing methods

Our method was compared against two state-of-the-art segmentation methods. The first one is an integrated edge detection and region growth (IER) algorithm^23,24 that has been shown superior to either single method. The second method is level set that has gained wide utilization for segmentation problems.²⁵

In IER, the Otsu method²⁶ using the Canny operator was applied with the lower and upper thresholds of 0.03 and 0.15, respectively. In the region growth algorithm, the original seed points were manually placed in the same way as the hGReS. The growth terminated when there was no more point with the grey scale value within the variance of the seed points. The variance thresholds var were set as 20, 25 and 28 for the liver, pancreas and stomach, respectively. Seed transfer from the initial frame to subsequent time frames was performed similar to the hGReS.

The level set method was first developed by Osher and Fedkiw²⁷ and Sethian²⁸ to describe wave propagation. The method was then applied for medical imaging processing²⁹ and has evolved to be one of the most important tools for imaging segmentation.³⁰ The level set evolution is derived as the gradient flow that minimizes energy functional with a distance regularization term and an external energy that drives the motion of the zero level set towards desired locations. In this study, initial contours were roughly manually drawn on the first imaging frames and propagated to subsequent time frames. The detail of the distance regularized level set method can be found in the study by Li et al.³⁰ Briefly, segmentation was performed based on a distance regularized level set energy function:

$$\epsilon \left(\mathrm{\varnothing }\right)=\mu {\mathrm{ℛ}}_{\text{p}}\left(\mathrm{\varnothing }\right)+\lambda {\mathrm{ℒ}}_{\text{g}}\left(\mathrm{\varnothing }\right)+\alpha {\mathcal{A}}_{\text{g}}\left(\mathrm{\varnothing }\right)$$ (7)

where ${\mathrm{ℛ}}_{\text{p}}\left(\mathrm{\varnothing }\right)$ is the level set regularization term, ${\mathrm{ℒ}}_{\text{g}}\left(\mathrm{\varnothing }\right)$ computes the line integral of the function along the zero-level contour, and ${\mathcal{A}}_{\text{g}}\left(\mathrm{\varnothing }\right)$ is introduced to speed up the motion of the zero level contour in the level set evolution process. We adopted the same parameter values for µ(0.2), λ(5) from the original publication, but the α value given by Li et al³⁰ resulted in poor segmentation performance. In this study, we set α as −2, 2.2 and −5, and the number as 40, 100 and 8 for the liver, stomach and pancreas, respectively. All algorithms were implemented using in-house programs implemented in MATLAB® (R2007; MathWorks®, Natick, MA). In addition to the automated segmentation methods, manual segmentation of the three ROIs was performed by an oncologist. The manual contour served as a common sense reference.

Validation of the segmentation

The segmentation results were evaluated both visually and quantitatively.

To quantitatively analyse the segmentation performance, Dice's similarity index was used:

$$S=2·\frac{\left|A_1\cap A_2\right|}{\left|A_1\right|\hspace{0.17em}+\hspace{0.17em}\left|A_2\right|}$$ (8)

where A₁ and A₂ were the binary masks from automated and manual segmentation, respectively.

The second quantitative validation was performed by calculating the motion trajectories of the ROI centroids, which were assumed to be highly correlative to the respiratory motion. To quantify the correlation, a highly conspicuous blood vessel in the lung was selected as the reference (Figure 2). The blood vessel was tracked using maximal cross-correlation. Details of lung blood vessel motion tracking can be found in previous publications.^15–18

The third quantification is the proximity between ROIs calculated as the average gap distance between two colour-coded line segments in adjacent ROIs that were closest to each other as shown in Figure 4. The 2-cm-long line segments consisted 11 pixel-pairs that were closest to each other. Different from gating based on a fixed breathing phase, the gap distance is useful for gating based on greatest separation between the tumour and the critical normal organs.

Figure 4.

The locations where two organs are closest to each other are marked by colour-coded line segments. The gap distance is defined as the average distance between these line pairs with matching colour.

RESULTS

Segmentation performance

Figure 5 shows the segmentation results for a select imaging frame. The hGReS results agreed well with our visual inspection despite the low conspicuity of organ boundaries. IER was oversensitive to the growth threshold and tended to over- or undergrow. The level set was better at separating the stomach and pancreas but showed slightly less accurate definition of the boundary.

Figure 5.

Automated segmentation results for a typical imaging frame. (a) Integrated edge detection and region growth; (b) level set; (c) hybrid gradient, region growth and shape constraint; (d) manual contour.

Comparison of different image segmentation methods

Figure 6 shows the comparison of IER, level set, hGReS and manual contouring methods for every 45th frame of the dMRI sequence. Clearly, the IER method was not robust to consistently separate the pancreas and stomach. The level set method resulted in more robust segmentation than did the first two methods but failed to consistently separate the stomach and the pancreas owing to under- and oversegmentation.

Figure 6.

Segmentation result of the first subject. Five imaging frames were selected with organ contours shown in binary masks. hGReS, hybrid gradient, region growth and shape constraint; IER, integrated edge detection and region growth.

The segmentation quality was quantified for the three methods using the Dice's index and referencing to the manual contours. The result is shown in Figure 7. All methods resulted in acceptable segmentation of the liver and stomach (>0.84), but hGReS resulted in consistently more accurate segmentation for all three organs (Table 1).

Figure 7.

Dice's index (DI) comparison for the three regions of interests over the 300 dynamic MRI frames. (a) Pancreas, (b) stomach, (c) liver. hGReS, hybrid gradient, region growth and shape constraint; IER, integrated edge detection and region growth.

Table 1.

Average Dice's coefficient for 300 frames

Segmentation methods	Pancreas	Stomach	Liver
Integrated edge detection and region growth	0.7093 ± 0.0131	0.8524 ± 0.0061	0.8474 ± 0.0009
Level set	0.7210 ± 0.0036	0.9051 ± 0.0008	0.8485 ± 0.0040
Hybrid gradient, region growth and shape constraint	0.7701 ± 0.0030	0.9026 ± 0.0005	0.9278 ± 0.0040

Region of interest motion trajectory tracking

Figure 8 shows the automatically tracked centroid motion trajectories of three ROIs and the reference point in the lung. All three methods were able to produce regular breathing motion profiles consistent with reference breathing motion. The abdominal organ motion was found to be well correlated to the respiratory motion as shown in Table 2 with >0.90 correlation coefficient between the upper abdominal organs and the lung blood vessel when manual contouring was used. Automated segmentation using the hGReS and level set methods produced a similar level of correlation. IER resulted in inferior correlation between the pancreas and the lung motion owing to increased inconsistency and noisy centroid tracking results.

Figure 8.

Estimated centroid motion trajectories on two-dimensional MRI of individual frames using (a) integrated edge detection and region growth, (b) level set, (c) hybrid gradient, region growth and shape constraint, and (d) manual contour. Horizontal axis is MRI frame number, and vertical axis is computed based on the $\text{centroid}\text{displacement}=\sqrt{x^2+y^2}$ with the unit (millimetres). Respiratory trace is superimposed on the plots.

Table 2.

Correlation coefficients of three organs and lung reference for four segmentation methods

Segmentation methods	Liver–lung reference	Stomach–lung reference	Pancreas–lung reference
Integrated edge detection and region growth	0.8981	0.9250	0.8119
Level set	0.8389	0.9350	0.9218
Hybrid gradient, region growth and shape constraint	0.9163	0.9261	0.8906
Manual contour	0.9379	0.9201	0.9372

Boundary-based evaluation

The gap distances between ROIs are shown in Figure 9. Both the hGReS and level set methods produced regularly varying gap distances between the liver and pancreas correlated to the breathing cycles, but only the hGReS method was able to discern the varying gap distances between the pancreas and stomach. IER was not sensitive to the gap distance changes. The varying gap distances were less clearly captured by the human observer but subsequently validated after viewing the automated segmentation results. For example, in the fifth row of Figure 6, the pancreas touched the stomach, resulting in a zero-millimetre gap distance that was reflected in the automated segmentation, but the human operator kept a gap in between the first pass.

Figure 9.

Gap distances between region of interest boundaries vs imaging frame for five segmentation methods using (a) integrated edge detection and region growth, (b) level set, (c) hybrid gradient, region growth and shape constraint and (d) manual contour. Respiratory signal is overlaid on the distance.

Robustness test on the second healthy volunteer

The result of applying hGReS on the second subject is shown in Figure 10. dMRI of the second subject showed both higher contrast and noise. The images were first denoised using a three-dimensional (3D) non-local means filter.³¹ Robust segmentation hGReS results were observed for all imaging frames with small deviation compared with the manual contouring results. The average Dice's index of the automated and manual contouring results for the pancreas, stomach and liver are 0.82, 0.92 and 0.93, all slightly superior to the first subject.

Figure 10.

Hybrid gradient, region growth and shape constraint (hGReS) and manual segmentation comparison for Subject 2. The third and fourth columns show contours superimposed on the denoised images.

The second imaging planes of this subject showed the scenario where the pancreas was intersected by the plane twice owing to folding. This resulted in two parts for the same organ in the same imaging plane as shown in Figure 11. hGReS was able to consistently segment the entire organ presented in this plane unaffected by this separation.

Figure 11.

(a) A frame of dynamic MRI shows folded pancreas intercepted by the imaging plane twice. (b) Segmentation results superimposed on the denoised image.

Computational time

All calculation was performed on a personal computer with an i3 core central processing unit and 2-GB memory. Table 3 shows the total calculation time and the time used to segment each frame. Edge detection, region growth and hGReS were significantly faster than the level set method.

Table 3.

Time used for segmentation

Segmentation methods	Total (s)	Per frame (s)
Integrated edge detection and region growth	533.62	1.8
Level set	9944.28	33.0
Hybrid gradient, region growth and shape constraint	483.36	1.6

DISCUSSION

Geometrical accuracy in radiotherapy is paramount to the quality of treatment. Extensive research has been performed to characterize the internal organ motion. Passive and active motion adaptive treatment protocols, such as internal target volume, gating^32,33 and four-dimensional (4D) treatment,^34–36 have been developed to mitigate the adverse effects of organ motion.

Among tumours exhibiting significant intrafractional motion, lung tumours have been most thoroughly studied both because of their significant internal motion driven by the breathing motion and high CT contrast that allows for CT-based motion modelling and tracking. An intrinsic challenge in both the gated and 4D radiotherapy is to determine the precise tumour location during treatment. For lung tumours, it has been reported that the gating position can drift significantly to outside the gating window despite using the same respiratory phase and verification of the tumour position at the beginning of treatment using cone beam CT.^15,37 The uncertainties and tightened geometrical margin in the gated radiotherapy may have reduced local tumour control rates in some studies.³⁸ The same problem is expected to exist in pancreatic tumour treatment without a reliable way to determine the target location during treatment. Compared with lung cancer therapy, the ability to continuously monitor the tumour location during treatment for gated pancreas radiotherapy was technically limited by the low soft-tissue X-ray contrast despite its great clinical relevance.

MRI-guided radiotherapy provides the ability to image soft tissues during treatment, but these images need to be segmented. Automated segmentation of the pancreas and surrounding anatomies is challenging as evidenced by the scarcity in related literature. In the limited number of reports based on either CT with contrast or MRI, significantly poorer accuracy of pancreas segmentation was observed than with other abdominal organs.^39,40 We have used a method combining region growth, gradient and shape constraints to more robustly segment these organs as evidenced by consistent and high similarity to manual contours.

In this study, we modified an IER method by imposing additional shape constraints. As a result, the new segmentation method, which we termed hGReS, outperformed both IER and a state-of-the-art level set method. hGReS is particularly more sensitive to the gap distance between organs than the other two methods. The new method is computationally efficient, making it suitable for online imaging segmentation.

Varying gap distances between ROIs from automated segmentation were found and correlated to respiratory motion. The variation may be exploited in maximizing the dosimetric gains in gated radiotherapy. Instead of gating at a fixed respiratory phase, the dose should be delivered when there is a maximal separation between the pancreas and the surrounding critical organs. Interestingly, the distance variation was less clearly reflected by the manual contours. The discrepancy may attribute to that the human operator had a tendency to maintain the contour copied from previous imaging frames when the motion is perceived to be small. To better understand the phenomenon, the situation where the ground truth is known, such as phantom studies, is needed.

The study has several limitations. The main limitation of this study is that the segmentation method was tested on a small number of healthy volunteers. Although the limitation is compensated to a certain degree by the large number of imaging frames for the same subject, the pancreas and its surrounding anatomies are highly complex and variable between subjects. Post-operative pancreas can exhibit substantially different imaging characteristics that may require modification of the segmentation parameters. This study shows the feasibility of segmenting intact pancreas, with potential applicability to pre-operative patients.

Another limitation is the shape constraints that facilitate the segmentation of abutting stomach and pancreas with little contrast. The highly engineered approach may be inapplicable to an empty/collapsed stomach. To solve the problem, manifold clustering⁴¹ can be introduced as a more general approach for anatomies substantially different from these healthy volunteers including empty/collapsed stomachs. The third and comparably minor limitation is the manual annotation step required to initialize automated segmentation. On the other hand, this step does not require high precision and should take very little time to accomplish owing to the small number of organs involved in pancreas treatment.

It will be interesting to test the feasibility of 3D MRI segmentation. Currently, 3D MRI is not dynamic owing to limited acquisition speed but multiple breath-hold images should enable testing of some of the algorithms. The changes of organ shapes between MRI slices are more drastic than the changes between sequential 2D dMRI frames and may pose a challenge to the seed transfer algorithm. However, the organ continuity in the 3D space offers additional information to assist segmentation.

CONCLUSION

We showed the feasibility of employing a hGReS method to segment pancreas and surrounding normal organs of two healthy volunteers based on dMRI. This method resulted in superior segmentation accuracy than do edge detection and region growth and was more robust and computationally efficient than the level set segmentation.

FUNDING

This study was funded in part by the National Institutes of Health grant #R21CA161670.