Fast internal marker tracking algorithm for onboard MV and kV imaging systems

Abstract

Intrafraction organ motion can limit the advantage of highly conformal dose techniques such as intensity modulated radiation therapy (IMRT) due to target position uncertainty. To ensure high accuracy in beam targeting, real-time knowledge of the target location is highly desired throughout the beam delivery process. This knowledge can be gained through imaging of internally implanted radio-opaque markers with fluoroscopic or electronic portal imaging devices (EPID). In the case of MV based images, marker detection can be problematic due to the significantly lower contrast between different materials in comparison to their kV-based counterparts. This work presents a fully automated algorithm capable of detecting implanted metallic markers in both kV and MV images with high consistency. Using prior CT information, the algorithm predefines the volumetric search space without manual region-of-interest (ROI) selection by the user. Depending on the template selected, both spherical and cylindrical markers can be detected. Multiple markers can be simultaneously tracked without indexing confusion. Phantom studies show detection success rates of 100% for both kV and MV image data. In addition, application of the algorithm to real patient image data results in successful detection of all implanted markers for MV images. Near real-time operational speeds of ∼10 frames∕sec for the detection of five markers in a 1024×768 image are accomplished using an ordinary PC workstation.

INTRODUCTION

Modern conformal radiation therapy techniques, such as intensity-modulated radiation therapy (IMRT), can provide radiation doses that closely conform to the tumor dimensions while sparing sensitive structures.^{1, 2} To be optimally effective, these techniques require a high geometric precision in both tumor localization and patient treatment setup. The presence of inter and intrafraction organ motion uncertainties can therefore reduce the benefit of using a highly conformal radiotherapy technique. For instance, intrafractional respiratory or prostate based tumor motion can lead to tumor displacements up to 2–3 cm over the course of routine radiotherapy.^{3, 4, 5, 6, 7, 8, 9} The use of image-guided radiation therapy (IGRT) is a promising candidate to ensure proper targeting in radiation treatment deliveries.¹⁰ Due to the dynamical nature of human anatomy, it is most advantageous when IGRT can be performed in real-time in order to ensure an accurate delivery of the planned conformal dose distribution.¹¹

Several methods of obtaining real-time tumor position are available, and these can be categorized as being either indirect (external surrogate-based) or direct (fiducial∕image) in nature. In general, indirect tumor location methods, such as external skin marker tracking or breath monitoring techniques, rely on the correlation between external body parameters and the tumor.^{5, 12} In reality, the relationship between external parameters and internal organ motion is complex and a large uncertainty may be present in predicting the tumor location based on external signals. A direct tumor position measurement is therefore highly desirable for therapeutic guidance. In the last decade, a number of direct real-time 3D tumor tracking methods have been implemented, primarily using fluoroscopy^{5, 11, 13} or magnetic field localization.¹⁴ In addition, the feasibility of using an electronic portal imaging device (EPID) and stereoscopic x-ray imaging for tumor tracking has been explored.^{3, 5, 6, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26}

To be clinically useful, an internal marker tracking algorithm should reliably segment markers from varying complex anatomic image backgrounds, be able to track multiple markers simultaneously without indexing confusion, and be able to operate at near real-time speeds. Generally, marker segmentation algorithms based on pixel intensity tend to fail when markers are in the vicinity of high contrast structures such as bone. A more reliable solution is the use of template matching, as demonstrated by Shirato et al., in the tracking of a spherical gold marker using multiple kV fluoroscopic imaging systems.²⁵ Tang et al. further extended the template matching technique to include detection of cylindrically shaped markers on an in-house built stereoscopic kV imaging system.²⁷ In their algorithm, the user first manually located and defined a region of interest (ROI) around each marker and determined the orientation of every cylindrical marker, and then a template matching algorithm was applied to the ROIs. The normalized cross correlation between the template and correlate pixels was calculated for every pixel and the highest cross correlation yielded the marker location. The detection failure rate for their method could be up to 12%.

As of yet, few works have presented algorithms suitable for tracking internal markers reliably using MV image data.¹⁸ This is a consequence of the low inherent contrast between different materials in MV based images, making image based marker detection difficult.

This work proposes a novel pattern matching algorithm specifically designed to work with both kV and MV imaging systems. Unlike previous algorithms reported by Shiratoet al. and Tang et al. where only the cross correlation is used for marker detection, this algorithm employs a criteria system based not on only the correlation score, but also the scaling factor and their combination. This is found to allow 100% successful marker detection rates, even on low contrast MV images. This algorithm is highly desirable since it can combine MV imaging with kV fluoroscopy imaging to locate real-time 3D tumor position during the actual radiotherapy process, as recently demonstrated by Wiersmaet al.²⁸ Unlike other tracking techniques, which require two or more kV sources for 3D marker positioning,^{3, 5, 6, 25, 26, 27, 29} this technique has the inherent benefit in that only one kV source is required for full 3D marker geometric information since the actual MV treatment beam is also used for positioning.

MATERIALS AND METHOD

Hardware setup

A Varian Trilogy (Varian Medical System, Palo Alto, CA) operating in the 6 MV photon mode was used for the study. Images of the MV beam were acquired by an aSi EPID (PortaVision aS-500, Varian Medical System, Palo Alto, CA) attached to the LINAC, as shown in Fig. 1. The kV imaging was accomplished using the onboard imaging system located perpendicular to the treatment beam (Fig. 1). The kV imaging system consisted of a 125 kV x-ray tube together with an aSi flat panel imager (PaxScan 4030CB, Varian Medical Systems, Salt Lake City, UT). Pixel sizes of the kV and MV detectors were 0.388 mm and 0.392 mm, respectively. Both detectors had a resolution of 1024×768, corresponding to an effective area of detection of approximately 40 cm×30 cm. The source-to-axis distances (SAD) were set to 100 cm and source-to-imager distances (SID) were set to 150 cm for both MV and kV systems (Fig. 1).

Figure 1.

Varian Trilogy with kV and MV imagers in extended positions. The system’s frame of reference is denoted by arrows. A head phantom is located on the couch.

The markers used were either stainless steel ball bearings (BB) or gold (Au) cylinders. The BB diameters varied from 1.57 mm to 4 mm, whereas the Au cylinders were 1.2 mm in diameter and 5 mm in length (North West Medical Physics Equipment, MED-TEC Company, Orange City, IA). Efficiency of the algorithm in segmenting markers from complex anatomical image background was tested using a head phantom (Fig. 1). The cylindrical fiducials were internally embedded into the phantom, whereas BBs were mounted externally on the phantom.

CT based region-of-interest definition

Prior planning CT knowledge can provide valuable marker information that can be used to reduce the marker search space. Reduction of search space in turn removes unnecessary image processing resulting in increased fiducial detection speeds. As displayed by the flow diagram in Fig. 2, a simple intensity based search for the markers was initially done on the planning CT. Due to the large CT numbers of the metallic markers relative to other anatomical structures, the markers were easily segmented from the image background. The displacement vector relating the CT isocenter to the treatment isocenter was then used to transform the marker CT coordinates to that of the treatment isocenter such that each marker was given a 3D position (x_B,y_B,z_B) relative to the machine isocenter. The expected projection location (u,v) of each marker on either the kV detector or the EPID can then be predicted by the following relationships

$$u=F\frac{\cos \left(\varphi \right)x_B+\sin \left(\varphi \right)y_B}{R-\sin \left(\varphi \right)x_B+\cos \left(\varphi \right)y_B},$$ (1)

$$v=F\frac{z_B}{R-\sin \left(\varphi \right)x_B+\cos \left(\varphi \right)y_B},$$ (2)

where ϕ was the gantry angle, R was SAD, and F was SID. As seen in Fig. 1, the x-axis was in the lateral direction of patient couch, the y-axis was in the anterior-posterior direction, the z-axis was the superior-interior direction, and the origin was the LINAC’s isocenter. The coordinates of the imagers were defined in the (u,v) plane in Fig. 1.

Figure 2.

(a) Flow chart algorithm computational path. (b) To the right is shown the eight-bin orientation segmentation filter, (c) spherical pattern, and (d) a cross section pattern for a vertical cylindrical marker.

Having located the expected positions of the marker projections, the search region was reduced to a small circular ROI around each marker projection center. Typically, a ROI with a radius of 75 pixels on the imager, or about 2 cm around the fiducial position, was found to be adequate in locating the marker. For markers located near each other, overlapping ROIs would be grouped together. The fiducial search was then performed group by group in order to avoid redundancy.

Marker orientation

With a cylindrical marker of fixed length (l) and width (w), the unsymmetrical shape could lead to a host of different possible projection images depending on the marker’s particular orientation relative to the source∕imager setup. Parameters subject to variation were the marker’s projection length and orientation (from 0°–180°). The marker projection length might vary from “1.5^*w” (if the projection direction was along its longitudinal direction) to “1.5^*1” (if the projection direction was perpendicular to its longitudinal direction) with an amplification factor of 1.5 (=SID∕SAD). To take into account the different possible orientations, the 180° rotation was divided into a number of bins, as shown in Fig. 2b. As seen in the figure, each of the eight bins represented two possible opposite directions, such that each bin covered an angle of 22.5°. The center of this orientation filter was placed on each pixel in a ROI group, and all adjacent pixels from the center pixel were then grouped into eight bins corresponding to their particular angle. The average intensity of each bin was computed and the bin with the highest number was taken as the orientation of the center pixel. It should be noted that the orientation results carried no information for most of the pixels and it was only valid for pixels on a cylindrical object. The eight-bin pattern was chosen based on the limited pixel resolution of a projected marker. Due to the small marker size, a detected projection was composed of a limited number of pixels and therefore mosaic in nature (typically with a width of ∼5 pixels and a length of ∼20 pixels or less). Therefore, for small angles of rotation, only a few pixels would change. Using bin numbers greater than eight generally did not lead to a more accurate angle determination since now the pixel noise could be the deciding factor in a particular angle bin.

Pattern matching algorithm

Different patterns were used for spherical and cylindrical markers. Due to rotational symmetry, a simple spherical marker pattern [Fig. 2c] was universally used for all BBs with the same physical diameter. For cylindrically shaped markers, a trapezoidal pattern [Fig. 2d] was used. The rationale behind this pattern was that even though a cylindrical marker projection might undergo a wide variety of rotational and length changes, the signal intensity distribution along the cross section of the cylinder projection remained constant. This cross section had a unique trapezoidlike pattern that was experimentally found to be dependent on the marker’s width, but independent of projection length. Searching for these unique cross-sectional patterns along the orientation of every pixel in each ROI allowed for identification of fiducials.

Depending on the sought-after marker, either a spherical or cylindrical cross-sectional pattern {p_i,j} is used for the matching procedure. At each pixel location (x,y) within a ROI group, a comparison was made between the pattern and the surrounding pixels ({f_x+i,y+j}). Two basic criteria were calculated: the square of the correlation coefficient $R_{x,y}^2$ and the scaling factor H_x,y

$$R_{x,y}^2=\frac{{\sum }_{(i,j)∊\mathrm{Pattern}}(f_{x+i,y+j}-\overline{f_{x,y}})(p_{i,j}-\overline{p})}{\sqrt{{\sum }_{(i,j)∊\mathrm{Pattern}}{(f_{x+i,y+j}-\overline{f_{x,y}})}^2}\cdot \sqrt{{\sum }_{(i,j)∊\mathrm{Pattern}}{(p_{i,j}-\overline{p})}^2}},$$ (3)

$$H_{x,y}=\frac{{\sum }_{(i,j)∊\mathrm{Pattern}}(f_{x+i,y+j}-\overline{f_{x,y}})(p_{i,j}-\overline{p})}{{\sum }_{(i,j)∊\mathrm{Pattern}}{(p_{i,j}-\overline{p})}^2},$$ (4)

where $\overline{f_{x,y}}$ was the average intensity of the pattern region around pixel (x,y) and $\overline{p}$ was the average intensity of the pattern distribution as given by

$$\overline{f_{x,y}}=\frac{1}{N}\sum _{(i,j)∊\mathrm{Pattern}}{f}_{x+i,y+j},$$ (5)

$$\overline{p}=\frac{1}{N}\sum _{(i,j)∊\mathrm{Pattern}}{p}_{i,j},$$ (6)

with N being the total pixel number of the pattern. The square of the coefficient of correlation (R²) for a linear regression could vary from 0 (no correlation) to 1 (perfect correlation). The scaling factor H indicated the relative intensity of the object compared to the background. As an example, in an ideal case, the image was scaled from the pattern after a background shift, f_x+i,y+j=k⋅p_i,j+b, where k and b were constants. The pattern matching result would be $R_{xy}^2=1$ and H_x,y=k.

Because the cylindrical cross sectional pattern is unable to determine correct marker lengths, it was necessary to group adjacent qualified cross sections and reconstruct their overall lengths. If the calculated length was found to be longer than the maximum projection length, the feature would be rejected. Here the maximum projection length was defined by multiplying the actual physical marker length by the imager magnification factor (=1.5) plus a reasonable margin.

Marker identification

With multiple markers it was easy to confuse the individual marker labeling for kV or MV projections at different gantry angles. The simplest case was when only one marker exists in each ROI group. From Eqs. 1, 2 there was a direct correlation of the ROI group to a specific marker. In the case of multiple markers existing in the same ROI group, the detected marker positions in this ROI group were correlated to the planning CT markers corresponding to this ROI group, while indexing was based on the shortest distances between detected and predicted marker projections by using Eqs. 1, 2. At certain gantry angles, it was possible that two or more markers may be projected on the same (u,v) location. In this case the number of detected markers in the ROI group would be fewer than the number of expected markers, indicating projection overlapping. This was resolved by comparing the measured to the predicted projections. If one or more of the projections was found missing, but was calculated to be in close proximity to another marker, this projection would be double counted.

Experimental validation and patient data analysis

To evaluate the algorithm’s efficiency when markers overlap or near different anatomical structures, a 360° gantry rotation was performed around a head phantom. Images were acquired every 0.56° for the kV imager (∼640 images in total) and every 10° with the MV imager (36 images in total). Having obtained the projection locations for each marker, 3D spatial information could be calculated by using projections at different gantry angles together with Eqs. 1, 2. Particularly, the results from pairs of MV and kV projections were calculated.

As a preliminary test, this algorithm was also applied to actual patient images previously acquired for patient setup by onboard MV EPID. Five prostate patients with implanted cylindrical fiducial markers were treated in the past two years. Every patient had two or three fiducials (with a diameter of 1.2 mm and a length of 3 mm) implanted. They were treated on a conventional LINAC with MV EPID only. A total of 196 MV projection images were acquired for patient setup at anterior∕posterior (AP) and lateral (LAT) directions before every treatment fraction. After all images were analyzed, the 3D spatial positions were calculated from every pair of AP and LAT images based on Eqs. 1, 2.

An in-house software (C language) was specified to analyze projection images and obtain fiducial positions. All calculations were performed on a Dell Precision 470 workstation (3.4 GHz Xeon CPU and 4 GB RAM).

RESULTS

As a demonstration of the orientation and cross-sectional pattern matching, a MV image of a head phantom with five embedded cylindrical markers was examined [Fig. 3a]. For a selected marker enlarged in Fig. 3b, the eight-bin orientations are shown in Fig. 3c. As can be seen, for pixels located around the fiducial, the ∼45° angle was favored, corresponding to the actual projection orientation. Having determined the orientation, a 90° rotation was made and the marker’s cross section was segmented from the image background [Fig. 3d]. Then the cross-sectional pattern at 45° orientation [Fig. 3e] was applied to calculate $R_{x,y}^2$ and H_x,y.

Figure 3.

(a) MV image of a head phantom with embedded Au cylindrical fiducial. (b) On the right shows magnified image of the selected marker displaying orientation and cross section (dotted line). (c) Orientation map with pixel intensity corresponding to adjacent intensity orientation for the selected marker. (d) Segmented cross-section. (e) Application of the cross-section pattern.

Figure 4 is a side-by-side comparison of the kV and MV detection process for five cylindrical Au markers embedded in a head phantom. The kV and MV images were acquired at a MV gantry angle of 220°. It should be noted that the onboard kV x-ray source is always rotated 90° relative to the MV source (Fig. 1). Application of the algorithm uses prior planning CT information to first define the ROI for each fiducial, as shown by the circular highlighted regions in Figs. 4a, 4b. After ROIs were defined and grouped, orientation and pattern matching were applied to each ROI group and resulted in only image data that conformed to the cross-sectional pattern [Figs. 4c, 4d, 4e, 4f]. The scaling of the ROIs led to greater background image segmentation for the kV case [Fig. 4e] over the MV case [Fig. 4f]. The product of the scaling factor H and the correlation R² led to complete segmentation of the markers, as displayed in Figs. 4g, 4h. In this case a threshold of 0.6 and 0.006 for R² and H, respectively, were determined by previous trials and used through all our analyses.

Figure 4.

Side-by-side kV (left column)/MV (right column) comparison of marker detection for five gold markers embedded in a head phantom. (a) and (b) Projection image and the predicted BB ROIs were highlighted, (c) and (d) R² results (threshold applied), (e) and (f) scaling factors (threshold applied), and (g) and (h) searching index=R^2* scaling factor.

Both kV and MV projections were analyzed using our detection algorithm, where it was found that all five markers were correctly detected for every kV and MV projection. Figure 5 plots the (u,v) coordinates on both the kV and MV detectors for one of the markers. Spatial positions of every marker were calculated for all MV projection images with their corresponding kV partner images gathered over the 360° gantry rotation. Table 1 summarizes these results, where it can be seen that the standard deviation of the location is better 0.5 mm and the errors are within 1 mm.

Figure 5.

Projection locations of one cylindrical fiducial on kV and MV images as functions of kV and MV gantry angles, respectively.

Table 1.

Variation of FM 3D positions calculated from MV∕kV pair projections at 36 different gantry angles.

	FM #1			FM #2			FM #3			FM #4			FM #5
	x	y	z	x	y	z	x	y	z	x	y	z	x	y	z
Mean	−22.9	22.3	−17.2	20.7	24.1	0.0	2.5	−31.9	2.5	−17.3	−32.7	22.7	19.5	22.8	30.4
Max−Mean	0.7	0.7	0.3	0.4	0.6	0.4	0.5	0.6	0.3	0.6	0.7	0.3	0.5	0.6	0.6
Min−Mean	−0.6	−0.7	−0.3	−0.7	−0.6	−0.3	−1.0	−0.6	−0.3	−0.8	−0.8	−0.4	−0.7	−0.9	−0.4
Standard Dev	0.4	0.4	0.2	0.3	0.4	0.2	0.4	0.4	0.2	0.4	0.4	0.2	0.3	0.4	0.2

All patient images were analyzed similarly. In all cases the detection success rate was 100%. Figure 6 displays AP (left column) and LAT (right column) images of three markers embedded in a prostate. Three-dimensional positions of markers were calculated from pairs of AP and LAT projections. Figure 7 shows a fiducial marker’s various 3D positions for 14 fractions. It should be noted that those images were taken at the beginning of patient setup and these positions were not yet the treatment position.

Figure 6.

Analyzing five cylindrical fiducials in a prostate patient MV image at AP (left column) and LAT (right column) directions. (a) and (b) DRR images, (c) and (d) Projection image and the predicted BB ROIs were highlighted, (e) and (f) R² results (threshold applied), (g) and (h) scaling factors (threshold applied), and (i) and (j) searching index=R^2* scaling factor.

Figure 7.

3D spatial position of a fiducial in a prostate patient before treatment setup at 14 treatment fractions.

In the case for symmetrical markers such as BBs, the algorithm can be easily modified by using the spherical pattern, as shown in Fig. 2c. This detection is more simplistic compared to cylindrical marker detection since the BB projection image is independent of the BB’s orientation. More than 2000 combined kV and MV projection images were made of four spherical BB markers (Fig. 8). For all images the algorithm was able to successfully detect the markers.

Figure 8.

Analyzing four BBs on kV (left column) and MV (right column) images. (a) and (b) Projection image and the predicted BB ROIs were highlighted, (c) and (d) R² results (threshold applied), (e) and (f) scaling factors (threshold applied), and (g) and (h) searching index=R^2* scaling factor.

DISCUSSION

The continued advancement of medical imaging technology is reaching the stage where procurement of high resolution anatomical images can be acquired rapidly and with low diagnostic dosages. In addition, utilizing the actual MV treatment beam for imaging has the potential to further reduce diagnostic doses. These various imaging modalities pave the way for real-time IGRT. For the tested projection images, the algorithm demonstrates high successful detection rates together with near real-time speeds. As seen in Figs. 468, reliance on only R² can lead to false positive marker detection, especially for MV based projections. The combination of these two criteria resulted in 100% success detection rates for all 3500 images kV or MV projections analyzed. It is important to emphasize that multiple markers could be successfully tracked on MV images.

Fast marker detection is a crucial component of real-time image based tracking. This algorithm quickens the search process in two aspects. First, the orientation and cross-sectional pattern matching for cylindrical markers simplifies the problem due to cylindrical orientation and projection length, which would otherwise require a large number of patterns to be tested.²⁷ Secondly, the search space is significantly reduced by incorporating prior CT marker location knowledge. For a typical 1024×768 image with five cylindrical markers, the processing time is reduced from ∼1 s to 0.1 s upon incorporating ROI selection. For fewer markers this time would be reduced even further. Generally, spherical marker detection requires approximately half the computational time as compared to detecting the same number of cylindrical markers. In addition, this process is completely automated and does not need any manual location initializations, which is another asset.

Currently, the EPID imager used in this study is hardware limited to a maximum frames-per-sec (fps) rate of ∼7.5 fps. It is, however, envisioned that as future faster kV and MV detectors become available, the algorithm’s detection time can be further reduced. In general, the maximum clinically seen velocity of a tumor is ∼4.0 cm∕s (respiratory based). With the current 7.5 fps imaging speed this would correspond to a between frame marker travel distance of ∼0.5 cm. This distance is small in comparison to the 2 cm in radius ROI used for detection. To increase computational speeds it is beneficial to reduce the search space defined by the ROI. This could be accomplished by recalculating a new 0.5 cm in radius ROI for each new frame analyzed. The location of this ROI would be centered on the marker’s location in the previous frame, thus ensuring that the marker is located within the ROI.

As demonstrated in Fig. 5, the orthogonal location of the kV imager relative the MV imager allows for calculation of the 3D position of the markers from the isocenter. Although this current work consists of retrospective analysis, it is expected that this tracking algorithm can provide real-time 3D tracking in such a system. This is especially convenient on treatment systems pre-equipped with kV and MV onboard imaging equipment as it should only require minimal hardware changes, thus being a cost effective solution for implementing IGRT.

CONCLUSION

Fast and reliable localization of implanted metallic fiducials of various shapes has been an important yet challenging problem in kV stereoscopic image guided RT. This problem is further aggravated by much reduced image contrast when MV beam imaging is involved for therapeutic guidance. A new pattern matching algorithm has been proposed to track multiple spherical or cylindrical fiducial markers on both MV and kV projection images. A completely automated detection, 100% detection efficiency, and fast detection speed (10 frames∕sec) enable tracking tumor motion in real-time on a LINAC with both kV and MV imaging systems. This algorithm makes it a suitable candidate for future image-guided radiosurgical procedures.