Automatic tracking of arbitrarily shaped implanted markers in kilovoltage projection images: A feasibility study

Abstract

Purpose:

Certain types of commonly used fiducial markers take on irregular shapes upon implantation in soft tissue. This poses a challenge for methods that assume a predefined shape of markers when automatically tracking such markers in kilovoltage (kV) radiographs. The authors have developed a method of automatically tracking regularly and irregularly shaped markers using kV projection images and assessed its potential for detecting intrafractional target motion during rotational treatment.

Methods:

Template-based matching used a normalized cross-correlation with simplex minimization. Templates were created from computed tomography (CT) images for phantom studies and from end-expiration breath-hold planning CT for patient studies. The kV images were processed using a Sobel filter to enhance marker visibility. To correct for changes in intermarker relative positions between simulation and treatment that can introduce errors in automatic matching, marker offsets in three dimensions were manually determined from an approximately orthogonal pair of kV images. Two studies in anthropomorphic phantom were carried out, one using a gold cylindrical marker representing regular shape, another using a Visicoil marker representing irregular shape. Automatic matching of templates to cone beam CT (CBCT) projection images was performed to known marker positions in phantom. In patient data, automatic matching was compared to manual matching as an approximate ground truth. Positional discrepancy between automatic and manual matching of less than 2 mm was assumed as the criterion for successful tracking. Tracking success rates were examined in kV projection images from 22 CBCT scans of four pancreas, six gastroesophageal junction, and one lung cancer patients. Each patient had at least one irregularly shaped radiopaque marker implanted in or near the tumor. In addition, automatic tracking was tested in intrafraction kV images of three lung cancer patients with irregularly shaped markers during 11 volumetric modulated arc treatments. Purpose-built software developed at our institution was used to create marker templates and track the markers embedded in kV images.

Results:

Phantom studies showed mean ± standard deviation measurement uncertainty of automatic registration to be 0.14 ± 0.07 mm and 0.17 ± 0.08 mm for Visicoil and gold cylindrical markers, respectively. The mean success rate of automatic tracking with CBCT projections (11 frames per second, fps) of pancreas, gastroesophageal junction, and lung cancer patients was 100%, 99.1% (range 98%–100%), and 100%, respectively. With intrafraction images (approx. 0.2 fps) of lung cancer patients, the success rate was 98.2% (range 97%–100%), and 94.3% (range 93%–97%) using templates from 1.25 mm and 2.5 mm slice spacing CT scans, respectively. Correction of intermarker relative position was found to improve the success rate in two out of eight patients analyzed.

Conclusions:

The proposed method can track arbitrary marker shapes in kV images using templates generated from a breath-hold CT acquired at simulation. The studies indicate its feasibility for tracking tumor motion during rotational treatment. Investigation of the causes of misregistration suggests that its rate of incidence can be reduced with higher frequency of image acquisition, templates made from smaller CT slice spacing, and correction of changes in intermarker relative positions when they occur.

I. INTRODUCTION

Cone beam computed tomography (CBCT) and on-board kilovoltage (kV) radiographic imaging have led to great improvements in the initial setup accuracy for patients receiving radiotherapy.^1,2 However, the overall accuracy of dose delivery can be compromised by intrafractional tumor motion due to overall patient motion and to physiological processes. This is especially problematic for tumors in organs that experience respiratory motion such as in the thorax and upper abdomen. Different techniques have been used to partly account for intrafraction respiratory motion, but clinically convenient monitoring and control of intrafraction tumor motion has not yet been achieved and is still an area of active research.^3–6

One solution is real time tracking of tumor motion using a series of orthogonal radiographs that monitors internal motion during treatment. This approach is used by the CyberKnife™ system (Accuray Incorporated, Sunnyvale, CA),⁷ which is a dedicated system with a robotic-mounted linear accelerator and a room-mounted radiographic image guidance system. However, tumor tracking is also of interest for treatments on modern conventional C-arm linacs. The recent advent of on-board kV imaging has improved the quality of radiographs on such machines. Treatment with rotational volumetric arc therapy on a linac with an on-board kV imager allows for the acquisition of radiographs at many different gantry angles.^8,9 Intrafraction images can be also taken for fixed gantry angle treatments.¹⁰ However, because many tumors are poorly visualized in radiographs, it is common to implant radio-opaque markers in or near the tumor.^11–13 Simple spherical¹⁴ or cylindrical¹⁵ gold markers (referred to hereafter as regularly shaped markers) have been used successfully in some anatomic locations, but there are cases when their use is discouraged because of the possibility of migration after implantation and the generation of artifacts on planning and future diagnostic images. In these situations, markers that are resistant to migration are preferred.^16,17 Migration-resistant markers, which are currently available in different shape and sizes, coil up and wrap around tissue after implantation (hereafter referred to as irregularly shaped markers) thus reducing the chance of migration. However, the use of a template that assumes a known marker shape before implantation is problematic for automatic tracking of irregularly shaped markers, since the size and shape of the markers after implantation is unpredictable. An additional challenge is the variation in the projection of irregularly shaped markers with gantry angle, which can affect template-based matching accuracy. Further, a means of generating templates for irregularly shaped markers will also naturally handle regularly shaped markers.

Different approaches have been used to extract the position of a marker from kV images. Shirato et al.¹⁸ tracked implanted spherical 2 mm gold markers with geometrical accuracy better than 1.5 mm and acceptably low amounts of diagnostic x-ray exposure. With a conventional template-based tracking algorithm, Tang et al.¹⁹ tracked cylindrical markers in two liver cancer patients with a failure rate (difference between ground truth and tracked position >1.5 mm) of more than 50%. However, with their multiple objects tracking algorithm, they found a maximum failure rate of 12% with 95% confidence level. Marchant et al.²⁰ tracked markers (1 mm in diameter, 10 mm in length) in two pancreatic cancer patients with a success rate of 98.9%. They also tracked markers (1 mm in diameter, 5 mm in length) in a prostate cancer patient with success rate of 94%. All the above work used a marker shape before implantation to create a template for detecting the marker in kV images. Mao et al.²¹ have tracked spherical (stainless steel ball bearings of diameter 1.57 to 4 mm) and cylindrical (gold cylinders with 1.2 mm in diameter, and 3 and 5 mm in length) markers in both kV and megavoltage (MV) images in phantom and prostate patients. They used a pattern matching algorithm, and found detection success rates of 100% in both phantom and patient studies. However, they did not quantify marker location accuracy since no ground truth was defined, which is different from our method of analysis. In addition, their study focused on tracking of regularly shaped spherical and cylindrical markers, and did not investigate irregularly shaped markers. Poulsen et al.²² reported on a semiautomatic method of creating templates for arbitrarily shaped markers, by generating a three dimensional (3D) marker model from CBCT projections. The method selects three to six CBCT projection images with good marker contrast on a uniform background and at larger gantry angle separation, and creates binary images of these selected projections by threshold-based segmentation. The binary images are aligned to each other in order to remove any marker motion between the projections. The binary images are used to construct a 3D model of the marker and to generate templates for all gantry angles. There were several limitations of this study. First, templates were created to match the CBCT images on the basis of only a few projection images, which could cause the size, shape, and orientation of the templates to differ from those seen in the projection images at some gantry angles. Second, evaluation of the method was limited to two CBCT scans of two lung cancer patients; thus its applicability to other disease sites, marker types, and images during treatment delivery was not addressed. Monitoring of internal motion during treatment is a primary clinical goal, but imaging conditions (such as acquisition rate) may differ and thereby affect tracking performance. Third, template creation required manual inspection of segmentations from CBCT projections and correction of any failed segmentations, which in clinical application must be performed at the time of treatment. Further, changes in intermarker relative positions may occur, caused by tissue deformation or marker migration, which may limit the applicability of templates from a CBCT scan on the first treatment day to subsequent days.

In this study, we describe a general method to create templates for any arbitrarily shaped markers whose appearance also varies with gantry angle. The templates are created using CT images for phantom studies and from breath-hold CT images taken at the patient's simulation for patient studies. The method creates templates over a full 360° of rotation and automatically tracks the markers, regardless of their shape, in planar kV images. We report on the accuracy of the automatic tracking using such templates in phantom, pancreas, gastroesophageal junction (GEJ), and lung cancer patients. The studies in phantom evaluate the applicability of the method to both regularly and irregularly shaped markers, whereas the patient studies evaluate tracking of irregularly shaped markers in CBCT projection images and in serial kV radiographs during treatment delivery.

II. METHODS AND MATERIALS

II.A. Phantom data

The phantom study examined automatic tracking in two types of markers: (1) a regularly shaped marker consisting of a cylindrical gold marker (Mick Radio-Nuclear Instruments, Mount Vernon, NY); and (2) an irregularly shaped marker consisting of a single gold Visicoil marker (RadioMed, Bartlett, TN). Table I summarizes the phantom data. The Visicoil in phantom was bent in a C-like shape, thus approximating a post-implant irregular shape. The markers were placed in an anthropomorphic phantom (The Phantom Laboratory, Salem, NY). For the purpose of generating reference image templates for automatic matching with the projection images (described in Sec. II C), CT images of the cylindrical and Visicoil markers in phantom were acquired on an eight-slice CT simulator (Lightspeed, GE Healthcare, Waukesha, WI). Two CT scans were performed, with slice spacing of 1.25 mm and 2.5 mm, for assessing the effect of slice spacing on automatic matching performance. CBCT projection images of the phantom were acquired on a linear accelerator equipped with kV imaging (TrueBeam™, Varian Medical Systems, Palo Alto, CA). The image detector was an amorphous silicon flat panel with size of 40 cm wide and 30 cm long, and effective pixel size of 0.388 mm. X-ray source-to-isocenter and source-to-detector distances were 100 cm and 150 cm, respectively, and the detector was oriented such that its longer dimension was in the transverse direction. The phantom was positioned initially such that the Visicoil marker was at isocenter, and CBCT projection images were obtained in half fan mode (detector laterally offset by 16 cm). The image acquisition was repeated with the cylindrical marker at isocenter. By positioning the marker at isocenter, its ground truth position was known in all projection images, and provided a test of automatic matching accuracy without requiring manual matching.

TABLE I.

Summary of phantom and patient data.

Target site	Number of cases	Marker type and dimensions (diameter, length) (mm)	kV image context (kV, mAs)	Acquisition rate (fps)	Number of images tracked
Phantom	1	Visicoil (0.75, 10)	CBCT (125, 1051)	11	638
Phantom	1	Gold cylindrical (1.2, 3)	CBCT (125, 1051)	11	657
Pancreas	4	Visicoil (0.75, 10)	CBCT before and after treatment (125, 1471)	11	4199
Gastroesophageal junction	6	Visicoil (0.75, 10)	CBCT before and after treatment (125, 1471)	11	7017
Lung	1	SuperLock band (0.9, 23)	CBCT before treatment (125, 728)	11	728
Lung	3	SuperLock band (0.9, 23)	Radiograph during treatment (80, 114–438)	0.17 – 0.2	702

II.B. Patient data

The patient study involved retrospective analysis of CBCT projection images of 11 patients, and analysis of serial kV radiographs taken during treatment of three lung cancer patients receiving volumetric modulated arc therapy (VMAT). Table I summarizes the patient data. Approximately one week prior to simulation, each patient received an implant of one-to-three markers. All patients received a breath-hold CT on a CT simulator about a week before the treatment. All patients also had a respiration-correlated CT (RCCT) scan as part of their simulation. Patient images were acquired with the same CT simulator, linac, and kV imaging equipment as in the phantom studies. Breath-hold CT was used for template creation to minimize the effect of respiratory motion on the templates. In the CBCT part of the study, there were four pancreatic cancer patients, six GEJ patients and one lung cancer patient. All the pancreas and GEJ patients were enrolled in an Institutional Review Board (IRB) approved imaging protocol, whereas all the lung patient data were IRB exempted. Each pancreas and GEJ patient had one to three implanted Visicoil markers, which in most cases assumed irregular (nonstraight) shapes after implantation. During a single treatment session, pancreas and GEJ patients received CBCT scans immediately before and after the treatment as a part of the imaging protocol. The CBCT was acquired over a 360° arc in half-fan mode. In 14 (out of 20) scans, markers lay outside the field of view (FOV) in some images (range 2%–33%).

For one lung patient, as a part of the normal treatment process, two CBCT scans were taken before the treatment over an arc of 200° in full-fan mode (i.e., image detector was not offset laterally). This lung patient had three SuperLock band markers (SuperDimension, Minneapolis, MN) implanted near the tumor, such that the markers were always inside the projection image FOV. This lung patient was one of three lung patients enrolled for intrafraction kV images studies described in the next paragraph.

Additionally, intrafraction kV images were acquired for three lung cancer patients, each with three SuperLock band markers implanted in or near the tumor and who were treated using VMAT. Version 1.0 of the TrueBeam system was capable of kV image acquisition during gated treatment (IMR™, Varian Medical Systems, Palo Alto, CA) to assess internal respiratory motion during treatment. In order to minimize the increase in treatment time, a wide amplitude gating threshold was used such that kV images were acquired near end inspiration and near end expiration (Fig. 1) while maintaining a high duty cycle. A research application (iTools Capture 1.1, Varian Medical Systems) served to passively record a high-quality version of the kV images. Images were acquired from a total of 11 treatment sessions. The data acquisition rate varied from 0.17 to 0.2 frames per second (fps), depending on the patient's breathing pattern.

FIG. 1.

Schematic diagram showing kV triggered imaging near end-expiration and end-inspiration.

II.C. Template creation and image processing

For the purpose of automatically tracking markers in the kV images (i.e., CBCT projections or serial intrafraction radiographs) and computing their deviation from the planned position, 2D image templates of the fiducial markers are generated from reference CT images.²³ The templates are produced at 1° gantry intervals, for a total of 360 templates. For the patient studies, the templates are derived from end-expiration breath-hold CT scans acquired at simulation, with 2.5 mm slice spacing and 0.9 mm pixel size for pancreas and GEJ patients, and with both 1.25 and 2.5 mm slice spacing and 0.9 mm pixel size for lung cancer patients. For phantom studies, templates were generated from CT images acquired with 1.25 mm and 2.5 mm slice spacing, and 0.9 mm pixel size. The method of template formation is the same for phantom and patient images.

Template generation involves the following steps: (1) segmentation of the markers in a CT image, (2) addition of margin to form an extended volume around each marker, and (3) projection of the extended volume onto the image detector plane to produce the template. Figure 2 is a schematic diagram describing a template creation from a CT image.

FIG. 2.

Schematic diagram of method to create templates.

II.C.1. Segmentation of the markers in the planning CT

The user manually draws a rough (e.g., rectangular or cylindrical) volume of interest (VOI) of arbitrary size to encompass the marker but should not contain background objects of similar intensity. A VOI is drawn around each marker, or around two or more markers if they are too close to allow separate VOIs without overlap. Each VOI is searched for its maximum intensity voxel, and a threshold of 60% of the maximum intensity voxel inside each VOI is applied to automatically delineate the marker on each slice within the VOI. The 60% threshold is empirically chosen such as to select voxels that are within the markers while avoiding voxels in regions of streaking artifacts.

II.C.2. Formation of extended volume

A 3-dimensional margin expansion is applied to each delineated marker to produce an extended volume (outlines indicated by arrows in leftmost panel of Fig. 2). The margin is 2 mm and 3 mm for CT slice spacing of 1.25 mm and 2.5 mm, respectively, which ensures expansion by one slice spacing in the superior-inferior direction. The extended volume spans at least 3 CT slices, as in most cases, a Visicoil or SuperLock marker usually is visible on more than one slice.

II.C.3. Template formation

Each extended volume is resliced into planar images perpendicular to the incident x-ray direction (leftmost panel of Fig. 2). Pixel size of the planar image is 0.26 mm and the spacing between planar images is also 0.26 mm. The planar image resolution is chosen to be the same as that of the kV images demagnified to isocenter to optimize the template matching algorithm, such as not to degrade the matching precision (lower resolution) or increase processing time (higher resolution). The pixel intensities of each of the planar images are computed using trilinear interpolation of the voxels inside the extended volume. Parallel rays at normal incidence to the planar images are cast through each pixel (second panel from left in Fig. 2) and the maximum intensity projection (MIP) along each ray was computed (third panel from left in Fig. 2) to produce a template image for this extended volume. The final template image is created by combining all templates generated from the individual extended volumes (rightmost panel in Fig. 2). Similarly, other templates are created at 1° gantry intervals throughout the rotation. On each template, each extended volume is projected onto the template to define a region of interest (ROI) for the automatic match procedure. Multiple ROIs that overlap are merged into a single ROI. The CT image display and processing to produce the templates were performed with purpose-built software developed at our institution.

The kV images taken during the treatment and CBCT projection images taken before and after the treatment are pre-processed to minimize misregistration caused by anatomical and other background features in the images. The images are processed with horizontal and vertical Sobel filters, which enhance the marker edges, and then smoothed with a Gaussian filter to remove noise. The resultant images have enhanced marker edges and suppressed low gradient intensity variations.

II.D. Correction of intermarker relative positions

Since templates generated from a CT scan at simulation are registered with images acquired on treatment days, it is possible that in patients with multiple markers the relative positions of the markers to one another in the kV images may differ from those in the templates due to tissue deformation or marker migration. These changes can introduce errors in the template-based registration. To correct for this, we assume that the position of one marker is unchanged and adjust the position of the remaining markers relative to it. For the purpose of comparing automatic tracking to manual tracking as ground truth, the choice of unchanged marker is the same as one of the markers used in the manual matching. We select a pair of projection images in such a way that they are approximately 90° apart, all markers are within the image FOV, and markers are not overlapping. In each image, each marker is registered manually to its corresponding marker in the template to obtain a corrected 2D position on the template. From the pair of 2D positions so determined, a corrected 3D position is calculated for each marker, which is then projected at 1° intervals to obtain a corrected position for each marker and its associated ROI on each template. The templates are not recalculated; instead, the computed offset in each ROI position (from uncorrected to corrected) on the template is incorporated into the cost function minimization used in matching kV images to templates, described further in Sec. II E. Figure 3 shows template ROI positions before and after correction, superimposed on a kV image from a lung patient case. In this example, the ROI of the upper marker is fixed and corrections are applied to the remaining ROI positions. In cases where markers close together were included in the same extended volume (Sec. II C), no correction was applied between those markers.

FIG. 3.

Example correction of marker relative positions on a kV image of a lung patient. Left: Marker regions of interest (ROIs) defined on the template before correction and overlaid on the kV image are shown as dashed lines. Misalignment of ROIs with two lower markers is evident, caused by changes in intermarker relative positions between simulation and treatment. Right: ROIs after correction relative to uppermost marker are indicated by solid lines.

II.E. Tracking algorithm

The markers in the processed kV images, f (i, j), were automatically matched with the markers in templates, g (i, j), inside the predefined ROIs using a normalized cross-correlation cost function and simplex minimization. The cost function (F) is given by

$$\begin{array}{ccc}\hfill F(a,b)& =& -\frac{1}{n}\sum _k^{Nroi}\sum _{i,j\in RO{I}_k}\hfill \\ & & \times \frac{[f(i+a+c_k,j+b+d_k)-\overline{f}][g(i,j)-\overline{g}]}{{\sigma }_f{\sigma }_g},\hfill \end{array}$$ (1)

where $f(i,j),\overline{f}$, and σ_f are the intensity of the pixel at position (i, j), mean, and standard deviation (SD) of the pixel values respectively over the ROIs for the kV images to be tracked, and the notation is the same for the template with pixel g (i, j). Quantities $\overline{f},{\sigma }_f,\overline{g},{\sigma }_g$ are computed over pixels in all ROIs, N_roi, and n is the total number of pixels inside all the ROIs. Index j is along the longitudinal (superior-inferior) direction, i is in the transverse (TR) direction in the image, and a and b are the displacements of the kV image relative to the template image. The first summation is over N_roi, whereas the second summation is over pixels inside the kth ROI; thus the match is performed using all markers simultaneously. The ROI-dependent offsets c_k and d_k correct for changes in intermarker relative positions (Sec. II D): in effect, the kV image pixels associated with the kth ROI are shifted by the offset such as to bring them into concordance with the ROI in the template. The tracking program reports the longitudinal and TR coordinates of the centroid of all the markers relative to the x-ray central axis for further analysis. The reported centroid coordinates are computed from the centroid coordinates relative to isocenter in the template, and adjusted for the template displacement relative to the kV image in the matching procedure [corresponding to a and b in Eq. (1)] and changes in intermarker relative positions (corresponding to c_k and d_k).

The tracking overview is shown in Fig. 4. In the first kV image of the sequence, the kV image and template are initially aligned according to the x-ray central axis. The search region in the first frame is defined to be large enough to account for expected initial deviations of the markers from the planned position, i.e., 1 × 1 cm for CBCT projection images in this study. In order to avoid the simplex minimization becoming trapped in a local minimum, a grid of starting positions is defined within the search region with 4 mm grid spacing. The cost is calculated at each grid point and the five points with the lowest costs are selected. Simplex minimization is started at each of these points, each with an initial distance of 4 mm between simplex vertices, and the minimization with the best match is selected. During the iterative matching procedure, the kV projection image is translated by varying a and b so as to minimize Eq. (1).

FIG. 4.

Schematic overview of image processing and fiducial marker tracking method. In rightmost panel, template ROIs before and after marker position corrections are schematically indicated by dashed and solid lines, respectively.

Upon completion of the match, the position of the template relative to the kV image serves as the initial starting point for the matching procedure in the next angular projection, but with an additional correction to account for the apparent TR shift of the marker positions owing to the change in gantry angle, which is calculated as follows. First, the centroid position of the markers at the image plane is computed for the current image. Next, the 3D position of the centroid is estimated from triangulation of its position on the current projection image with those in recent prior images. The prior images are chosen approximately one breathing cycle prior to the current image, and such that the superior-inferior position of the centroid in the prior images agrees within ±2 mm of its position in the current image. The first condition limits the choice to recent prior images, with sufficient opening angle (approximately 20°–30° at 1 rpm gantry rotation speed) for accurate triangulation, whereas the second condition imposes a requirement of consistency in the position of the markers between images. At least four prior images must satisfy the above two conditions, in order to reduce sensitivities to outliers. To compute the 3D position, a ray from the source to the centroid position in the current image is constructed. This is repeated for each of the prior images. Since the marker may have moved (e.g., owing to respiration) between images, the rays may not intersect. Instead, the point of nearest approach between the current ray and each of the rays in the prior images is computed, and the point with the median position along the direction of the current ray is selected as the current 3D position of the centroid. Next, the 3D position is projected onto the plane of the next image. Defining (x, y, z) as the 3D coordinates of the centroid in the room system with the isocenter as origin, x points toward the right, y toward the gantry, and z toward the ceiling, and x’ and y’ are the coordinates of the centroid on the image plane with x-ray central axis as origin, x’ points toward the image right and y’ toward the gantry, the centroid position on the image plane is given by

$$\begin{array}{ccc}\hfill x^{\prime }& =& (x·\cos \theta -z·\sin \theta )·M\hfill \\ \hfill y^{\prime }& =& y·M,\hfill \end{array}$$ (2)

where M = (s + d)/(s − x · sin θ − z · cos θ), s is the source-to-isocenter distance (100 cm), d is the isocenter-to-detector distance (50 cm), and θ is the kV source angle of incidence, measured from the 12 o'clock position in the clockwise direction when facing the gantry. The starting position of the template for the next image is adjusted such that the centroid position of the markers on the template coincides with the (x′, y′) coordinates on the kV image. We note that in this study, the prior image conditions stated above for computing 3D position applied only to the CBCT data sets but not to the intrafraction radiograph data sets, because in the latter there was only a single image acquired per respiratory gate.

The purpose-built automatic tracking program, developed at this institution, processes each kV image in a data set by selecting the nearest template to the current x-ray orientation for the matching, and uses Eq. (1) to perform the automatic match. For the CBCT projection images (11 fps image acquisition rate), the amount of motion between successive images is small and a search area of 5 × 5 mm² (2 mm grid spacing and initial simplex step size) at the plane of the image detector is sufficient to capture the range of motion. For intratreatment images in which successive kV images occur at two extremes of the breathing cycle, there may be larger motion of the markers between successive images (Fig. 1). Therefore, for this type of image, the search region was chosen on the basis of maximum displacement of markers in transverse and superior-inferior direction as observed in the RCCT scan from simulation; this area varied from 5 × 8 mm² (2 mm grid spacing and initial simplex step size) to 30 × 45 mm² (4 mm grid spacing and initial simplex step size) depending on patient. The search region estimated from the RCCT scan was adequate for tracking markers in most (>90%) of the treatment sessions. In the remaining intratreatment sequences the search region had to be increased by 10 mm in superior-inferior direction to account for larger breathing motion than was observed in the RCCT.

II.F. Evaluation

II.F.1. Phantom

We estimated the uncertainty in auto tracking by automatically registering the Visicoil marker and cylindrical marker detected in the CBCT projection images with templates created from CT images. The phantom was repositioned between CBCT scans such that the Visicoil marker was at isocenter in one scan, and the cylindrical marker in the other. The measurement uncertainty was determined from deviations in the detected marker position with respect to the isocenter position (0, 0). We define Δ2D as the difference in auto and ground truth in both transverse u and longitudinal v directions of the imaging plane, added in quadrature:

$$\Delta 2D=\sqrt{{(u_{\mathrm{auto}}-u_{gt})}^2+{(v_{\mathrm{auto}}-v_{gt})}^2}.$$ (3)

In the phantom studies, u_gt and ν_gt were set to the isocenter position.

II.F.2. Patients

In the patient data, manual registration was used as the ground truth for evaluating auto-matching accuracy. Thus u_gt and ν_gt were set to the manually tracked positions. We assume that 2 mm difference between auto tracking and manual tracking is clinically acceptable and use Δ2D < 2 mm as the criterion for successful auto-tracking. Intraobserver variation was estimated by repeating the manual matching in a subset of all patient data sets with Visicoils. Approximately 20 images from each of the ten patients with Visicoils were selected randomly throughout a scan of each patient to evaluate intraobserver uncertainty in manual registration. Reproducibility of the automatic tracking was also tested on 657 projection images of a cone beam scan.

CBCT projection images of one lung patient and intrafraction kV images of all three lung patients were tracked by using templates made from both 1.25 mm and 2.5 mm slice spacing planning CT scans. However, CBCT projection images of pancreas and GEJ patients were tracked using templates made from 2.5 mm slice spacing planning CT scans only due to the unavailability of 1.25 mm slice spacing CT scans.

III. RESULTS

III.A. Phantom

Figure 5 shows plots of the transverse and longitudinal positions of the Visicoil and cylindrical markers in the CBCT projection images, using templates derived from CT scans with 2.5 mm slice spacing, as a function of gantry angle. Using template from 2.5 mm slice spacing CT scans, mean ± SD variation in Δ2D was 0.26 ± 0.24 mm and 0.29 ± 0.12 mm for Visicoil and cylindrical markers, respectively. Using templates derived from CT scans with 1.25 mm slice spacing, mean ± SD variation in Δ2D was 0.14 ± 0.07 mm and 0.17 ± 0.08 mm respectively (data not shown). The results show that tracking uncertainty is similar for irregularly and regularly shaped markers in phantom, and that smaller slice spacing reduces the uncertainty.

FIG. 5.

Position of automatically tracked irregularly shaped Visicoil marker in anthropomorphic phantom and placed at the isocenter as a function of gantry angle, along the (a) transverse and (b) longitudinal directions at the imaging plane. Templates were generated from multislice CT scans with 2.5 mm slice spacing. Panels (c) and (d): Positions of regularly shaped cylindrical gold marker under the same measurement conditions.

III.B. Pancreas cancer patients

CBCT projection images were analyzed from four pancreatic cancer patients, each having two to three implanted Visicoil markers in the pancreas. Correction for intermarker relative positions was applied to all four patients, and the magnitude of the corrections in 3D varied from 0.2 mm to 1.8 mm. The markers were tracked without manual intervention in all the patients. To evaluate the accuracy of auto tracking, manual tracking of all images served as an approximate ground truth. Figure 6 compares automatic and manually matched centroid positions of the markers at the plane of the image detector, the results from one CBCT scan. Similar marker motion trajectories were observed in the other patients with a range of motion varying from 9 to 14 mm. Since the CBCT scans were performed in half fan mode and the markers were situated several cm lateral of patient midline, the markers were outside the projection images at gantry angles between approximately 110° and 235° in this example. The low frequency oscillation (sinusoidal with gantry angle) in Fig. 6(a) is a consequence of the rotating imaging system and the marker positions away from the isocenter, whereas the higher frequency superior-inferior oscillations (period of approximately 5 s), are due to respiratory motion. In transverse and longitudinal directions, mean and SD differences between automatic and manual matching were 0.00 ± 0.02 mm and 0.00 ± 0.05 mm, respectively, implying that auto tracking agreed well with manual tracking. Similar results were found in the other pancreas patients. In all images (eight CBCT scans of four patients), the maximum Δ2D, calculated from Eq. (3), was less than 2 mm. Assuming a criterion for successful tracking of Δ2D < 2mm, the overall success rate of automatic tracking in four pancreas patients was 100%. Tracking results for the pancreas patients are summarized in Table II. In addition, we simulated tracking of intrafraction images by analyzing a subsample of CBCT projection images at 0.3 fps, and found the success rate to be 100% for all four patients.

FIG. 6.

Centroid position of makers as a function of gantry angle for a pancreas cancer patient in (a) transverse and (b) longitudinal directions.

TABLE II.

Summary of patient tracking results. Mean and range of success rates is across patients.

Target site	Marker type	kV image context	CT slice spacing (mm)	Mean success rate (range) (%)
Pancreas	Visicoil	CBCT before and after treatment	2.5	100
Gastroesophageal junction	Visicoil	CBCT before and after treatment	2.5	99.1 (98–100)
Lung	SuperLock band	CBCT before treatment	1.25	100
			2.5	98.3
Lung	SuperLock band	Radiograph during treatment	1.25	98.2 (97.5–100)
			2.5	94.3 (92.9–97.3)

III.C. Gastroesophageal junction patients

CBCT projection images were analyzed from six gastroesophageal junction patients, each with one to three implanted Visicoil markers. Correction for intermarker relative positions was applied to only three patients and varied from 0.6 mm to 8.7 mm. In the remaining three patients, correction was not applied since two of them had only one marker and one of them had markers overlapped. Figure 7 compares automatic and manual tracking from one CBCT scan of a patient with two markers, in which the markers were within the FOV in all projection images. However, markers were outside the FOV for some images in four of the six GEJ patients. The range of motion in longitudinal direction varied from 11 to 43 mm in all GEJ patients. In Fig. 7, mean and SD differences between automatic and manual matching were 0.00 ± 0.03 mm in transverse and 0.00 ± 0.05 mm in longitudinal, implying that auto tracking agreed well with manual tracking. Similar results were found in the other GEJ patients. The percentage of registrations with Δ2D > 2 mm was up to 2.0%. Overall, the success rate of auto-tracking for GEJ patients was 99.1% (failure in 62 out of 7017 images). Tracking results for the GEJ patients are summarized in Table II. Tracking of simulated intrafraction images, by analyzing a subsample of CBCT projection images at 0.3 fps, yielded an average success rate of 93% (range 82%–100%) in all six GEJ patients.

FIG. 7.

Centroid position of markers as a function of gantry angle for a gastroesophageal junction patient in (a) transverse and (b) longitudinal directions.

Overall, automatic tracking of irregularly shaped Visicoils in the pancreas and GEJ cancer patients had a success rate (Δ2D < 2 mm) of 99.4% (range 98%–100%). One SD uncertainty in the repeatability of automatic registration was 0.02 mm, and intraobserver repeatability in manual registration was found to be 0.08 mm (SD) in both transverse and longitudinal directions.

III.D. Lung cancer patients

Data consisted of intrafraction images from three lung cancer patients, each with three implanted SuperLock band markers near the tumor. Correction for intermarker relative position was applied to all three patients and varied from 1.2 mm to 7.0 mm. Since the kV images of these lung patients were taken during VMAT treatment and limited to a single image at the start of each gating interval (Fig. 1), the data acquisition rate was approximately 0.2 fps. Figure 8 compares automatic and manual tracking for one VMAT field of approximately 220° arc length. Similar trends were seen for the other patients with range of motion varying from 6 to 42 mm. Since intrafraction kV images were taken near end-expiration and end-inspiration, there are no data points between these two phases as seen in Fig. 8. We also examined the effect of CT slice spacing on automatic tracking performance. Using templates made from the 1.25 mm slice spacing CT scans, we found Δ2D > 2 mm in 13 out of total of 702 images with an average tracking success of 98.2% (range 97.5% to 100%). Using templates made from 2.5 mm slice spacing CT scans, the average success rate was 94.3% (range 92.9% to 97.3%). In addition, analysis of CBCT projection images from one lung patient yielded Δ2D < 2mm of 100% and 98.3% for automatic tracking with templates made from 1.25 and 2.5 mm slice spacing CT scans, respectively. Tracking results of the lung patients are summarized in Table II.

FIG. 8.

Centroid position of markers as a function of gantry angle for a lung cancer patient in (a) transverse and (b) longitudinal directions. In longitudinal direction, points near 0 cm (−20 cm) are at end expiration (end inspiration).

III.E. Correction of intermarker relative positions

Analysis of changes in intermarker positions was focused on the 11 patients with CBCT scans. Of these, measurement of changes was not possible in three GEJ patients (two with single marker, one with overlapping markers). Table III summarizes the intermarker 3D corrections and success rate of tracking with and without correction in the remaining eight patients in which corrections were applied. In each patient case, results are shown for two different choices of gantry angles for the image pairs. In six out of eight patients (numbered 1 through 6), corrections were between 0.4 and 2.6 mm, and the already high success rates without correction were marginally improved with correction. In one GEJ and one lung patient (numbered 7 and 8), the changes in intermarker position were sufficiently large, and therefore, it was not possible to track all fiducials simultaneously without correction. Following correction, all fiducials could be tracked simultaneously in both patients. In all cases, the choice of gantry angles for the image pairs had a negligible effect on the intermarker 3D corrections and tracking success rates with correction.

TABLE III.

Tracking success rate before and after intermarker relative position correction.

Target site	Patient	Number of markers	Intermarker 3D correction (mm)	Success rate without correction	Success rate with correction	Gantry angles
Pancreas	1	2	0.4	99.7%	100%	160°, 250°
			0.6		100%	200°, 290°
	2	3	0.9, 1.0	100%	100%	135°, 225°
			0.2, 0.3		100%	200°, 290°
	3	2	1.4	100%	100%	160°, 250°
			0.7		100%	200°, 290°
	4	3	0.8, 1.8	99.8%	100%	160°, 250°
			1.1, 1.8		100%	200°, 290°
Gastro-	5	2	2.6	99.8%	100%	160°, 250°
esophageal			1.9		100%	200°, 290°
junction	6	2	0.6	97.6%	98.7%	160°, 250°
			1.4		98.7%	230°, 320°
	7	3	4.1, 8.7	NPa	100%	160°, 250°
			3.5, 8.3		100%	200°, 290
Lung	8	3	2.8, 6.2	NPa	100%	0°, 90°
			3.3, 7.0		100%	50°, 140°

^aMeasurement of Δ2D not possible with all markers simultaneously.

IV. DISCUSSION

Our findings indicate that arbitrarily shaped implanted markers, as visualized in kV planar images acquired during CBCT or during rotational gantry angle treatments, can be automatically tracked with high accuracy in phantom, abdominal, and thoracic patients. The primary advantages of the proposed method of template construction are twofold. First, the method does not require assumptions of marker shape; instead, a breath-hold CT scan serves as a model of marker shape, from which templates representing marker projections at different orientations are derived. Thus the method is applicable to both regularly and irregularly shaped markers. In the event the planning CT scan is not breath held, an additional short breath-hold CT scan, encompassing only the markers, is sufficient for generating the templates. Alternatively, the RCCT image at end expiration may serve for this purpose, although irregular breathing may introduce artifacts in the RCCT image and hence the templates. Second, in most cases the templates can be prepared in advance of treatment, since they are derived from a CT scan acquired at simulation. This could minimize potential delays at treatment, in contrast to the method by Poulsen et al.,²² which requires manual interaction and review of a 3D marker model from CBCT projections acquired immediately prior to treatment. Cases involving large intermarker changes in position between simulation and treatment require additional processing in our method and are discussed further later in this section.

Our results from phantom studies demonstrate that, first, the proposed template-based tracking method achieves similar accuracy with Visicoil and gold cylindrical markers, thus confirming the method's applicability to both irregularly and regularly shaped markers. Second, tracking accuracy is improved with higher CT spatial resolution (i.e., smaller slice spacing). It is important to note, however, that the phantom studies did not include motion, and background features in phantom images were less challenging than those in patient images.

In patient data, automatic tracking worked best in pancreatic cases, with a success rate of 100% in all four patients. Auto registration was slightly worse in the GEJ patients, although the success rate was still high (varying from 98% to 100%). Although GEJ patients had the same frequency of data acquisition (11 fps) as in the pancreas cancer patients, occasional misregistration was caused by strong anatomical background (anatomical structures with high density). GEJ tumors lie near the diaphragm and lung, which caused a larger range of motion and more variation in contrast between different regions of the images. Application of the edge enhancing filter results in enhancement of regions having large contrast variation. Even with a small search window, when markers are inside a region with strong anatomical background, templates may match with the background instead of markers. Figure 9(a) shows an example where misregistration is caused by strong anatomical background in a GEJ cancer patient. Misregistration was highest (2%) in one GEJ patient with only one marker. We therefore investigated tracking of multiple (2 and 3) markers versus a single marker in four scans of two GEJ patients. The two GEJ patients were chosen because the distance between adjacent markers was greater than the capture range throughout the scan, so that single markers could be tracked independently. The average success rate of tracking a single marker was 99.8% whereas for multiple marker tracking, the success rate of tracking was 100%. Clearly further study with larger number of patients is needed. In some of the GEJ patients, misregistration was due to conflicting contrast between marker and the background as well. An example of misregistration due to this situation is shown in Fig. 9(b). The effect of imaging frequency on tracking success rate, by analyzing all CBCT projection images and a subsample of them, was found to depend on site: success rate was unchanged (100% at 11 fps and 0.3 fps) in pancreas, whereas in GEJ the success rate decreased at lower frequency (mean of 99% at 11 fps, 93% at 0.3 fps). At the lower frequency of data acquisition (0.3 fps), marker positions in consecutive images were further apart due to respiratory motion, which required an enlargement of the search region. The resultant increased incidence of high density background caused more misregistrations and thus a lower success rate of tracking. The lower imaging frequency is representative of intrafraction imaging during gated treatment.

FIG. 9.

Misregistration due to (a) high density background features and (b) low contrast between background and marker in a GEJ patient.

The conditions for marker tracking in lung patients differed in several ways as compared to those of pancreas and GEJ patients: intrafraction image acquisition rate (∼0.2 fps) was substantially lower than for CBCT; markers implanted in lung were bigger; and templates for tracking were made from both 1.25 mm and 2.5 mm slice spacing CT scans. Processed kV images resulted in high-contrast anatomical structures, since the markers were implanted in lower lung tumors and were near the diaphragm. In addition, since the frequency of data acquisition was sparse for intrafraction images, a larger search window had to be used between images, which included more background in the search region. Nevertheless, the overall success rate was high (mean 98%, range 97%–100%) when using templates from 1.25 mm slice spacing CT scans. The success rate was reduced (mean 94%, range 93%–97%) with templates from 2.5 mm slice spacing CT scans. Tracking success rate in CBCT scans of one lung patient was also reduced with larger slice spacing (100% and 98% for 1.25 and 2.5 mm, respectively). These findings illustrate the benefit of using smaller slice spacing, even in instances where marker is long (23 mm) relative to slice spacing.

One limitation of our method is the inability to recover in some cases in which it has matched to background, thus requiring manual adjustment of the starting position for tracking of subsequent frames. An area of current investigation is to incorporate a method of assessing the goodness of a match. The goodness of match would be used in decision rules, such as whether to use the current match as a starting point for the next image, or whether to increase the search range in the next image if current match quality is poor.

A second limitation of our method is in cases in which there are sufficiently large (several millimeter) changes in intermarker positions between simulation and treatment, such that corrections to marker relative positions must be applied to the templates prior to tracking. In this study, two patients (numbers 7 and 8 in Table III) had large intermarker changes that required correction in order to simultaneously track all markers. Our current method requires manual determination of correction from an image pair, which can be selected from the CBCT projection images on the iTools workstation. Thus the manual correction procedure would require about 2-to-3 min between completion of the CBCT scan and start of treatment. Alternatively, an orthogonal kV image pair prior to treatment can serve for this purpose. An area of current investigation is to develop a two-step automatic matching of the image pair, in which the current template-based matching with all markers (first step) is followed by matching of individual markers (second step) to compute the relative offsets. In the second step, only one ROI at a time, corresponding to a single marker or overlapped markers, is included in the cost function minimization in Eq. (1). In our current method, one marker is assumed fixed. Alternatively, one can compute marker changes relative to the centroid of all the markers. In either case, to verify targeting accuracy, a volumetric image (CT, CBCT, or MR) is desirable to reestablish marker positions relative to soft tissue targets.

V. CONCLUSIONS

The accuracy of a proposed template-based automatic tracking method has been evaluated in kV planar images in phantom and in patients. Tracking accuracy computed in CBCT projection images of a stationary phantom was found to be similar for regularly (gold cylindrical) and irregularly (Visicoil) shaped markers. The success rate of automatic tracking, relative to manual tracking as an approximate ground truth, was evaluated in CBCT projection images of GEJ and pancreas cancer patients, and CBCT and intrafraction images of lung cancer patients having irregularly shaped markers. The success rate was high in all cases (range 98%–100% CBCT; 93-100% intrafraction), with no misregistration in pancreas patients, and only occasional misregistration in GEJ and lung cancer patients. Investigation of the causes of misregistration suggests that its rate of incidence can be reduced with higher frequency of image acquisition, templates made from smaller slice spacing CT scans, and correction of changes in intermarker relative positions when they occur.