Real-time soft tissue motion estimation for lung tumors during radiotherapy delivery

Abstract

Purpose: To provide real-time lung tumor motion estimation during radiotherapy treatment delivery without the need for implanted fiducial markers or additional imaging dose to the patient.

Methods: 2D radiographs from the therapy beam's-eye-view (BEV) perspective are captured at a frame rate of 12.8 Hz with a frame grabber allowing direct RAM access to the image buffer. An in-house developed real-time soft tissue localization algorithm is utilized to calculate soft tissue displacement from these images in real-time. The system is tested with a Varian TX linear accelerator and an AS-1000 amorphous silicon electronic portal imaging device operating at a resolution of 512 × 384 pixels. The accuracy of the motion estimation is verified with a dynamic motion phantom. Clinical accuracy was tested on lung SBRT images acquired at 2 fps.

Results: Real-time lung tumor motion estimation from BEV images without fiducial markers is successfully demonstrated. For the phantom study, a mean tracking error <1.0 mm [root mean square (rms) error of 0.3 mm] was observed. The tracking rms accuracy on BEV images from a lung SBRT patient (≈20 mm tumor motion range) is 1.0 mm.

Conclusions: The authors demonstrate for the first time real-time markerless lung tumor motion estimation from BEV images alone. The described system can operate at a frame rate of 12.8 Hz and does not require prior knowledge to establish traceable landmarks for tracking on the fly. The authors show that the geometric accuracy is similar to (or better than) previously published markerless algorithms not operating in real-time.

INTRODUCTION

Tumors in the thorax and upper abdomen can exhibit large intrafractional motion due to respiration.¹ Real-time knowledge of the tumor location during radiotherapy treatment delivery is desirable for a range of applications aimed at quantifying and reducing the negative effect of intrafractional tumor motion on the treatment outcome. In particular, for motion mitigation approaches such as dynamic multileaf collimator (DMLC) tracking or robotic couch table tracking, the real-time availability of target motion information is imperative.

A variety of techniques have been proposed to assess intrafractional tumor motion during treatment delivery. Most approaches that are in clinical use today rely on the implantation of fiducial markers. Real-time fiducial marker segmentation from kV fluoroscopic images has been utilized clinically to drive tumor motion mitigation during therapy delivery to the lung.² Another study used electromagnetic transponders implanted in a phantom for real-time localization.³ For lung tumors, the high risk of pneumothorax associated with percutaneous implantation of fiducial markers is a limitation of this procedure.⁴ Endobronchial implantation techniques can avoid this risk but are limited to small markers and some peripheral tumors may not be reachable.

Using an external surrogate, e.g., chest motion, to estimate tumor motion in real-time has been proposed in various scenarios (usually also incorporating occasional internal motion information to update a correlation model between internal and external motion).^{5, 6} Using an external surrogate signal alone may however provide insufficient tumor position information as it has been shown that the correlation between external surrogate signal and tumor motion can be unstable over the course of a treatment fraction.^{7, 8}

A real-time markerless template matching technique was developed for tracking lung tumor motion on fluoroscopic kV images.⁹ A classification algorithm has been proposed to track soft tissue motion without fiducial markers in kV fluoroscopic images.¹⁰ Markerless template matching approaches to track lung tumors on MV images have been proposed as well.^{11, 12} However, both published algorithms rely upon retrospective image analysis and are therefore not suitable for real-time applications.

Classification algorithms require appropriate training data and cannot quickly adapt to unexpected changes of motion range (baseline drifts) or motion pattern. For 2D motion estimation, the beams-eye-view perspective yields the dosimetrically most relevant information.¹³ However, kV-imaging is not commercially available in this configuration and may also be problematic considering the accumulated imaging dose over the course of a multifraction treatment.^{14, 15, 16} For the current study, we have chosen a markerless direct motion estimation approach by capturing the exit fluence of the MV therapy beam with a linac mounted electronic portal imaging device (EPID).

The aim of this study is to investigate the use of the MV-EPID for real-time soft tissue motion estimation of lung tumors during radiotherapy delivery.

MATERIALS AND METHODS

Our tumor motion estimation system consists of two components: a frame grabber and a real-time image processor. The frame grabber (iTool), provided by Varian Medical Systems (Palo Alto, CA), allows direct RAM access to the EPID acquisition image buffer from the LINAC control system computer (see left of Fig. 1). The image processor accepts incoming MV-EPID frames and estimates tumor motion by means of an in-house developed soft tissue localizer (STiL). This algorithm is based on a previously published method of multiregion tracking.¹² However, a redesign and extension was necessary for real-time operation due to the old algorithm's reliance on prior knowledge of the image sequence to find suitable landmarks. We have removed this requirement by introducing a new landmark selection algorithm and a geometric regularization that is applied to each frame. In this paper, we consider an algorithm to be real-time if it can generate an output signal from a given input signal before the next input signal arrives, i.e., if the target position can be calculated faster than (1/imaging-frequency). We make one exception to this requirement: the initialization of the algorithm with suitable landmarks for tracking. This step may take 1-2 s during which no motion information is available.

Figure 1.

(Left) Data flow of the tracking platform. (Middle) EPID image of the tumor model (in solid water phantom). The center location of each landmark is marked by a cross. As an example, the smaller rectangle shows the location of one of the landmarks and the larger rectangle shows its corresponding search region. (Right) EPID image of the tumor model (in dynamic thorax phantom) with the landmark center locations marked as crosses. Both images are cropped and show axes units in pixels. The window/level was adjusted for maximal visibility of the tumor model.

In Sec. 2A, a brief overview of the new algorithm is given.

Soft tissue localization (STiL) algorithm

The STiL algorithm consists of three components: a landmark detector, a landmark tracking algorithm, and geometric regularization to filter out poor performing landmarks on each frame. The input is a stream of incoming images from the frame grabber and the output is a displacement vector for each image with respect to the first acquired image.

Landmark selection

The first acquired image is used as a reference image to automatically define a set of landmarks. Each landmark consists of a small region of the reference image (template) and a surrounding search region. We keep the same metrics for selecting an initial set of landmark candidates, i.e., regions of maximal texture. We use a local variance filter as a texture measure and select local maxima of this quantity. To avoid bunching, a minimal distance between landmarks is required.

To ensure the uniqueness of each landmark j within its search region, an auto-similarity map ${\mathcal{S}}_j(x,y)$ is calculated for each landmark using normalized cross correlations (NCCs). If we consider ${\mathcal{S}}_j$ as a 2D surface, the stipulation for uniqueness will be satisfied if the surface has an elliptic point at its origin, i.e., the auto-similarity is maximal at the origin and falls off in all directions. This is equivalent to requiring the Gaussian curvature K_j¹⁷ at the origin of ${\mathcal{S}}_j$ to be positive: K > 0. We therefore maximize local variance var_j and Gaussian curvature K_j simultaneously. For each landmark, a score is assigned by

$${\xi }_j={\text{var}}_j·K_j$$ (1)

Here K_j is the Gaussian curvature (of the auto-similarity map) and var_j is the local variance (of the first acquired image) for each landmark j. The scale for both measures is transformed to the interval [0, 1] before the multiplication. The landmark candidate set is then picked from the highest scoring landmarks.

To minimize computation time of the tracker and to maximize uniqueness of each landmark within its search region the latter is chosen to be asymmetric (i.e., longer in the superior-inferior direction). These parameters can be chosen as fixed values as long as they can cover the entire motion range (estimated, for example, from a 4DCT, other pretreatment imaging or generically assuming 20 mm maximum motion¹⁸). For noncoplanar fields, the search box dimensions projected to the imager plane are assigned as (in EPID image coordinates):

$$s={\left[\begin{array}{c}x\\ y\end{array}\right]}_{g,c}=\left[\begin{array}{c}{r}_{si}·\sin \left({\varphi }_c\right)·\cos \left({\varphi }_g\right)\\ r_{si}·\cos \left({\varphi }_g\right)\end{array}\right]$$ (2)

Here the indices g and c stand for gantry and couch, respectively and r_si stands for the superior–inferior motion range. The minimum search box dimension is defined by the template size.

The automatic landmark selection process requires only one image. However, it takes about 1-2 s until a landmark candidate set is readily available for tracking. One can therefore either hold the beam for this time or has to discard the first 1-2 s of continuous EPID imaging.

Tracker

The tracking algorithm calculates the best matching location of each landmark on a subsequent frame by maximizing a similarity measure within the landmark's search region. We use normalized cross correlations as a similarity measure to tackle this template matching problem (see Ref. 12 for more detail).

Regularization

Robust regularization is essential because tracking failures of some landmark regions may be expected on each image for several reasons:

• – Landmark candidates are chosen solely based on the texture measure without considering temporal gray value gradient, so there may be landmark candidates that are pinned to static structures contained in the candidate set.
• – Landmarks are attached to a structure that is not sufficiently unique within the search window throughout the entire breathing cycle, e.g., due to deformation of the target.
• – Occlusion caused by the restricted field of view (landmark becomes partially occluded at times when close to the treatment aperture).

The regularization works as follows: once a set of initial landmark positions $\left\{p_j^{*}\right|j=1,\cdots ,N\}$ has been selected, their relative geometric relationships are stored as a symmetric N × N matrix

$$A_{ij}^{*}=\Vert p_i^{*}-p_j^{*}\Vert .$$ (3)

On each subsequent image the new landmark positions are expected to have a similar geometric relationship to each other as on the initialization image. The deviation from the initial relation can be calculated for t > 0 as another symmetric matrix:

$$A_{ij}\left(t\right)=\left|A_{ij}^{*}-\Vert p_i\left(t\right)-p_j\left(t\right)\Vert \right|.$$ (4)

While a certain degree of deformation and rotation are allowed, the relationship of the landmarks should stay similar as in the initial image. This is used as the regularization criterion: iteratively, the landmark with the greatest value of

$$a_j\left(t\right)=\sum _i\frac{A_{ij}\left(t\right)}{N^2}$$ (5)

that is larger than an empirically derived threshold is removed from the landmark set for the current image and not used for calculation of the average motion trajectory. The landmark removal process is repeated until all values a_j(t) are below the threshold and we are left with a subset $\left\{p_{j_k}\right|j_k=1,\cdots ,N\left(t\right)\}$. Here N(t) is the current number of landmarks remaining on image I(t) (acquired at time t) after regularization is applied.

From the N(t) remaining landmarks for image I(t) an average displacement vector p(t) is calculated which is the final output of each acquisition loop:

$$p\left(t\right)=\frac{1}{N\left(t\right)}\sum _{j_k}{p}_{j_k}\left(t\right).$$ (6)

Figure 2 (middle and right) gives an example illustrating the rejection of failing landmarks by means of the regularization procedure that has been described here.

Figure 2.

Illustration of the regularization method for a case with N = 30 landmark candidates. (a) and (b) show EPID images acquired during lung SBRT delivery of a patient. The initial position of each landmark candidate $\left\{p_j^{*}\right|j=1,\cdots ,N\}$ is marked with a red cross and the individual displacement vectors p_j(t) are drawn in green. The average displacement vector p(t) is drawn as red arrow in the center. In (a), the displacement vectors of all landmark candidates are shown before regularization (b) after regularization. In (c), A_ij(t) is depicted as color coded matrix: blue refers to small deviations, and red to large deviations.

Implementation

A graphical user interface (GUI) was developed to allow an observer to follow the tracking process in real-time. We used C++ and Matlab (Mathworks, Inc) for the implementation of the STiL algorithm and the communication with the frame grabber.

Characterization of geometric accuracy

We characterize the STiL algorithm's accuracy with EPID images from both dynamic phantom experiments and a patient's lung SBRT delivery. In order to compare the real-time tracking performance to retrospective EPID tracking performance,¹² we also include an analysis of the same data set as was previously used.

The following setup parameters are the same for all three studies: 6 MV photons are delivered with a Varian TX clinical linear accelerator featuring a gantry mounted AS-1000 EPID detector which is operated at half resolution (512 × 384 pixel) and a source to imager distance of SID = 180 cm at SAD = 100 cm. We chose a larger SID compared to the often clinically used SID = 150 cm. This was due to a study we conducted to optimize continuous EPID imaging contrast during lung SBRT delivery.

1. Dynamic solid water phantom: A realistic resin tumor model¹⁹ is placed on 10 cm of solid water that is moved by a programmable translation stage along a patient's lung tumor trajectory.²⁰ The frame rate is set to 12.8 fps and the tumor motion is estimated in real-time during the delivery of a small circular field.

2. Dynamic thorax phantom: A realistic breathing thorax phantom,¹⁹ with the resin tumor model inserted, is driven by a patient's lung tumor trajectory. The image acquisition frame rate is set to 2.0 fps. These data were acquired for a previous study¹² to which we want to compare the accuracy of the STiL algorithm; therefore, the data had to be analyzed retrospectively at the lower frame rate.

3. Patient (lung SBRT): The delivery of a lung SBRT treatment is monitored from the beam's-eye-view (BEV) with continuous EPID imaging. A right posterior oblique (RPO) field is selected for (retrospective) motion analysis with the STiL algorithm. For the patient data analysis, image acquisition in the LINAC's clinical mode was necessary. The number of frames that can be acquired during one treatment fraction in this mode is currently limited due to an internal cache size constraint. To avoid an internal cache overflow in the treatment control software, it is necessary to reduce the frame rate of the patient data acquisition to 2 fps. This restriction does not apply to the LINAC's service mode which we use to deliver the phantom studies. The implications of the different frame rates are discussed in Sec. 4.

The signal to noise ratio (SNR) depends on the integration time. Although frame averaging improves SNR, it also introduces motion blur. We did not use frame averaging for study (i) and chose to average every two frames for studies (ii) and (iii).

As a quality measure, we use the deviation of the trajectory p^(a)(t) estimated by the STiL algorithm from a reference trajectory p^(m)(t) defined manually by an expert and calculate the (geometric) root mean square (rms) deviation:

$$\begin{array}{ccc}\hfill {\text{rms}}^{\left(a\right)}& =& \sqrt{\frac{{\sum }_t{\left({\Delta }^{\left(a\right)}\left(t\right)\right)}^2}{N}}\text{with}\hfill \\ \hfill {\Delta }^{\left(a\right)}\left(t\right)& =& ‖p^{\left(a\right)}\left(t\right)-p^{\left(m\right)}\left(t\right)‖.\hfill \end{array}$$ (7)

For the manually defined reference trajectory, we randomized the image sequence presented to the expert examiner in order to minimize any bias arising from the image order. To estimate an operator uncertainty Δ^(m)(t), we presented each EPID image N_m = 3 times and calculate the standard deviation (error bars in Figs. 34):

$${\Delta }^{\left(m\right)}\left(t\right)=\sqrt{\frac{{\sum }_{k=1}^{N_m}{\left(p_k^{\left(m\right)}\left(t\right)-⟨p^{\left(m\right)}\left(t\right)⟩\right)}^2}{N_m}}.$$ (8)

Figure 3.

(Left) Prospective real-time motion analysis of a realistic lung tumor model on a dynamic phantom (AP field) at 12.8 fps. (Right) Histogram of the deviation between the estimated motion from the STiL algorithm and the manually defined reference trajectory. The maximum absolute deviation is smaller than 0.6 mm with a rms^(a) = 0.3 mm.

Figure 4.

Comparison of prospective vs retrospective tracking. (Left) Realistic breathing thorax phantom (AP field): we calculated a deviation of (0.7 ± 0.3) mm from the manual verification trajectory. This compares to (0.6 ± 0.3) mm with the retrospective algorithm. (Right) Lung SBRT patient (RPO field): we observe an accuracy of (0.9 ± 0.5) mm for the STiL algorithm which compares to (0.6 ± 0.4) mm with the retrospective tracking (see Table 1 for more details).

RESULTS

Dynamic phantom study (real-time)

A comparison of the STiL algorithm's (prospective) real-time motion estimation and a manual verification registration for 121 portal images of an AP field (ϕ_g = 0°, ϕ_c = 0°) is shown in Fig. 3. The prospective tracking with the STiL algorithm resulted in a maximal tracking error of 0.6 mm with a rms^(a) = 0.3 mm.

Realistic dynamic thorax phantom

In order to compare the real-time implementation of the STiL algorithm with the previously published retrospective algorithm, the image analysis was performed on the same data.¹² On the left side of Fig. 4, the results for the first 110 images of the EPID sequence is shown for an AP field (ϕ_g = 0°, ϕ_c = 0°). The maximal deviation between the STiL algorithm and the reference trajectory is found to be smaller than 1.5 mm with rms^(a) = 0.8 mm. These results are in a similar range as the values reported in Ref. 12, i.e., an absolute mean of (0.6 ± 0.3) mm.

Patient (lung SBRT delivery)

The application of the STiL algorithm to patient data is shown on the right side of Fig. 4 for a noncoplanar field (ϕ_g = 30°, ϕ_c = 270°). The motion range is about 20 mm and the maximal deviation between STiL output and reference trajectory is smaller than 2.7 mm with rms^(a) = 1.0 mm, which is in a similar range as the manual verification uncertainty (rms^(m) = 0.6 mm) and the accuracy reported in Ref. 12 for a retrospective algorithm that performed at an absolute mean deviation of (1.0 ± 0.5) mm.

DISCUSSION

The real-time localization system presented in this paper provides an estimation of lung tumor motion from a stream of images acquired from the beams-eye-view during radiotherapy delivery. The STiL algorithm does not rely on fiducial markers or expose the patient to additional dose from the image acquisition.

This work is the first demonstration of a markerless tumor tracking system working with MV-EPID images alone that can operate in real-time during beam delivery. Compared to previous publications reporting on retrospective tissue tracking from portal images,^{11, 12} we achieve a similar level of accuracy but at a much higher frame rate (12.8 fps) and without the need for prior knowledge or user intervention. The geometric accuracy of the motion estimation was experimentally validated with a dynamic phantom study. We show that the accuracy is on the same order as the uncertainty in reference trajectories created by manual registration of an expert viewer (cf. Table 1).

Table 1.

Experimental verification for the performance of the STiL algorithm from 2 phantom studies and a lung SBRT treatment. We give the mean value and standard deviation of the absolute deviation between calculated trajectory and reference trajectory for comparison.

	Motion	STiL uncertainty (mm)				Reference uncertainty (mm)
Experiment	Range (mm)	rms(a)	max	mean	σa	rms(m)	max	mean	σm
Solid water phantom	8 - 10	0.3	0.6	0.2	0.1	0.3	0.7	0.3	0.2
Thorax phantom	8 - 10	0.8	1.5	0.7	0.3	0.5	1.0	0.4	0.3
Lung SBRT patient	≈20	1.0	2.7	0.9	0.5	0.6	1.3	0.6	0.3

The algorithm presented in this paper builds on a previously published algorithm that was developed to demonstrate retrospective markerless EPID tracking.¹² That method relied on prior knowledge of the image sequence in order to find suitable landmarks. We have removed this requirement by introducing a new landmark selection algorithm and a geometric regularization applied in each frame. Furthermore, we accelerate the image acquisition more than sixfold. However, during the time interval needed to find suitable landmarks (1-2 s) tracking is not possible. Therefore, it would be necessary to either refrain from using the first 2 s of EPID images or hold the beam for the same time after the first image is acquired. The localization accuracy of the prospective algorithm was found to be similar to that of the retrospective method.

Due to memory restrictions in the clinical mode of the commercial hardware (cf. also Sec. 2B), we were only able to acquire patient images at 2 fps. The analysis of the clinical data was performed offline with the data passing through the STiL algorithm as if coming from the online frame grabber. The markerless localization algorithm proceeded without prior knowledge and at the same processing speed as stated earlier in this paper. Acquisition at a higher frame-rate will likely improve clinical tracking due to less motion between frames.

We assume for the automatic landmark selection process that structures which are unique within a search region, show maximal texture and have nonzero motion represent tumor motion. Therefore, we cannot entirely exclude the tracking of normal structures that are moving with the tumor. A future implementation of the STiL algorithm will include the discrimination of physician-identified target structures based on clinical contours.

This study utilizes only two beam directions: AP fields for the phantom studies and a RPO field for the patient data study. Although one would expect a strong correlation between gantry angle and tumor visibility, Richter et al.¹¹ were not able to confirm this hypothesis in their patient data study of lung SBRT portal image sequences.

We have shown that the algorithm presented in this paper can provide adequate motion estimation accuracy for potential use as position input for motion mitigation techniques such as couch tracking or dynamic multileaf collimator tracking.

CONCLUSIONS

We have proposed and characterized a markerless prospective tumor tracking system consisting of a frame grabber and a soft tissue localization algorithm that can estimate lung tumor motion during radiotherapy delivery in real-time. The motion estimate shows rms errors of 1 mm or smaller in phantom studies and a patient data sample. Due to the real-time computation speed, the algorithm is potentially suitable for applications in motion mitigation techniques like couch tracking or dynamic multileaf collimator tracking.