An Assessment of a Low-Cost Visual Tracking System (VTS) to Detect and Compensate for Patient Motion during SPECT

Abstract

Patient motion is inevitable in SPECT and PET due to the lengthy period of time patients are imaged and patient motion can degrade diagnostic accuracy. The goal of our studies is to perfect a methodology for tracking and correcting patient motion when it occurs. In this paper we report on enhancements to the calibration, camera stability, accuracy of motion tracking, and temporal synchronization of a low-cost visual tracking system (VTS) we are developing. The purpose of the VTS is to track the motion of retro-reflective markers on stretchy bands wrapped about the chest and abdomen of patients. We have improved the accuracy of 3D spatial calibration by using a MATLAB optical camera calibration package with a planar calibration pattern. This allowed us to determine the intrinsic and extrinsic parameters for stereo-imaging with our CCD cameras. Locations in the VTS coordinate system are transformed to the SPECT coordinate system by a VTS/SPECT mapping using a phantom of 7 retro-reflective spheres each filled with a drop of Tc^99m. We switched from pan, tilt and zoom (PTZ) network cameras to fixed network cameras to reduce the amount of camera drift. The improved stability was verified by tracking the positions of fixed retro-reflective markers on a wall. The ability of our VTS to track movement, on average, with sub-millimeter and sub-degree accuracy was established with the 7-sphere phantom for 1 cm vertical and axial steps as well as for an arbitrary rotation and translation. The difference in the time of optical image acquisition as decoded from the image headers relative to synchronization signals sent to the SPECT system was used to establish temporal synchrony between optical and list-mode SPECT acquisition. Two experiments showed better than 100 ms agreement between VTS and SPECT observed motion for three axial translations. We were able to track 3 reflective markers on an anthropomorphic phantom with a precision that allowed us to correct motion such that no loss in visual quality was noted in motion corrected slices relative to motion free slices.

I. INTRODUCTION

In cardiac SPECT imaging, body motion has been determined to occur in ~25% of studies and ~5% of the time motion is significant enough to cause artifacts which can mislead diagnosis [1,2]. Only a small amount of motion is needed to impact diagnostic accuracy. Generally it has been reported that motion of 2 or more pixels (>13 mm) was required to create minor to moderate defects in the tomographic data [1–3]. One study that analyzed Tl-201 polar maps has shown that motion of 1 pixel (~6.4 mm) can induce significant artifacts [4].

A number of approaches relying solely on SPECT data to detect and compensate for motion have been described [5–10]. However, the use of external tracking devices providing additional information independent of SPECT data might be expected to result in more robust correction than using solely emission data. Following the work of others for head motion compensation in SPECT [11] and PET [12, 13], our group has been working towards developing a robust method to track and compensate patient body and respiratory motion in cardiac SPECT with a visual tracking system (VTS). Patient motion is estimated by the VTS through the use of stereo-imaging of retro-reflective markers on stretchy bands wrapped about the patient chest and abdomen [14–17]. The motion of these markers is separated into respiratory (repetitive) and assumed rigid-body (RB) components using an adaptive approach we have developed [18]. RB motion is then corrected within reconstruction by breaking the acquisition data up into subsets defined by similarity in patient position. We then account for the six-degree-of-freedom (6DOF) motion of the patient between these states using three-dimensional (3D) Gaussian interpolation. This modifies the location of the attenuation maps and current estimate of the emission slices prior to projection, and the updates following backprojection [19]. The respiratory component of motion from the central markers on the band around the abdomen of volunteers has been shown to be very similar to that of the signal from a pneumatic bellows device wrapped about their abdomen [18]. It can thus be used to form respiratory amplitude or phase binned sets of projection data from list-mode acquisitions. For cardiac SPECT, the axial motion of the heart can then be determined and corrected within the projection data itself [20], or the 6DOF motion assessed post-reconstruction between the different binned slice sets and then correction performed during a second pass of reconstruction [21, 22].

Commercial tracking systems such as the “Polaris” system of Northern Digital have been previously employed for the tracking of motion in emission imaging [12, 23]. The purpose of our investigation was to develop a low cost system with greater flexibility it terms of positioning the optical cameras making-up the stereo-pairs relative to each other, and ultimately the number of cameras employed for stereo-tracking. The low-cost and flexibility of positioning are needed since we believe the robust tracking of motion in SPECT will ultimately require the use of stereo-cameras tracking the motion of markers from both the head and foot end of the SPECT system. This is due to the closeness of approach for the SPECT heads to the patient and marker obstruction by patient anatomy [23].

In this paper we report on some recent modifications to our low-cost VTS which improve its accuracy for motion tracking and its stability of operation. We also provide results documenting the operation of the VTS system and a test of its use to estimate and correct the RB motion of a phantom. Finally we discuss modifications to how we position retro-reflective markers on the patients.

II. BACKGROUND

The basic pin-hole camera model and the theory of using a pair of cameras for photogrammetric measurements are aptly covered in the classic text by Faugeras [24]. The pin-hole camera projection equation which maps 3D world points (X^World) onto 2D camera projections (X^Cam) is given by:

$${\stackrel{\rightharpoonup }{X}}^{\mathit{\text{Cam}}}=\mathit{\text{CP}}·{\overrightarrow{X}}^{\mathit{\text{World}}}$$ (1)

where the projection matrix, CP, describe the intrinsic and extrinsic parameters that are unique to a particular camera and its pose. CP is a 3×4 matrix that can be separated into (C) a 3×3 matrix of intrinsic camera parameters and (P) a 3×4 matrix of extrinsic camera parameters as in:

$$C=\left[\begin{array}{ccc}{f}_x& s{f}_x& C_x\\ 0& f_y& C_y\\ 0& 0& 1\end{array}\right]$$ (2)

$$P=\left[\begin{array}{cccc}\text{cos}\alpha \text{cos}\gamma -\text{cos}\beta \text{sin}\alpha \text{sin}\gamma & -\text{cos}\beta \text{cos}\gamma \text{sin}\alpha -\text{cos}\alpha \text{sin}\gamma & \text{sin}\alpha \text{sin}\beta & T_x\\ \text{cos}\gamma \text{sin}\alpha -\text{cos}\alpha \text{cos}\beta \sin \gamma & \text{cos}\alpha \text{cos}\beta \text{cos}\gamma -\text{sin}\alpha \text{sin}\gamma & -\text{cos}\alpha \text{sin}\beta & T_y\\ \text{sin}\alpha \text{sin}\gamma & \text{cos}\gamma \text{sin}\beta & \text{cos}\beta & T_z\end{array}\right]$$

The 5 intrinsic parameters of C describe the physical properties of each camera: focal lengths (f_x and f_y), the center of image plane (C_x and C_y) and the skew (s) between the x and y pixel axis of the camera sensor which is 0 if the axes are perpendicular. The extrinsic parameters of P relate the pose between the two camera image planes and can be written as three principal angles (α, β, γ about x,y,z) and a translation vector T. These parameters will vary slightly depending on how the lens distortion is modeled. Lens distortion is an aberration that causes straight lines to appear curved. In most cases, a 1^st order radial lens distortion (k₁) is sufficient for today’s higher-quality CCD cameras. This is modeled using only the first 2 terms of the power series expansion for the undistorted radii (r_u) in terms of the distorted radii (r_d) where r²=x²+y²:

$$r_u=r_d+k_1{r}_d^3+k_2{r}_d^5+\cdots$$ (3)

Calibration is a process of determining all these camera parameters. Accurate 3D measurements are a function of camera resolution, distance to the imaging volume, camera separation relative to that distance and, most importantly, the quality of the calibration. As detailed below, an important advancement in our current version of our VTS was to improve the accuracy of calibration over our past methodology.

III. METHODS

As noted in the preceding, the key to stereo-imaging is camera calibration [25, 26] or the establishment of a relationship such that the 3D positions of the retro-reflective markers can be determined from their projections onto 2D optical images. We previously performed calibration using a phantom with markers at whose center we could position a small volume of Tc^99m. The location of the centers of the spheres in 2D for each optical camera as well as the location of the centers as determined from a SPECT acquisition were then input to a program which used singular value decomposition (SVD) to estimate the transformation between optical and SPECT coordinate system locations [15, 17]. This is an ill-posed problem and these methods were plagued with sensitivity to noise, which ultimately limited the accuracy of determining marker positions. To address this sensitivity to noise and thereby improve the accuracy with which we determine the 3D location of the markers we have switched to a 2-step procedure for calibration [26, 27]. In the first step, the intrinsic and extrinsic parameters for each camera are determined and then optimized for the particular stereo-setup. The software necessary to perform these steps is contained in a freely available MATLAB toolbox [28]. For use in the calibration, we fabricated a black and white checkerboard grid on rigid foam board with a flexible base that can be positioned in the SPECT imaging volume and fixed-in-place. Calibration is thus performed by taking simultaneous snapshots of this grid pattern for 15 to 20 orientations within the volume of SPECT reconstruction as illustrated in Figure 1. These images are processed for each camera with a corner extraction algorithm. Given the known dimensions of the grid and accurate corner extractions, the positions of the grid relative to the image planes are determined. These data are used in a stereo algorithm that determines a set of 5 intrinsic and 6 extrinsic parameters for each camera. A set of corresponding pixel points from two optical cameras projected into 3D with the camera parameters will result in a point in space whose true location depends on the precision of the former. The steps involved in processing the video streams are C programs that parse the video into JPEGs, extract the timestamps from the JPEGs, and convert from VTS to SPECT coordinates; and MATLAB programs that threshold the images and find the blob centers, calculate the time differences between images, sort the pairs of blob centers, and triangulate the points using the stereo parameters.

Figure 1.

The planar calibration pattern that we fabricated supported with a weighted-flexible base in between the heads of our SPECT system.

Calibration provides the location in 3D of objects within the SPECT system FOV in a coordinate system specific to the VTS. Thus to make measurements meaningful for use in correcting motion in SPECT images, the transformation from the VTS coordinate system to the coordinate system used in SPECT needs to be determined. This is done via imaging with both the SPECT and VTS systems a phantom consisting of 7 retro-reflective spheres each with a small volume of Tc^99m at the center of the spheres as we have previously described [17]. This phantom is shown in Figure 2 which is a repeat of a figure from our previous publication [17]. From the VTS coordinates of the 7 spheres and the 7 coordinates of the reconstructed SPECT data, we can determine a unique rotation and translation matrix to map VTS locations to SPECT locations via SVD [17].

Figure 2.

The calibration phantom consists of seven retro-reflective spheres screwed onto the top of rods of various heights each with a well that can receive a drop of radioactivity. During the calibration step, the phantom is placed in the field of view of the SPECT system. Snapshots are taken with the optical cameras and a SPECT acquisition is performed.

To assess the spatial accuracy of our VTS, the phantom with 7 reflective spheres labeled with Tc^99m used to establish the VTS to SPECT coordinates transformation was moved through 10 steps of approximately 1.0 cm in the vertical and then in the axial direction via translating the table of the SPECT system. An arbitrary translation and rotation was performed on the phantom after the last axial step. A SPECT acquisition was performed for each position along with a snapshot of the phantom with each optical camera. SPECT slices were reconstructed with filtered backprojection with no attenuation or scatter compensation. The location of the centroids of the 7 spheres were found from the reconstructed SPECT data and compared to the location of the spheres as determined by the VTS from the projections of the spheres in the snapshots.

In order to synchronize the VTS data with a planar list-mode SPECT acquisition, we created a LabVIEW program that triggers both systems with short circuits as it sends a repeating square-wave voltage signal to SPECT. Our SPECT data contains 100ms frames with 10ms timing marks as well as the values of the voltage signal. The list-mode file is converted to raw data and processed with a program that looks for gaps in the voltage signal to calculate the initial SPECT delay as well as the dead-time during head rotations. This timing information is used to plot the center-of-mass (COM) of counts on a timescale that correlates with our VTS data which has no gaps or delays.

In the previous version of our VTS we employed Axis PTZ 2130 network cameras (Axis Communications AB, Lund, Sweden) with pan, tilt and zoom capabilities to allow us to remotely select the orientation of the camera as well as the size of the field-of-view (FOV) within the region of interest for SPECT imaging. Subsequently, we determined these cameras to drift in orientation over time thereby causing decreased accuracy in the determination of marker location by stereo imaging. Although we developed a method to correct for this drift by imaging the location of reflective markers attached to the opposed wall [29], it was decided to replace the PTZ cameras with cameras whose zoom and orientation can be rigidly locked such that significantly less drift occurs. To maintain as much as possible software compatibility we choose Axis 211 CCD network cameras with Tamron 13VG550T 5–50 mm zoom lenses as shown in Figure 3 from which to construct our modified VTS. An LED light source is mounted on top of each camera via a specially constructed ring attachment to provide a source of light which is retro-reflected back to the camera by the special coating on the markers. This allows the camera to see the markers with excellent contrast independent of room lighting as is needed for clinical operation [17]. As shown in Fig 3, the cameras are attached with a Manfrotto 482 micro-ball tripod head to a support which is mounted on the wall of the SPECT room. The tripod head allows the camera to be freely angled making up for the lack of pan and tilt capability. When the desired orientation is obtained, the camera can be locked in place by tightening the knob seen on the left side in Fig 3. The network cameras are mounted about 7 feet high to see over the patients head and 12 feet away focused on the imaging volume within the SPECT system. They are connected via Ethernet cables to a network hub connected to a computer that records snapshots or video streams set to 640×480 resolution which we determined to be optimal.

Figure 3.

One of our Axis 211 cameras with LED on top which was powered via camera and controlled through Axis software API’s. The camera is securely mounted on wall pointed in the desired direction with a Manfrotto tripod head.

Stability of orientation for the new cameras was assessed via imaging retro-reflective markers (diameter= 25 mm) on the wall opposite the cameras which were visible through the opening in the SPECT gantry. Images of the visible markers from a pair of cameras are shown in Figure 4. Positions of these markers were tracked for two months with periodic snapshots with each camera at 640×480 resolution. Centroids were calculated for each marker, after applying a threshold to images to mask the background, and the changes were reported in mean values and SD of pixels where 1 pixel= 1.25mm at that distance.

Figure 4.

Images of retro-reflective markers from both cameras visible through gantry of our SPECT system. The markers are 25 mm in diameter with 1 pixel= 1.25 mm at that distance and were tracked for two months with only about 1 pixel drift in each camera.

The change to new cameras necessitated a modification in our method of synchronization between the optical images recorded on the PC and emission events in the SPECT system [17]. The time in ms between the two successive JPEG images or “delta time” is no longer available directly from the extra information stored with the images when the individual JPEG files are concatenated together to form an MJPEG file. The new cameras instead store a timestamp in the header of each JPEG image indicating when the frame was captured in units of 10 ms. We wrote C code to extract these timestamps and then calculate the “delta time”. We checked this new method of synchronization by manually moving a Tc^99m source with a retro-reflective marker attached during simultaneous list-mode SPECT and VTS imaging. The time relative to the start of acquisition at which the axial location of the marker changed as determined from the VTS, on average, 40 ms recordings and the position of the axial COM of the counts in 100 ms frames reformatted from the list-mode SPECT data was compared to determine the accuracy of synchronization between the two systems.

As a final test of the VTS system we used the VTS to determine the location of the Data Spectrum Anthropomorphic Phantom and used this knowledge of the change in location as a function of projection angle to correct for RB motion during reconstruction using 3D Gaussian interpolation projection and backprojection [19]. The relative concentration of Tc^99m activity in the heart, liver, and background compartments of the phantom was 1.0, 0.5, and 0.125. We acquired four motion-free SPECT projection sets with the anthropomorphic phantom in different positions. The six degree-of-freedom (6-DOF) motion between the sets was determined from the change in location of 3 retro-reflective markers attached to the phantom. The projections from these 4 studies were used to create multiple projection sets with various combinations of motion. Comparison was made between the reconstructed slices of the first projection set which was taken as the standard, the slices reconstructed without motion correction and slices reconstructed with RB motion correction.

IV. RESULTS AND DISCUSSION

We found good agreement between the change in position of the 7-sphere phantom as assessed via stereo-imaging with the VTS and as determined from the COM of counts in SPECT slices for ten vertical (Fig. 5) and ten axial (Fig. 6) imaging table movements of approximately 1.0 cm. Besides these data points, shown on these plots, in each case, are the regression line, the determined regression equation between the two methods, and the square of the correlation coefficient (R²). Notice that in each case, R² is essentially 1.00 indicating very good agreement between the two measures of relative motion. This is further seen in that the differences in Euclidean distance as assessed via SPECT and VTS of the 70 (seven spheres times 10 steps) measurements of position were max= 0.11 cm and mean= 0.03 cm in the vertical direction, and were max= 0.13 cm and mean= 0.04 cm in the axial direction. The overall relative accuracy of our VTS was calculated as the mean difference between VTS and SPECT determined translations of the centroid of our phantom. The mean disagreement between VTS and SPECT for both vertical and axial motion was 0.04 cm with a standard deviation of 0.002 cm. These results document the ability of our VTS to track marker movement, on average, with sub-millimeter accuracy compared with SPECT.

Fig. 5.

VTS position (cm) versus SPECT position (cm) for all seven spheres of the phantom in the vertical direction (Y-axis) as the SPECT table is moved along that axis in 10 steps of about 1.0 cm. Data is fitted with a linear regression line and the square of the correlation coefficient is shown.

Fig. 6.

VTS position (cm) versus SPECT position (cm) for all seven spheres of the phantom in the axial direction (Z-axis) as the SPECT table is moved along that axis in 10 steps of about 1.0 cm. Data is fitted with a linear regression line and the square of the correlation coefficient is shown.

Besides the studies of repeated motion in the axial and vertical directions, we also used the 7-sphere phantom to assess the ability of the VTS to determine an arbitrary motion consisting of a mixture of translation and rotation. We determined the 6-DOF RB transformation from the initial to final position using the centers of the spheres as determined by the VTS and from SPECT reconstructions. The VTS angular rotations about the x, y and z axes were 2.8°, −66.0° and −1.5°. From the SPECT slices, the rotations were 2.2°, −66.1° and −1.4°. Note the difference between SPECT and VTS determined rotations about the y and z axes are 0.1°, whereas the difference about the x axis is 0.6°. The translations determined from the VTS data were −0.67 cm, 0.71 cm and 4.82 cm in the x, y and z directions. For SPECT, the translations were −0.66 cm, 0.70 cm and 4.75 cm. Note the largest disagreement (0.07 cm) occurs about the z axis which is the hardest axis to calculate translations with stereo for our geometry. Small rotations about x axis and translations along the z axis move mostly perpendicular to the CCD image planes. Thus the 6-DOF transformations as determined via VTS and SPECT were in good agreement being sub-degree for rotation and sub-millimeter for translation.

The stability of our new fixed-mount cameras is illustrated by the smaller amount of camera drift. Mean drift, during a 2 month period, for the left camera being 1.39 ± 0.12 pixels in the x-direction and 0.70 ± 0.01 pixels in the y-direction. For the right camera, the mean drift was 0.26 ± 0.02 pixels in the x-direction and 1.55 ± 0.02 pixels in the y-direction. The zoom employed was such that at a distance approximately in the center of the SPECT imaging volume a pixel corresponds to about 1.11 mm. This is far better than the 4-to-10 pixel drifts we observed with the older software-controlled PTZ cameras [29]. Although we developed a method for correcting for camera drift [29], it still involved an additional step that we felt could be avoided with more stable video cameras. The stability of the new cameras was also verified by noting that there was little change in the intrinsic and extrinsic parameters from successive calibrations over the 2 month period.

Data from two synchronization experiments with a Tc^99m labeled marker moved three times during VTS and planar list-mode acquisition showed good temporal synchronization for all the detected motions. The agreement between VTS and SPECT was within 100 ms for the first experiment (Fig. 7) and also within 100 ms for the second (Fig. 8). This level of temporal sensitivity, given our sub-millimeter spatial accuracy, should allow us to correct for RB motion as well as periodic respiratory motion during our cardiac SPECT studies.

Fig. 7.

Overlay of the temporal synchronization between our VTS and SPECT for three axial translations. The differences in time for the three observed motions is negligible.

Fig. 8.

Overlay of the temporal synchronization between our VTS and SPECT for three axial translations. The differences in time for the three observed motions is negligible.

Using data from four motion-free SPECT studies, with the anthropomorphic phantom in different positions, we created acquisitions with artificial motion via mixing the acquisitions. The motion of the phantom between these four sets was measured by the VTS and used in reconstruction to correct for RB motion [19]. Three sets of SPECT transverse slices spanning much of the heart wall chamber within the anthropomorphic phantom reconstructed from SPECT acquisitions are shown in Figure 9. The top row of slices is for motion with no VTS correction. The next row shows slices for motion with VTS correction. The bottom row are the motion-free reference slices. All SPECT data are reconstructed with OSEM over 204° using scatter and attenuation compensation as well as VTS motion compensation for the motion-corrected slices. It is clear that the slices with simulated motion and no VTS correction have significant motion-related artifacts and the motion-corrected slices look very similar to the motion-free reference images.

Fig. 9.

SPECT slices of an anthropomorphic phantom reconstructed with OSEM over 204° using scatter and attenuation compensation for (top row) simulated motion with no VTS correction, (middle row) with VTS motion correction, and (bottom row) motion-free references images.

Initially we employed a reusable “Spandex” vest that had 16 retro-reflective spherical markers permanently attached [17]. This proved to be time consuming to wrap around patients, and there was concern relating to the possibility of contamination being passed on from one patient to the next. Thus we now attach, via “Velcro” stickers, the markers to stretchy bands made of self-adhesive bandage material. The bands, with 4 markers each, are wrapped about the patient chest and abdomen over the clothing or hospital gown prior to SPECT imaging. With practice this process takes less than one minute. The bands are inexpensive and can be disposed of to help with infection control.

V. CONCLUSIONS

The total cost of the hardware components of a 2-camera VTS system was under $3,000. Although the development costs were high for us, avoiding the mistakes we made with camera and calibration-method choices, one should be able to quickly implement a VTS whose accuracy is sufficient for motion correction with SPECT imaging and is significantly less expensive than commercial systems. The disagreement between VTS and SPECT marker movement, calculated as the mean difference of measured motion, is less than 1.0 mm for translation and 1.0 degree for rotation. Two experiments showed better than 100 ms agreement between VTS and SPECT observed motions. VTS-tracked marker motion enabled us to correct for motion artifacts when reconstructing SPECT data from an anthropomorphic phantom. Thus our VTS is a stable, accurate, and inexpensive system for determining the 3D locations of retro-reflective spherical markers for use in tracking patient motion during SPECT imaging. It is also flexible in that the two cameras do not have to be in a fixed geometry relative to each other, and we can add more cameras to provide more stereo-pairs thus allowing us to more robustly track the motion of the markers clinically.

Fig. 10.

Illustration of the current “Velcro” attachment of retro-reflective markers on disposable stretchy bands made from self-adhesive bandage material. The bands with 4 markers each are wrapped about the abdomen and chest of patients prior to SPECT imaging.