Comparison of 4-Dimensional Computed Tomography Ventilation With Nuclear Medicine Ventilation-Perfusion Imaging: A Clinical Validation Study

Abstract

Purpose

Four-dimensional computed tomography (4DCT) ventilation imaging provides lung function information for lung cancer patients undergoing radiation therapy. Before 4DCT-ventilation can be implemented clinically it needs to be validated against an established imaging modality. The purpose of this work was to compare 4DCT-ventilation to nuclear medicine ventilation, using clinically relevant global metrics and radiologist observations.

Methods and Materials

Fifteen lung cancer patients with 16 sets of 4DCT and nuclear medicine ventilation-perfusion (VQ) images were used for the study. The VQ-ventilation images were acquired in planar mode using Tc-99m-labeled diethylenetriamine-pentaacetic acid aerosol inhalation. 4DCT data, spatial registration, and a density-change-based model were used to compute a 4DCT-based ventilation map for each patient. The percent ventilation was calculated in each lung and each lung third for both the 4DCT and VQ-ventilation scans. A nuclear medicine radiologist assessed the VQ and 4DCT scans for the presence of ventilation defects. The VQ and 4DCT-based images were compared using regional percent ventilation and radiologist clinical observations.

Results

Individual patient examples demonstrate good qualitative agreement between the 4DCT and VQ-ventilation scans. The correlation coefficients were 0.68 and 0.45, using the percent ventilation in each individual lung and lung third, respectively. Using radiologist-noted presence of ventilation defects and receiver operating characteristic analysis, the sensitivity, specificity, and accuracy of the 4DCT-ventilation were 90%, 64%, and 81%, respectively.

Conclusions

The current work compared 4DCT with VQ-based ventilation using clinically relevant global metrics and radiologist observations. We found good agreement between the radiologist’s assessment of the 4DCT and VQ-ventilation images as well as the percent ventilation in each lung. The agreement lessened when the data were analyzed on a regional level. Our study presents an important step for the integration of 4DCT-ventilation into thoracic clinical practice.

Introduction

Ventilation imaging can provide useful lung function information for thoracic patients treated with radiation therapy. A new and exciting method has been proposed to calculate ventilation maps (1-4) from phase-resolved 4-dimensional computed tomography (4DCT) data (5). 4DCT-ventilation is attractive because in current clinical practice most thoracic patients undergo 4DCT simulations as part of routine clinical care; therefore, acquiring lung function information from 4DCT-based ventilation would come at no extra monetary or radiation exposure cost to the patient. Several clinical uses of 4DCT-ventilation have been proposed (6-12). The most widely proposed use of 4DCT-ventilation is to create functional avoidance plans (6, 9, 12, 13). Vinogradskiy et al (12) tested the potential benefit of using 4DCT-ventilation functional avoidance by correlating dose and 4DCT ventilation-based function with clinical radiation pneumonitis. Several groups have used 4DCT-ventilation to evaluate lung function changes during (11) and after (14) radiation therapy.

Before 4DCT-ventilation can be implemented in the clinic it needs to be validated. Studies have attempted to validate 4DCT-ventilation by comparing it to other ventilation imaging modalities (2, 3, 15-17). Overall, validation studies have provided promising results but have been limited by the number of patients used in the study and the technical challenges of voxel-by-voxel image correlation.

In current clinical practice, nuclear medicine ventilation-perfusion (VQ) scans are used by physicians to assess global lung function of lung cancer patients (18). Radiation oncology clinicians order VQ scans to assess major ventilation defects for complicated lung cancer cases where lung function may be compromised because of the location and size of the tumor, re-irradiation, or nononcologic lung disease. In these instances, the radiation oncologist will look for major ventilation defects and rely on the clinical assessment of the VQ scan from the radiologist to help aid with clinical management of the patient’s radiation treatment plan. In instances where a major ventilation defect is present, the physician may opt to perform a simple version of functional avoidance by deciding not to place any beams through the functional portion of the lung. 4DCT-ventilation has the potential to provide the same global and clinical lung function information as VQ scans with the available 4DCT data with the added benefits of no extra imaging requirement for the patient, improved spatial resolution, and a functional image inherently registered to the patient’s planning CT. Before 4DCT-ventilation can be used to assess global lung function, a validation study is needed that compares 4DCT-ventilation to the established method of nuclear medicine VQ imaging using clinically relevant metrics. Therefore, the purpose of this work was to perform a global and clinical validation of 4DCT-ventilation imaging by comparing the modality with currently used methods. Specifically, we performed a retrospective study comparing 4DCT-ventilation with nuclear medicine VQ-ventilation, using radiologist observations and clinically relevant metrics of global lung function.

Methods and Materials

Patient population

Fifteen lung cancer patients with 16 sets of 4DCT and VQ images were used for the study (1 patient had 2 sets of both 4DCT and VQ scans). All patients were treated at the University of Colorado from 2010 to 2013. Patients were chosen retrospectively and were included in the study if they had both a 4DCT simulation and a VQ scan. Patient and clinical characteristics are shown in Table 1. Four image sets (4DCT and VQ) were taken on the same day, 15 of 16 cases had image sets taken within 50 days of each other, and 1 case had a 62-day time difference between the 4DCT and VQ scans. None of the patients had any chemotherapy or surgery between the acquired image sets. Two patients had started receiving radiation therapy between the simulation 4DCT and the acquired VQ scan. One of the patients received 6 of 30 2-Gy fractions, and another patient received 18 of 30 1.5-Gy fractions between the scans.

Table 1.

Patient and clinical parameters for the patient population used for the study

Parameter	Median (range) or number (%)
Age	65 (45-87)
Sex
Female	8 (53)
Male	7 (47)
COPD
Yes	10 (67)
No	5 (33)
Smoking status
No	2 (13)
Yes	6 (40)
Former	7 (47)
Tumor location
Right	6 (40)
Left	9 (60)
Tumor stage
I	3 (20)
II	3 (20)
III	6 (40)
IV	3 (20)

Abbreviation: COPD = chronic obstructive pulmonary disease.

4DCT images and VQ scans

The patient’s simulation pretreatment 4DCT scan was used to calculate the ventilation images. Patients were scanned with a Brilliance Big Bore CT scanner (Philips Healthcare, Andover, MA) under free-breathing conditions using the gated lung protocol along with the bellows belt to track the breathing trace. The 4DCT imaging protocol used a kV of 120,800 mAs/slice and a varied pitch based on the patient’s breathing cycle. An attempt was made by the CT technician to ensure the patient’s breathing was not erratic during scan acquisition.

Ventilation images were calculated from the 4DCT data by using a density change-based approach as previously described (1, 2, 11, 12, 15). The first step is to segment the lungs in the end-inhale and end-exhale phases of the 4DCT dataset. Deformable image registration was used to link lung voxel elements from the inhale to the exhale phase (19). The accuracy of the deformation algorithm was reported to be 1 mm for thoracic registration (19). Each deformation field was manually reviewed for discontinuities or errors. Once the inhale and exhale voxels were linked, Hounsfield units (HU) for each voxel were input into a density change-based model (1). The model can be written as

$$\frac{V_{\mathit{in}}-V_{\mathit{ex}}}{V_{\mathit{ex}}}=1000\frac{{\mathit{HU}}_{\mathit{in}}-{\mathit{HU}}_{\mathit{ex}}}{{\mathit{HU}}_{\mathit{ex}}(1000+{\mathit{HU}}_{\mathit{in}})},$$ (1)

where V_in and V_ex are the inhale and exhale volumes, respectively, and HU_in and HU_ex are the inhale and exhale of the individual lung voxels. Equation 1 calculates the local change in air content for each voxel and produces a 3D map of ventilation function (a sample coronal slice shown in Fig. 1B).

Fig. 1.

Examples of a patient VQ-ventilation scan (A) and a 4DCT-ventilation scan (B) and the quantitative metrics used for regional ventilation comparisons (C). Theoretical lines on each image (A and B) illustrate the concept of dividing the lung into individual lung and lung thirds.

The nuclear medicine VQ-ventilation scans were acquired with a Siemens scanner (ECAM or Symbia T16; Siemens, Malvern, PA) in planar mode, using 1.0 mCi Tc-99m-labeled diethylenetriamine-pentaacetic acid aerosol. Immediately after aerosol administration, 8 planar view ventilation images were acquired: 1 anterior, 1 posterior, 4 oblique, and 2 lateral projections.

Comparison metrics

The goal of our work was to evaluate clinically relevant ventilation metrics. In our clinic, we use commercially available software (Xeleris version 2.1753 software; GE Healthcare, Milwaukee, WI) to derive quantitative metrics to aid the radiologist in interpreting the VQ-ventilation scan. The software uses the anterior and posterior projections to determine the regional percentage of counts in each individual lung as well as each volumetric third of each lung (Fig. 1 A and C). The value of each individual lung and each individual lung third is given as a percentage of the total number of counts. To match the metrics of the VQ-ventilation scan, we calculated the percentage of ventilation in each individual lung and each lung third using the 4DCT-based ventilation maps (Fig. 1 B and C). The entire 3D ventilation image was used to calculate the 4DCT-based comparison metrics. The average left and right regional ventilation values were determined by applying a lung mask to the ventilation image and calculating the average ventilation in each lung. Volumetric thirds were calculated from the mask by determining the extreme points in the superior-inferior orientation and dividing by 3. The average ventilation in each third was calculated as a percentage of the total ventilation. The individual lung and volumetric third metrics were compared between the VQ-ventilation and the 4DCT-ventilation using correlation coefficients and linear regression analysis.

In addition to the quantitative ventilation metrics, we incorporated the radiologist’s observation in our analysis. A dual board-certified nuclear medicine radiologist read both the VQ-ventilation (using all 8 projections) and 4DCT-ventilation (using the entire 3D image) and assessed the scans in 2 ways. First, the presence or absence of a ventilation defect in each lung for each patient was noted. A binary yes or no was used to score the presence or absence of a defect. The second metric used was a value that represented an “abnormality” score, which was based on the heterogeneity of the ventilation distribution. A value of 1 represented “normal,” a value of 2 represented “mildly abnormal,” and a value of 3 represented “severely abnormal.” The presence or absence of a defect and an abnormality score were intended to reflect the clinical VQ readings at our institution. Both the VQ and the 4DCT ventilation images were read by the same radiologist, who interpreted the VQ scan independently from the 4DCT-ventilation scan, blinded to patient identity, with no knowledge of corresponding pairs of scans for a given patient. The presence of ventilation defects was compared between the VQ and 4DCT-based ventilation scans by using receiver operating characteristics (ROC). The VQ observations were set as the gold standard, and the sensitivity, specificity, and accuracy were calculated. In addition, we compared readings of the presence of defects by using the McNemar test (evaluating significance at the .05 level). The abnormality scores were compared between the 2 different imaging modalities using rank-correlation methods.

To study the effect of comparing planar images (VQ-ventilation) with 3D images (4DCT-ventilation), an additional comparison was made between planar VQ-ventilation images and planar 4DCT-ventilation images. Planar 4DCT-ventilation images were calculated by integrating the 3D image data along the coronal orientation. The ROC analysis was repeated using the radiologist’s observation of planar VQ images and planar 4DCT-ventilation images.

Results

Representative cases of good agreement between the VQ and 4DCT-ventilation are shown in Figures 1 and 2. Figure 1 shows a ventilation defect in the left upper lobe in both the VQ- and 4DCT-based ventilation images. The percent ventilation values were 61.1% and 38.9% in the right and left lung, respectively, for the VQ-ventilation and 61.8% and 38.2% for right and left lung ventilation, respectively, for the 4DCT scan. Figure 2 also shows good qualitative and quantitative agreement between the VQ scan and 4DCT-based ventilation.

Fig. 2.

Patient example of the comparison of the VQ-ventilation scan (A), the 4DCT-ventilation scan (B), and the quantitative metrics used for regional ventilation comparisons (C).

A scatter plot is shown comparing the percent ventilation calculated in each lung for the VQ and 4DCT-ventilation scans (Fig. 3). The correlation coefficient comparing the percent ventilation in each lung was 0.68 (P<.01), and the P value of the linear regression analysis was <.01. The scatter plot showing the regional ventilation value comparison for each lung third is shown in Figure 4. Data in Figure 4 are grouped according to middle third regions and non-middle third regions. Qualitatively, most of the middle third points have greater percent ventilation in the VQ scan than in the 4DCT-ventilation, whereas the non-middle-third volumes have decreased ventilation in the VQ image compared to the 4DCT-ventilation. The correlation coefficient for the lung third analysis was 0.45 (P<.01), and the P value of the linear regression was <.01.

Fig. 3.

Scatter plot comparing the percent ventilation in each lung between the VQ and 4DCT-ventilation. Line y = x shown for reference.

Fig. 4.

Scatter plot comparing the percent ventilation in each lung third between the VQ and 4DCT-ventilation. Middle third volumes are shown as squares, and non-middle-third volumes are shown as circles. Line y = x shown for reference.

The radiologist noted the presence or absence of a ventilation defect on each lung by using both the VQ and 4DCT-ventilation scans. A comparison was made between observations using 4DCT-ventilation and the observations using VQ-ventilation (Table 2). The sensitivity value (both the 4DCT and VQ ventilation predicted ventilation defects) was 90% (19 of 21 cases), and the specificity was 64%. The overall accuracy of the ability of 4DCT-ventilation readings to predict VQ-ventilation readings was 81%. Using the McNemar test to compare 4DCT- and VQ-based defect observations, we found the P value was .29, meaning that the observations of the 2 imaging modalities were not significantly different. The Spearman rank correlation value comparing the abnormality scores was calculated to be 0.61 (P<.01). The radiologist’s comparisons between the planar VQ-ventilation images and planar 4DCT-ventilation images yielded a sensitivity, specificity, and accuracy value of 90%, 73%, and 84%, respectively.

Table 2.

ROC analysis metrics used to compare observations of VQ- and 4DCT-based ventilation images

ROC metric	Description	No. of cases (%)
Raw ROC data
TP	Both 4D and VQ predicted a defect	19 (59)
FP	4D predicted defect but not VQ	4 (13)
TN	Both 4D and VQ predicted no defect	7 (22)
FN	VQ predicted defect but not 4D	2 (6)
Derived ROC metrics
Sensitivity	True positive rate	19 of 21 (90)
Specificity	True negative rate	7 of 11 (64)
PPV	Positive predictive value	19 of 23 (83)
NPV	Negative predictive value	7 of 9 (78)
Accuracy	Accuracy of 4D in predicting the same outcome as VQ	26 of 32 (81)

Abbreviations: 4DCT = 4-dimensional computed tomography; FN = false negative; FP = false positive; ROC = receiver operating characteristic; TN = true negative; TP = true positive; VQ = ventilation-perfusion.

Discussion

A comparison was made between 4DCT and VQ-ventilation using clinically relevant global metrics and radiologist observations. The 2 representative cases qualitatively show the similarity between the VQ and 4DCT-ventilation images. The scatter plot (Fig. 3), linear regression analysis, and a correlation value of 0.68 indicate good agreement between metrics assessing the ventilation of each individual lung. When ventilation metrics are compared on a more regional level by using values for each volumetric third, the agreement lessens, with a correlation coefficient value of 0.45. This result can be partially explained by a known phenomenon, where the radioactive aerosol used in the VQ scans gets stuck in the airway, producing false “hot spots” (2, 15, 20). The aerosol deposition in the airway can be seen in both of the patient examples (Figs. 1 and 2) and caused the middle lobes to have an artificially larger ventilation value and the upper and lower lobes to have an artificially lower value in the VQ scans when compared to the 4DCT-ventilation (Fig. 4). Overall, our global results are in line with previously reported data (2, 3, 15-17). Reinhardt et al (3) and Ding et al (17) compared 4DCT-ventilation to xenon-CT measures of ventilation in mechanically ventilated sheep and reported correlation co-efficients in the range of 0.52 to 0.94 (3). Castillo et al (2) compared 4DCT-ventilation with single-photon emission computed tomography (SPECT) ventilation in 7 patients and reported Dice similarity coefficients ranging from 0.10 to 0.65. They also noted that the best correlation between 4DCT and SPECT ventilation was found in the lowest ventilation percentiles (corresponding to regions of ventilation defects) and that the disagreement in the higher functioning regions was likely caused by the previously noted aerosol deposition phenomenon. Our data and previous work show a clear relationship between ventilation calculated from 4DCT and ventilation imaged with other modalities, albeit with limited statistical results because of low patient sample sizes and other study uncertainties.

To the best of our knowledge, this is the first study to include radiologist clinical observations in the validation of 4DCT-ventilation. This represents an important step in the validation process because in many situations it is ultimately the radiologist’s observation of the images that are clinically meaningful. Our ROC, McNemar test results, and abnormality score results indicate good agreement between the radiologist’s observations of the 4DCT and VQ-ventilation scans. The ROC metrics comparing the radiologist’s observations of the 4DCT and VQ-ventilation images were 90%, 64%, and 81% for the sensitivity, specificity, and accuracy, respectively. One of the most important outcomes of our work is a sensitivity value of 19 of 21 cases (90%). This result indicates that of 21 cases (individual lungs) that were determined to have ventilation defects in the VQ scan, 19 were found to have matching defects in the 4DCT-ventilation image. The primary goal of a VQ-ventilation scan is to identify any major ventilation defects present, and our data seem to indicate that any defect that can be seen on the VQ scan can be expected to occur in the 4DCT-ventilation image. This congruence is in line with previous work which yielded the best numerical agreement in regions of the lowest ventilation percentiles (2, 15, 16). We also noted that the specificity value was 7 of 11 (64%), meaning that there were 4 cases where the radiologist noted a defect on 4DCT-ventilation but not on the VQ scan. This result can potentially be explained by the fact that 4DCT-ventilation contains 3D functional and anatomical information whereas the VQ scan contains planar images of ventilation. To study the issue of comparing observations of planar images to observation of 3D images, we repeated the analysis using a planar-to-planar image comparison and noted improved specificity values of 73%. Overall, an accuracy of 81% indicates good agreement between the defect observation comparisons of the 2 imaging modalities.

Our study builds on prior work (2, 3, 15-17) and extends 4DCT-ventilation validation in several important ways. First, the current study of 16 cases is larger than previously published series (4 to 11 cases). In addition, the current study includes a more global and clinical comparison of 4DCT-ventilation with VQ-ventilation. Prior work has used metrics for validation that can be challenging to interpret clinically. Our work provides data for the validation of 4DCT-ventilation using metrics (radiologist’s observations and regional percentage of ventilation) that are used directly for clinical assessment. Because 4DCTs are routinely used for clinical simulations, the acquired lung ventilation images could provide clinicians with the necessary information without burdening the patient with extra imaging.

There were several uncertainties in our study that may have contributed to the observed discrepancies between the 2 imaging modalities. The VQ-ventilation scans showed aerosol deposition in the airway, which reduced the agreement at the regional level (2, 15, 20). VQ-ventilation provides planar images while 4DCT-ventilation is a 3D map. Although, we attempted to study these differences by performing a planar-to-planar analysis, there are still inherent differences in image acquisition that cannot be remedied by image processing. Since 4DCT-ventilation is an emerging imaging technology, work is still underway to improve the quality of the 4DCT-ventilation images. Du et al (21) evaluated the reproducibility of 4DCT-ventilation by performing repeat 4DCT scans and reported mean Jacobian ratios of 1.02. Yamamoto et al (22) reported more modest reproducibility results with correlation coefficients on the order of 0.5. Yamamoto et al (23) reported correlation coefficients of 0.77, comparing 4DCT-ventilation images calculated with different registration algorithms. Although we attempted to mitigate these affects, the noted uncertainties may have affected the correlation values. With improved segmentation, registration, and ventilation image processing techniques, we believe the validation results could be improved. The current study focused on global and clinical metrics to validate 4DCT ventilation. Although this method of validation is sufficient for the use of 4DCT-ventilation to assess global lung function, other potential uses of 4DCT-ventilation such as tracking radiation-induced lung changes will require different validation strategies. Because our study was retrospective, there were patients who had 4DCT and VQ scans acquired on different days. We verified that medical events had a minimal effect on the results, but we believe validation data could be improved as the time differences between the 4DCT and VQ imaging is minimized. Although a 16-sample study cohort represents progress from previous work, more patients are needed to make stronger statistical conclusions on a more regional level.

Conclusions

There is a need to validate 4DCT-ventilation before it can be used in the clinic. Our study attempted to validate 4DCT-ventilation against VQ-ventilation using clinically relevant global metrics and radiologist observations. We found good agreement between the radiologist’s assessments of the 4DCT and VQ-Ventilation images, as well as good agreement using the percent ventilation in each individual lung. The agreement lessened when the data were analyzed on a more local level. Our study builds on prior validation work and provides a clinically relevant comparison of 4DCT-ventilation against an established ventilation imaging modality.

Summary.

The current work compared 4-dimensional computed tomography (4DCT) ventilation to nuclear medicine ventilation-perfusion scans by using clinically relevant global metrics and radiologist observations. We found good agreement between the radiologist’s assessment of the 4DCT and ventilation-perfusion images and good agreement between the percent ventilation in each lung. 4DCT ventilation imaging provides lung function information for lung cancer patients undergoing radiation therapy. Our study presents an important step toward integration of 4DCT-ventilation into thoracic clinical practice.