With current commercially available iterative reconstruction techniques, radiation dose reductions of 25%–50% can reduce the low-contrast spatial resolution relative to that achieved by using full dose and filtered back projection.
Abstract
Purpose
To determine the dose reduction that could be achieved without degrading low-contrast spatial resolution (LCR) performance for two commercial iterative reconstruction (IR) techniques, each evaluated at two strengths with many repeated scans.
Materials and Methods
Two scanner models were used to image the American College of Radiology (ACR) CT accreditation phantom LCR section at volume CT dose indexes of 8, 12, and 16 mGy. Images were reconstructed by using filtered back projection (FBP) and two manufacturers’ IR techniques, each at two strengths (moderate and strong). Data acquisition and reconstruction were repeated 100 times for each, yielding 1800 images. Three diagnostic medical physicists reviewed the LCR images in a blinded fashion and graded the visibility of four 6-mm rods with a six-point scale. Noninferiority and inferiority-superiority analyses were used to interpret the differences in LCR relative to FBP images acquired at 16 mGy.
Results
LCR decreased with decreasing dose for all reconstructions. Relative to FBP and full dose, 25%–50% dose reductions resulted in inferior LCR for vendors 1 and 2 for FBP and 25% dose reductions resulted in inferior and equivalent performance for vendor 1 and equivalent and superior performance for vendor 2 at moderate and strong IR settings, respectively. When dose was reduced by 50%, both IR techniques resulted in inferior LCR at both strength settings.
Conclusion
For radiation dose reductions of 25% or more, the ability to resolve the four 6-mm rods in the ACR CT accreditation phantom can be lost.
© RSNA, 2015
Introduction
Iterative reconstruction (IR) is now available from all major manufacturers of clinical computed tomographic (CT) scanners. IR techniques allow substantial noise reduction while maintaining high-contrast spatial resolution (1). However, IR techniques affect the noise and spatial resolution properties in a nonlinear manner. As a result, the spatial resolution of low-contrast objects can be degraded by IR without changes to the spatial resolution of high-contrast objects; the amount of degradation is determined by the desired level of noise reduction (2). Thus, the dose reduction potential of IR is highly dependent on the diagnostic task. For diagnostic tasks involving high-contrast objects, such as bony anatomy or relatively large vessels containing iodinated contrast agents, substantial noise reduction is possible without compromising diagnostic performance (3). This ability to substantially reduce image noise allows for marked dose reduction (3). However, for diagnostic tasks involving low-contrast objects, such as liver lesions or hypoattenuated regions of the brain secondary to stroke, it is critical to determine how much low-contrast spatial resolution (LCR) is affected by IR, such that as dose is reduced, the noise reduction caused by IR does not compromise the ability to detect and characterize low-contrast objects.
A familiar example of the assessment of LCR is the LCR test of the American College of Radiology (ACR) CT Accreditation Program (4). The program requires submission of images of the LCR test pattern that have been acquired and reconstructed by using protocol parameters for the relevant clinical examinations. The passing criteria that were established early in the program were based on the LCR performance of generally accepted protocols for routine brain and abdomen scanning (5). The minimum performance level required that all four 6-mm rods were deemed to be visible by the physicist reviewer. This ensured that practices receiving ACR CT accreditation achieved a minimal, albeit somewhat subjective, level of LCR as directly determined by human observers. To facilitate more objective review of submitted phantom images, the ACR CT Accreditation Program recently changed its LCR criterion from requiring that the reviewer be able to visualize all four 6-mm rods to requiring that the contrast-to-noise ratio measured in the 25-mm rod be greater than 1.0. There is evidence, however, that the use of contrast-to-noise ratio is an inadequate measure of LCR when IR techniques are used (6,7).
As IR-equipped scanners were added to our institution’s practice, we noticed degraded LCR quality testing performance for reduced-dose images reconstructed with IR relative to the performance measured by using full-dose filtered back projection (FBP) images, even though the same measurement and evaluation methods were used for FBP and IR images. Others have also reported degraded LCR performance with use of IR and decreased radiation doses (8–10). It is not clear, however, which IR parameter settings and dose reduction amounts, if any, can be used without sacrificing LCR. Thus, the purpose of this work was to determine the dose reduction that could be achieved without degrading LCR performance for two commercial IR techniques, each evaluated at two strengths with a large number of repeated scans.
Materials and Methods
Data Acquisition and Image Reconstruction
Image data were acquired by using two scanner models from two different manufacturers: a Lightspeed VCT scanner (GE Healthcare, Waukesha, Wis) and a Somatom Definition Flash scanner (Siemens Healthcare, Forchheim, Germany) (Table 1). The latter was used in a single-source mode. Scans were performed in the LCR section of the ACR CT accreditation phantom (Fig 1) (Gammex 464; Gammex, Middleton, Wis) at three different volume CT dose index (CTDIvol) values: 16, 12, and 8 mGy. Three different reconstructions were performed for each acquisition, two of which were IR techniques. For the Lightspeed VCT scanner, the default FBP reconstruction technique and the standard (or STD) reconstruction algorithm were used. Additionally, the Adaptive Statistical Iterative Reconstruction (ASIR; GE Healthcare) technique was used to reconstruct IR images. ASIR reconstruction was performed at moderate and strong settings (50% and 100% ASIR, respectively). For the Definition Flash scanner, reconstructions were completed by using Siemens’ FBP and Sinogram AFfirmed Iterative Reconstruction (SAFIRE; Siemens Healthcare) techniques. The FBP reconstruction kernel used was B40. Two SAFIRE reconstructions were performed by using the I40 kernel, which has a high-contrast spatial resolution equivalent to the B40 kernel at moderate and strong settings (strength of 3 and 5, respectively). All images were reconstructed with an image thickness of 5 mm.
Table 1.
Reconstruction Settings and Dose Levels Used in This Study
Note.—A total of 100 images were acquired by using each combination of IR setting and dose level for each scanner. STD = standard.
*Physical detector collimation was 64 × 0.6 mm. Double sampling along the z direction was achieved with a flying focal spot.
†CTDIvol, as displayed on scanner console.
Figure 1:
CT image of the LCR section of the ACR CT Accreditation Program phantom acquired with a commercial CT scanner.
For each of the two scanner models, data acquisition and image reconstruction were repeated 100 times for each of the three dose levels and three reconstruction settings, yielding a total of 1800 images. A total of 100 repeated studies were acquired and reviewed for each reconstruction and dose configuration to reduce the effect of statistical variation. That is, the appearance of a low-contrast object can differ substantially from one image to another due to statistical variation (Fig 2), causing two images acquired with two separate scans but with the exact same scanning and reconstruction settings to yield very different LCRs.
Figure 2:
Example images show the effect of statistical variation on LCR. These images were acquired with two separate scans but the exact same scanning parameters and reconstruction settings. The 6-mm rods are clearly seen on the left image but not on the right image. Display window width and window level are 100 HU and 100 HU, respectively, as required by the ACR CT Accreditation Program.
Image Quality Evaluation
LCR images were reviewed independently in a randomized and blinded fashion by three diagnostic medical physicists with 6, 7, and 10 years of experience in reviewing ACR CT accreditation images (S.L., L.Y., and J.M.K., respectively). The following six-point scale was used to describe the ability of each reviewer to resolve the four 6-mm rods in each image: 1, definitely not visible; 2, not visible, a few rods barely visible; 3, some but not all rods visible; 4, all rods slightly visible; 5, all rods reasonably visible; and 6, all rods definitely visible. Image review was performed in a darkened room with a calibrated monitor. Images were presented in 18 sets of 100 randomized images, with an automated software tool used to record each score. The viewing screen was briefly turned black between each image to avoid persistence of the image on the reader’s retina, and each of the three or four viewing sessions was limited to 2 hours to prevent reviewer fatigue.
Statistical Analysis
The goal of the analysis was to provide estimates of the change in reviewer scores for the altered dose and reconstruction configurations relative to the reference image (FBP, 16 mGy). The primary end point was a composite mean reviewer score, which was calculated by averaging the scores from the three reviewers for each specific combination of dose level and reconstruction technique. The differences in composite scores, along with their associated 95% confidence intervals (CIs), were estimated by using contrasts obtained from a one-way analysis of variance model. A sensitivity analysis was conducted by using 99.4% (ie, 1–0.05/8) CIs to account for the eight comparisons performed within each vendor.
The principles of noninferiority analysis were used to guide the interpretation of the findings (11). The limit of noninferiority was determined before we analyzed the data and was set to be 25% of the hypothesized standard deviation of the composite score based on the plausible range of data. Specifically, the limit of noninferiority was set at 0.25 (ie, 0.25·(6–1)/5). For interpretations of noninferiority, the lower limit of the 95% CI had to be greater than the limit of noninferiority. If this did not occur, the data were considered significantly inconclusive from the noninferiority perspective. A secondary interpretation of the CIs was also possible: If the upper limit of the 95% CI was less than 0, the standard statistical test (two-sided superiority test) would lead to a rejection of the null hypothesis of equality in favor of the alternative hypothesis, which was that the altered imaging conditions were significantly inferior to the reference configuration. Similarly, if the lower limit of the 95% CI was greater than 0, the standard statistical test (two-sided superiority test) would lead us to reject the null hypothesis of equality in favor of the alternative hypothesis, which was that the altered imaging conditions were significantly superior to the reference configuration.
All analyses were conducted by using the SAS system (SAS, version 9.3; SAS Institute, Cary, NC).
Results
Figures 3 and 4 show LCR images at each dose and reconstruction setting for vendors 1 and 2, respectively. Images were from one of the 100 repeated studies. The mean pooled reader scores showed a general decrease in LCR with decreasing dose (Fig 5).
Figure 3:
Example images acquired with the Lightspeed VCT scanner and FBP, moderate IR, and strong IR reconstructions. Images were from one of the 100 repeated scans at each of the three dose levels. Display window width and window level are 100 HU and 100 HU, respectively, as required by the ACR CT Accreditation Program. Dark circular regions visible in the center of the phantom and behind the low-contrast rods are typical of the appearance of this phantom on GE Healthcare scanners and persisted even though all manufacturer CT number and uniformity tolerances were met.
Figure 4:
Example images acquired with a Definition Flash scanner and FBP, moderate IR, and strong IR reconstructions. Images were from one of the 100 repeated scans at each of the three dose levels. Display window width and window level are 100 HU and 100 HU, respectively, as required by the ACR CT Accreditation Program.
Figure 5a:
Distribution of mean reader scores for image acquisitions from (a) vendor 1 and (b) vendor 2.
Figure 5b:
Distribution of mean reader scores for image acquisitions from (a) vendor 1 and (b) vendor 2.
For vendor 1, the moderate and strong IR reconstructions were shown to be noninferior to the FBP reconstruction at a dose level of 16 mGy, with differences of 0.307 (95% CI: 0.149, 0.464) and 0.333 (95% CI: 0.176, 0.491), respectively (Fig 6). Noninferiority was not shown for IR or FBP reconstructions at any other dose level.
Figure 6a:
The results of noninferiority analysis for mean human observer score for reconstructions from (a) vendor 1 and (b) vendor 2. The reference in the noninferiority analysis for the Lightspeed VCT scanner was an FBP reconstruction acquired at 16 mGy. The reference for the Definition Flash scanner was an FBP reconstruction acquired at 16 mGy. 95% CIs are shown. Interpretations of the 95% and 99.4% CIs are shown in Tables 2 and 3.
For vendor 2, the moderate and strong IR reconstructions were shown to be noninferior at dose levels of 16 and 12 mGy (Fig 6b). The difference for the moderate IR setting was 0.093 (95% CI: −0.062, 0.248) at 12 mGy and 0.670 (95% CI: 0.515, 0.825) at 16 mGy. The difference for the strong IR setting was 0.478 (95% CI: 0.323, 0.633) at 12 mGy and 0.977 (95% CI: 0.822, 1.132) at 16 mGy.
Figure 6b:
The results of noninferiority analysis for mean human observer score for reconstructions from (a) vendor 1 and (b) vendor 2. The reference in the noninferiority analysis for the Lightspeed VCT scanner was an FBP reconstruction acquired at 16 mGy. The reference for the Definition Flash scanner was an FBP reconstruction acquired at 16 mGy. 95% CIs are shown. Interpretations of the 95% and 99.4% CIs are shown in Tables 2 and 3.
Relative to a 16-mGy CTDIvol acquisition and FBP reconstruction, 25%–50% dose reductions resulted in inferior LCR for both vendors 1 and 2 for FBP. Relative to FBP and full dose, 25% dose reductions resulted in inferior and equivalent performances for vendor 1 at moderate and strong IR settings, respectively, and equivalent and superior performance for vendor 2 at moderate and strong IR settings, respectively (Tables 2, 3). When dose reduction was 50% (8 mGy), both IR strengths resulted in inferior LCR for both vendors.
Table 2.
Summary of Noninferiority and Superiority-Inferiority Analyses for 95% and 99.4% CIs for the LightSpeed VCT Scanner
*CTDIvol.
Table 3.
Summary of Noninferiority and Superiority-Inferiority Analyses for 95% and 99.4% CIs for the Definition Flash Scanner
*CTDIvol.
†With the multiplicity adjusted 99.4% CI, this result changed to inferiority/superiority not shown.
For all comparisons, the sensitivity analyses resulted in wider CIs. In spite of these wider CIs, the same qualitative conclusions were reached, with one exception; for vendor 2 at 8 mGy, the strong IR reconstruction was no longer significantly inferior. The upper limit of the 99.4% CI for this configuration increased from −0.048 to 0.013.
Discussion
In a number of clinical evaluations of IR, researchers have concluded that use of IR facilitates radiation dose reduction, without a detrimental effect on image quality (12–15). Most of these studies have assessed image quality in a semiquantitative fashion based on acceptability of the image to individual readers. Such studies are strongly influenced by individual reader training and preference, particularly in terms of the noise level and image texture deemed acceptable. In our study, we used objective criteria for image quality, with a primary focus on LCR. Specifically, we evaluated the ability of human observers to resolve the four 6-mm rods in images of the ACR CT accreditation phantom. By using this specific criterion as our measure of acceptable LCR, we found that for vendor 1 and at both IR strengths, images with noninferior LCR were not produced at either a 25% or a 50% reduction in CTDIvol. For vendor 2, a 25% reduction in CTDIvol (from 16 mGy to 12 mGy) did produce images with noninferior LCR relative to the full-dose FBP images. Of note, superiority was shown for both IR techniques at both strength settings at the full-dose level.
The results of our study may have important clinical implications. In spite of the more pleasing low-noise appearance of IR images, LCR may be degraded with reduced-dose IR images relative to that achieved with full dose levels. It is imperative that the ability to detect and characterize low-contrast lesions not be compromised in the pursuit of reduced radiation dose levels. Until sufficient studies have been performed to show that clinically relevant diagnostic performance for low-contrast imaging tasks is maintained or improved at reduced dose levels, use of IR techniques to decrease the appearance of image noise may increase the likelihood of missing clinically important findings, particularly those that are subtle in appearance. To address this concern, we recommend that practices perform a pilot study, similar to but smaller than the study described herein, and evaluate the images in both a blinded and side-by-side manner. If the ability to resolve the four 6-mm rods appears to decline with use of reduced-dose images either with or without use of IR, we recommend that the routine dose level be used for examinations performed to evaluate small and subtle structures or abnormalities. Additionally, when purchasing a new scanner, evaluation of the IR technique with the ACR phantom and this study design should be performed in conjunction with a qualified medical physicist, as the results of this study show that differences in performance can exist between manufacturers.
This study was limited in several aspects. First, diagnostic performance was assessed for a somewhat artificial task, namely visualization of all four 6-mm low-contrast rods against an otherwise uniform background. Subtle low-contrast lesions would be expected to be more difficult to visualize in a structured background, as occurs in patient images. Thus, the noninferiority and superiority conclusions for the 25% dose reduction and IR technique of Siemens Healthcare may be overly optimistic when compared with some clinical tasks. Second, only two IR algorithms were assessed. A recent article on low-contrast performance in clinical studies for a third manufacturer (Toshiba Medical Systems) similarly concluded that LCR performance is degraded at reduced dose levels with IR techniques (9). Thus, while we have not assessed all manufacturers or algorithms, it appears that the current findings may not be unique to the two manufacturers evaluated in our article. Also, newer techniques from each of the evaluated manufacturers are now available and may yield improved low-contrast performance related to what was shown in this article. However, the number of CT scanners in use in clinical practice that are equipped with the two algorithms evaluated here is large, making the findings relevant to a large number of practices. Finally, although the findings of our study raise important questions regarding the ability of IR to preserve LCR at reduced dose levels, the clinical effect of our findings needs to be assessed for various diagnostic tasks by using clinical images in a conventional multireader multicase study design.
In summary, LCR performance of CT images reconstructed with IR techniques can be substantially different than that when FBP techniques are used, particularly as dose levels are reduced. With current commercially available IR techniques, radiation dose reductions of 25%–50% can reduce the LCR relative to that achieved by using the full dose and FBP. The improved appearance of the images, in terms of decreased noise levels, can conceal this fact, as can quantitative measurements, such as contrast-to-noise ratios (8). This is due to the nonlinear relationship between image noise and spatial resolution for IR techniques. With FBP, the spatial resolution is the same, regardless of the contrast of the object. Thus, LCR is improved as noise is decreased. However, with IR, the spatial resolution changes as a function of object contrast (2). For objects with very low contrast, such as resolution of the four 6-mm rods evaluated in this work, the well-defined edges of the low-contrast rods become more defuse when IR is used, making the rods more difficult to resolve, even when the overall image noise is decreased by the IR algorithm. It is important, therefore, that practices exercise caution when implementing IR techniques and radiation dose reduction in clinical practice for diagnostic tasks involving low-contrast objects.
Advances in Knowledge
■ Dose reductions of 25%–50% degraded the low-contrast spatial resolution compared with what was obtained with filtered back projection reconstruction at full dose levels, even with use of iterative reconstruction techniques.
■ For radiation dose reductions of 25%–50%, the ability to visualize all four 6-mm low-contrast rods in the American College of Radiology CT accreditation phantom can be lost.
Implications for Patient Care
■ For diagnostic tasks requiring the ability to detect and resolve subtle low-contrast lesions, the use of iterative reconstruction techniques at decreased radiation dose levels may degrade diagnostic performance.
■ The use of iterative reconstruction techniques to compensate for decreased dose should be adopted with caution for clinical examinations requiring excellent low-contrast spatial resolution, such as unenhanced brain CT in the evaluation of possible stroke, or contrast-enhanced abdominal CT in the detection of liver metastases.
Received August 26, 2014; revision requested October 6; revision received October 28; accepted December 15; final version accepted January 12, 2015.
Funding: This research was supported by the National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health (grants EB017095 and EB017185).
The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Abbreviations:
- ACR
- American College of Radiology
- CI
- confidence interval
- CTDIvol
- volume CT dose index
- FBP
- filtered back projection
- IR
- Iterative reconstruction
- LCR
- low-contrast spatial resolution
See also the article by Fletcher et al in this issue.
Disclosures of Conflicts of Interest: C.H.M. Activities related to the present article: none to disclose. Activities not related to the present article: received a grant from Siemens Healthcare. Other relationships: none to disclose. L.Y. disclosed no relevant relationships. J.M.K. disclosed no relevant relationships. S.L. disclosed no relevant relationships. Y.Z. disclosed no relevant relationships. Z.L. disclosed no relevant relationships. R.E.C. disclosed no relevant relationships.
References
- 1.Thibault JB, Sauer KD, Bouman CA, Hsieh J. A three-dimensional statistical approach to improved image quality for multislice helical CT. Med Phys 2007;34(11):4526–4544. [DOI] [PubMed] [Google Scholar]
- 2.Yu L, Vrieze TJ, Leng S, Kofler J, Fletcher JG, McCollough CH. Measuring the contrast-dependent spatial resolution of CT iterative reconstruction methods at very low contrast values [abstr]. In: Radiological Society of North America Scientific Assembly and Annual Meeting Program. Oak Brook, Ill: Radiological Society of North America, 2013; 212. [Google Scholar]
- 3.Ehman EC, Yu L, Manduca A, et al. Methods for clinical evaluation of noise reduction techniques in abdominopelvic CT. RadioGraphics 2014;34(4):849–862. [DOI] [PubMed] [Google Scholar]
- 4.CT accreditation phantom instructions . American College of Radiology. http://www.acr.org/∼/media/ACR/Documents/Accreditation/CT/PhantomTestingInstruction.pdf. Published May 23, 2013. Accessed May 7, 2014.
- 5.McCollough CH, Bruesewitz MR, McNitt-Gray MF, et al. The phantom portion of the American College of Radiology (ACR) computed tomography (CT) accreditation program: practical tips, artifact examples, and pitfalls to avoid. Med Phys 2004;31(9):2423–2442. [DOI] [PubMed] [Google Scholar]
- 6.Kofler JM, Jr, Yu L, Leng S, Zhang Y, Li Z, McCollough CH. Iterative reconstructions and low contrast resolution measurements: are ACR accreditation threshold values still valid? [abstr]. In: Radiological Society of North America Scientific Assembly and Annual Meeting Program. Oak Brook, Ill: Radiological Society of North America, 2013; 201. [Google Scholar]
- 7.Yu L, Kofler JM, Jr, Leng S, et al. CT radiation dose reduction and iterative reconstruction (IR): how low can you go and still achieve ACR CT accreditation [abstr]. In: Radiological Society of North America Scientific Assembly and Annual Meeting Program. Oak Brook, Ill: Radiological Society of North America, 2012; 215. [Google Scholar]
- 8.Baker ME, Dong F, Primak A, et al. Contrast-to-noise ratio and low-contrast object resolution on full- and low-dose MDCT: SAFIRE versus filtered back projection in a low-contrast object phantom and in the liver. AJR Am J Roentgenol 2012;199(1):8–18. [DOI] [PubMed] [Google Scholar]
- 9.Schindera ST, Odedra D, Raza SA, et al. Iterative reconstruction algorithm for CT: can radiation dose be decreased while low-contrast detectability is preserved? Radiology 2013;269(2):511–518. [DOI] [PubMed] [Google Scholar]
- 10.Goenka AH, Herts BR, Obuchowski NA, et al. Effect of reduced radiation exposure and iterative reconstruction on detection of low-contrast low-attenuation lesions in an anthropomorphic liver phantom: an 18-reader study. Radiology 2014;272(1):154–163. [DOI] [PubMed] [Google Scholar]
- 11.Evaluation of Medicines for Human Use . Points to consider on switching between superiority and non-inferiority (CPMP/EWP/482/99). London, UK. The European Agency for the Evaluation of Medicinal Products. http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500003658.pdf. Published July 27, 2000. Accessed May 7, 2014. [Google Scholar]
- 12.Pickhardt PJ, Lubner MG, Kim DH, et al. Abdominal CT with model-based iterative reconstruction (MBIR): initial results of a prospective trial comparing ultralow-dose with standard-dose imaging. AJR Am J Roentgenol 2012;199(6):1266–1274. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Hara AK, Paden RG, Silva AC, Kujak JL, Lawder HJ, Pavlicek W. Iterative reconstruction technique for reducing body radiation dose at CT: feasibility study. AJR Am J Roentgenol 2009;193(3):764–771. [DOI] [PubMed] [Google Scholar]
- 14.Singh S, Kalra MK, Hsieh J, et al. Abdominal CT: comparison of adaptive statistical iterative and filtered back projection reconstruction techniques. Radiology 2010;257(2):373–383. [DOI] [PubMed] [Google Scholar]
- 15.Kalra MK, Woisetschläger M, Dahlström N, et al. Radiation dose reduction with Sinogram Affirmed Iterative Reconstruction technique for abdominal computed tomography. J Comput Assist Tomogr 2012;36(3):339–346. [DOI] [PubMed] [Google Scholar]