Superior objective and subjective image quality of deep learning reconstruction for low-dose abdominal CT imaging in comparison with model-based iterative reconstruction and filtered back projection

Abstract

Objective:

This study aimed to conduct objective and subjective comparisons of image quality among abdominal computed tomography (CT) reconstructions with deep learning reconstruction (DLR) algorithms, model-based iterative reconstruction (MBIR), and filtered back projection (FBP).

Methods:

Datasets from consecutive patients who underwent low-dose liver CT were retrospectively identified. Images were reconstructed using DLR, MBIR, and FBP. Mean image noise and contrast-to-noise ratio (CNR) were calculated, and noise, artifacts, sharpness, and overall image quality were subjectively assessed. Dunnett’s test was used for statistical comparisons.

Results:

Ninety patients (67 ± 12.7 years; 63 males; mean body mass index [BMI], 25.5 kg/m²) were included. The mean noise in the abdominal aorta and hepatic parenchyma of DLR was lower than that in FBP and MBIR (p < .001). For FBP and MBIR, image noise was significantly higher for obese patients than for those with normal BMI. The CNR for the abdominal aorta and hepatic parenchyma was higher for DLR than for FBP and MBIR (p < .001). MBIR images were subjectively rated as superior to FBP images in terms of noise, artifacts, sharpness, and overall quality (p < .001). DLR images were rated as superior to MBIR images in terms of noise (p < .001) and overall quality (p = .03).

Conclusions:

Based on objective and subjective comparisons, the image quality of DLR was found to be superior to that of MBIR and FBP on low-dose abdominal CT. DLR was the only method for which image noise was not higher for obese patients than for those with a normal BMI.

Advances in knowledge:

This study provides previously unavailable information on the properties of DLR systems and their clinical utility.

Introduction

CT imaging has considerable diagnostic utility; however, due to the widespread use of CT and the development of advanced CT systems that enable scanning over wide areas and multiphase imaging, concerns remain regarding the increasing exposure of patients to radiation.^1,2 According to the Biologic Effects of Ionizing Radiation VII report, CT scanning can increase the lifetime risk of cancer by 0.34 to 1.30%, depending on age, sex, and site of examination.³ In response to concerns about radiation exposure during CT scanning, the radiological community has been working to develop low-dose imaging techniques. This has been challenging because with filtered back projection (FBP), which is a widely used method for CT image reconstruction, image noise increases as the radiation dose decreases, making it difficult to assess structures in detail.

To solve this problem, CT vendors are developing technologies to improve image quality at a lower radiation dose.^4–14 Iterative reconstruction techniques are classified as hybrid and model based. While the former features more widespread use, its performance can still be improved—especially when compared against the superior image noise, image texture, and spatial resolution yielded by model-based iterative reconstruction (MBIR). Although time-consuming when reconstructing images, MBIR is not an analytical-based, simple solution like the filtered back projection, which has a very high spatial resolution and a high capacity for noise reduction.^8,9,15 Importantly, this technique enables images of diagnostic quality to be generated at a low radiation dose; MBIR supports a reduction in radiation exposure of 22–70% in abdominal CT imaging.^9,13 However, a disadvantage of MBIR is that image reconstruction is a time-consuming process.

In recent years, artificial intelligence, especially deep learning, has attracted considerable attention, and multiple applications have been developed across various fields. One of the most advanced applications of deep learning is the improvement of image quality through noise reduction.^16–18 In the field of CT image reconstruction, new deep learning reconstruction (DLR) algorithms have recently been introduced. The Advanced Intelligent Clear-IQ Engine (AiCE), the first commercially available DLR tool developed for CT, uses deep convolutional neural networks to distinguish the true signal from the noise in the image.¹⁹ For this deep learning approach, a high-dose MBIR image and image data corresponding to 12.5–75% of the dose of the target image are provided as training pairs, and statistical features that distinguish the signal from noise and artifacts are learned in the training process.²⁰ While DLR has great potential, there are still insufficient data available on the properties of DLR systems and their clinical utility. The purpose of this study was to perform objective and subjective comparisons of the use of FBP, vendor-specific MBIR (Forward Projected Model-Based Iterative Reconstruction Solution [FIRST]), and DLR (AiCE) systems for the reconstruction of low-dose abdominal CT scans.

Methods

Patients and study design

Patients were included in the study if they were aged 18 years or older. Exclusion criteria were contraindications to iodinated contrast media, emergency cases, and hemodialysis or renal failure. The study design was approved by the Institutional Review Board, and the requirement for informed consent was waived due to the retrospective nature of the study.

CT technique

All scans were performed using a 320 row multidetector CT scanner (Aquilion ONE PRISM; Canon Medical Systems, Otawara, Japan) with the following parameters: 80 row × 0.5 mm, pitch of 0.813, 0.6 s rotation time (fixed), 120 kVp tube voltage, and automatic exposure control with a noise index of 20. The standard care of abdominal CT in our hospital is the MBIR protocol (FIRST BODY Mild), which is designed for low exposure, with a mean volume CT dose index (CTDIvol) of 7.9 mGy. This value was obtained by retrospectively evaluating the doses of 125 consecutive patients prior to the start of this study. Iohexol (Omnipaque 300; Daiichi-Sankyo, Tokyo, Japan), iomeprol (Iomeron 350; Eisai, Tokyo, Japan), or iopamidol (Iopamiron 370; Nihon Schering, Osaka, Japan) was administered via an antecubital vein using a 20-gauge catheter at a dose of 600 mgI/kg; the injection procedure was standard across all patients. The scan delays for arterial and portal venous phase imaging were determined using an automatic bolus-tracking program (Canon Medical Systems). The region of interest (ROI) cursor was placed in the aorta at the level of the diaphragmatic dome, and scanning for the arterial and portal venous phases began automatically at 20 and 60 s, respectively, after the trigger threshold of 100 Hounsfield Units was reached.

CT image reconstruction

Images were reconstructed from raw data using FBP (FC13), MBIR (FIRST BODY Mild; reconstruction time, 7 min), and DLR (AiCE BODY Mild; reconstruction time, 1 min) algorithms. Reconstructed images were created with soft tissue window settings, with a 3-mm slice thickness and interval. FBP, FIRST, and AiCE images were obtained for each of the 90 patients.

Analysis of image quality

Objective assessment: One CT technologist (M.K., with 30 years’ experience in abdominal CT) measured the mean attenuation of the abdominal aorta, hepatic parenchyma, and bilateral erector spinae muscles within circular ROIs for each patient. The area of each ROI was set at approximately 150 ± 25 mm². We standardized the size of the ROIs using the copy-and-paste function of the workstation (Ziostation2; Ziosoft, Tokyo, Japan). Aortic attenuation in the arterial phase was measured immediately proximal to the level of the celiac trunk. Hepatic attenuation was measured in three separate areas (the left lobe and the anterior and posterior segments of the right lobe) at the level of the main portal vein in the portal venous phase. Areas that included focal changes in parenchymal density, large vessels, and prominent artifacts were avoided. Attenuation of the two erector spinae muscles was measured in the arterial and portal venous phases, and areas of macroscopic fat infiltration were avoided. Image noise was defined as the standard deviation of the attenuation value of the abdominal aorta and hepatic parenchyma. The contrast-to-noise ratio (CNR) of the abdominal aorta was calculated by subtracting the mean attenuation of the ROI of the erector spinae muscle from that of the ROI of the aorta and dividing this difference by the image noise of the erector spinae muscle. The CNR of the hepatic parenchyma was calculated similarly, using the ROI of the liver rather than the aorta (Figure 1).

Figure 1.

Axial contrast-enhanced CT images obtained in a 57-year-old male during the arterial phase (a) show regions of interest (ROIs) manually drawn on aorta and bilateral erector spinae muscles; the portal phase (b) shows ROI drawn on liver and bilateral erector spinae muscles.

To evaluate the modulation transfer function (MTF), image data were acquired from the CTP404 module of the Catphan 504 phantom (The Phantom Laboratory, Inc., Salem, NY) using a sensitometry target of Delrin^®, a setting of 80 × 0.5 mm collimation, pitch of 0.813, 0.6 s rotation time, and 120 kVp/60 mA. Images reconstructed with FBP (FC13), FIRST BODY Mild, and AiCE BODY Mild, were acquired with a 3-mm slice thickness and interval for measuring the MTF. The MTF was calculated for each dataset via the radial edge method using a CT measure software (version 0.97b; Japanese Society of CT Technology, Hiroshima, Japan).

Subjective assessment: The three image sets were presented randomly to two blinded radiologists (M.N. and A.T.: 4 and 13 years’ experience in abdominal CT, respectively). CT images were viewed with standard abdominal window settings (width, 300 Hounsfield Units; level, 50 Hounsfield Units). The radiologists independently graded the quality of portal phase axial CT images using a 5-point scale for image noise and artifacts, sharpness, and overall image quality, as follows: Image noise and artifacts: 1, unacceptable; 2, major; 3, minor; 4, very little; and 5, none. Sharpness, defined as the clarity of the liver contour: 1, blurry; 2, poorer than average; 3, average; 4, better than average; and 5, very sharp. Overall image quality: 1, unacceptable; 2, suboptimal; 3, average; 4, above average; and 5, excellent. Before image quality assessment, a training session including abdominal CT images from 10 patients was provided to familiarize the radiologists with the scoring system.

Statistical analysis

All numerical values are reported as mean ± standard deviation. We used Dunnett’s test to compare quantitative and qualitative image analysis data. To evaluate the association between BMI and image noise or CNR in the portal phase, we categorized patients into four subgroups based on the World Health Organization classification²¹: BMI <18.5 kg/m², underweight (n = 1); 18.5 kg/m² ≤ BMI<25 kg/m², normal range (n = 45); 25 kg/m² ≤ BMI<30 kg/m², pre-obese (n = 33); and ≥30 kg/m², obese (n = 11). The weighted κ score was calculated to determine the interobserver agreement and was classified as follows: <0.20, slight; 0.21–0.40, fair; 0.41–0.60, moderate; 0.61–0.80, substantial; and 0.81–1.00, almost perfect or perfect agreement.²² Statistical analysis was performed using the SPSS software (version 24; IBM Corp., Armonk, NY). A <i>p value < 0.05 was considered statistically significant.

Results

Patients

Raw CT image data were obtained from 90 patients (63 males; age range, 19–89 years; mean age, 67 years; body weight range, 41–155 kg; mean body weight, 68.4 kg; height range, 145–185 cm; mean height, 163.5 cm; body mass index [BMI] range, 17.8–51.8 kg/m²; mean BMI, 25.5 kg/m²) who underwent liver CT (volume CT dose index range, 3.1–29.4 mGy; mean volume CT dose index, 8.2 mGy; dose-length product range, 78–1065 mGy-cm; mean dose-length product, 247.9 mGy-cm).

Objective assessment of image quality

The mean attenuation, image noise, and CNR for images reconstructed with each algorithm are presented in Table 1. There were no significant differences in the mean attenuation values measured in the aorta and hepatic parenchyma among the reconstruction algorithms (p = .90–.97). However, noise levels were significantly different among the three algorithms. Overall, the mean image noise measured in the abdominal aorta and hepatic parenchyma for AiCE images (11.1 ± 2.5 and 10.0 ± 0.9, respectively) was significantly lower than for FBP images (27.7 ± 5.0 and 23.1 ± 2.6, respectively) and FIRST images (16.9 ± 2.9 and 14.7 ± 1.6, respectively) (p < .001 for all comparisons). The CNR for the abdominal aorta and hepatic parenchyma was significantly higher for AiCE images (26.6 ± 6.7 and 4.0 ± 1.6, respectively) than for FBP (13.3 ± 3.0 and 2.1 ± 0.8, respectively) and FIRST images (18.9 ± 4.9 and 2.9 ± 1.1, respectively) (p < .001 for all comparisons). Table 2 shows the mean image noise and CNR for the three reconstruction algorithms when patients were classified according to the BMI group. Since there was only one underweight patient, we opted not to include this patient’s data in the statistical analysis. For both FBP and FIRST, image noise was significantly higher in obese patients than in those with normal BMI (p = .001 and p = .008, respectively), but this effect was not observed for AiCE (p = .13). For all three reconstruction methods, the CNR was significantly lower in obese than in pre-obese patients, and in the pre-obese group than in the normal BMI group. MTF data for each algorithm are presented in Figure 2. The spatial resolution of images reconstructed with FIRST and AiCE was consistently higher than that of those reconstructed with FBP.

Table 1.

Quantitative assessment of image quality for the sets of images reconstructed using FBP, FIRST, and AiCE

Parameter	FBP	FIRST	AiCE	FBP vs FIRST p value	FIRST vs AiCE p value
Mean CT number
Aorta	340.1 ± 55.1	337.8 ± 55.5	336.1 ± 56.2	0.90	0.97
Hepatic parenchyma	105.1 ± 16.4	104.6 ± 16.4	103.8 ± 16.3	0.95	0.96
Image noise
Aorta	27.7 ± 5.0	16.9 ± 2.9	11.1 ± 2.5	<0.001	<0.001
Hepatic parenchyma	23.1 ± 2.6	14.7 ± 1.6	10.0 ± 0.9	<0.001	<0.001
CNR
Aorta	13.3 ± 3.0	18.9 ± 4.9	26.6 ± 6.7	<0.001	<0.001
Hepatic parenchyma	2.1 ± 0.8	2.9 ± 1.1	4.0 ± 1.6	<0.001	<0.001

AiCE, Advanced intelligent clear-IQ engine; CNR, Contrast-to-noise ratio; FBP, Filtered back projection; FIRST, Forward projected model-based iterative reconstruction solution.

Data are presented as mean values ± standard deviation. Dunnett’s test was used for statistical comparisons. The mean volume CT dose index was 8.2 mGy.

Table 2.

Variation in image noise and CNR in the hepatic parenchyma with respect to BMI category

Parameter	Normal BMI (n = 45)	Pre-obese (n = 33)	Obese (n = 11)	Obese vs normal p value	Obese vs pre-obese p value
Image noise
FBP	22.5 ± 1.9	23.2 ± 1.5	25.6 ± 5.5	0.001	0.01
FIRST	14.3 ± 1.5	14.9 ± 1.5	15.8 ± 1.5	0.008	0.16
AiCE	9.9 ± 0.8	10.0 ± 0.8	10.4 ± 1.2	0.13	0.18
CNR
FBP	2.2 ± 0.7	2.2 ± 0.8	1.4 ± 0.8	0.002	0.002
FIRST	3.1 ± 0.9	3.0 ± 1.1	1.7 ± 1.3	<0.001	0.001
AiCE	4.3 ± 1.2	4.0 ± 1.6	2.4 ± 2.0	<0.001	0.006

AiCE, Advanced intelligent clear-IQ engine; BMI, body mass index; CNR, contrast-to-noise ratio; FBP, filtered back projection; FIRST, Forward projected model-based Iterative reconstruction solution.

Data are presented as mean values ± standard deviation. Dunnett’s test was used for statistical comparisons. The mean volume CT dose index was 5.4 mGy, 9.4 mGy, and 16.5 mGy for the normal BMI, pre-obese, and obese groups, respectively.

Figure 2.

Modulation transfer functions for CT images reconstructed with FBP, FIRST, and AiCE. Compared with FBP images, FIRST and AiCE images had a higher spatial resolution. AiCE, Advanced intelligent clear-IQ engine; FBP, Filtered back projection; FIRST, Forward projected model-based iterative reconstruction solution.

Subjective assessment of image quality

Table 3 shows the results of the subjective image quality assessments. On the 5-point scales, FIRST images received significantly higher scores than FBP images regarding noise, artifacts, sharpness, and overall image quality (p < .001). AiCE images received significantly higher scores than FIRST images with respect to noise (p < .001) and overall image quality (p = .03). Figure 3 shows an example of the superior noise reduction with AiCE reconstruction. In contrast to the noise and overall quality ratings, artifact scores were significantly higher for FIRST images than for AiCE images (p < .001). In one instance only, both readers scored the FBP image higher than the corresponding FIRST image in terms of artifacts (Figure 4). There was no significant difference between FIRST and AiCE scores in terms of the sharpness of the liver contour. Between the two readers, there was substantial to almost perfect interobserver agreement for all subjective image quality metrics with κ = 0.86, 0.90, 0.81, and 0.88 for noise, artifacts, sharpness, and overall image quality, respectively.

Table 3.

Subjective assessment of image quality for the sets of images reconstructed using FBP, FIRST, and AiCE

Parameter	FBP	FIRST	AiCE	FBP vs FIRST p value	FIRST vs AiCE p value
Noise	2.5 ± 0.8	3.5 ± 0.7	4.0 ± 0.6	<0.001	<0.001
Artifacts	2.4 ± 0.6	4.3 ± 0.6	3.8 ± 0.5	<0.001	<0.001
Sharpness	2.6 ± 0.7	3.9 ± 0.7	4.0 ± 0.6	<0.001	0.60
Overall image quality	2.5 ± 0.6	3.8 ± 0.6	3.9 ± 0.5	<0.001	0.03

AiCE, Advanced intelligent clear-IQ engine; FBP, Filtered back projection; FIRST, Forward projected model-based iterative reconstruction solution.

Data are presented as mean values ± standard deviation. Dunnett’s test was used for statistical comparisons. κ = 0.86, 0.90, 0.81, and 0.88 for noise, artifacts, sharpness, and overall image quality, respectively.

Figure 3.

Axial contrast-enhanced abdominal CT images obtained in a 38-year-old female with liver metastases from gastric cancer. Images were obtained with a preset soft tissue window (width, 300 HU; level, 50 HU). Differences in image quality can be observed between (a) FBP, (b) FIRST, and (c) AiCE reconstructions of the same anatomical location. The noise level is appreciably lower in (c). AiCE, Advanced intelligent clear-IQ engine; FBP, Filtered back projection; FIRST, Forward projected model-based iterative reconstruction solution.

Figure 4.

A 42-year-old male with chronic hepatitis. Axial contrast-enhanced abdominal CT images of the same anatomical location were reconstructed with (a) FBP, (b) FIRST, and (c) AiCE. A strong augmented ring artifact is visible in the image reconstructed using FIRST, but not in those reconstructed using FBP or AiCE. AiCE, Advanced intelligent clear-IQ engine; FBP, Filtered back projection; FIRST, Forward projected model-based iterative reconstruction solution.

Discussion

The aim of this study was to perform both objective and subjective comparisons of image quality of low-dose abdominal CT images using DLR, MBIR, and FBP. Lower noise and a higher CNR were observed with deep learning than with the other two algorithms (p < .001). Furthermore, DLR was the only method for which image noise was not higher for obese patients than for those with a normal BMI.

This study found that the noise reduction capability and low-contrast resolution of AiCE was superior to FIRST, and that the objective image noise measurements corresponded well with subjective image assessments. Akagi et al reported a comparison of DLR, hybrid-IR, and MBIR in ultra-high-definition CT of the abdomen and found significant noise reduction and CNR improvement in DLR.²² Our results are similar to their report, but importantly, with AiCE, we further demonstrated that the image noise did not differ among the BMI groups. This is of particular clinical relevance because reducing noise in the CT images of patients with obesity can be challenging. Several techniques can improve the quality of the CT images of obese patients; BMI-adaptive scan protocols typically use higher kilovoltage acquisition and dual-source CT to reduce noise.^23,24 Unfortunately, both of these techniques involve a markedly increased radiation dose. In abdominal CT images, high image noise may obscure subtle low-contrast lesions in parenchymal organs,^25,26 increasing the risk of missing liver tumors.^27–30 In this regard, our observation that DLR, in contrast to MBIR and FBP, was able to maintain low noise levels even in patients with a higher BMI is encouraging. Further investigation of the ability of AiCE to support the diagnosis of liver lesions in obese patients is recommended.

An analysis of the subjective results revealed a lower level of artifacts on FIRST and AiCE images than on FBP images. FIRST performed better than AiCE regarding artifacts in general. MBIR models the optics and geometry of the CT system, including the response of the X-ray tube and detector, and incorporates the physical properties of scattering and crosstalk.³¹ This modeling reduces image noise and artifacts and improves spatial resolution. In contrast, it is important to note that the accuracy of DLR reproduction is highly dependent on the training data set that the algorithm uses to train the model. In the AiCE, high-quality MBIR images and low-dose images are used as teacher data, but at this point, it is thought that the network of deep learning is being constructed with priority given to noise removal rather than artifact improvement. Simulation-based studies have shown that such algorithms can lead to serious misdiagnosis and incorrect addition of anatomical features when tested against lesions or anatomical textures that are not present in the training dataset.³² In the present clinical study, no such cases were found in AiCE, but very strong augmentation ring artifacts were found in the FIRST images of one patient. This artifact was not observed on the AiCE images of the same patient. The cause of this artifact remains uncertain, but a possible explanation could be that the noise model in the FIRST algorithm misrecognized the artifact as a signal and, therefore, performed an inappropriate iteration. Gaddikeri et al reported strong artifacts due to thyroid shielding in head and neck CT images reconstructed with MBIR, but not with hybrid iterative reconstruction, and suggested that these artifacts are partly due to the MBIR noise model.³³

Subjective scores regarding the clarity of the liver contour were high for both FIRST and AiCE images. This may reflect the MTF data obtained from the phantom experiment, which showed that both the FIRST and AiCE have a higher spatial resolution than FBP. The spatial resolution of FIRST images was slightly but consistently higher than that of AiCE images at all spatial frequency values. However, there was no appreciable difference between the two on visual assessment.

The clinical benefits of CT are clear, and the number of scans conducted is increasing every year.¹ However, there are concerns regarding radiation-induced malignancies resulting from CT use. In the last decade, the development of hybrid iterative reconstruction and MBIR has made it possible to achieve a substantial dose reduction.^4–14,34 In this study, overall abdominal CT image quality was higher for AiCE than FIRST images; therefore, the use of AiCE may potentially allow for further radiation dose reductions while maintaining the same image quality. Several noise reduction algorithms have been developed with the aim of reducing radiation dose; these have typically been applied to CT images obtained using routine tube voltage settings (120 kV). In contrast-enhanced CT, the use of low tube voltage imaging in combination with various noise reduction methods can reduce exposure without compromising image quality or contrast.^6,7,35,36 However, owing to increased noise and artifacts, these techniques have a limited use in overweight patients.^6,7,37 While BMI-adaptive scan protocols have been developed to improve image quality in obese patients, these typically use higher kilovoltage acquisition and dual-source CT.^23,38 Unfortunately, both of these techniques involve a markedly high radiation dose. Since AiCE allows for noise reduction even in patients with a BMI ≥ 25 kg/m², the use of AiCE may potentially support adequate noise reduction in CT images of overweight patients, even when using low tube voltages. The reconstruction times with MBIR were originally between 15 min and 1 h, but due to recent technological advances, reconstruction times are now more appropriate for a clinical context.^8,9,15 Since AiCE reconstruction takes only approximately 1 min and as such takes only slightly more time than FBP, AiCE may be more appropriate than FIRST for clinical use, particularly given its superior noise reduction, CNR, and overall image quality compared with the latter.

There are several limitations of this study. The main limitation is the fact that because the AiCE is vendor specific, it might not be possible to generalize the findings to other vendors. Recently, GE has also developed the DRL algorithm “TrueFidelity” and has reported on their image evaluation. Jensen et al reported that TrueFidelity improves the image quality through noise reduction and decreased CNR without altering the texture on abdominal CT.³⁹ DLR seems to be a promising technology, but the methods still constitute a developing field of technology; hence, this study could be considered preliminary. Second, the mean BMI in our study was lower than the global average.⁴⁰ Third, we did not compare the reconstruction algorithms with respect to diagnostic accuracy because the number of patients with liver tumors was low. Future studies are needed to evaluate the use of this DLR algorithm in the diagnosis of liver tumors. In addition, we used CNR to evaluate the low-contrast resolution, but CNR may not always correlate with visual evaluation. A promising way to evaluate the dose reduction capability of IR algorithms is to use computational models of human observers. These models reproduce the response of the human visual system to the image system and, unlike CNR, calculate a detectability index (dʹ) that incorporates the attributes of image resolution and noise texture.⁴¹

In summary, our data suggest that images reconstructed with the AiCE, a novel deep learning CT reconstruction method, are of considerably higher quality than images reconstructed with either MBIR or traditional FBP. Considering the excellent performance of this novel algorithm observed during phantom testing as well as on abdominal CT in clinical patients regardless of BMI, DLR appears to be promising for low-dose CT applications. Further studies are required to develop reliable low-dose CT protocols for routine clinical use and to systematically analyze the performance of the AiCE in the diagnosis of various conditions.