Evaluation of Image Quality and Detectability of Deep Learning Image Reconstruction (DLIR) Algorithm in Single- and Dual-energy CT

Abstract

This study is aimed to evaluate effects of deep learning image reconstruction (DLIR) on image quality in single-energy CT (SECT) and dual-energy CT (DECT), in reference to adaptive statistical iterative reconstruction-V (ASIR-V). The Gammex 464 phantom was scanned in SECT and DECT modes at three dose levels (5, 10, and 20 mGy). Raw data were reconstructed using six algorithms: filtered back-projection (FBP), ASIR-V at 40% (AV-40) and 100% (AV-100) strength, and DLIR at low (DLIR-L), medium (DLIR-M), and high strength (DLIR-H), to generate SECT 120kVp images and DECT 120kVp-like images. Objective image quality metrics were computed, including noise power spectrum (NPS), task transfer function (TTF), and detectability index (d′). Subjective image quality evaluation, including image noise, texture, sharpness, overall quality, and low- and high-contrast detectability, was performed by six readers. DLIR-H reduced overall noise magnitudes from FBP by 55.2% in a more balanced way of low and high frequency ranges comparing to AV-40, and improved the TTF values at 50% for acrylic inserts by average percentages of 18.32%. Comparing to SECT 20 mGy AV-40 images, the DECT 10 mGy DLIR-H images showed 20.90% and 7.75% improvement in d′ for the small-object high-contrast and large-object low-contrast tasks, respectively. Subjective evaluation showed higher image quality and better detectability. At 50% of the radiation dose level, DECT with DLIR-H yields a gain in objective detectability index compared to full-dose AV-40 SECT images used in daily practice.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10278-023-00806-z.

Introduction

Computed tomography (CT) examinations have been broadly applied for clinical diagnosis and treatment in modern medicine as it is a fast, non-invasive imaging method with high resolution. However, with the increasing concern on the potential cancer-inducing risk of ionizing radiation, a conservative approach to reduce radiation exposure as low as reasonably achievable during CT studies is recommended [1]. On the other hand, the reduction in radiation dose leads to an increased image noise and results in a degraded image quality and diagnostic acceptability. To preserve diagnostic confidence of radiologists, several techniques have been employed for CT image reconstruction to reduce the radiation dose while preserve image quality [2, 3]. Filtered back projection (FBP), which yielded high-quality images at a high radiation dose, has been the method of choice for decades, while decreasing the radiation dose would lead to unsatisfied quality of images reconstructed by FBP [4]. With the increasing of computation power, hybrid iterative reconstruction (IR) algorithms, such as adaptive statistical iterative reconstruction-V (ASIR-V), combing FBP and model-based IR have been introduced into CT image reconstruction allowing lower radiation dose while maintaining image quality. Nevertheless, hybrid IR algorithms occasionally resulted in overly-smoothing, artificial-looking images, because hybrid IR algorithms do not model the complete CT system [5]. With the development of artificial intelligence, it is possible to apply artificial intelligence into image reconstruction. One of such image reconstruction algorithms is the deep learning image reconstruction (DLIR) algorithm by GE Healthcare, which employs convolutional neural networks to learn from high-quality labelled FBP CT images and images in clinical settings [6]. DLIR could better distinguish signal and noise to generate better detailed images with more radiation reduction, while the output image characteristics might vary with the input conditions [7].

These image reconstruction algorithms have been used not only in single-energy CT (SECT), but also in dual-energy CT (DECT) recently. To objectively assess their image quality, a task-based image quality assessment has been employed for both SECT [8–14] and DECT [15–18] with metrics such as noise power spectrum (NPS), task-based transfer function (TTF), and detectability index (d′). To employ this tool in DECT, the virtual monoenergetic images (VMI) were generated to mimic corresponding SECT-like images [15–18]. Both DLIR and hybrid IR algorithms can reduce the image noise of VMIs [16, 17], and even more with DLIR [17, 18], but the detectability of VMI using DLIR varied depending on the contrast [18]. Indeed, model-based image reconstruction (MBIR) did yield better objective low-contrast detectability compared to hybrid IR, but worse confidence and subjective low-contrast detectability [19–21]. Since in vitro performance of image reconstruction algorithms does not always translate into clinical performance and acceptability, it is important to correlate the two aspects.

Therefore, this study was aimed to evaluate the effects of DLIR on image quality and clinical acceptance in SECT (120 kVp) and DECT (VMIs at 70 keV) using comprehensive objective imaging metrics and to confirm the results by subjective ratings of multiple human observers, with ASIR-V images as reference standards.

Materials and Methods

Phantom

The workflow of this study is shown in Fig. 1. The ethics approval was not required due to the nature of this phantom study. A 20-cm diameter American College of Radiology quality assurance Gammex 464 phantom was scanned for image quality evaluation. This phantom is composed of four modules. The module 3 is a uniform, water-equivalent material, and was used to compute NPS. The module 1 consists of four inserts of 25-mm-diameter placed into a water-equivalent background material. The bone (955 Hounsfield unit value, HU) and acrylic (120 HU) inserts were used for TTF computation. The module 4 is composed of a series of high contrast patterns of 4- to 12- line pairs per cm. The module 2 is composed of a series of cylinders with 2-mm to 25-mm in diameters of 6 HU. These two modules (module 4 and 2) were used for high-contrast and low-contrast subjective detectability assessments, respectively.

Fig. 1.

Study workflow. The ACR 464 phantom was scanned with two scan modes (SECT and DECT) at three dose levels (5, 10, and 20 mGy) and then reconstructed using six image reconstruction algorithms (FBP, AV-40, AV-100, DLIR-L, DLIR-M, and DLIR-H), which resulted 36 sets of images in total. Objective image quality evaluation (noise power spectrum, task transfer function, and detectability index) and subjective image quality evaluation (image noise, texture, sharpness, overall quality, and low- and high-contrast detectability) have been performed. The results of objective and subjective evaluation have been compared

Image Acquisition and Reconstruction

The phantom was scanned on a 256-slice dual-energy CT scanner (Revolution Apex CT, GE Healthcare). This CT scanner was used because DLIR is currently only available on this scanner. The acquisition parameters are present in Table 1. DECT and SECT scans were performed with the fast kVp switching technology (80/140 kVp) and conventional 120 kVp, respectively, at three different dose levels (volume CT dose indexes, CTDI_Vol: 5, 10, and 20 mGy). The 20 mGy dose was selected as the reference full dose level for an adult abdomen based on the clinical practice in our institution [22, 23]. The 10 mGy and 5 mGy dose levels were used to reflect the radiation dose reduction conditions. Different combinations of tube current and rotation time were selected to obtain these three dose levels.

Table 1.

Image acquisition parameters

Scan mode	Tube voltage (kVp)	Milliamperage (mA)	Rotation time (s)	Revolution time (s)	Pitch	CTDIVol (mGy)
SECT	120	90	0.8	0.8	0.98438	5.00
SECT	120	180	0.8	0.8	0.98438	10.00
SECT	120	215	1.227	0.7	0.51563	19.98
DECT	80/140	145	0.3	0.6	0.98438	5.00
DECT	80/140	335	0.3	0.6	0.98438	10.00
DECT	80/140	370	0.5	1	0.98438	19.75

For SECT scans, the conventional 120 kVp images were used for evaluation. For the DECT scans, the 120 kVp-like virtual monochromatic images (VMI) were generated around 70 keV [23]. The scan raw data were all reconstructed on advanced workstation (AW, version 4.7, GE Healthcare) with a standard kernel, commonly used in clinical practice, and is currently the only available kernel for DLIR. The images were generated using six different image reconstruction algorithms: FBP, ASIR-V with a blending factor of 40% (AV-40) and 100% (AV-100), and DLIR (TrueFidelity™, GE Healthcare) with a low (DLIR-L), medium (DLIR-M) or high (DLIR-H) strength level. The AV-40 image at 20 mGy dose level was selected as the reference standard for comparison since it is currently employed at our institution in daily clinical practice. Subsequently, thirty-six sets of images were generated in total.

Objective Image Evaluation

The objective image evaluations were carried out using imQuest software version 7.1 (Duke University) by a radiologist with 4-year-experience in CT image reading. imQuest is a widely accepted image analysis tool for task-based CT image quality assessment. This tool allows noise evaluation in terms of NPS, resolution evaluation in terms of TTF, and detectability index (d′) computation to stimulate clinical tasks. This tool has been used repeatedly in previous studies and is recognized by American Association of Physicists in Medicine task group report, TG-233 [8–18]. The non-prewhitening with an eye filter and internal noise model was used, integrating human visual systems and internal noise [24].

NPS was measured to show the noise texture and magnitude [10, 25], by placing five square regions of interest (ROIs) in module 3 of the phantom. To improve the measurement statistics [10], the ensemble NPS was computed within 40 consecutive axial slices, which resulted in a total of 200 ROIs. For noise texture evaluation, the average of spatial frequency of NPS (f_average) was measured. For noise magnitude, the square root of the area under the curve (AUC) of the NPS was measured [26]. The spatial frequencies of 0.01 to 0.2 mm⁻¹, and 0.5 to 0.7 mm⁻¹ were employed to present noise magnitude in the low and high spatial frequency ranges, respectively [15].

TTF was computed for spatial resolution using bone and acrylic inserts [7, 10], in module 1. Two circular ROIs were placed around the two inserts, and circular-edge technique was applied to compute the edge spread function [27]. The line-spread function was acquired by derivation of the edge spread function ensemble. To avoid the effect of noise in edge spread function calculation [10], 40 consecutive axial slices were used for the calculation of the spatial frequency at which TTF values reach 50% (f_50%).

Detectability index (d′) was calculated as a task-based figure of merit surrogate for imaging performance, which uses a simple template-matching strategy to determine if a given image contains the signal of interest or not [10]. Each task function is defined as a 2D Gaussian form with a full width at half maximum equal to the task diameter. Two tasks were considered, included a small high-contrast feature and a large low-contrast feature [8]. The small high-contrast feature was set as a 1.5-mm-diameter 500 HU feature, mimicking calcifications, or identifying high-contrast tissue boundaries. The large low-contrast feature was defined as a 25-mm-diameter 120 HU feature, modeling a large mass in the liver or a nodule in the lungs. The details of objective image quality evaluation method are presented in Supplementary Note S1.

Subjective Image Evaluation

The subjective image evaluation was performed by six radiologists blinded to the acquisition and reconstruction parameters [28]. The images obtained with SECT and DECT scans at three dose levels using six algorithms resulted in thirty-six sets of images in total. The thirty-six sets of images were read by six radiologists with 1, 3, 3, 4, 4, 6 years of experience in CT image reading, in a random and independent fashion. The images were presented to readers with a default window width and level for four modules of the phantom. Readers were allowed to adjust the window width and level and viewing distance as they preferred and had no time limits to complete the image review. One radiologist with 4-year-experience in CT image reading re-assessed all the images two weeks after the first readout.

Before the assessment of image quality, all readers were trained on how to conduct the analysis. The image quality rating was performed on a five-point scale, from 1 to 5 (worst to best), in terms of image noise, texture, sharpness, and overall quality [29–31]. The image noise referred to mottle or graininess of image. The image texture review included the smoothness, plastic feeling of the images. The sharpness reflected the contour definition and delineation of inserts. The overall image quality was to show reader’s personal preference of the image quality. Readers were told that a value of less than 3 was deemed unsatisfactory for clinical use.

For subjective detectability assessments for high-contrast task and low-contrast task, module 4 and 2 of phantom was used, respectively. Images were displayed in random position and order to avoid memorization bias during readouts. Readers were instructed to tell the number of detectable low-contrast cylinders with a contrast of 6HU from the background (from 0 to 5, 2 to 6 mm in diameter), and the number of clearly displayed high-contrast line pairs (from 0 to 8, 4- to 12- line pairs per cm), but the readers were not asked to localize them [8]. The details of subjective image quality evaluation method are presented in Supplementary Note S2.

Statistical Analysis

The statistical analysis was performed with R language version 4.1.3 (https://www.r-project.org/) within RStudio version 1.4.1106 (https://www.rstudio.com/). The assessment results of image noise and spatial resolution were presented in percentages of variations comparing with corresponding FBP since they were highly correlated to the dose levels, while the detectability and subjective assessment results were compared with the SECT 20 mGy AV-40 image sets, to show the potential of algorithms for clinical practice and dose reduction. The intra- and inter-reader agreements were assessed by linear weighted kappa and Kendall's W test, respectively. Degrees of agreement based on kappa values and Kendall’s W statistics were interpreted using the following criteria: 0 to 0.20, poor; 0.21 to 0.40, fair; 0.41 to 0.60, moderate; 0.61 to 0.80, good; and 0.81 to 1.00, excellent.

Results

Objective Image Assessment

NPS

The absolute values of objective image assessments are presented in Table 2. The NPS curves and corresponding metrics are presented in Figs. 2 and 3. In general, the noise magnitude decreased as the increase of dose level and reconstruction strength (Table 3). Compared to FBP, ASIR-V showed comparable average image noise reduction in SECT and DECT (AV-40: -27.45% vs. -24.58% and AV-100: -63.15% vs. -59.76%), while the DLIR of all strength showed higher average reduction of image noise in DECT than SECT (DLIR-L: -34.66% vs -29.74%, DLIR-M: -46.48% vs -40.37%, and DLIR-H: -59.21% vs -51.19%). The DLIR-H provided additional noise reduction over AV-40 (-59.21% vs. -24.58% in DECT, and -51.19% vs. -27.45% in SECT), but similar noise reduction as AV-100 in DECT (-59.21% vs. -59.76%), and slightly less than AV-100 in SECT (-51.19% vs. -63.15%), using those of FBP as baselines.

Table 2.

Absolute values of objective image assessments

Metric	SECT			DECT
	5 mGy	10 mGy	20 mGy	5 mGy	10 mGy	20 mGy
Noise (Hu)
FBP	24.4	17.4	11.9	22.3	16.6	12.4
AV40	17.6	12.6	8.7	17.3	12.5	9.1
AV100	8.5	6.3	4.7	9.2	6.5	5
DLIR-L	16	12.4	8.8	15.4	10.9	7.6
DLIR-M	13.2	10.6	7.6	12.7	8.9	6.2
DLIR-H	10.2	8.7	6.5	9.7	6.8	4.7
FBP	0.315	0.283	0.252	0.299	0.283	0.268
AV40	0.220	0.220	0.173	0.189	0.173	0.173
AV100	0.126	0.126	0.126	0.126	0.110	0.110
DLIR-L	0.252	0.252	0.205	0.268	0.220	0.173
DLIR-M	0.252	0.236	0.173	0.205	0.189	0.094
DLIR-H	0.220	0.220	0.142	0.094	0.094	0.079
f average
FBP	0.319	0.315	0.306	0.306	0.303	0.296
AV40	0.288	0.284	0.272	0.274	0.269	0.263
AV100	0.193	0.193	0.188	0.184	0.183	0.181
DLIR-L	0.304	0.301	0.287	0.288	0.284	0.278
DLIR-M	0.295	0.293	0.277	0.277	0.274	0.268
DLIR-H	0.280	0.282	0.263	0.259	0.257	0.247
TTF bone f 50%
FBP	0.429	0.431	0.429	0.402	0.395	0.400
AV40	0.434	0.437	0.433	0.407	0.400	0.404
AV100	0.442	0.445	0.440	0.415	0.407	0.409
DLIR-L	0.432	0.436	0.435	0.426	0.425	0.426
DLIR-M	0.431	0.436	0.435	0.425	0.425	0.427
DLIR-H	0.430	0.435	0.434	0.425	0.425	0.427
TTF acrylic f 50%
FBP	0.430	0.411	0.414	0.368	0.370	0.363
AV40	0.408	0.403	0.424	0.404	0.391	0.372
AV100	0.375	0.404	0.438	0.458	0.447	0.392
DLIR-L	0.460	0.436	0.452	0.458	0.470	0.487
DLIR-M	0.457	0.431	0.447	0.464	0.475	0.493
DLIR-H	0.450	0.432	0.443	0.470	0.482	0.498
d' 1.5 mm 500 Hu
FBP	23.273	32.245	44.789	23.906	31.337	40.689
AV40	28.020	38.587	53.144	26.532	35.701	47.638
AV100	38.308	52.199	68.981	33.580	46.913	61.748
DLIR-L	33.531	42.947	56.858	32.587	44.921	62.805
DLIR-M	38.990	48.442	62.591	37.349	52.552	74.175
DLIR-H	47.371	56.258	69.854	45.355	64.249	90.022
d' 25 mm 120 Hu
FBP	275.268	382.729	477.089	280.637	357.741	444.233
AV40	312.741	426.642	533.312	294.353	381.316	486.420
AV100	368.858	508.831	614.655	326.332	443.314	567.957
DLIR-L	353.110	459.127	553.239	343.244	454.915	601.506
DLIR-M	387.314	495.557	587.152	373.240	504.970	674.945
DLIR-H	433.851	550.363	627.794	416.384	574.662	756.503

Fig. 2.

NPS curves. NPS curves were obtained with six image reconstruction algorithms (FBP, AV-40, AV-100, DLIR-L, DLIR-M, and DLIR-H) at three dose levels (5, 10, and 20 mGy). top, SECT; bottom, DECT. from left to right, 5, 10, and 20 mGy. Note the AV-100 algorithm obviously reduced the noise in the high spatial frequency range

Fig. 3.

NPS metrics. a Noise, b Square root of AUC low, c Square root of AUC high, d f average Note the AV-100 algorithm obviously reduced the noise in the high spatial frequency range, which resulted in large shifts toward lower frequencies in f peak, and f average

Table 3.

Percentage variations of noise magnitude and noise texture relative to FBP images

Reconstruction	SECT				DECT
	5 mGy	10 mGy	20 mGy	average	5 mGy	10 mGy	20 mGy	average
Noise (HU)
FBP	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
AV-40	-27.87%	-27.59%	-26.89%	-27.45%	-22.42%	-24.70%	-26.61%	-24.58%
AV-100	-65.16%	-63.79%	-60.50%	-63.15%	-58.74%	-60.84%	-59.68%	-59.76%
DLIR-L	-34.43%	-28.74%	-26.05%	-29.74%	-30.94%	-34.34%	-38.71%	-34.66%
DLIR-M	-45.90%	-39.08%	-36.13%	-40.37%	-43.05%	-46.39%	-50.00%	-46.48%
DLIR-H	-58.20%	-50.00%	-45.38%	-51.19%	-56.50%	-59.04%	-62.10%	-59.21%
Square root of AUC NPS low
FBP	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
AV-40	-14.52%	-13.76%	-13.34%	-13.87%	-7.51%	-9.64%	-12.33%	-9.83%
AV-100	-33.46%	-32.46%	-29.69%	-31.87%	-22.81%	-27.49%	-29.24%	-26.52%
DLIR-L	-28.65%	-23.00%	-18.90%	-23.52%	-23.96%	-26.75%	-32.13%	-27.61%
DLIR-M	-37.86%	-31.08%	-25.59%	-31.51%	-32.80%	-36.60%	-41.83%	-37.08%
DLIR-H	-47.89%	-39.95%	-32.45%	-40.10%	-43.54%	-47.23%	-51.12%	-47.30%
Square root of AUC NPS high
FBP	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
AV-40	-34.51%	-34.44%	-34.45%	-34.47%	-30.48%	-32.63%	-34.10%	-32.40%
AV-100	-84.07%	-83.11%	-81.41%	-82.86%	-82.24%	-82.80%	-81.93%	-82.32%
DLIR-L	-37.37%	-32.34%	-30.92%	-33.54%	-34.96%	-37.55%	-40.74%	-37.75%
DLIR-M	-49.42%	-43.23%	-41.39%	-44.68%	-47.64%	-50.15%	-53.28%	-50.36%
DLIR-H	-62.58%	-55.16%	-52.68%	-56.80%	-62.23%	-63.88%	-66.67%	-64.26%
f average
FBP	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
AV-40	-9.73%	-10.10%	-10.93%	-10.25%	-10.62%	-11.06%	-11.17%	-10.95%
AV-100	-39.63%	-38.82%	-38.54%	-39.00%	-40.08%	-39.62%	-38.64%	-39.44%
DLIR-L	-4.77%	-4.60%	-6.27%	-5.21%	-5.97%	-6.15%	-5.83%	-5.98%
DLIR-M	-7.62%	-7.05%	-9.46%	-8.04%	-9.63%	-9.56%	-9.53%	-9.57%
DLIR-H	-12.38%	-10.69%	-14.09%	-12.39%	-15.61%	-15.20%	-16.42%	-15.74%

Negative values indicated reduction

In terms of noise reduction distribution as spatial frequency, the noise in both the low and high spatial frequency ranges decreased with the increase of DLIR strength and ASIR-V blending factor (Table 3). Compared to FBP, the noise in the low spatial frequency range was decreased with AV-100 by an average reduction of -31.87% and -26.52% in SECT and DECT, respectively, which is lower than that of -40.10% and -47.30% with DLIR-H. In contrast, the AV-100 provided an average reduction of noise in the high spatial frequency range of -82.86% and -82.32% in SECT and DECT, respectively, much higher than that of -62.23% and -64.26% with DLIR-H, compared to FBP. AV-40 exhibited the same characteristics as AV-100: higher noise reduction in the high frequency range (-34.47% in SECT and -32.40% in DECT), but much lower noise reduction in the low frequency range (-13.87% in SECT and -9.83% in DECT).

The SECT and DECT images exhibited almost identical shifts in f_average for all scan protocols comparing to FBP (Table 3). The f_average shifted toward lower frequencies with increasing DLIR strength and ASIR-V blending factor. The shift of f_average in SECT and DECT with DLIR-H (-12.39% and -15.74%) were lower than that of f_average with AV-100 (-39.00% and -39.44%).

TTF

The TTF curves of bone and acrylic inserts and their corresponding metrics are presented in Figs. 4 and 5. For the bone insert, the TTF_50% values obtained with DLIR and ASIR-V algorithms were not different compared to those with FBP in both SECT and DECT (Table 4), and were not affected by the dose level. With the acrylic insert, all DLIR images improved TTF_50% values compared with FBP and ASIR-V in both SECT and DECT. ASIR-V images also improved TTF_50% values in DECT compared with those of FBP images.

Fig. 4.

TTF curves. TTF curves were obtained with six image recnstruction algorithms (FBP, AV-40, AV-100, DLIR-L, DLIR-M, and DLIR-H) at three dose levels (5, 10, and 20 mGy). top, SECT; bottom, DECT. from left to right, 5, 10, and 20 mGy. a Bone insert. the f_50% values obtained with DLIR and ASIR-V algorithms were not different compared to those obtained with FBP in both SECT and DECT, and were not obviously affected by the dose level. b Acrylic insert. All DLIR images improved f_50% values compared with FBP and ASIR-V in both SECT and DECT. ASIR-V images also improved f_50% values in DECT compared with FBP images

Fig. 5.

TTF metrics. a TTF bone f 50%, b TTF acrylic f 50%

Table 4.

Percentage variations of TTF relative to FBP images

Reconstruction	SECT				DECT
	5 mGy	10 mGy	20 mGy	average	5 mGy	10 mGy	20 mGy	average
Bone TTF f50%
FBP	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
AV-40	1.18%	1.21%	1.04%	0.00%	1.81%	1.24%	0.91%	0.00%
AV-100	3.04%	3.08%	2.65%	1.14%	3.74%	3.22%	2.29%	1.32%
DLIR-L	0.89%	1.14%	1.44%	2.92%	6.39%	7.59%	6.60%	3.08%
DLIR-M	0.67%	1.01%	1.37%	1.16%	6.29%	7.63%	6.64%	6.86%
DLIR-H	0.44%	0.86%	1.31%	1.02%	6.27%	7.60%	6.69%	6.85%
Acrylic TTF f50%
FBP	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
AV-40	-5.07%	-1.93%	2.51%	0.00%	9.73%	5.81%	2.51%	0.00%
AV-100	-12.77%	-1.67%	5.85%	-1.50%	24.43%	20.91%	8.02%	6.02%
DLIR-L	6.94%	5.96%	9.10%	-2.86%	24.46%	27.02%	33.97%	17.79%
DLIR-M	6.29%	4.80%	7.95%	7.33%	26.14%	28.50%	35.67%	28.48%
DLIR-H	4.58%	4.94%	7.08%	6.35%	27.77%	30.29%	37.20%	30.10%

Negative values indicated reduction and positive values indicated improvement

Detectability Index

The detectability indexes for small high-contrast feature (1.5-mm-diameter, 500 HU) and large low-contrast feature (25-mm-diameter, 120 HU) are presented in Fig. 6. The detectability indexes were increased with the increasing dose level (Fig. 7). To provide insights for clinical practice, the detectability indexes were compared with that of SECT 20 mGy AV-40 images (Table 5). For the small high-contrast feature, the SECT 10 mGy DLIR-H images provided comparable detectability (5.86%), and the DECT 10 mGy DLIR-H images showed higher detectability (20.90%), comparing to the SECT 20 mGy AV-40 images. Similarly, for the large low-contrast feature, the SECT 10 mGy DLIR-H images (3.20%) and DECT 10 mGy DLIR-H images (7.75%) provided comparable detectability, comparing to the SECT 20 mGy AV-40 images. For both features, the DECT 20 mGy AV-40 images showed lower detectability (-10.36% and -8.79%), but DECT 20 mGy AV-100 and DECT 20 mGy DLIR images all showed increased detectability and values increased with the reconstruction strength, comparing to the SECT 20 mGy AV-40 images.

Fig. 6.

Detectability index. a Small high-contrast feature and, b Large low-contrast feature

Fig. 7.

Detectability index and dose level. Comparisons of detectability index (d′) values obtained with a, c SECT and b, d DECT scan modes for a, b small high-contrast feature, and c, d large low-contrast feature

Table 5.

Percentage variations of d’ relative to SECT 20 mGy AV-40 images

Reconstruction	SECT			DECT
	5 mGy	10 mGy	20 mGy	5 mGy	10 mGy	20 mGy
d’ (1.5 mm, 500 HU)
FBP	-56.21%	-39.33%	-15.72%	-55.02%	-41.03%	-23.44%
AV-40	-47.28%	-27.39%	n/a	-50.07%	-32.82%	-10.36%
AV-100	-27.92%	-1.78%	29.80%	-36.81%	-11.73%	16.19%
DLIR-L	-36.90%	-19.19%	6.99%	-38.68%	-15.47%	18.18%
DLIR-M	-26.63%	-8.85%	17.78%	-29.72%	-1.11%	39.57%
DLIR-H	-10.86%	5.86%	31.44%	-14.66%	20.90%	69.39%
d’ (25 mm, 120 HU)
FBP	-48.39%	-28.24%	-10.54%	-47.38%	-32.92%	-16.70%
AV-40	-41.36%	-20.00%	n/a	-44.81%	-28.50%	-8.79%
AV-100	-30.84%	-4.59%	15.25%	-38.81%	-16.88%	6.50%
DLIR-L	-33.79%	-13.91%	3.74%	-35.64%	-14.70%	12.79%
DLIR-M	-27.38%	-7.08%	10.10%	-30.01%	-5.31%	26.56%
DLIR-H	-18.65%	3.20%	17.72%	-21.92%	7.75%	41.85%

SECT 20 mGy AV-40 images was considered as the standard radiation dose and representative reconstruction method for an adult abdomen with single-energy CT in clinical practice. Negative values indicated reduction and positive values indicated improvement

Subjective Image Assessment

Subjective Image Quality Assessment

The subjective image quality assessment showed good to excellent intra-reader agreement and fair to good inter-reader agreement (Table 6). The results of subjective image quality assessment are presented in Fig. 8 and Table 7. The image noise and overall quality rating usually increased with increasing dose level, while the texture and sharpness rating did not show obvious increase trend with increasing dose level. The image noise rating increased with increasing DLIR strength and ASIR-V blending factor, but the other subjective image quality rating did not present an increase trend with increasing DLIR strength or ASIR-V blending factor. Notably, the DECT 10 mGy DLIR-H images showed better image quality than SECT 20 mGy AV-40 images in terms of image noise (10.00%), texture (5.88%), sharpness (5.00%), and overall quality (5.00%).

Table 6.

Intra-reader agreements of subjective image quality evaluation

Rating	Intra-reader agreement		Inter-reader agreement
	Weighted kappa	P value	Kendall’s W statistics	P value
Image noise	0.743	< 0.001	0.660	< 0.001
Texture	0.689	< 0.001	0.389	< 0.001
Sharpness	0.748	< 0.001	0.446	< 0.001
Overall quality	0.674	< 0.001	0.563	< 0.001
High-contrast detectability	0.735	< 0.001	0.421	< 0.001
Low-contrast detectability	0.813	< 0.001	0.745	< 0.001

Fig. 8.

Subjective image quality and detectability assessment. Bar plots of a image noise, b image texture, c border sharpness, d overall image quality, e high-contrast detectability, and f low-contrast detectability subjectively evaluated by six radiologists

Table 7.

Subjective evaluation results

Reconstruction	SECT 5 mGy	SECT 10 mGy	SECT 20 mGy	DECT 5 mGy	DECT 10 mGy	DECT 20 mGy
Image noise (1 to 5)
FBP	1.33 ± 0.52	2.17 ± 0.98	2.67 ± 1.03	1.50 ± 0.55	2.17 ± 0.75	2.17 ± 0.75
AV-40	1.83 ± 1.17	2.33 ± 1.03	3.33 ± 0.82	2.00 ± 1.10	2.50 ± 0.84	3.00 ± 0.63
AV-100	2.17 ± 0.75	2.50 ± 1.05	3.67 ± 1.03	2.00 ± 0.63	3.00 ± 0.89	3.33 ± 1.21
DLIR-L	2.17 ± 0.75	2.67 ± 1.03	3.17 ± 1.17	2.33 ± 0.82	2.83 ± 0.98	3.50 ± 0.84
DLIR-M	2.67 ± 1.03	3.00 ± 0.89	3.33 ± 1.03	2.33 ± 0.82	3.17 ± 0.75	3.67 ± 1.03
DLIR-H	2.83 ± 0.41	3.33 ± 0.82	4.00 ± 1.10	2.67 ± 1.03	3.67 ± 1.37	4.17 ± 1.33
Texture (1 to 5)
FBP	2.00 ± 0.89	2.50 ± 0.55	2.83 ± 0.41	2.67 ± 1.37	2.67 ± 0.52	2.67 ± 0.82
AV-40	2.17 ± 0.75	2.67 ± 0.82	2.83 ± 0.41	2.83 ± 1.47	2.67 ± 1.03	3.17 ± 0.75
AV-100	2.50 ± 0.84	2.17 ± 1.17	3.33 ± 0.82	2.50 ± 1.38	2.83 ± 0.41	2.83 ± 0.98
DLIR-L	2.83 ± 0.41	3.33 ± 0.52	3.00 ± 0.63	3.00 ± 0.89	3.00 ± 0.63	3.67 ± 0.52
DLIR-M	2.83 ± 0.98	3.17 ± 0.41	3.00 ± 0.63	2.83 ± 1.17	3.00 ± 0.63	3.33 ± 0.82
DLIR-H	3.33 ± 0.52	3.00 ± 0.63	3.50 ± 0.55	3.00 ± 1.10	3.00 ± 0.63	3.17 ± 0.75
Sharpness (1 to 5)
FBP	1.67 ± 0.52	2.67 ± 0.52	3.17 ± 0.75	2.50 ± 1.38	2.67 ± 0.82	2.67 ± 0.82
AV-40	2.33 ± 0.82	2.67 ± 0.52	3.33 ± 0.82	2.83 ± 1.17	2.50 ± 1.05	3.00 ± 0.73
AV-100	2.00 ± 0.00	2.33 ± 1.03	3.50 ± 0.55	2.33 ± 1.03	2.67 ± 0.82	3.00 ± 0.89
DLIR-L	2.83 ± 0.41	3.33 ± 0.52	3.17 ± 0.75	3.00 ± 0.89	3.17 ± 0.75	3.67 ± 0.52
DLIR-M	2.67 ± 0.82	3.00 ± 0.00	3.00 ± 0.63	2.83 ± 1.17	3.17 ± 0.75	3.83 ± 0.98
DLIR-H	3.17 ± 0.41	3.17 ± 0.98	3.50 ± 0.55	3.00 ± 1.26	3.50 ± 0.84	3.33 ± 0.82
Overall (1 to 5)
FBP	1.67 ± 0.52	2.50 ± 0.55	3.00 ± 0.63	2.00 ± 1.10	2.33 ± 0.52	2.67 ± 0.82
AV-40	2.00 ± 0.63	2.50 ± 0.55	3.33 ± 0.82	2.83 ± 1.47	2.67 ± 1.03	3.17 ± 0.75
AV-100	2.17 ± 0.41	2.67 ± 1.03	3.83 ± 0.75	2.33 ± 0.82	2.83 ± 0.41	3.33 ± 0.82
DLIR-L	2.83 ± 0.41	3.50 ± 0.55	3.33 ± 0.82	2.50 ± 0.55	3.00 ± 0.63	3.83 ± 0.75
DLIR-M	2.83 ± 0.75	3.00 ± 0.63	3.17 ± 0.75	2.83 ± 1.33	3.00 ± 0.63	3.83 ± 0.98
DLIR-H	3.33 ± 0.52	3.33 ± 0.82	3.83 ± 0.75	3.00 ± 1.10	3.50 ± 0.84	3.50 ± 0.55
High-contrast detectability (0 to 8)
FBP	3.50 ± 0.55	3.50 ± 0.55	3.83 ± 0.41	3.50 ± 0.84	3.50 ± 0.55	3.67 ± 0.52
AV-40	3.50 ± 0.55	3.50 ± 0.55	4.00 ± 0.00	3.83 ± 0.41	4.00 ± 0.00	4.17 ± 0.41
AV-100	3.67 ± 0.52	3.50 ± 0.55	4.00 ± 0.00	3.83 ± 0.41	4.00 ± 0.00	4.00 ± 0.00
DLIR-L	3.50 ± 0.55	3.50 ± 0.55	3.83 ± 0.41	3.83 ± 0.41	4.00 ± 0.00	4.17 ± 0.41
DLIR-M	3.50 ± 0.55	3.50 ± 0.55	4.00 ± 0.00	4.00 ± 0.00	4.00 ± 0.00	4.00 ± 0.00
DLIR-H	3.33 ± 0.52	3.33 ± 0.52	4.00 ± 0.00	4.00 ± 0.00	4.17 ± 0.41	4.00 ± 0.00
Low-contrast detectability (0 to 5)
FBP	0.17 ± 0.41	0.50 ± 0.55	2.17 ± 0.98	0.33 ± 0.52	1.00 ± 0.89	1.17 ± 0.75
AV-40	0.50 ± 0.55	0.83 ± 0.41	2.67 ± 1.03	1.17 ± 0.75	1.17 ± 1.17	1.67 ± 1.03
AV-100	0.83 ± 0.41	1.33 ± 1.03	2.83 ± 1.47	1.17 ± 0.75	1.33 ± 1.03	2.17 ± 0.98
DLIR-L	0.83 ± 0.41	1.33 ± 0.82	2.33 ± 1.51	0.67 ± 0.82	1.83 ± 1.17	2.33 ± 1.03
DLIR-M	1.00 ± 0.63	1.50 ± 0.55	2.50 ± 1.64	1.00 ± 1.10	1.83 ± 0.75	2.50 ± 0.55
DLIR-H	1.83 ± 0.41	1.83 ± 0.75	2.83 ± 1.33	1.17 ± 0.98	2.67 ± 1.03	2.83 ± 1.17

SECT 20 mGy AV-40 images was considered as the standard radiation dose and representive reconstruction method for an adult abdomen with single-energy CT in clinical practice. Negative values indicated reduction and positive values indicated improvement. Present as mean ± standard deviation

Subjective Detectability Assessment

The subjective detectability for high-contrast and low-contrast tasks were compared to the SECT 20 mGy AV-40 images (Fig. 9 and Table 7). SECT 10 mGy images could not provide comparable high-contrast (3.33 vs. 4.00, -16.75%) and low-contrast (1.83 vs. 2.67, -31.46%) detectability even with DLIR-H. In contrast, DECT 10 mGy DLIR-H images provided comparable detectability for both high-contrast (4.17 vs. 4.00, 4.25%) and low-contrast (2.67 vs. 2.67, 0.00%) task, comparing with SECT 20 mGy AV-40 images. DLIR algorithms provided the same detectability in both SECT and DECT at the dose level of 20 mGy, while ASIR-V algorithms performed slightly worse in DECT than in SECT.

Fig. 9.

Objective and subjective detectability assessment. Heatmaps of percentage of variation of objective and subjective detectability assessment compared to SECT 20 mGy AV-40 images. a objective detectability index for small high-contrast feature, b detectability index for small high-contrast feature, c subjective detectability assessment results for high-contrast task, d subjective detectability assessment results for low-contrast task

Discussion

Our study presented the image quality and detectability improvement with DLIR in both SECT and DECT, compared with the commonly used ASIR-V in routine clinical applications. DLIR could reduce the image noise in a more balanced manner in both the low and high spatial frequency ranges compared with FBP and ASIR-V algorithms, and provide improved TTF value compared with ASIR-V. DLIR provided higher detectability index for both small high-contrast feature and large low-contrast feature in DECT than those with ASIR-V. At 50% (10 mGy) dose level, DLIR-H images in DECT had similar image quality and diagnostic ability in terms of subjective detectability assessments, and objective detectability index estimation as the AV-40 images in SECT obtained at 20 mGy dose level that are commonly used in daily practice.

The noise magnitude evaluation indicated that DLIR provided deeper noise reduction in the low spatial frequency range compared to ASIR-V, which was similar to those of previous studies [18, 29]. On the other hand, AV-100 had higher noise reduction in the high spatial frequency range than DLIR. Similar characteristics of hybrid IR were observed in a previous study [32]. More reduction in high spatial frequency range than that in low spatial frequency range caused a shift of f_average toward lower frequencies. This unbalanced image noise decreasing affected the image texture and potentially resulted in lower detectability of diagnostic tasks [7]. In contrast, DLIR showed lower shifts of f_average, since DLIR decreased noise in a more balanced manner regarding both low and high spatial frequency ranges. Therefore, the DLIR could preserve the image texture, which was similar to that reconstructed using FBP, and provide images that are not too overly-smoothing or artificial-looking to radiologists [12, 18].

In the current study, the TTF_50% values obtained in DECT images processed by DLIR did not show obvious change compared to those of FBP for the bone insert (955 HU). while these values were higher for the acrylic insert (120 HU). In a previous study, the DECT images processed by DLIR showed higher spatial resolution than those of SECT with the same algorithm for a contrast object of 300 HU, but the TTF_50% values resulting from DLIR were lower than those obtained by FBP at contrast of 60 HU [18]. These results indicated that the spatial resolution of DECT images processed by DLIR showed different characteristics depending on the object contrast. SECT images processed by FBP have linear properties, and the spatial resolution of these images did not change with the object contrast [27]. As a contrast, the spatial resolution of DECT images produced by FBP differed depending on the object contrast [15], since the noise reduction was performed on the raw data of DECT images with FBP [15, 18]. We suspected that the varying properties of DLIR on the spatial resolution of DECT images was related to the fact that DLIR was applied to the original images in which noise reduction had already been performed [18].

We further subjectively assessed the image quality. The visual assessment indicated a higher noise magnitude of images obtained with FBP than those with hybrid IR and DLIR for the same dose level, which was in accordance with subjective image noise rating, and supported by objective measurements. Images processed by AV-100 were smoothed out. Similar results have been described in SECT images [13]. However, image texture, sharpness and overall quality rating did not obviously differ visually among reconstruction algorithms. Since the phantom images might not feature the same exact characteristics as the anatomical structures present in patients [17], we considered that subjective assessments on clinical image might better reflect the preference of the radiologists [28]. Indeed, within clinical abdominal images, radiologists have showed their preference for DLIR images both with SECT mode [14, 30, 33–37], and DECT mode [38–44].

The results of subjective detectability assessment were not always in accordance with the corresponding objective detectability indexes. Objective assessment showed higher detectability index for both small high-contrast feature and large low-contrast feature with 20 mGy images with all DLIR reconstructions in SECT and DECT. However, with DECT 10 mGy DLIR-H images, objective assessments showed more improvements than subjective evaluation in both high-contrast (20.90% vs 4.17%) and low-contrast (7.75% vs 0.00%) tasks. Meanwhile, with DECT 20 mGy DLIR-H images, the differences between objective and subjective assessment were even more obvious in both high-contrast (69.39% vs 0.00%) and low-contrast (41.85% vs 6.25%) tasks. These results indicated that objective in vitro performance improvements did not always translate into clinical acceptability to naked eyes.

Therefore, we believe that the subjective evaluation should be considered important before applying DLIR algorithms in clinical practice. Previous studies using MBIR algorithm [19] and the deep learning reconstruction (DLR) by Canon in SECT have demonstrated that subjective evaluation did not always show corresponding improvements in detection tasks as objective assessments suggested [20, 21]. A recent study implied dramatic detectability improvement of DECT images with DLIR in terms of objective assessment without subjective validation [18], which left the doubt whether these promising objective improvements could translate into clinical acceptance of DLIR algorithm in DECT images. Our study also indicated that although there were improvements in image quality perceived by human observers for DLIR algorithms, the improvements were not as large as the objective measurements in both SECT and DECT images.

The detectability index varied depending on the contrast of the object being investigated. The detectability indexes of DECT 10 mGy DLIR-H images were higher than those of SECT 20 mGy AV-40 images, for both small high-contrast (500 HU) feature and large low-contrast (120 HU) feature. We believe that DLIR-H did not decrease the diagnostic performance of these two detection tasks when the dose level was reduced by half. The results of subjective detectability assessments supported the objective measurements. At 50% of the radiation dose, only DECT images processed by DLIR-H were deemed to be of similar detectability for both high-contrast and low-contrast tasks, compared with those of SECT 20 mGy AV-40 images. Similar results were obtained in SECT that DLIR-H could provide higher detectability index with approximately half the dose reduction for features with objective contrast of 10 HU, 120 HU, and 500 HU [17]. However, DLIR-H were not recommended in DECT images for objective contrast of 60 HU, simulating soft tissue or organs [18]. Meanwhile, DLIR-M were recommended for objective contrast of 300 HU, to improve image quality of CT angiography and bone images [18]. The detectability index estimation relied on NPS and TTF [10]. Considering the dependence of spatial resolution on objective contrast, it is not strange that the detectability index might differ according to tasks in DECT images. Therefore, further investigation is needed to reveal the potential influence of DLIR on spatial resolution and detectability index of different object contrast in DECT images. On the other hand, since the objective metrics are not always in accordance with the subjective evaluation, investigations with clinical images might provide insights for this contrast-depending variation, to directly guide the usage of DLIR in diagnostic tasks with different contrast.

There were several limitations in our study. First, our study was performed with a phantom instead of clinical images. There were only two lesions used in our study to mimic calcifications or identifying high-contrast tissue boundaries. To confirm the results of this study, further clinical studies are needed. Second, only standard reconstruction kernel was available for DLIR in DECT. The influence of different reconstruction kernels on the image quality reconstructed using DLIR needs further investigation. Third, our study was performed with only one CT system. Other manufacturers provide different deep learning-based algorithms for clinical use and may induce different characteristics of images from those obtained in current study [45]. Finally, it is also important to note that the DECT assessment performed in this study is very limited in scope, focusing only on VMIs similar to SECT acquisitions.

Conclusion

To summarize, compared to the ASIR-V at 40% strength commonly used in clinical practice, DLIR further reduces image noise in a more balanced manner in both SECT and DECT, and showed better objective and subjective detectability. DLIR-H has the potential to provide similar or better image quality and preserve the diagnostic performance at 50% of the radiation dose compared to images used in clinical practice. Nevertheless, the spatial resolution and detectability of DECT images processed with DLIR varied depending on the contrast of the object being investigated, and further clinical investigation is needed to guide the clinical use of DLIR in DECT. The potential to provide additional 50% radiation dose reduction with DLIR might accelerate the clinical acceptance of DECT.

Supplementary Information

Below is the link to the electronic supplementary material.

Abbreviations

ASIR-V

Adaptive statistical iterative reconstruction-V

AUC

Area under the curve

DECT

Dual-energy computed tomography

DLIR

Deep learning image reconstruction

DLR

Deep learning reconstruction

FBP

Filtered back-projection

HU

Hounsfield unit value

IR

Iterative reconstruction

MBIR

Model-based image reconstruction

NPS

Noise power spectrum

ROI

Region of interest

SECT

Single-energy computed tomography

VMI

Virtual monoenergetic images

Author Contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Jingyu Zhong, Hailin Shen, Yong Chen, Yihan Xia, Yue Xing, Yangfan Hu, Xiang Ge, Defang Ding, and Zhenming Jiang. The first draft of the manuscript was written by Jingyu Zhong and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Funding

This work was supported by National Natural Science Foundation of China (82271934, 82101986), Yangfan Project of Science and Technology Commission of Shanghai Municipality (22YF1442400, 20YF1427200), Shanghai Science and Technology Commission Science and Technology Innovation Action Clinical Innovation Field (18411953000), Medicine and Engineering Combination Project of Shanghai Jiao Tong University (YG2021QN08, YG2019ZDB09), Research Fund of Tongren Hospital, Shanghai Jiao Tong University School of Medicine (TRKYRC-XX202204, TRGG202101, TRYJ2021JC06, 2020TRYJ(LB)06, 2020TRYJ(JC)07), Guangci Innovative Technology Launch Plan of Ruijin Hospital, Shanghai Jiao Tong University School of Medicine (2022–13).

Data Availability

The data that support the findings of this study are available from the corresponding author, upon reasonable request.

Declarations

Ethics Approval

Institutional Review Board approval was not required because of the nature of our study, which was a phantom study.

Consent to Participate

Written informed consent was not required for this study because of the nature of our study, which was a phantom study.

Consent to Publish

Consent to publish was not required because of the nature of our study, which was a phantom study.

Competing Interests

Mr. Wei Lu and Dr. Jianying Li are employees of GE Healthcare. However, they neither had access nor control on the data acquisition and analysis. All other authors of this manuscript have no relevant financial or non-financial interests to disclose.