CT Super-Resolution GAN Constrained by the Identical, Residual, and Cycle Learning Ensemble (GAN-CIRCLE)

Abstract

In this paper, we present a semi-supervised deep learning approach to accurately recover high-resolution (HR) CT images from low-resolution (LR) counterparts. Specifically, with the generative adversarial network (GAN) as the building block, we enforce the cycle-consistency in terms of the Wasserstein distance to establish a nonlinear end-to-end mapping from noisy LR input images to denoised and deblurred HR outputs. We also include the joint constraints in the loss function to facilitate structural preservation. In this process, we incorporate deep convolutional neural network (CNN), residual learning, and network in network techniques for feature extraction and restoration. In contrast to the current trend of increasing network depth and complexity to boost the imaging performance, we apply a parallel 1 × 1 CNN to compress the output of the hidden layer and optimize the number of layers and the number of filters for each convolutional layer. The quantitative and qualitative evaluative results demonstrate that our proposed model is accurate, efficient and robust for super-resolution (SR) image restoration from noisy LR input images. In particular, we validate our composite SR networks on three large-scale CT datasets, and obtain promising results as compared to the other state-of-the-art methods.

I. INTRODUCTION

X-RAY computed tomography (CT) is one of the most popular medical imaging methods for screening, diagnosis, and image-guided intervention [1]. Potentially, high-resolution (HR) CT (HRCT) imaging may enhance the fidelity of radiomic features as well. Therefore, super-resolution (SR) methods in the CT field are receiving a major attention [2]. The image resolution of a CT imaging system is constrained by x-ray focal spot size, detector element pitch, reconstruction algorithms, and other factors. While physiological and pathological units in human bodies are on an order of 10 microns, the in-plane and through-plane resolution of clinical CT systems are on an order of submillimeter or 1 mm [2]. Even though the modern CT imaging and visualization software can generate any small voxels, the intrinsic resolution is still far lower than what is ideal in important applications such as early tumor characterization and coronary artery analysis [3]. Consequently, producing HRCT images at a minimum radiation dose level is highly desirable in the CT field.

In general, there are two strategies for improving CT image resolution: (1) hardware-oriented and (2) computational. First, more sophisticated hardware components can be used, including an x-ray tube with a fine focal spot size, detector elements of small pitch, and better mechanical precision for CT scanning. These hardware-oriented methods are generally expensive, increase CT system costs and radiation dose, and compromise imaging speeds. Especially, it is well known that high radiation dosage in a patient could induce genetic damages and cancerous diseases [4]–[6]. As a result, the second type of methods for resolution improvement [7]–[11] is more attractive, which is to obtain HRCT images from LRCT images. This computational deblurring job is a major challenge, representing a seriously ill-posed inverse problem [2], [12]. Our neural network approach proposed in this paper is computational, utilizing advanced network architectures. More details are as follows.

To improve the signal-to-noise ratio and image resolution, various algorithms were proposed. These algorithms can be broadly categorized into the following classes: (1) Model-based reconstruction methods [13]–[17]: These techniques explicitly model the image degradation process and regularize the reconstruction according to the characteristics of projection data. These algorithms promise an optimal image quality under the assumption that model-based priors can be effectively imposed; and (2) Learning-based (before deep learning) SR methods [18]–[20]: These methods learn a nonlinear mapping from a training dataset consisting of paired LR and HR images to recover missing high-frequency details. Especially, sparse representation-based approaches have attracted an increasing interest since it exhibits strong robustness in preserving image features, suppressing noise and artifacts. Dong et al. [20] applied adaptive sparse domain selection and adaptive regularization to obtain excellent SR results in terms of both visual perceptions and PSNR. Zhang et al. [19] proposed a patch-based technique for SR enhancements of 4D-CT images. These results demonstrate that learning-based SR methods can greatly enhance overall image quality but outcomes may still lose image subtleties and yield blocky appearance.

Recently, deep learning (DL) has been instrumental for computer vision tasks [21], [22]. Hierarchical features and representations derived from a convolutional neural network (CNN) are leveraged to enhance the discriminative capacity of visual quality, thus people have started developing SR models for natural images [23]–[27]. The key to the success of DL-based methods is its independence from explicit imaging models and backup by big domain-specific data. The image quality is optimized by learning features in an end-to-end manner. More importantly, once a CNN-based SR model is trained, achieving SR is a purely feed-forward propagation, which demands a very low computational overhead.

In the medical imaging field, DL is an emerging approach which has exhibited a great potential [28]–[30]. For several imaging modalities, DL-based SR methods were successfully developed [31]–[38]. Chen et al. [31] proposed a deep densely connected super-resolution network to reconstruct HR brain magnetic resonance (MR) images. More recently, Yu et al. [32] developed two advanced CNN-based models with a skip connection to promote high-frequency textures which are then fused with up-sampled images to produce SR images.

Very recently, adversarial learning [39], [40] has become increasingly popular, which enables CNNs to learn feature representations from complex data distributions, with unprecedented successes. Adversarial learning is performed based on a generative adversarial network (GAN), defined as a mini-max game in which two competing players are a generator $G$ and a discriminator $D$. In the game, $G$ is trained to learn a mapping from source images $\mathit{x}$ in a source domain $X$ to target images $\mathit{y}$ in the target domain $Y$. On the other hand, $D$ distinguishes the generated images $\stackrel{\mathit{ˆ}}{\mathit{y}}$ and the target images $\mathit{y}$ with a binary label. Once well trained, GAN is able to model a high-dimensional distribution of target images. Wolterink et al. [41] proposed an unsupervised conditional GAN to optimize the nonlinear mapping, successfully enhancing the overall image quality. Also, Mardani et al. [38] adopted a Compressed Sensing (CS) based MRI reconstruction method into the GAN-based network termed GANCS to reconstruct high-quality MR images. Also, in order to ensure data consistency in the learned manifold domain, a least-square penalty was applied to the training process.

However, there are still several major limitations in the DL-based SR imaging. First, existing supervised DL-based algorithms cannot address blind SR tasks without LR-HR pairs. In clinical practice, a limited number of LR and HR CT image pairs makes the supervised learning methods impractical since it is infeasible to ask patients to take multiple CT scans with additional radiation doses for paired CT images. Thus, it is essential to resort to semi-supervised learning. Second, utilizing the adversarial strategy can push the generator to learn an inter-domain mapping and produce compelling target images [42] but there is a potential risk that the network may yield features that are not exhibited in target images due to the degeneracy of the mapping. Since the optimal $G$ is capable of translating $X$ to $\hat{Y}$ distributed identically to $\mathit{y}$, the GAN network cannot ensure that the noisy input $\mathit{x}$ and predicted output $\stackrel{\mathit{ˆ}}{\mathit{y}}$ are paired in a meaningful way - there exist many mappings $G$ that may yield the same distribution over $\hat{Y}$. Consequently, the mapping is highly under-constrained. Furthermore, it is undesirable to optimize the adversarial objective in isolation: the model collapse problem may occur to map all inputs $\mathit{x}$ to the same output image $\stackrel{\mathit{ˆ}}{\mathit{y}}$ [40], [43], [44]. To address this issue, Cycle-consistent GANs (cycleGAN) was designed to improve the performance of generic GAN, and utilized for SR imaging [27]. Third, other limitations of GANs were also pointed out in [45]–[48]. How to steer a GAN learning process is not easy since G may collapse into a narrow distribution which cannot represent diverse samples from a real data distribution. Also, there is no interpretable metric for training progress. Fourth, as the number of layers increases, deep neural networks can derive a hierarchy of increasingly more complex and more abstract features. Frequently, to improve the SR imaging capability of a network, complex networks are often tried with hundreds of millions of parameters. However, given the associated computational overheads, they are hard to use in real-world applications. Fifth, local feature parts in the CT image have different scales. This feature hierarchy can provide more information to reconstruct images, but most DL-based methods [24], [25] neglect to use hierarchical features. Finally, the $L_2$ distance between $\stackrel{\mathit{ˆ}}{\mathit{y}}$ and $\mathit{y}$ is commonly used for the loss function to guide the training process of the network. However, the output optimized by the $L_2$ norm may suffer from over-smoothing as discussed in [49], [50], since the $L_2$ distance means to maximize the peak signal-to-noise rate (PSNR) [23].

Motivated by the aforementioned drawbacks, in this study we made major efforts in the following aspects. First, we present a novel residual CNN-based network in the CycleGAN framework to preserve high-resolution anatomical details with no task-specific regularization. Specially, we utilize the cycle-consistency constraint to enforce a strong across-domain consistency between $X$ and $Y$. Second, to address the training problem of GANs [40], [46], we use the Wasserstein distance or “Earth Moving” distance (EM distance) instead of the Jensen-Shannon (JS) divergence. Third, inspired by the recent work [51], we optimize the network according to several fundamental designing principles to alleviate computational overheads [52]–[54], which also helps prevent the network from over-fitting. Fourth, we cascade multiple layers to learn highly interpretable and disentangled hierarchical features. Moreover, we enable the information flow across skip-connected layers to prevent gradient vanishing [52]. Finally, we employ the $L_1$ norm instead of $L_2$ norm to refine deblurring, and we propose to use a jointly constrained total variation-based regularization as well, which leverages the prior information to reduce noise with a minimal loss in spatial resolution or anatomical information. Extensive experiments with three real datasets demonstrate that our proposed composite network can achieve an excellent CT SR imaging performance comparable to or better than that of the state-of-the-art methods [23]–[26], [55].

II. METHODS

Let us first review the SR problems in the medical imaging field. Then, we introduce the proposed adversarial nets framework and also present our SR imaging network architecture. Finally, we describe the optimization process.

A. Problem Statement

Let $\mathit{x}\in X$ be an input LR image and a matrix $\mathit{y}\in Y$ an output HR image, the conventional formulation of the ill-posed linear SR problem [18] can be formulated as

$$\mathit{x}=SH\mathit{y}+\mathit{ϵ},$$ (1)

where $SH$ denote the down-sampling and blurring system matrices, and $\mathit{ϵ}$ the noise and other factors. Note that in practice, both the system matrix and not-modeled factors can be non-linear, instead of being linear (i.e., neither scalable nor additive).

Our goal is to computationally improve noisy LRCT images obtained under a low-dose CT (LDCT) protocol to HRCT images. The main challenges in recovering HRCT images can be listed as follows. First, LRCT images contain different or more complex spatial variations, correlations and statistical properties than natural images, which limit the SR imaging performance of the traditional methods. Second, the noise in raw projection data is introduced to the image domain during the reconstruction process, resulting in unique noise and artifact patterns. This creates difficulties for algorithms to produce high image quality results. Finally, since the sampling and degradation operations are coupled and ill-posed, SR tasks cannot be performed beyond a marginal degree using the traditional methods, which cannot effectively restore some fine features and suffer from the risk of producing a blurry appearance and new artifacts. To address these limitations, here we develop an advanced neural network by composing a number of non-linear SR functional blocks for SR CT (SRCT) imaging along with the residual module to learn high-frequency details. Then, we perform adversarial learning in a cyclic manner to generate perceptually and quantitatively superior SRCT images.

B. Deep Cycle-Consistent Adversarial SRCT Model

1). Cycle-Consistent Adversarial Model:

Current DL-based algorithms use feed-forward CNNs to learn non-linear mappings parametrized by $\mathit{\theta }$, which can be written as:

$$\stackrel{\mathit{ˆ}}{\mathit{y}}=G_{\mathit{\theta }}\left(\mathit{x}\right).$$ (2)

In order to obtain a decent $\stackrel{\mathit{ˆ}}{\mathit{y}}$, a suitable loss function must be specified to encourage $G_{\mathit{\theta }}$ to generate a SR image based on the training samples so that

$$\stackrel{\mathit{ˆ}}{\mathit{\theta }}=\underset{\mathit{\theta }}{\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}}\sum _i ℒ\left({\stackrel{\mathit{ˆ}}{\mathit{y}}}_i\left(\mathit{\theta }\right),{\mathit{y}}_i\right),$$ (3)

where $\left({\mathit{x}}_i,{\mathit{y}}_i\right)$ are paired LRCT and HRCT images for training. To address the limitations mentioned in II-A, our cyclic SRCT model is shown in Fig. 1. The proposed model includes two generative mappings $G:X\to Y\left(G\right)$ and $F:Y\to X\left(F\right)$ given training samples ${\mathit{x}}_i\in X$ and ${\mathit{y}}_i\in Y$. The two mappings $G$ and $F$ are jointly trained to produce synthesized images in a way that confuse the adversarial discriminators $D_Y$ and $D_X$ respectively, which intend to identify whether the output of each generative mapping is real or artificial. i.e., given an LRCT image $\mathit{x}$, $G$ attempts to generate a synthesized image $\stackrel{\mathit{ˆ}}{\mathit{y}}$ highly similar to a real image $\mathit{y}$ so as to fool $D_Y$. In a similar way, $D_X$ attempts to discriminate between a reconstructed $\stackrel{\mathit{ˆ}}{\mathit{x}}$ from $F$ and a real $\mathit{x}$. The key idea is that the generators and discriminators are jointly/alternatively trained to improve their performance metrics synergistically. Thus, we have the following optimization problem:

$$\underset{G,F}{\mathrm{m}\mathrm{i}\mathrm{n}} \underset{D_Y,D_X}{\mathrm{m}\mathrm{a}\mathrm{x}} {ℒ}_{\mathrm{G}\mathrm{A}\mathrm{N}}\left(G,D_Y\right)+{ℒ}_{\mathrm{G}\mathrm{A}\mathrm{N}}\left(F,D_X\right).$$ (4)

Fig. 1.

Proposed GAN framework for SR CT imaging. Our approach uses two generators $G$ and $F$, and the corresponding adversarial discriminators $D_X$ and $D_Y$ respectively, where $X$ denotes a LR CT image and $Y$ is the HR CT counterpart. To regularize the training and deblurring processes, we utilize the generator-adversarial loss (adv), cycle-consistency loss (cyc), identity loss (idt), and joint sparsifying transform loss (jst) synergistically. In the supervised/semi-supervised mode, we also apply a supervision loss (sup) on $G$ and $F$. For brevity, we denote $G:X\to Y$ and $F:Y\to X$ as $G$ and $F$ respectively. * denotes that the loss is only trained in the supervised fashion.

To enforce the mappings between the source and target domains and regularize the training procedure, our proposed network combines four types of loss functions: adversarial loss (adv); cycle-consistency loss (cyc); identity loss (idt); joint sparsifying transform loss (jst).

2). Adversarial Loss:

For marginal matching [39], we employ adversarial losses to urge the generated images to obey the empirical distributions in the source and target domains. To improve the training quality, we apply the Wasserstein distance with gradient penalty [56] instead of the negative log-likelihood used in [39]. Thus, we have the adversarial objective with respect to $G$:

$$\begin{array}{r}\underset{G}{\mathrm{m}\mathrm{i}\mathrm{n}} \underset{D_Y}{\mathrm{m}\mathrm{a}\mathrm{x}} {ℒ}_{\mathrm{W}\mathrm{G}\mathrm{A}\mathrm{N}}\left(D_Y,G\right)={\mathbb{E}}_{\mathit{x}}[D(G(\mathit{x}))]-{\mathbb{E}}_{\mathit{y}}[D(\mathit{y})]\\ +\lambda {\mathbb{E}}_{\tilde{\mathit{y}}}[{\left({‖{\nabla }_{\tilde{\mathit{y}}}D(\tilde{\mathit{y}})‖}_2-1\right)}^2],\end{array}$$ (5)

where $\mathbb{E}(\cdot )$ denotes the expectation operator; the first two terms are in terms of the Wasserstein estimation, and the third term penalizes the deviation of the gradient norm of its input relative to one, $\tilde{\mathit{y}}$ is uniformly sampled along straight lines for pairs of $G\left(\mathit{x}\right)$ and $\mathit{y}$, and $\lambda$ is a regularization parameter. A similar adversarial loss ${\mathrm{m}\mathrm{i}\mathrm{n}}_F {\mathrm{m}\mathrm{a}\mathrm{x}}_{D_X} {ℒ}_{\text{WGAN}}\left(D_X,F\right)$ is defined for marginal matching in the reverse direction.

3). Cycle Consistency Loss:

Adversarial training is for marginal matching [39], [40]. However, in these earlier studies [43], [57], it was found that using adversarial losses alone cannot ensure the learned function can transform a source input successfully to a target output. To promote the consistency between $F\left(G\right(\mathit{x}\left)\right)$ and $\mathit{x}$, the cycle-consistency loss can be express as:

$$\begin{array}{l}{ℒ}_{\mathrm{C}\mathrm{Y}\mathrm{C}}(G,F)={\mathbb{E}}_{\mathit{x}}[\Vert F(G\left(\mathit{x}\right))\left.-x{\Vert }_1\right]\\ +{\mathbb{E}}_{\mathit{y}}\left[\Vert G\left(F\right(\mathit{y}\left)\right)-\mathit{y}{\Vert }_1\right],\end{array}$$ (6)

where $\Vert \cdot {\Vert }_1$ denotes the ${ℒ}_1$ norm. Since the cycle consistency loss encourages $F\left(G\right(\mathit{x}\left)\right)\approx \mathit{x}$ and $G\left(F\right(\mathit{y}\left)\right)\approx \mathit{y}$, they are referred to as forward cycle consistency and backward cycle consistency respectively. The domain adaptation mapping refers to the cycle-reconstruction mapping. In effect, it imposes shared-latent space constraints to encourage the source content to be preserved during the cycle-reconstruction mapping. In other words, the cycle consistency enforces latent codes deviating from the prior distribution in the cycle-reconstruction mapping. Additionally, the cycle consistency can help prevent the degeneracy in adversarial learning [58].

4). Identity Loss:

Since an HR image should be a refined version of the LR counterpart, it is necessary to use the identity loss to regularize the training procedure [43], [44]. Compared with the ${ℒ}_2$ loss, the ${ℒ}_1$ loss does not over-penalize large differences or tolerate small errors between estimated and target images. Thus, the ${ℒ}_1$ loss is preferred to alleviate the limitations of the ${ℒ}_2$ loss in this context. Additionally, the ${ℒ}_1$ loss enjoys the same fast convergence speed as that of the ${ℒ}_2$ loss. The ${ℒ}_1$ loss is formulated as follows:

$${ℒ}_{\mathrm{I}\mathrm{D}\mathrm{T}}\left(G,F\right)={\mathbb{E}}_{\mathit{y}}\left[\Vert G\left(\mathit{y}\right)-\mathit{y}{\Vert }_1\right]+{\mathbb{E}}_{\mathit{x}}\left[\Vert F\left(\mathit{x}\right)-\mathit{x}{\Vert }_1\right].$$ (7)

We follow the same training baseline as in [44]; i.e., in the bi-directional mapping, the size of $G\left(\mathit{y}\right)$ (or $F\left(\mathit{x}\right)$) is the same as that of $\mathit{y}$ (or $\mathit{x}$).

5). Joint Sparsifying Transform Loss:

The total variation (TV) has demonstrated the state-of-the-art performance in promoting image sparsity and reducing noise in piecewise-constant images [59], [60]. To express image sparsity, we formulate a nonlinear TV-based loss with the joint constraints as follows:

$${ℒ}_{\mathrm{J}\mathrm{S}\mathrm{T}}\left(G\right)=\tau ‖G\left(\mathit{x}\right){\Vert }_{\mathrm{T}\mathrm{V}}+\left(1-\tau \right)‖\mathit{y}-G\left(\mathit{x}\right){\Vert }_{\mathrm{T}\mathrm{V}},$$ (8)

where $\tau$ is a scaling factor. Intuitively, the above constrained minimization combines two components: the first component is used for sparsifying reconstructed images and alleviating conspicuous artifacts, and the second helps preserve anatomical characteristics by minimizing the difference image $\mathit{y}-G\left(\mathit{x}\right)$. Essentially, these two components require a joint minimization under the bidirectional constraints. In this paper, the control parameter $\tau$ was set to 0.5. In the case of $\tau =1$, the ${ℒ}_{\mathrm{J}\mathrm{S}\mathrm{T}}\left(G\right)$ is regarded as the conventional TV loss.

6). Overall Objective Function:

In the training process, our proposed network is fine-tuned in an end-to-end manner to minimize the following overall objective function:

$$\begin{array}{l}{ℒ}_{\mathrm{G}\mathrm{A}\mathrm{N}-\mathrm{C}\mathrm{I}\mathrm{R}\mathrm{C}\mathrm{L}\mathrm{E}}={ℒ}_{\mathrm{W}\mathrm{G}\mathrm{A}\mathrm{N}}\left(D_Y,G\right)+{ℒ}_{\mathrm{W}\mathrm{G}\mathrm{A}\mathrm{N}}\left(D_X,F\right)\\ +{\lambda }_1{ℒ}_{\mathrm{C}\mathrm{Y}\mathrm{C}}(G,F)+{\lambda }_2{ℒ}_{\mathrm{I}\mathrm{D}\mathrm{T}}(G,F)\\ +{\lambda }_3{ℒ}_{\mathrm{J}\mathrm{S}\mathrm{T}}\left(G\right),\end{array}$$ (9)

where ${\lambda }_1,{\lambda }_2$, and ${\lambda }_3$ are parameters for balancing among different penalties. We call this modified cycleGAN as the GAN-CIRCLE as summarized in the title of this paper.

7). Supervised Learning With GAN-CIRCLE:

In the case where we have access to paired dataset, we can render SRCT problems to train our model in a supervised fashion. Given the training paired data from the true joint, i.e., $(\mathit{x},\mathit{y})\sim P_{data}(\mathit{X},\mathit{Y})$, we can define a supervision loss as follows:

$${ℒ}_{\mathrm{S}\mathrm{U}\mathrm{P}}\left(G,F\right)={\mathbb{E}}_{\left(\mathit{x},\mathit{y}\right)}\left[\Vert G\left(\mathit{x}\right)-\mathit{y}{\Vert }_1\right]+{\mathbb{E}}_{\left(\mathit{x},\mathit{y}\right)}\left[\Vert \left(F\left(\mathit{y}\right)-\mathit{x}{\Vert }_1\right]\right..$$ (10)

C. Network Architecture

Although more layers and larger model size usually result in the performance gain, for real application we designed a lightweight model to validate the effectiveness of GAN-CIRCLE. The full architecture and details of GAN-CIRCLE are provided in the supplementary material.

III. EXPERIMENTS AND RESULTS

We discuss our experiments in this section. We first introduce the datasets we utilize and then describe the implementation details and parameter settings in our proposed methods. We also compare our proposed algorithms with the state-of-the-art SR methods quantitatively and qualitatively. We further evaluate our results in reference to the state-of-the-art, and demonstrate the robustness of our methods in the real SR scenarios. Then, we present the detailed diagnostic quality assessments from expert radiologists. Next, we progressively modify some of key elements to investigate the best trade-off between performance and speed, and evaluate the relations between performance and parameters. Finally, we further illustrate the effect of the filter size, of the number of layers, and of the training patch size with respect to the training and testing datasets. Note that we use the default parameters of all the evaluated methods. The source code is released at https://github.com/charlesyou999648/GAN-CIRCLE.

A. Training Datasets

In this study, we used two high-quality sets of training images to demonstrate the fidelity and robustness of the proposed GAN-CIRCLE. As shown in Figs. 2–5, these two datasets are of very different characteristics. Note the descriptions of detailed data preprocessing are provided in the supplementary material.

Fig. 2.

Visual comparison of SRCT Case 1 from the Tibia dataset. The restored bony structures are shown in the red and yellow boxes in Fig. 3. The display window is [−900, 2000] HU.

Fig. 5.

Visual comparison of SRCT Case 2 from the abdominal dataset. The display window is [−160, 240] HU. The restored anatomical features are shown in the red and yellow boxes. (Zoomed for visual clarity).

1). Tibia Dataset:

This micro-CT image dataset reflects twenty-five fresh-frozen cadaveric ankle specimens which were removed at mid-tibia from 17 body donors (mean age at death ± SD: 79.6 ± 13.2 Y; 9 female). After the soft tissue were removed and the tibia was dislocated from the ankle joint, each specimen was scanned on a Siemens microCAT II (Preclinical Solutions, Knoxville, TN, USA) in the cone beam imaging geometry. The micro-CT parameters are briefly summarized as follows: a tube voltage 100 kV, a tube current 200 mAs, 720 projections over a range of 220 degrees, an exposure time of 1.0 sec per projection, and the filter backprojection (FBP) method was utilized to produce 28.8 μm isotropic voxels. Since CT images are not isotropic in each direction, for convenience of our previous analysis [61], we convert micro-CT images to 150 μm using a windowed sync interpolation method. In this study, the micro-CT images we utilized as HR images were prepared at 150 μm voxel size, as the target for SR imaging based of the corresponding LR images at 300 μm voxel size. The full description is in [61]. We target 1X resolution improvement.

2). Abdominal Dataset:

This clinical dataset is authorized by Mayo Clinic for 2016 NIH-AAPM-Mayo Clinic Low Dose CT Grand Challenge. The dataset contains 5, 936 full dose CT images from 10 patients with the reconstruction interval and slice thickness of 0.8 mm and 1.0 mm respectively. The original CT images were generated by multidetector row CT (MDCT) with the image size of 512 × 512. The projection data is from 2, 304 views per scan. The HR images, with voxel size 0.74 × 0.74 × 0.80 mm³, were reconstructed using the FBP method from all 2, 304 projection views. More detailed information of the dataset is given in [62].

B. Performance Comparison

In this study, we compared the proposed GAN-CIRCLE with the state-of-the-art methods: adjusted anchored neighborhood regression A⁺ [55], FSRCNN [24], ESPCN [26], LapSRN [25], and SRGAN [23]. For clarity, we categorized the methods into the following classes: the interpolation-based, dictionary-based, PSNR-oriented, and GAN-based methods. Especially, we trained the publicly available FSRCNN, ESPCN, LapSRN, and SRGAN with our paired LR and HR images. To demonstrate the effectiveness of the DL-based methods, we first denoised the input LR images and then super-resolved the denoised CT image using the typical method A⁺. BM3D [63] is one of the classic image domain denoising algorithms, which is efficient and powerful. Thus, we preprocessed the noisy LRCT images with BMD3, and then super-solved the denoised images by A⁺.

We evaluated three variations of the proposed method: (1) G-Forward (G-Fwd), which is the forward generator of GAN-CIRCLE, (2) G-Adversarial (G-Adv), which uses the adversarial learning strategy, and (3) the full-fledged GAN-CIRCLE. To emphasize the effectiveness of the GAN-CIRCLE structure, we first trained the three models using the supervised learning strategy, and then trained our proposed GAN-CIRCLE in the semi-supervised scenario (GAN-CIRCLE^s), and finally implement GAN-CIRCLE in the unsupervised manner (GAN-CIRCLE^u). In the semi-supervised settings, two datasets were created separately by randomly splitting the dataset into the paired and unpaired dataset with respect to three variants: 100%, 50%, and 0% paired. To better evaluate the performance of each method, we use the same size of the dataset for training and testing.

We validated the SR performance in terms of three widely-used image quality metrics: Peak signal-to-noise ratio (PSNR), Structural Similarity (SSIM) [64], and Information Fidelity Criterion (IFC) [65]. Through extensive experiments, we compared all the above-mentioned methods on the two benchmark datasets described in Section III-A. Due to the space limit, we present network architecture details, and the implementation details are presented in the supplementary material.

C. Experimental Results With the Tibia Dataset

We evaluated the proposed algorithms against the state-of-the-art algorithms on the tibia dataset. We present typical results in Fig. 2. It can be seen that our proposed GAN-CIRCLE recovers more fine subtle details and captures more anatomical information in Fig. 3. It is worth mentioning that Fig. 2 shows that there are severe distortions of the original images but SRGAN generates compelling results in Figs. 5–8, which indicate VGG network is a task-specific network which can generate images with excellent image quality. We argue that the possible reason is that the VGG network [66] is a pre-trained CNN-based network based on natural images with structural characteristic correlated with the content of medical images [67]. Fig. 3 shows that the proposed GAN-CIRCLE^s can predict images with sharper boundaries and richer textures than GAN-CIRCLE^u which learns additional anatomical information from the unpaired samples. The difference images are shown in the Figs. 4. The difference images were generated by subtracting the generated image from the reference image. We compared our method with adjusted anchored neighborhood regression A⁺ [55], FSRCNN [24], ESPCN [26], LapSRN [25], SRGAN [23], G-Forward (G-Fwd), G-Adversarial (G-Adv). The quantitative results are in Table I. The results demonstrate that the G-Forward achieves the highest scores using the evaluation metrics, PSNR and SSIM, which outperforms all other methods. However, it has been pointed out in [68], [69] that high PSNR and SSIM values cannot guarantee a visually favorable result. Non-GAN based methods (FSRCNN, ESPCN, LapSRN) may fail to recover some fine structure for diagnostic evaluation, such as shown by zoomed boxes in Fig. 3. Quantitatively, GAN-CIRCLE achieves the second best values in terms of SSIM and IFC. It has been pointed out in [70] that IFC value is correlated well with the human perception of SR images. Our GAN-CIRCLE^s obtained comparable results qualitatively and quantitatively. Table I shows that the proposed semi-supervised method performs similarly compared to the fully supervised methods on the tibia dataset. In general, our proposed GAN-CIRCLE can generate more pleasant results with sharper image contents.

Fig. 3.

Zoomed regions of interest (ROIs) marked by the red rectangle in Fig. 2. The restored image with GAN-CIRCLE reveals subtle structures better than the other variations of the proposed neural network, especially in the marked regions. The display window is [−900, 2000] HU.

Fig. 8.

Visual comparison of SRCT Case 4 from the real dataset. The display window is [139, 1913] HU. The restored bony structures are shown in the red and yellow boxes. (Zoomed for visual clarity).

Fig. 4.

Absolute difference images relative to the original HR image from the Tibia dataset. The display window is [0, 900] HU.

TABLE I.

Quantitative Evaluation of State-of-the-Art SR Algorithms. Red and Blue Indicate the Best and the Second Best Performance, Respectively

	Tibia Case			Abdominal Case			Real Case 1			Real Case 2
	PSNR	SSIM	IFC	PSNR	SSIM	IFC	PSNR	SSIM	IFC	PSNR	SSIM	IFC
A+	26.496	0.696	3.028	28.154	0.589	1.899	27.877	0.804	0.249	27.037	0.778	0.236
FSRCNN	28.360	0.923	3.533	30.950	0.924	2.285	35.384	0.830	0.265	33.643	0.805	0.237
ESPCN	28.361	0.921	3.520	30.507	0.923	2.252	35.378	0.830	0.278	33.689	0.805	0.245
LapSRN	28.486	0.923	3.533	30.985	0.925	2.299	35.372	0.830	0.277	33.711	0.805	0.244
SRGAN	21.924	0.389	1.620	28.550	0.871	1.925	33.002	0.737	0.232	31.775	0.701	0.220
G-Fwd	28.649	0.931	3.618	31.282	0.925	2.348	35.227	0.829	0.276	33.589	0.803	0.236
G-Adv	26.945	0.676	2.999	26.930	0.889	1.765	32.518	0.725	0.199	31.712	0.700	0.210
GAN-CIRCLE	27.742	0.895	3.944	30.720	0.924	2.435	-	-	-	-	-	-
GAN-CIRCLEs	28.402	0.907	3.943	29.988	0.902	2.367	33.194	0.829	0.285	31.252	0.804	0.245
GAN-CIRCLEu	27.255	0.891	2.713	28.439	0.894	2.019	32.138	0.824	0.283	30.641	0.796	0.232

D. Experimental Results on the Abdominal Dataset

We further compared the above-mentioned algorithms on the abdominal benchmark dataset. A similar trend can be observed on this dataset. Our proposed GAN-CIRCLE can preserve better anatomical informations and more clearly visualize the portal vein as shown in Fig. 5. These results demonstrate that PSNR-oriented methods (FSRCNN, ESPCN, LapSRN) can significantly suppress the noise and artifacts. However, it suffers from low image quality as judged by the human observer since it assumes that the impact of noise is independent of local image features, while the sensitivity of the Human Visual System (HVS) to noise depends on local contrast, intensity and structural variations. Fig. 5 displays the LRCT images processed by GAN-based methods (SRGAN, G-Adv, GAN-CIRCLE, GAN-CIRCLE^s, and GAN-CIRCLE^u) with improved structural identification. It can also observed that the GAN-based models also introduce strong noise into results. For example, there exist tiny artifacts on the results of GAN-CIRCLE^u. As the SR results shown in Fig. 5, our proposed approaches (GAN-CIRCLE, GAN-CIRCLE^s) are capable of retaining high-frequency details to reconstruct more realistic images with relatively lower noise compared with the other GAN-based methods (G-Adv, SRGAN). In the Figs. 6, we showed the difference images by subtracting the generated image from the reference image. We compared our method with adjusted anchored neighborhood regression A+ [55], FSRCNN [24], ESPCN [26], LapSRN [25], SRGAN [23], G-Forward (G-Fwd), G-Adversarial (G-Adv). Table I show that G-Fwd achieves the best performance in PSNR. Our proposed methods GAN-CIRCLE and GAN-CIRCLE^s both obtain the pleasing results in terms of SSIM and IFC. In other words, the results show that the proposed GAN-CIRCLE and GAN-CIRCLE^s generate more visually pleasant results with sharper edges on the abdominal dataset than the competing state-of-the-art methods.

Fig. 6.

Absolute difference images relative to the original HR image from the abdominal dataset. The display window is [100, 280] HU.

E. Super-Resolving Clinical Images

We analyzed the performance of the SR methods in the simulated SRCT scenarios in Sections III-C and III-D. These experimental results show that the DL-based methods are very effective in addressing the ill-posed SRCT problems with two significant features. First, SRCT aims at recovering a HRCT image from a LRCT images under a low-dose protocol. Second, most DL-based methods assume the paired LRCT images and HRCT images are matched, an assumption which is likely to be violated in clinical practice. In other words, the above-evaluated datasets were simulated, and thus the fully supervised algorithms can easily cope with SRCT tasks, with exactly matched training samples. Our further goal is to derive the semi-supervised scheme to handle unmatched/unpaired data with a relative lack of matched/paired data to address real SRCT tasks. In this subsection, we demonstrate a strong capability of the proposed methods in the real applications using a small amount of mismatched paired LRCT and HRCT images as well as a high flexibility of adapting to various noise distributions.

1). Practical SRCT Implementation Details:

We first obtained 3 LRCT and HRCT image pairs using a deceased mouse on the same scanner with two scanning protocols. The micro-CT parameters are as follows: X-ray source circular scanning, 60 kVp, 134 mAs, 720 projections over a range of 360 degrees, exposure 50 ms per projection, and the micro-CT images were reconstructed using the Feldkamp-Davis-Kress (FDK) algorithm [71] in practice: HRCT image size 1450 × 1450, 600 slices at 24 μm isotropic voxel size, and the LRCT image size 725 × 725, 300 slices at 48 μm isotropic voxel size. Then, we compared with the state-of-the-art super-resolution methods. Since the real data are unmatched, we accordingly evaluated our proposed GAN-CIRCLE^s and GAN-CIRCLE^u networks for 1X resolution improvement.

2). Comparison With the State-of-the-Art Methods:

The quantitative results were summarized for all the involved methods in Table I. The PSNR-oriented approaches, such as FSRCNN, ESPCN, LapSRN, and our G-Fwd, yield higher PSNR and SSIM values than the GAN-based methods. It is not surprising that the PSNR-oriented methods obtained favorable PSNR values since their goal is to minimize per-pixel distance to the ground truth. However, our GAN-CIRCLE^s and GAN-CIRCLE^u achieved the highest IFC among all the SR methods. Our method GAN-CIRCLE^s obtained the second best results in term of SSIM. The visual comparisons are given in Figs. 7 and 8. To demonstrate the robustness of our methods, we examined anatomical features in the lung regions and the bone structures of the mice, as shown in Figs. 7 and 8 respectively. It is observed that the GAN-based approaches performed favorably over the PSNR-oriented methods in term of perceptual quality as illustrated in Figs. 7 and 8. Fig. 7 confirms that the PSNR-oriented methods produced blurry results especially in the lung regions, while the GAN-based methods restored anatomical contents satisfactorily. In Fig. 8, it is notable that our methods GAN-CIRCLE^s and GAN-CIRCLE^u performed better than the other methods in terms of recovering structural information and preserving edges. These SR results demonstrate that our proposed methods can provide better visualization of bone and lung microarchitecture with sharp edge and rich texture.

Fig. 7.

Visual comparison of SRCT Case 3 from the real dataset. The display window is [139, 1913] HU. The restored anatomical features are shown in the red and yellow boxes. (Zoomed for visual clarity).

F. Diagnostic Quality Assessment

We invited three board-certified radiologists with mean clinical CT experience of 12.3 years to perform independent qualitative image analysis on 10 sets of images from two benchmark dataset (Tibia and Abdominal Dataset). Each set includes the same image slice but generated using different methods. We label HRCT and LRCT images in each set as reference. The 10 sets of images from two datasets were randomized and deidentified so that the radiologists were blind to the post-processing algorithms. Image sharpness, image noise, contrast resolution, diagnostic acceptability, and overall image quality were graded on a scale from 1 (worst) to 5 (best). A score of 1 refers to a ‘non-diagnostic’ image, while a score of 5 means an ‘excellent’ diagnostic image quality. The mean scores with their standard deviation are presented in Table III. The radiologists confirmed that GAN-based methods (G-Adv, SRGAN, GAN-CIRCLE, GAN-CIRCLE^s and GAN-CIRCLE^u) provide sharper images with better texture details, while PSNR-oriented algorithms (FSRCNN, ESPCN, LapSRN, G-Fwd) give the higher noise suppression scores. Table III shows that our proposed GAN-CIRCLE and GAN-CIRCLE^s achieve comparable results, while outperforming the other methods in terms of image sharpness, contrast resolution, diagnostic acceptability, and overall image quality.

TABLE III.

Diagnostic Quality Assessment in Terms of Subjective Quality Scores for Different Algorithms (Mean±Stds). Red and Blue Indicate the Best and the Second Best Performance, Respectively

	Tibia Dataset					Abdominal Dataset
	Image Sharpness	Image Noise	Contrast Resolution	Diagnostic Acceptance	Overall Quality	Image Sharpness	Image Noise	Contrast Resolution	Diagnostic Acceptance	Overall Quality
A+	2.34±0.47	2.54±0.68	2.52±0.67	1.98±0.59	2.37±0.97	2.74±0.75	3.07±0.96	2.61±0.69	2.35±0.57	2.74±0.71
FSRCNN	2.85±0.94	3.16±0.57	2.54±0.96	2.77±0.69	3.27±0.76	3.07±0.89	3.55±0.50	2.94±0.78	2.92±0.58	3.09±0.53
ESPCN	2.82±0.86	3.18±0.51	2.58±0.46	2.95±0.46	3.49±0.66	2.95±1.43	3.39±0.80	2.85±0.63	2.76±0.83	3.06±0.85
LapSRN	2.91±0.88	3.49±0.70	2.69±0.56	3.01±0.78	3.63±0.61	3.01±0.56	3.58±0.81	2.83±0.71	3.25±0.92	3.11±0.78
SRGAN	1.94±0.37	2.71 ±0.23	1.91±0.71	1.75±0.83	1.93±1.01	3.35±0.97	3.23±1.01	3.27±0.92	3.46±1.11	3.41 ±0.94
G-Fwd	2.99±0.42	3.59±0.57	3.07±0.91	3.45 ±1.02	3.70±0.71	3.25±0.94	3.53±0.70	2.95±0.57	3.38±0.93	3.09±0.55
G-Adv	2.89±0.86	3.13Ü.02	3.02±0.58	3.29±0.69	3.62±0.67	3.45±1.12	3.34±0.81	3.31±0.86	3.48±0.77	3.32±0.82
GAN-CIRCLE	3.12±0.73	3.40±0.43	3.17Ì0.46	3.6Ü0.36	3.79±0.72	3.59±0.41	3.41 ±0.42	3.51 ±0.66	3.64±0.54	3.62±0.41
GAN-CIRCLEs	3.02±0.78	3.14±0.68	3.12±0.88	3.47±0.67	3.71±0.76	3.48±0.81	3.29±0.80	3.42±0.78	3.57±0.68	3.51 ±0.46
GAN-CIRCLEu	2.91±0.82	3.32±0.89	3.08±0.94	3.32±0.48	3.57±0.52	3.46±0.73	3.39±1.04	3.39±0.50	3.54±0.53	3.34±1.01

G. Model and Performance Trade-Offs

1). Filter Size:

Here we compared the GAN-CIRCLE network sensitivities with respect to different filter sizes. First, we fixed the filter size of the reconstruction network and enlarged the filter size of the feature extraction network to 5×5 and 7×7 respectively. Note that all the other settings remained the same as that in Supplementary Material Section I–B. The average PSNR with the filter size 3 × 3 is slightly higher than that with the filter size 5 × 5 and 7 × 7 shown in Fig. 9a. In general, the reasonably larger filter size could help capture larger structural features, leading to a performance gain [72]. In the case of the Tibia dataset, utilizing filter size 3 × 3 is sufficient to grasp small structural information. Considering the tiny structural texture with small pixel size in the case of bone images, the filter size 3 × 3 is already good enough.

Fig. 9.

The test convergence curve of GAN-CIRCLE on the Tibia dataset. (a) The influence of different filter sizes, (b) The influence of different layers, (c) The influence of different training patch sizes, and (d) Average PSNR results over GAN-CIRCLE on the Tibia dataset with respect to the parameters λ₁, λ₂, λ₃. Note that the parameter λ = 0 (λ = ∞) indicates that the SR model was only optimized with respect to the corresponding loss.

2). Number of Layers:

Recent studies reported in [72], [73] suggest that training a network could benefit from increasing the network depth moderately. Here we evaluated different network depths by adjusting the number of non-linear mapping layers in the feature extraction network to 6, 9, 12, 15 layers respectively in the case of the Tibia dataset. Note that all the other settings remained the same as that in Supplementary Material Section I–B, and our proposed GAN-CIRCLE used the twelve-layer network. It can be seen in Fig. 9b that the twelve-layer network is superior to the six-layer and nine-layer networks, respectively. Furthermore, it is found that deeper networks cannot always do better. Specifically, the performance of the fifteen-layer network did not outperform the twelve-layer network. The observation that “deeper” does not mean “better” was also reported in [74], [75]. Therefore, we have selected the twelve-layer networks in this study.

3). Training Patch Size:

In general, the benefit of training a CNN-based network with patches is two-fold. First, a properly truncated receptive field can reduce the complexity of the network while still capturing the richness of local anatomical information [66], [76], [77]. Second, the use of patches significantly increases the number of training samples [41], [77]. Here we respectively experimented with patch sizes 48 × 48, 64×64, 80×80, and 96×96 respectively on the Tibia dataset. The results are shown in Fig. 9c. It is observed that large training patch sizes do not show any improvement in term of the average PSNR. As a trade-off, we used the patch size 64 × 64 in our investigation.

IV. DISCUSSIONS

SR imaging promises multiple benefits in medical applications; i.e., depicting bony details, lung structures, and implanted stents, and potentially enhancing radiomics analysis. As a result, X-ray computed tomography can provide compelling practical benefit in biological evaluation.

High resolution micro-CT is well-suited for bone imaging. Osteoporosis, characterized by reduced bone density and structural degeneration of bone, greatly diminishes bone strength and increases the risk of fracture [78]. Histologic studies have convincingly demonstrated that bone micro-structural properties are strong determinants of bone strength and fracture risk [79]–[81]. Modern whole-body CT technologies, benefitted with high spatial resolution, ultra-high speed scanning, relatively-low dose radiation, and large scan length, allows quantitative characterization of bone micro-structure [61]. However, the state-of-the-art CT imaging technologies only allow the spatial resolution comparable or slightly higher than human trabecular bone thickness (100 − 200 μm⁶ [82]) leading to fuzzy representations of individual trabecular bone micro-structure with significant partial volume effects that add significant errors in measurements and interpretations. The spatial resolution improvements in bone micro-structural representation will largely reduce such errors and improve the generalizability of bone micro-structural measures from multi-vendor CT scanners by homogenizing spatial resolution.

Besides revealing micro-architecture, CT scans of the abdomen and pelvis are diagnostic imaging tests used to help detect diseases of the small bowel and colon, kidney stone, and other internal organs, and are often used to determine the cause of unexplained symptoms. With rising concerns over increased lifetime risk of cancer by radiation dose associated with CT, several studies have assessed manipulation of scanning parameters and newer technologic developments as well as the adoption of advanced reconstruction techniques for radiation dose reduction [83]–[85]. However, in practice, the physical constraints of system hardware components and radiation dose considerations constrain the imaging performance, and computational means are necessary to optimize image resolution. For the same reason, high-quality/high-dose CT images are not often available, which means that there are often not enough paired data to train a hierarchical deep generative model.

Our results have suggested an interesting topic on how to utilize unpaired data so that the imaging performance could be improved. In this regard, the use of the adversarial learning as the regularization term for SR imaging is a new mechanism to capture anatomical information. In this work, we have confirmed the following expected performance order: GAN-CIRCLE > GAN-CIRCLE^s > GAN-CIRCLE^u, and even in the unsupervised context we still have decent deblurring effects. Our proposed semi-supervised learning method has achieved the compelling results with abdominal and mouse datasets. Specifically, as listed in Tables I, II, and III, the proposed semi-supervised methods achieved promising quantitative results. However, it should be noted that the existing GAN-based methods introduce additional noise to the results, as seen in Section III-C and III-D. To cope with this limitation, we have incorporated the cycle-consistency so that the network can learn a complex deterministic mapping to improve image quality. The enforcement of identity and supervision allows the model to master more latent structural information to improve image resolution. Also, we have used the Wasserstein distance to stabilize the GAN training process. Moreover, typical prior studies used complex inference to learn a hierarchy of latent variables for HR imaging, which is hard to be utilized in medical applications. Thus, we have designed an efficient CNN-based network with skip-connection and network in network techniques. In the feature extraction network, we have optimized the network structures and reduced the computational complexity by applying a small amount of filters in each Conv layer and utilizing the ensemble learning model. Both local and global features are cascaded through skip connections before being fed into the restoration/reconstruction network.

TABLE II.

Statistical Properties of the Images in Figs. 2, 5, 7, and 8. Red and Blue Indicate the Best and the Second Best Performance, Respectively

	Tibia Case		Abdominal Case		Real Case 1		Real Case 2
	Mean	SDs	Mean	SDs	Mean	SDs	Mean	SDs
HRCT	−321.571	676.683	125.427	176.351	485.584	305.261	484.394	332.121
LRCT	−310.300	648.118	129.262	172.767	308.147	288.917	308.789	314.800
A+	−316.080	637.457	128.724	170.591	301.891	288.222	302.525	313.994
FSRCNN	−333.806	657.914	125.102	174.567	480.237	291.184	481.097	318.084
ESPCN	−330.660	660.538	125.500	174.771	482.686	291.445	483.427	317.809
LapSRN	−322.968	667.776	125.397	174.883	483.903	292.359	484.605	319.162
SRGAN	−255.065	628.754	122.264	167.347	469.796	296.98	469.641	321.566
G-Fwd	−323.492	670.247	125.130	174.607	484.833	292.336	485.426	319.307
G-Adv	−314.155	643.952	123.992	173.391	471.312	302.897	471.853	328.565
GAN-CIRCLE	−320.519	676.526	125.330	175.607	-	-	-	-
GAN-CIRCLEs	−320.417	680.264	125.304	175.389	472.359	304.110	473.182	330.182
GAN-CIRCLEu	−316.080	681.024	125.961	176.120	467.897	288.376	308.538	334.277

Although our model has achieved compelling results, there still exist some limitations. First, the proposed GAN-CIRCLE requires much longer training time than other standard GAN-based methods, which generally requires 1–2 days. Future work in this aspect should consider more principled ways of designing more efficient architectures that allow for learning more complex structural features with less complex networks at less computational cost and lower model complexity. Second, although our proposed model can generate more plausible details and better anatomical details, all subtle structures may not be always faithfully recovered. It has been also observed that the recent literature [86] mentions that the Wasserstein distance may yield the biased sample gradients, is subject to the risk of incorrect minimum, and not well suitable for stochastic gradient descent searching. In the future, experiments with the variants of GANs are highly recommended. Finally, we notice that the network with the adversarial training can generate more realistic images. However, the restored images cannot be uniformly consistent to the original high-resolution images. Also, the recent literature [87]demonstrates that CycleGAN model learn to hide reconstruction details in imperceptible noise (high-frequency signal). This could theoretically be avoided by strictly enforcing the latent space assumption with added losses. It is also mentioned that the cycle-consistency loss may make the CycleGAN network vulnerable to adversarial attacks. Increasing the domain entropy with additional hidden variables is recommended. To make further progress, we may also undertake efforts to add more constraints such as the sinogram consistence and the low-dimensional manifold constraint to decipher the relationship between noise, blurry appearances of images and the ground truth, and even develop an adaptive and/or task-specific loss function.

V. CONCLUSIONS

In this paper, we have established a cycle Wasserstein regression adversarial training framework for CT SR imaging. Aided by unpaired data, our approach learns complex structured features more effectively with a limited amount of paired data. At a low computational cost, the proposed network G-Forward can achieve the significant SR gain. In general, the proposed GAN-CIRCLE has produced promising results in terms of preserving anatomical information and suppressing image noise in a purely supervised and semi-supervised learning fashion. Visual evaluations by the expert radiologists confirm that our proposed GAN-CIRCLE networks have brought superior diagnostic quality, which is consistent with systematic quantitative evaluations in terms of traditional image quality measures.