Reverberation Noise Suppression in Ultrasound Channel Signals Using a 3D Fully Convolutional Neural Network

Abstract

Diffuse reverberation is ultrasound image noise caused by multiple reflections of the transmitted pulse before returning to the transducer, which degrades image quality and impedes the estimation of displacement or flow in techniques such as elastography and Doppler imaging. Diffuse reverberation appears as spatially incoherent noise in the channel signals, where it also degrades the performance of adaptive beamforming methods, sound speed estimation, and methods that require measurements from channel signals. In this paper, we propose a custom 3D fully convolutional neural network (3DCNN) to reduce diffuse reverberation noise in the channel signals. The 3DCNN was trained with channel signals from simulations of random targets that include models of reverberation and thermal noise. It was then evaluated both on phantom and in-vivo experimental data. The 3DCNN showed improvements in image quality metrics such as generalized contrast to noise ratio (GCNR), lag one coherence (LOC) contrast-to-noise ratio (CNR) and contrast for anechoic regions in both phantom and in-vivo experiments. Visually, the contrast of anechoic regions was greatly improved. The CNR was improved in some cases, however the 3DCNN appears to strongly remove uncorrelated and low amplitude signal. In images of in-vivo carotid artery and thyroid, the 3DCNN was compared to short-lag spatial coherence (SLSC) imaging and spatial prediction filtering (FXPF) and demonstrated improved contrast, GCNR, and LOC, while FXPF only improved contrast and SLSC only improved CNR.

I. Introduction

RECENT epidemiological studies in obesity found that 71.2% of the American population is either overweight or obese [1]. Overweight and obese individuals have been shown to have increased incidence of poor image quality and inadequate visualization of anatomy with ultrasound [2]. There are numerous factors that contribute to poor image quality in ultrasound, and many of them are present in obese individuals. These include the inability to obtain a good acoustic window, the high attenuation of fatty or scar tissue, and the thickness of the fat layers, inhomogeneities in attenuation, variations in the speed of sound of tissue, off-axis scattering, and diffuse reverberation [3], [4].

Diffuse reverberation noise arises when an ultrasound wave-front reflects multiple times between subcutaneous tissue layers before returning to the transducer, causing unwanted signal that overlays deeper tissue and imaging targets. Diffuse reverberation noise is a spatially incoherent noise across the receiving aperture of an ultrasound system with a temporal bandwidth comparable to the ultrasound transducer [5], [6].

Overweight individuals have more subcutaneous fat and supporting connective tissue than normal individuals, and the acoustic impedance differences between these connective tissues and the surrounding fat is a large contributor to reverberation noise [7], [8]. This can manifest as a loss of contrast in hypoechoic or anechoic regions, such as cysts and blood vessels, and make it difficult to resolve small features during imaging because diffuse reverberation noise frequently has an appearance very similar to tissue speckle [9]. In addition to degrading B-mode image quality, diffuse reverberation noise introduces errors in techniques that rely on radiofrequency (RF) channel or beamformed data, such as phase aberration correction [7], adaptive beamforming techniques [10], sound speed estimation [11], shear-wave estimation [12], [13] and Doppler imaging [14].

Many promising reverberation removal techniques have been proposed. These include multi-phase apodization with cross-correlation (MPAX) [5] and the short lag spatial coherence (SLSC) beamformer [15], [16]. However, these techniques do not provide channel signals as an output and cannot be used to as a preprocessing filter for techniques that rely on channel signals. In addition, SLSC does not preserve the phase information, and cannot be used to preprocess signals for techniques such as Doppler and elastography imaging. Coupling high frequency pulses onto lower frequency carrier waves (SURF) [17] has also been proposed with promising in-vivo results, but the frequency range required for this method is not achievable with current conventional transducers [18].

Reverberation suppression methods that preserve the channel signals include tissue harmonic imaging (THI) [19], aperture domain model image reconstruction (ADMIRE) [20] and spatial prediction filtering (FXPF) [21]. THI is currently the standard of practice for removing reverberation noise, however it does not completely eliminate this source of noise [9]. FXPF, which uses an auto-regressive model to create a spatial filter in the frequency domain to remove off-axis signals, shows promising results in reducing reverberation noise in simulation and phantom channel signals [22]. However, this method requires a matrix inversion for every pixel and temporal frequency for multiple iterations and is therefore computationally expensive. ADMIRE decomposes the aperture domain signal into a sum of signals from different scattering sources and only preserves the signal from the region of interest. This technique achieves excellent reduction in reverberation, but is also computationally expensive and has thus far been difficult to implement in real time.

Recent studies have investigated deep learning as a tool to address reverberation noise. Various beamforming neural networks that take channel signals as input to create noise free images have been developed. Networks employing fully convolutional neural networks [6] and fully connected neural networks for anechoic region classification [23] have shown promising results, however these approaches also do not preserve the phase information necessary for other ultrasound imaging and processing techniques. Luchies et al. [24] proposed a deep learning beamforming technique that utilized a fully connected architecture to classify and remove offaxis scatterers from the channel signals. This network showed reduced reverberation in the channel signals and the resulting images demonstrated reduced noise in simulated and in-vivo data, and is a promising approach for real-time imaging with suppression of reverberation noise, but the off-axis scattering model used in training the network does not fully capture the characteristics of reverberation noise.

It has been shown in the computer vision field that the use of convolutional layers in neural networks can greatly improve the performance of neural networks, which recognize physical features in images. Because these layers assume that the network’s task is location invariant across the convolution dimensions, this greatly reduces the number of parameters in the network, and therefore simplifies the learning task [25].

The backscatter from an ultrasound pulse is a wide-sense stationary process in the aperture domain [26]. We have previously shown that the spatial coherence of backscattered ultrasound waves is a linear combination of an impulse function at zero-lag (due to reverberation noise) and the spatial covariance function of the underlying tissue (see equation 16 of [27]). Thus, the presence of reverberation noise can be identified through a local comparison across the channel dimension. Notably, the presence of reverberation noise is best identified in the first lag of this spatial coherence function [28]. Using this knowledge, we introduced a 3D fully convolutional neural network (3DCNN), that used a series of 3D convolutions across the lateral, axial and channel dimension of ultrasound channel data to eliminate reverberation noise in channel signals [29].

While this proposed 3DCNN successfully filtered reverberation noise from tissue signals, it was unable to filter noise in anechoic regions. In this paper, we introduce a modified dataset to accommodate anechoic targets and expand the performance of the 3DCNN. The performance and limitations of the 3DCNN is explored using quantitative metrics on simulated and phantom lesions. The network is then tested on in-vivo data and compared with SLSC and FXPF methods.

II. Methods

A. Training and Validation Dataset

To create a dataset to train the neural network, an approach similar to Hyun et al. [6] was used. The Field II Pro simulation package [30] was used to simulate ultrasound channel signals from 128 elements of a Verasonics L12–3v linear array transducer. A multi-static synthetic aperture data set was simulated using single-element transmits, with settings listed in Table I. Multi-static synthetic aperture data was chosen to provide a more spatially invariant point spread function in the dataset, which is beneficial for the chosen network architecture, detailed in Section II-B. Speckle was simulated with random uniformly distributed scatterers in a 1cm×1cm×3mm (axial by lateral by elevation) phantom centered laterally and at the elevation focus of the array (2cm). The scatterer density was randomly varied between 20 and 30 scatterers per resolution voxel, and the scatterer amplitude was Gaussian distributed and weighted according to a ground truth echogenicity map.

TABLE I.

Simulation Settings

Center Frequency	8 MHz
Bandwidth	60%
Sampling Frequency	160 MHz
# Elements	128 elements
Pitch	200 μm
Kerf	25 μm

For simulation content, real photographic images from the ImageNet [31] dataset were used as ground truth echogenicity maps. These real world images were used for their varying patterns and contrasts, to create a diverse training set that trains for any target geometry or underlying structure. Each image was converted to grayscale and cropped into a 224 × 224 pixel patch. To create anechoic regions in the training set, a random threshold value was generated between 0 and 70% of maximum brightness of the image. All pixels below this value were set to the threshold and the resulting image was then rescaled to the range [0,1]. The patch was mapped to match the lateral and axial extent of the phantom and the pixel intensity was used to weight the phantom scatterer amplitudes according to their positions via bilinear interpolation. For each of the images, an independent distribution of scatterers were used. After simulation, the received radiofrequency (RF) channel signals were demodulated to baseband in-phase and quadrature (I/Q) signals.

To create the ground truth channel signals for this training set, geometrical focal delays were applied to the multistatic data to obtain 224 × 224 pixel grid, with quarter wavelength pixel spacing axially and laterally, using synthetic focusing on transmit and dynamic focusing on receive, and then summing across the transmit dimension. The dimensions of the resulting channel signal cube were 224 axial samples × 224 lateral samples × 128 channels of focused, demodulated I/Q data.

To create noisy input training samples from this simulated data, noise was added as was done in [15]. Diffuse reverberation noise was approximated by generating a matrix of Gaussian distributed random noise, and then bandpass filtering it in the axial dimension, similar to how it was done in This bandpass filter used the same center frequency and bandwidth as the L12–3V transducer. This noise was scaled by a random gain in the range of −20 to 10 dB before adding it to the simulated RF channel signals. Additionally, thermal noise was included by adding Gaussian distributed random noise scaled by a random gain in the range of −20 to 10 dB to the RF channel signals. These channel signals were then demodulated to I/Q signals and focused in the same way as the ground truth channel signals.

Using this method, 4,096 unique simulations were performed to create noisy and ground truth pairs of complex I/Q focused channel signals. These focused channel signals were then cropped axially and laterally to obtain data of size 100×100×128 to train on. Because the data is complex, the real and imaginary components were separated into two separate features, resulting in a sample size of 100×100×128×2. Using a common preprocessing step in deep learning, each sample pair had it’s mean subtracted to give mean of 0, and the result was divided by the sample amplitude variance to give a variance of one. This normalization was performed on the reference and input data separately, with the mean and variance caluclated across all dimensions. A total of 3584 simulations were used to train the 3DCNN, and the remaining 512 simulations were used as an independent validation set for hyperparameter optimization. The training and validation sets had no overlap in simulations or source images. A table describing the simulation parameters that were varied during dataset generation are detailed in Table II. These parameters were uniformly distributed between the specified range.

TABLE II.

Parameter Ranges During Training Dataset Generation

Attribute	Minimum Value	Maximum Value
Scatterer Density	20 scat/res cell,	30 scat/res cell
Thermal Noise	−20dB	0dB
Reverberation Noise	−20dB	10dB
Anechoic Threshold %	0%	70%

In addition to the validation set for hyperparameter optimization, a second anechoic lesion test set was generated to evaluate the 3DCNN using common ultrasound metrics. This data was simulated using the same method as the training data, with the exception that the content image was replaced by a homogeneous echogenicity background and a 5mm anechoic lesion in the center. This validation set is available for download, and is provided in the additional media section of this publication.

B. Neural Network Architecture

A 3D fully convolutional neural network (3DCNN) was selected to remove reverberation noise from ultrasound channel signals. This architecture was chosen to simplify the learning task by assuming that the task is invariant across all three dimensions in the channel signals. In this way, the learned weights are applied across the entire data, instead of relearning the same weights for every pixel of the input. Because of this, a multi-static synthetic aperture pulse sequence was chosen, in order to provide a spatially invariant point spread function for the imaging system. The custom network architecture is illustrated in Fig. 1, and consists of sequential 3D convolutional layers with relu activation. Each 3D convolution was implemented with a stride of 1 in all dimensions without zero padding. The number of convolutional layers in the network and the size of each convolutional layer were varied as a hyperparameter during training, as described in Sec. II-C. All 3DCNNs were implemented in Python using TensorFlow [32].

Fig. 1.

Diagram of the 3DCNN architecture. The network consists of a series of 3D convolutions. The size of each of the convolution kernels and number of convolutional layers was varied as a hyperparameter.

C. Training and Analysis

Let $x\in {ℂ}^{PQN}$represent the multistatic baseband I/Q data focused in a P×Q pixel grid as captured by the N elements of a transducer array. Let $y\in {ℂ}^{PQN}$denote the ground truth (i.e. noise free) data. The output of the noise removal 3DCNN is represented as $\widehat{y}=f_{\text{NN}}(x;\theta )$, where θ are the parameters of the network and x is the input channel signal. The goal of the 3DCNN is to find the optimal parameters, θ^⋆, that minimize the error between the estimated channel signal, $\widehat{y}$, and the true channel signals, y, as defined by some cost function,$J(y,\widehat{y})$;

$${\theta }^{\star }=\underset{\theta }{\arg \min }J\left(y,f_{\text{NN}}(x;\theta )\right).$$ (1)

For the training optimization cost function, two losses were experimented with. The first is the standard L2 loss:

$${\mathcal{L}}_2(y,\widehat{y})={\left(\frac{1}{PQN}\sum _{p=1}^{PQN}{\left(y_p-{\widehat{y}}_p\right)}^2\right)}^{\frac{1}{2}},$$ (2)

where the p-th pixels of y and $\widehat{y}$ are respectively denoted as y_p and ${\widehat{y}}_p$.

Because ultrasound channel signals have a large dynamic range, standard loss functions will overemphasize errors in high-amplitude regions over errors in low-amplitude regions (e.g. emphasize background errors over hypoechoic lesion errors). To avoid this bias, we introduce a sign-preserving logarithmic compression:

$$g(y)=y\odot \frac{\log (|y|)}{\left|y\right|},$$ (3)

where $\odot$ denotes element-wise multiplication, and $|\cdot |$ denotes the absolute value. This is used to form a modified ${\mathcal{L}}_2$ optimization loss function, ${\mathcal{L}}_2(g(y),g(\widehat{y}))$, that we refer to as the log compressed L2 loss.

This loss minimization problem was solved using the Adam optimizer [33]. Regularization was included by adding the norm of the convolutional kernels to the cost function during training to reduce over fitting. The regularization was scaled by a coefficient, λ, to give the following cost function.

$$J={\mathcal{L}}_2(g(y),g(\widehat{y}))+\lambda \parallel \theta {\parallel }_1$$ (4)

This term is varied as a hyperparameter during tuning, along with the optimizer learning rate, batch size, 3D convolution kernel size, number of filters in each layer and number of convolutional layers. Skip connections were also inserted at various locations in the network to experiment if these connections aided in further reducing the loss. Hyperparameter tuning was performed using Bayesian Optimization [34] on the learning rate, regularization, batch size, number of filters, and loss function. The kernel size, number of layers and skip connections were varied empirically and hyperparameter tuning was performed on each architecture. In the empirical tuning of the architecture, permutations that examined kernel sizes in the range of (3,3,3) to (11,11,11) were used. Network architectures were designed based on knowledge of previously successful architectures that had decreasing kernel size with deeper layers [35]. The number of layers was varied from 1 to 10. A table of hyperparameter ranges used for Bayesian optimization are given in Table III, in addition to the loss function, which was a binary option between the L2 loss and the log compressed L2 loss. The final 3DCNN was selected as the network that gave the lowest log compressed L2 loss on the independent validation set.

TABLE III.

Hyperparameter Optimization Ranges

Layer	Min	Max
Regularization	1 × 10−2	1 × 10−8
Learning rate	1 × 10−2	1 × 10−8
Batch size	1	8
Hidden layer size	3	20

The code used to train the network, the model weights of the trained network presented in this work, and preprocessing code for the dataset after it is simulated, including the code that adds thermal and reverberation noise, is provided publicly through a github repository [36].

D. Phantom and in-vivo Experiments

In addition to the simulated validation and test sets, the 3DCNN’s ability to generalize from simulated data to physical ultrasound data was evaluated using a test set of phantom and in vivo experiments.

For the phantom experiments, 3 mm and 8 mm anechoic lesions were targeted in an ATS 549 phantom with a Verasonics Vantage 256 research scanner and an L12–3v linear array transducer. Single element transmissions with an 8MHz center frequency were sampled at 24MHz and demodulated to I/Q baseband. Synthetic focusing on transmit and dynamic focusing on receive were applied to the channel signals, which were then summed across the transmit aperture to obtain synthetic-focused receive channel signals.

Because the 3DCNN requires an input cube of data with a constant number of element signals as input, at depths where aperture growth was applied to the beamformer to maintain a constant F/#, the channel signals outside the active aperture were interpolated using cubic spline interpolation. Interpolated channel signals were used rather than the actual channel signals because the neural network was only exposed to the maximum spatial frequencies associated with an F/0.7 system during training, and inclusion of signals with higher spatial frequencies could result in artifacts.

The channel signals correspond to a 4 cm axial×2 cm lateral area in the phantom centered at the transducer’s elevation focus (2 cm). The image pixel spacing of this data is λ/2 in both dimensions. Reference channels were acquired from the phantom using a water gap between the transducer and phantom. Noisy data was acquired by replacing the water gap with bovine tissue of the same thickness consisting of skeletal muscle and fatty tissue. This bovine tissue induced both aberration and reverberation noise.

In-vivo imaging was assessed in the carotid artery and thyroid of 3 healthy volunteers. Because it is not possible to have reverberation-free reference in-vivo data, the carotid artery and thyroid were chosen to provide large anechoic regions of the artery and the common presence of benign, anechoic nodules in the thyroid. Anechoic regions are known to contain signal which is predominantly diffuse reverberation noise [8]. In this way, the performance of the 3DCNN is evaluated in its ability to remove signal from these anechoic regions. Subjects provided written informed consent and imaging was performed under an approved Stanford University IRB protocol. Channel signals were acquired in-vivo using the same system and method as described for the phantom data.

E. Image Reconstruction Quality Metrics

To evaluate the 3DCNN’s performance on reverberation reduction, B-mode images were formed from the resulting channel signals. Reverberation and thermal noise suppression by the 3DCNN filter was assessed using contrast, contrast-tonoise ratio (CNR), generalized contrast-to-noise ratio (GCNR) [37] and lag-one coherence (LOC) of the channel signals [28]. Contrast and CNR were computed as

$$\text{contrast}=20{\log }_{10}\left(\frac{{\mu }_t}{{\mu }_b}\right)$$ (5)

$$\text{CNR}=\frac{\left|{\mu }_t-{\mu }_b\right|}{\sqrt{{\sigma }_t^2+{\sigma }_b^2}},$$ (6)

where μ_t and μ_b denote the means and σ_t and σ_b the standard deviations of the target and background, respectively.

GCNR is a metric for lesion detectability based on the overlap of the probability density function between the background and target tissue. This metric is robust to rescaling of the image and has shown consistency with non-linear algorithms [37]. It is defined as

$$\text{GCNR}=1-\int \min \left(p_i(x),p_o(x)\right)dx,$$ (7)

where p_i and p_o are the probability density functions of the signal inside and outside the target region, respectively.

Because the spatial coherence computed from the ultrasound channel data will be a linear combination of an impulse function (due to reverberation noise) and the spatial coherence function of the tissue, the strength of the reverberation noise will be inversely proportional to the spatial coherence function at the first spatial lag (i.e. equal to the distance between neighboring elements). This so-called lag-one coherence (LOC) can therefore be used as a local measure of the strength of reverberation and thermal noise present in the data [28] and can be used as a metric to evaluate the network’s performance. The LOC is defined as the normalized ensemble correlation between adjacent transducer elements. The normalized ensemble correlation is defined as

$$\widehat{R}(p,m)=\frac{1}{N}\sum _{n=1}^N\frac{\langle S_p(n)S_p{(n+m)}^{\ast }\rangle }{\sqrt{\langle {\left|S_p(n)\right|}^2{\left|S_p(n+m)\right|}^2\rangle }},$$ (8)

where $S_p(n)$is the focused channel signal at pixel p of channel n. The LOC is defined as the ensemble correlation with lag 1:

$$\text{LOC}(p)=\widehat{R}(p,1)$$ (9)

Contrast, CNR, and GCNR were computed from anechoic regions, lesions, or vessel targets in the non-compressed B-mode images, while lag one coherence (LOC) [28] was computed from the channel signals of homogenous background.

F. Comparison Methods

The filtering of reverberation and thermal noise by the 3DCNN was compared against spatial prediction filtering (FXPF) [22] and short lag spatial coherence (SLSC) beamforming [7], [38]. These two techniques were applied to the validation and test datasets, and the metrics described above were computed on the resulting images to compare performance with the 3DCNN output. The prediction error filter length utilized in FXPF was 5 times the wavelength. The SLSC image was formed using 13 lags and a kernel size of 1, following the methodology of Hyun et al. [38].

III. RESULTS

A. Optimization Results

The final optimized 3DCNN after tuning is given in Table IV. This network is seven convolutional layers deep and contains no skip connections. The convolutional kernel dimensions refer to the axial, lateral, and channel dimensions respectively. It was found that the 3DCNN performed best when the earlier layers had a larger extent in the channel dimension. In this 3DCNN, the first two layers have a kernel size of (3,3,11), whereas the three layers at the end of the network have a kernel size of (3,3,3). The 3DCNN with the lowest validation loss was trained on the regular L2 loss with a learning rate of 1 × 10⁻⁴, batch size of 10, a regularization coefficient ( λ) of 1 × 10⁻³, and was trained for 60 epochs.

TABLE IV.

Optimized 3DCNN Architecture

Layer	Kernel Size	Number of Filters
Convs 1 & 2	(3,3,11)	5
Convs 3 & 4	(3,3,7)	5
Convs 5 & 6	(3,3,3)	5
Conv 7	(3,3,3)	2

B. Simulated Test Set Results

Fig. 2 shows the 3DCNN performance on simulations of anechoic lesions. Fig. 2 a) shows the B-mode image created from noise-free (reference) channel signals. Fig. 2 b) shows the resulting image from the noisy, unfiltered channel signals, with −5 dB each of thermal and reverberation noise, and Fig. 2 c) displays the corresponding image from the 3DCNN filtering. Fig. 2 d) & e) show the unfiltered and 3DCNN filtered output with 5dB each of thermal and reverberation noise. All images are shown with 60 dB of dynamic range. From visual inspection, the lesion contrast appears to improve in the −5 dB case and the noise visible in the lesion is removed, although the background brightness appears slightly reduced. In the 5 dB case, the lesion contrast is improved, but is not fully restored. The background speckle pattern is preserved in the 3DCNN filtered images, however.

Fig. 2.

3DCNN performance on simulated test data. a) The B-mode image from the noise-free reference channel signals. The resulting images from channel signals with b) −5dB and d) 5dB thermal and reverberation noise each. c & e) The resulting images from the 3DCNN filtering when given the noisy unfiltered channel data from b) and d) as input, respectively. All images are compressed and show 60dB of dynamic range. The orange dashed lines show the location of the plots in Fig. 3. The lesion and background ROIs shown in red are used for the metric calculations in Fig. 5.

Fig. 3 shows line plots through the background speckle and lesion, respectively, for the reference, unfiltered, and 3DCNN filtered images with −5dB each of added reverberation and thermal noise (orange dashed lines, Fig. 2a). These plots reveal that the 3DCNN maintains the brighter amplitude of speckle, but suppresses the dark regions of speckle and incoherent signal in the anechoic region.

Fig. 3.

Line plots through a) background speckle and b) anechoic lesion for reference, noisy, and 3DCNN filtered images shown in Fig. 2 (orange dashed lines, Fig. 2 a). The unfiltered channel signals had −5 dB each of reverberation and thermal noise added.

Fig. 4 shows the in-phase component of example I/Q channel data from the background region of the test simulations from Figs. 2a–c). Fig. 4 a) shows the reference channel signals, Fig. 4 b) shows the same channel data with added noise, and Fig. 4 c) shows the output of the 3DCNN filtering. Visually, the channel signals from the filtered data closely match the reference signals and the noise is greatly reduced. Across the entire lesion validation set, the correlation of the filtered channel signals with the noise-free reference signals increased from 0.85 before filtering to 0.92 after filtering. The L2 error with the reference channel signals was reduced from 0.53 before to 0.47 after filtering.

Fig. 4.

The real part of the demodulated I/Q channel signals from simulations. a) The noise-free signals, b) unfiltered channel signals with 5dB each of thermal and reverberation noise, and c) channel signals filtered by the 3DCNN. The 3DCNN has removed most of the noise from the channel signals.

The average CNR, GCNR, LOC, and contrast of 15 simulated anechoic lesions as a function of noise-to-signal ratio (NSR) are shown in Fig. 5; the ROIs used for metric calculations are shown in Figure 2 a). The 3DCNN improves GCNR, LOC, and contrast for all noise levels, however the contrast is decreased below the reference value for NSRs below 2.6dB. Additionally, the 3DCNN decreases the CNR for all NSRs below 5.8dB, even in the extremely low noise cases.

Fig. 5.

Average B-mode image quality metrics over 15 simulations of anechoic lesions as a function of NSR for the unfiltered noisy input and 3DCNN filtered output channel signals. Added noise includes both thermal and reverberation noise. GCNR, CNR, LOC and contrast are calculated using the ROI’s shown in Figure 2a. The green dashed line shows the noise-free reference value. Error bars show the standard deviation across 15 simulations.

C. Phantom Test Experiments

Figure 6 shows images of two lesions from the ATS 549 phantom experiment. Fig 6 a) and d) show the reference B-mode images made from the channel signals acquired with no bovine tissue layer (water gap only). Fig. 6 b) and e) show the B-mode image created from the unfiltered noisy input channel signals due to the bovine tissue. Fig. 6 c) and f) show the resulting B-mode images from channel signals filtered by the 3DCNN. All images are shown with 40 dB of dynamic range. After 3DCNN filtering, a visible contrast improvement is observed in both lesions. The CNR, GCNR, LOC, and contrast averaged over 5 independent phantom lesions before and after the 3DCNN are shown in Table V. Improvement is observed in all four metrics after 3DCNN filtering of the channel signals and the filtered channel data has higher contrast than the reference data. Unlike the simulated cases, the 3DCNN increased the CNR. Additionally, in Figures 6 c) and f), there is aberration present and therefore the speckle pattern does not match that of the reference. The remaining decrease in image quality is likely due to aberration from the bovine tissue layer, which has been mostly preserved by the 3DCNN.

Fig. 6.

B-mode images showing 3DCNN performance on 8 mm (top) and 3 mm (bottom) anechoic lesions. a & d) B-mode images of the lesions from the reference, noise-free channel signals. b & e) Unfiltered, noisy B-mode images produced with the bovine tissue. c & f) B-mode images with filtering from the 3DCNN. All images are compressed and shown with 40dB of dynamic range.

TABLE V.

Averaged Image Quality Metrics for Phantom Lesions

	Reference	Unfiltered	Filtered
GCNR	0.94 ± 0.00	0.84 ± 0.04	0.98 ± 0.01
CNR	1.63 ± 0.04	1.47 ± 0.12	1.55 ± 0.14
LOC	0.92 ± 0.00	0.84 ± 0.02	0.98 ± 0.00
contrast (dB)	−20.8 ± 0.3	−15.8 ± 2.4	−33.4 ± 6.9

Figure 7 a, b) show the in-phase component of demodulated I/Q channel data from Fig 6 e) and f), respectively. Fig. 7 a) shows the unfiltered channel data acquired with the bovine tissue and Fig. 7 b) shows the image after 3DCNN filtering. As in the simulated case, the 3DCNN removes reverberation and thermal noise while preserving the underlying signal. Although subtle to the eye, Fig. 7 a) also includes aberration. Fig. 7 b) shows by example that the 3DCNN preserves the low spatial frequencies of this aberration, however the high spatial frequencies of the aberration have been suppressed.

Fig. 7.

The in-phase component of demodulated I/Q channel signals from the phantom a) before and b) after 3DCNN filtering. As in the simulations, the wavefronts are smoother across the channels and have reduced noise. Some of the high spatial frequencies of the aberration have been removed by the 3DCNN.

D. In-vivo Test Experiments

Figure 8 compares images with 3DCNN filtering to the original B-mode image, short lag spatial coherence (SLSC) imaging, and spatial prediction filtering (FXPF) on in-vivo carotid artery and thyroid of healthy volunteers. Rows 1, 2, and 4 show saggital views of the common carotid artery and row 3 shows a transverse view. Rows 2–4 also show benign thyroid nodules, commonly present in healthy, but older, individuals. The first column displays conventional B-mode images created from the original unfiltered channel signals. Columns 2, 3 and 4 show the images formed using the SLSC beamformer, delay-and-sum beamforming with the channel signals after FXPF filtering, and delay-and-sum beamforming with the channel signals after 3DCNN filtering, respectively. All images were scaled to have the same mean value and are compressed and displayed using 50 dB of dynamic range.

Fig. 8.

Reverberation removal techniques applied to 4 different in-vivo images. Rows 1,2, and 4 show a saggital view of the common carotid artery and row 3 shows a transverse view. Rows 2–4 also show healthy thyroid containing benign nodules. The first column displays unfiltered B-mode images using the original channel signals. Rows 2, 3 and 4 show the images from the SLSC beamformer, images with FXPF filtering, and images with 3DCNN filtering, respectively. All images are displayed using 50dB of dynamic range. a), e) and m) show the ROI of the background and anechoic regions used for metric calculations.

Visual inspection of the lumen of the common carotid artery and thyroid nodules in Figs. 8 a), e) and m) show that they are overlayed with moderate amounts of reverberation. In the corresponding SLSC images, there is some improvement in the distal part of the lumen (Figs. 8 b) and n)) along with the thyroid nodules. Similarly, FXPF shows reverberation reduction in some regions (Fig 8 c), distal lumen), but for both SLSC and FXPF, a moderate amount of reverberation still remains. After 3DCNN filtering, the carotid lumen contrast improves significantly and has greatly reduced reverberation (see Table VI). Many of the benign nodules in the thyroid also show improved contrast after 3DCNN denoising, with the 2nd row showing a small, high-contrast feature in the center above the nodule that is not visible in the B-mode, SLSC, or FXPF images.

TABLE VI.

Metrics on in-vivo carotid Results

	Unfiltered	SLSC	FXPF	3DCNN
GCNR	0.86 ± 0.01	0.92 ± 0.05	0.88 ± 0.06	0.95 ± 0.05
CNR	1.43 ± 0.11	2.77 ± 0.53	1.37 ± 0.05	1.33 ± 0.06
LOC	0.95 ± 0.03	–	0.95 ± 0.02	0.99 ± 0.01
contrast (dB)	−18.9 ± 5.8	−22.1 ± 2.8	−22.2 ± 6.1	−34.9 ± 8.1

In the case of the transverse image of the carotid in Figure 8 l), which contains little reverberation in the artery, the SLSC and FXPF images are visually appealing; however lowamplitude signal has been removed in the 3DCNN image, thereby reducing visibility of the artery wall.

Table VI shows the CNR, GCNR, LOC, and contrast averaged for rows 1, 2 and 4 of Fig. 8 for all methods. The ROIs used for these metric calculations are shown in Figs. 8 a), e) and m). The 3DCNN improves GCNR, LOC, and contrast more than the other methods; however, the 3DCNN generated lower CNR. SLSC improves the CNR, however it has decreased contrast and GCNR. FXPF slightly improves the contrast and preserves GCNR and LOC, but also decreases CNR.

On input channel data of size 224 × 224 × 128, the proposed 3DCNN required 0.84 seconds on a NVIDIA Quadro M4000 GPU using TensorFlow. The FXPF method was implemented unoptimized on MATLAB on a 3GHz Intel Xeon Processor E3–1220 v5. No direct comparison could be made in their computation times due to the different computational hardware, but given the computational complexity of the FXPF algorithm, it would be difficult to implement with a run time comparable to the 3DCNN.

IV. Discussion

Based on the simulated, phantom, and in-vivo data, the 3DCNN reduces reverberation noise and generally improves image and channel data quality. For the in-vivo results, many anechoic regions of the carotid lumen had significant improvement in contrast and visibility using the 3DCNN to filter noise from the channel signals, and showed greater noise reduction than the SLSC and FXPF techniques. Visually, the anechoic regions in the resulting simulated, phantom and in-vivo B-mode images had significantly improved contrast, indicating to a reduction in diffuse reverberation.

A. 3D Convolutional Neural Network Architecture

Through hyperparameter tuning, it was found that the 3D convolution kernels that were longer in the channel dimension improved validation performance. Of the experimented kernel sizes, a size of (3,3,11) for the earlier layers yielded the best performance. It is not surprising that the 3DCNN needed only a small receptive field in the axial and lateral dimensions, and a larger receptive field in the channel dimension. From previous studies of reverberation noise [27], it is known that such noise can be identified from the covariance of the channel signals. Unlike tissue signal, reverberation noise is decorrelated across the channels. Therefore, a larger receptive field in the channel dimension via longer convolution kernels allows the 3DCNN to better discern signal from reverberation noise.

B. Dataset Generation

For translation to in-vivo applications, the ImageNet dataset was used for Field II simulations to provide a large range of contrasts and shapes, instead of point targets or random cysts as is frequently used in ultrasound simulations. This content was chosen over anatomical data because the 3DCNN should not make decisions based on the target shape or reflectivity, but rather a local comparison of the ultrasound signal statistics. It has been shown that neural networks can learn significant information (intentionally or unintentionally) from the content of images. This has been leveraged in many radiology applications such as for super-resolution [39], denoising [40] and inpainting missing information [41]. In some cases, overtraining on the training dataset has been observed. This leads the network to inpaint data with the nearest neighbor in the training set [42]. By training on a wide variety of shapes aside from anatomical images, we avoid overtraining the network to recognize anatomical shapes and therefore avoid inpainting channel signals with potentially false information. In doing so, we make the assumption that removing diffuse reverberation noise from the signal is not dependent on the geometry of the imaging medium. The goal is to have the denoising task be applied only through a local comparison of ultrasound signal statistics that discern reverberation from true signal.

Though the addition of filtered random noise seemed to be successful in representing reverberation noise for this neural network, we do acknowledge that better models can be used to improve the accuracy of the reverberation noise in the training set. Future work will address this by creating a training dataset using the fullwave simulation package [43]. In this simulation package, diffuse reverberation based on realistic tissue models (e.g. derived from histological samples [44]) can be introduced in the channel data by placing the tissue model above the target region [8].

Additionally, for better translation to physical data, aberration should be added to the training set in both the reference and input data so that the 3DCNN will learn to better preserve channel data without removing the high spatial frequencies of the aberration. However, a full analysis of the impact of the 3DCNN on aberrated channel signals was not performed here and would be necessary before applying this 3DCNN as a preprocessing step for techniques such as phase aberration correction or sound speed estimation.

C. 3DCNN Effects on Speckle Variance and Low Amplitude Signal

Although the 3DCNN improves contrast in anechoic regions, Fig. 5 and Table V shows that the 3DCNN produces lesion contrast greater than that of the reference image. This indicates that the 3DCNN is not only reducing reverberation noise, but also suppressing off-axis scattering signal in the lesion, which is present in the reference image. This offaxis scattering has statistical properties similar to (but not the same as) reverberation noise, and suppression of this off-axis scattering is likely the cause of increased contrast.

Similarly, as seen in Figure 3, the 3DCNN tends to suppress desired signals that have low correlation across the channels such as the dark regions of speckle. This is observed in Fig. 5, where the LOC is greatly improved to 0.993 (with low variance) for all noise cases. Because the theoretical limit of the LOC for this data is 0.992 [28], this implies that the 3DCNN is slightly over-trained to output highly correlated channel signals. Over-training in this case means that the 3DCNN strongly removes low correlation signals, including desired signals such as the dark regions of speckle, in addition to removing low-correlation reverberation noise. This has the effect of increased background speckle variance. As a result, the 3DCNN decreases CNR (and speckle SNR) even though the contrast has been improved. This explains the reduction in CNR in Fig. 5 by the 3DCNN, even when very low (eg. −15 dB) amounts of noise are present. Conversely, there is an improvement in CNR for the phantom case, and only a small CNR decrease in the in-vivo case. We are unable to explain why the CNR does not decrease after filtering in the phantom case, however the difference in speckle patterns between the reference and 3DCNN images (due to the bovine layer) may be introducing statistical variation in the CNR or the noise may be sufficiently strong such that the 3DCNN improves CNR, as shown in the high NSR cases in Fig. 5. This discrepancy may potentially be avoided by including aberration noise in the training set. This will expose the 3DCNN to uncorrelated signal that should not be removed, thereby decreasing overfitting.

Not only is the 3DCNN sensitive to low correlation data, but it is also sensitive to low-amplitude signal. This can be seen in the third row of Figure 8, where the tissue on the right side of the image is visible in both the original, SLSC and FXPF images, but not in the 3DCNN image. This may be due to training the 3DCNN on a dataset containing too many anechoic regions, causing it to become sensitive to large changes in contrast and assume that that region is anechoic. This can be addressed by adjusting the training dataset to contain more low amplitude data, in addition to the adjusting the amount of anechoic regions present in the dataset.

Because the 3DCNN was trained on data with limited axial extent, lower signal amplitude due to attenuation at depth may be misinterpreted by the 3DCNN. This may lead to a suppression of signal from deep regions in the image. For some of the in-vivo images with a large axial extent, time gain compensation (TGC) applied to the focused channel data improved the stability of the network’s performance. However, TGC was not used on any of the images shown in this paper.

D. 3DCNN Response to Aberration

Although the 3DCNN reduced reverberation noise in the anechoic phantom experiments shown in Figure 6, the speckle pattern of the reference was not restored. The lesion structure has been preserved in Figs. 6 c) and f), but distortions to the lesions edges that are not present in the reference image remain. This is due to the sound speed inhomogeneities in the bovine tissue, leading to aberration, which the 3DCNN has largely preserved. This is an unexpected result, given that the training dataset did not include aberration. Currently, this is a desirable outcome for this 3DCNN, because this allows other techniques such as aberration correction and sound speed estimation to be applied after reverberation is removed. However, Fig. 7 d) and e) suggest that even though the aberration is still present, the high frequency components of the aberration are suppressed, leaving only the low spatial frequencies. This suppression of aberration can be avoided by including aberration in the training simulations, such as the aforementioned fullwave simulations with abdominal layer models.

E. Comparison with Other Techniques

Compared with SLSC and FXPF in Fig. 8 and as shown in Table VI, the 3DCNN shows better performance at removing reverberation noise in the carotid and thyroid nodules and improves GCNR and LOC. However, the 3DCNN performs poorer than SLSC and FXPF at low amplitude signals, thereby decreasing CNR. It is interesting to note that the FXPF technique also degrades the CNR of the in-vivo images (Table VI). A decrease in CNR with the FXPF method has been observed in previous simulations that contain no aberration or noise [22]. The reduction in CNR appears to be a common problem in reverberation suppression methods like FXPF, ADMIRE, and the 3DCNN. For example, the ADMIRE technique has demonstrated a similar problem of decreasing CNR in early versions of the method [45], although later improvements to the model have eliminated this problem [20]. Because CNR is essentially a function of contrast and speckle SNR, it is interesting to note that the neural network method by Luchies et al. [24] showed a significant decreased in speckle SNR despite an modest increase in CNR, indicating that suppression of the dark regions of speckle is present in this method as well.

The computation time of the 3DCNN is relatively fast at 0.84 seconds. Although not real-time, there are a few approaches that might slightly degrade 3DCNN performance but increase implementation speed. Training a neural network with fewer layers, or with 2D convolutions followed by 1D convolutions in the 3rd dimension instead of 3D convolutions would be a few ways to increase network speed [46]. Hyun et al. [47] have demonstrated a similar approach to achieve real time imaging with fully convolutional networks on ultrasound sub-aperture data in the task of speckle suppression.

One drawback of using the 3DCNN is that this method requires the use of multistatic synthetic aperture data, which has its own limitations, such as poor SNR. However, the poor SNR be overcome by techniques such as REFoCUS, which can recover a multi-static synthetic aperture data from conventional focused transmit data [48], [49]. This would make the 3DCNN more amenable to clinical ultrasound scanners and allow imaging in deep tissue regions.

V. Conclusion

A custom-designed, 3D fully-convolutional neural network (3DCNN) was trained on simulated ultrasound channel signals to remove reverberation and thermal noise. The 3DCNN was applied to phantom and in vivo experiments and showed significant visual improvement in image quality in both the phantom and in-vivo experiments, demonstrating that the reverberation noise was reduced. This was confirmed by quantitative improvements in GCNR, LOC, and contrast in anechoic regions. The contrast of anechoic regions was greatly improved in all cases, while the CNR was improved in some cases. The 3DCNN appeared to strongly remove low amplitude signal, including the dark regions of background speckle, which decreased CNR due to an increase in speckle variance. Refinement to the training dataset such as adjusting the percentage of training samples containing anechoic regions and including aberration may be necessary to reduce overfitting.