Resolution enhancement in medical ultrasound imaging

Abstract.

Image resolution enhancement is a problem of considerable interest in all medical imaging modalities. Unlike general purpose imaging or video processing, for a very long time, medical image resolution enhancement has been based on optimization of the imaging devices. Although some recent works purport to deal with image postprocessing, much remains to be done regarding medical image enhancement via postprocessing, especially in ultrasound imaging. We face a resolution improvement issue in the case of medical ultrasound imaging. We propose to investigate this problem using multidimensional autoregressive (AR) models. Noting that the estimation of the envelope of an ultrasound radio frequency (RF) signal is very similar to the estimation of classical Fourier-based power spectrum estimation, we theoretically show that a domain change and a multidimensional AR model can be used to achieve super-resolution in ultrasound imaging provided the order is estimated correctly. Here, this is done by means of a technique that simultaneously estimates the order and the parameters of a multidimensional model using relevant regression matrix factorization. Doing so, the proposed method specifically fits ultrasound imaging and provides an estimated envelope. Moreover, an expression that links the theoretical image resolution to both the image acquisition features (such as the point spread function) and a postprocessing feature (the AR model) order is derived. The overall contribution of this work is threefold. First, it allows for automatic resolution improvement. Through a simple model and without any specific manual algorithmic parameter tuning, as is used in common methods, the proposed technique simply and exclusively uses the ultrasound RF signal as input and provides the improved B-mode as output. Second, it allows for the a priori prediction of the improvement in resolution via the knowledge of the parametric model order before actual processing. Finally, to achieve the previous goal, while classical parametric methods would first estimate the model order and then the model parameters, our approach estimates the model parameters and the order simultaneously. The effectiveness of the methodology is validated using two-dimensional synthetic and in vivo data. We show that, compared to other techniques, our method provides better results from a qualitative and a quantitative viewpoint.

1. Introduction

Ultrasound imaging is one of the most commonly used medical imaging modalities. Its low cost, nonionizing characteristics, ease of use, and real-time nature make it the gold standard for many crucial diagnostic exams, especially in obstetrics and cardiology. However, there is a compromise to find between imaging depth and resolution. This compromise directly results from the fundamental principles of ultrasound imaging. Classically, the resolution of an imaging system is its ability to separate two close source points. A medical ultrasound image of a structure consists of a collection of individual signatures of the elementary components (echoes) of the tissue (scatterers). An individual signature can be captured through the point spread function (PSF) of the imaging system. The separability of two point sources is restricted by the Rayleigh diffraction limit.¹ For years, the enhancement of the resolution has been based on the optimization of either the transducers, e.g., Refs. ² and ³, or the devices, e.g., Refs. ^4–6. The basic idea is that the resolution of the ultrasound imaging system depends mainly on the characteristics of the transducer, such as its center frequency, bandwidth, and focusing properties. For example, considering two-dimensional ultrasound imaging, these resolutions in each spatial dimension are referred to as lateral ($r_l$) and axial ($r_a$) resolutions and can be expressed by:⁶

$$r_l=\lambda f_{\text{number}}=\lambda \frac{L}{D};r_a=\frac{1}{2}\frac{c}{B_w},$$ (1)

where $\lambda$ is the average ultrasound wavelength, $c$ is the speed of sound, and $f_{\text{number}}=L/D$, where $D$ and $L$ are, respectively, the diameter and the focal length of the transducer (sensor). $B_w$ is the bandwidth of the transducer. $r_l$ and $r_a$ are the actual spatial resolution accessible through the PSF of the imaging system. The higher the frequency, the better is the resolution; thus, the ultrasound frequency or bandwidth have to be increased to improve the spatial resolution of ultrasound images, e.g., Ref. 6. Unfortunately, increasing the ultrasound frequency results in a decreased imaging depth, thus, one way to improve image resolution without decreasing its depth is to use image processing techniques. Recently, this problem of resolution enhancement by the postprocessing of images rather than by device engineering has received a lot of interest. Investigations have been mostly limited to general purpose deconvolution techniques, e.g., Refs. ⁵ and ^7–10. This paper is a deepening of preliminary studies introduced in Refs. ¹¹ and ¹². The aim of this paper is twofold. The first aim is to predict, a priori, the achievable improvement in resolution from an image, given the associated imaging system. The second objective is to introduce an effective method for improving the resolution of ultrasound images. We show that autoregressive (AR) modeling is a technique that is pertinent in the context of resolution improvement in ultrasound imaging. However, two problems have to be solved. The first is the presence of colored or correlated artifact noise on the radio frequency (RF) lines. The second originates from the difficulty of simultaneously estimating both the parameters and orders of the AR model for real-time purposes. Interestingly, as shown in this paper, noise does not change anything regarding the resolution, provided the parameters of the model are estimated correctly. Otherwise, the parameters are biased if an unsuitable parameter estimation technique is used (for instance, if a classical least-square-type algorithm¹³ is used). We present a multidimensional instrumental matrix estimation technique to overcome these issues of correct parameter estimation. Thus, the contribution of this work is threefold: first it allows for automatic resolution improvement. No estimation of the PSF of an imaging system is needed. This also means that no specific manual algorithmic parameter tuning is used as is the case with common methods. Consequently, our technique simply and exclusively uses the ultrasound RF signal as input and provides the improved B-mode as output. Second, it allows the a priori prediction of the resolution improvement via the knowledge of the parametric model order before actual processing. To the best of our knowledge, no existing ultrasound resolution improvement technique has this ability. Third, to achieve the previous goal, while classical parametric methods would first estimate the model order and then the model parameters, our approach estimates the model parameters and orders simultaneously. This simultaneous estimation of the model parameters and orders is done through relevant factorization of the regression matrix. The remainder of this paper is arranged as follows. Section 2 presents the framework and the models used to analyze the problem of resolution. In Sec. 3, the RF signal processing is presented and the results of resolution improvement are shown in Sec. 4. Finally, Sec. 5 is devoted to the conclusion and general discussions.

2. Framework and Models

2.1. Background on Ultrasound Imaging and Resolution Quantification Through a Model

One of the most common ultrasound imaging representations is known as B-mode. The envelopes of individual signals received from structures to be imaged or RF signals are computed after beamforming, filtered, log-compressed, and, finally, displayed in gray levels (Fig. 1). Given an individual baseband RF echo $g(z)$, its envelope is, thus, defined by:

$$P_0(z)=|g_{I/Q}(z)|,$$ (2)

with

$$g_{I/Q}(z)=g(z)+j\mathrm{HT}[g(z)],$$ (3)

where $j^2=-1$ and $g_{I/Q}$ is referred to as the complex analytical signal [or nondemodulated In-phase and Quadrature ($I/Q$) signal]. The envelope is obtained through step B of Fig. 1 and $\mathrm{HT}[]$ denotes the Hilbert transform used in step A of Fig. 1. In step B, the magnitude of $g_{I/Q}$ is computed. For a real-value narrowband (echo) signal $g(t)$, the Hilbert transform is defined by:

$$\mathrm{HT}[g(z)]=-g(z)\otimes \frac{1}{\pi z},$$ (4)

where $\otimes$ stands for the convolution product. Step C of Fig. 1 is a log compression and step D is a conversion to gray levels. All the B-mode echoes are then juxtaposed to be displayed as an image. Here, we are interested in post-beamforming imaging.

Fig. 1.

Principle of B-mode imaging. Like the original ultrasound image, which is a collection of $g(z)$ echoes, the B-mode image is a collection of B-mode echoes.

2.2. Proposed Parametric Spectral Like Super-Resolution Approach

Starting from Eq. (2) giving the B-mode representation, we will show hereafter that an analogy may be done between the envelope estimation and the classical Fourier-based power spectrum density. In the following, the classical Fourier-based power spectrum density will be referred to as PSD. It is known that the PSD estimates have poor frequency resolution. One main solution to improve the resolution of the PSD is to use parametric methods, such as AR, to estimate the PSD.^13,14

Let us begin by recalling a classical result from parametric AR spectral modeling. The interested reader may refer to, e.g., Refs. ¹³ and ¹⁴. Let a given real- or complex-number signal $y(n)$ defined by $N$ samples be modeled by an AR process. The signal $y(n)$ may be rewritten as:

$$\text{blue}y(n)=-\sum _{i=1}^M{a}_{i}y(n-i)+\omega (n),$$ (5)

where $M$ is the number of parameters, also referred to as the model order, $\omega (n)$ is a zero mean white noise, with variance ${\sigma }^2$, and the parameters $a_i$ are complex number parameters. The methodology also stands for nonwhite noise.

From Eq. (5), it can be shown that the PSD is given by:

$${\mathrm{PSD}}_y(f)=\frac{{\sigma }^2}{{|1+\sum _{m=1}^M{a}_m\exp (-2\pi jfm)|}^2},$$ (6)

where $j^2=-1$ and $f$ stands for the normalized frequency such that $-(1/2)\le f\le (1/2)$. From Eq. (6), it can be seen that since ${\mathrm{PSD}}_y(f)$ is directly linked to the frequency variable $f$ via the exponential function, the frequency resolution may be theoretically not limited, provided the $a_m$ coefficients and the model order $M$ are known. The classical spectral analysis may be resumed as follows. We consider for better readability, without loss of generality, a noiseless signal $y(n)$ (when the signal is noisy, the results are valid, the signal is split on some short windows, and the PSD is averaged on these windows). To obtain the PSD of a time or space-varying signal $y(n)$, two possible paths exist.

• •Path 1: From the time or space-varying signal $y(n)$, first compute the Fourier transform; then take the magnitude (possibly averaged on short windows) to obtain the PSD, say $P(f)$.
• •Path 2: From the time or space-varying signal $y(n)$, perform an AR analysis by first estimating the AR parameter $a_i$ in Eq. (5); then estimate the PSD using Eq. (6).

It is well known that parametric spectral methods (as compared to some classical Fourier spectral analyses) improve the frequency resolution of a PSD estimation. Moreover, it results from Fig. 1 that the ultrasound B-mode computation is based on the estimation of the envelope image. Thus, obtaining the envelope of an ultrasound signal is very similar to obtaining the classical PSD (path 1). That is, they are the magnitudes of, respectively, the complex analytical signal [Eq. (2)] and the Fourier transform. To improve the resolution of the envelope of an ultrasound signal, we propose to use the AR modeling as in classical spectral (PSD) analysis, by following (path 2) through a domain change.

This is achieved by computing the inverse Fourier transform of the received echo signal. This is explained by the fact that in spectral analysis, the time or space signal and the PSD are in two differents domains (respectively, the time or space domain and frequency domain), whereas in the envelope estimation, the time or space signal and the envelope are in the same domain. Thus, for envelope estimation with AR modeling, we need to change the processing domain so that this processing domain differs from the envelope domain. To do so, we go from the time or space domain to the inverse Fourier domain so that applying AR modeling yields the original time or space domain. The parametric spectral model is, thus, applied to the result of the inverse Fourier transform of the time or space signals, and gives us an estimation of envelope of the ultrasound signal (instead of the PSD). This is the dual part of Eqs. (5) and (6). Equation (5) is in the time or space domain and Eq. (6) in the frequency domain. Thus, performing an inverse Fourier transform of the RF signal and then using Eqs. (5) and (6) on this inverse Fourier transform leads to a new (better resolved) envelope signal.

The domain change and the super-resolved envelope estimation method are further detailed below.

Let us consider the spatial resolution analysis problem and, especially, the evaluation of the resolution. Consider two point objects, separated by a distance $d$, receiving a beam and reflecting ultrasound toward the emitting transducer. Assume the PSF of this imaging system is $g$, and assume its envelope has an absolute maximum, causal and zero at infinity.

Also assume all the parameters of the ultrasound imaging system (speed of sound, frequency, attenuation, etc.) are fixed. The two point sources located at $d_1$ and $d_2=d_1+d$ produce the echoes $g(z-d_1)$ and $g(z-d_1-d)$, respectively (see Fig. 2).

Fig. 2.

Two-point sources.

Performing a trivial variable change and sampling above the Nyquist frequency, the received echo signal can be written as:

$$y(z_k)={\alpha }_{1}g(z_k+\frac{d}{2})+{\alpha }_{2}g(z_k-\frac{d}{2})+w(z_k),$$ (7)

where ${\alpha }_1$ and ${\alpha }_2$ denote the uncorrelated random magnitude of each point and are related to the acoustic properties of the point sources. $w(z_k)$ is an additive zero mean Gaussian white noise with variance ${\sigma }_w^2$. Each point’s response is, thus, a convolution of a band-limited PSF envelope with a Dirac peak located at $d_1=-(d/2)$ and $d_2=(d/2)$.

Applying the inverse Fourier transform to Eq. (7), we obtain:

$$Y(u_k)={\alpha }_{1}G(u_k)\exp (-j2\pi \frac{d}{2}{u}_k)+{\alpha }_{2}G(u_k)\exp \left(j2\pi \frac{d}{2}{u}_k\right)+W(u_k).$$ (8)

The AR model is then applied to Eq. (8) to improve the envelope resolution.

In the following, we consider different assumptions:

• •The speed of sound is constant over the imaged tissues.
• •The two point sources are sufficiently close to have the same attenuation.
• •We consider, without loss of generality, that the PSF has a unit energy.

What we are interested in here is finding a practical way to access the resolution.

Let us summarize the proposed method. From the received ultrasound RF signal $y(z_k)$, we first perform an inverse Fourier transform to obtain $Y(u_k)$, then we apply an AR model using $Y(u_k)$ [that is, we replace $y(n)$ in Eq. (5) by $Y(u_k)$ instead of $y(z_k)$], and, finally, we estimate the envelope by using Eq. (6) as shown below:

$$y(z_k)\stackrel{{\mathrm{FT}}^{-1}}{\to }Y(u_k)\stackrel{\mathrm{AR}\text{model}}{\to }{S}_{\mathrm{AR}}(z),$$

where $z_k$ and $z$ are time or space variables. $z_k$ is discrete time (or space), whereas $z$ is the continuous-time (or space) variable according to Eq. (6); $S_{\mathrm{AR}}(z)\in {\mathbb{R}}^{+}$. Instead of providing an estimated PSD, the parametric spectral method gives us $S_{\mathrm{AR}}(z)$, an estimated envelope of $y(z)$, thanks to the inverse Fourier transform previously computed. $S_{\mathrm{AR}}(z)$ is, thus, a parametric spectral like an envelope. Note that any other high-resolution spectral analysis method can be used instead of AR modeling, which is used here for convenience and simple derivations.

We now investigate the relevance of the method by analyzing the relationship among the resolution, signal-to-noise ratio (SNR), and the model features.

Let us, thus, define

$$\theta ={[a_1,a_2,\dots ,a_M]}^T.$$ (9)

The estimate of $\theta$ will be further denoted by $\widehat{\theta }$.

In this context, by ignoring the constant, which is not useful here, Eq. (6) can be rewritten as

$$S_{\mathrm{AR}}(z)=\frac{1}{A(z)},$$ (10)

where $S_{\mathrm{AR}}$ is the super-resolved envelope of the RF signal, and

$$A(z)={(1-z^H\widehat{\theta })}^{*}(1-z^H\widehat{\theta }),$$ (11)

where $T$ and $H$ stand, respectively, for the transpose and transpose conjugate operators, and

$$z={[\exp (jz),\exp (j2z)\dots \exp (jmz)]}^T.$$ (12)

A practical condition^15–17 for determining the resolution ability is analyzing $A(z)$ [rather than $S_{\mathrm{AR}}(z)$] at the center location of the interval $d$, i.e., compare $A(0)$ with the average of the envelope evaluated at locations $d_1$ and $d_2$. To do so, it is common to define the quantity

$$R=A(-d/2)+A(d/2)-2A(0),$$ (13)

and the resolution problem comes to the point of hypothesis testing:

$$\{\begin{array}{l}R<0\text{:}\text{the two points are resolved}\\ R\ge 0\text{:}\text{the two points are not resolved}\end{array}.$$ (14)

Actually, the resolution limit case corresponds to

$$R=0.$$ (15)

From Eqs. (5) and (8), let us define

$$\varphi (u_k-1)={[Y(u_k-1)\dots .Y(u_k-M)]}^T,$$ (16)

$$C_y=E[\varphi (u_k-1){\varphi }^H(u_k-1)],$$ (17)

$$\mathrm{\Psi }=E[\varphi (u_k-1)Y^{*}(u_k)],$$ (18)

$${\overline{z}}_p={[\exp (j{\overline{z}}_p),\exp (j2{\overline{z}}_p)\dots \exp (jM{\overline{z}}_p)]}^T,$$ (19)

$$\begin{gathered} \text{with}p=\{\mathrm{1,2},3\}\text{and} \\ {\overline{z}}_1=-\frac{d}{2},{\overline{z}}_2=\frac{d}{2},{\overline{z}}_3=0, \end{gathered}$$ (20)

where $C_y$ is the data covariance matrix. Note that $C_y$ explicitly depends on the model order $M$.

These provide the least square estimate given by

$$\widehat{\theta }=C_y^{-1}\mathrm{\Psi }.$$ (21)

Thus, when $\widehat{\theta }$ is estimated, $R$ in Eq. (13) can be estimated using Eq. (11). Thus, solving Eq. (15) makes it possible to establish a nonlinear relation among the resolution limit, the SNR, and the model order (see Sec. 2.3).

2.3. Resolution Analysis

Here, we perform resolution analysis with respect to the imaging features. For readability, from Eq. (7), we set ${\alpha }_1={\alpha }_2$, and $\mathrm{SNR}={\alpha }_1^2/{\sigma }_w^2$ in all the following. Here, we perform an analysis with different kinds of PSF.

2.3.1. Ideal case

First, we consider the ideal case in which we set $g(z)=\delta (z)$ in Eq. (7), which means $y$ is wide-band. When $y$ is wide-band, i.e., $G(u)\sim 1$, an explicit nonlinear expression, can be provided. Indeed, from Eqs. (8) and (17) comes

$$C_y={\alpha }_1^2{\overline{z}}_1^H{\overline{z}}_1+{\alpha }_2^2{\overline{z}}_2^H{\overline{z}}_2+{\sigma }_w^{2}I.$$ (22)

Define

$${\tau }_{kl}=\frac{{\overline{z}}_k{\overline{z}}_l^H}{M},$$ (23)

with $k,l\in \{\mathrm{1,2},3\}$. Using either the matrix inversion lemma or eigenvalue/vector decomposition,^16,17 it can be shown that $R$ in Eq. (13) yields

$$R(M,\mathrm{\Delta },\mathrm{SNR})={|B_1|}^2+{|B_2|}^2-2{|B_3|}^2,$$ (24)

where

$$B_1={\beta }_{1}M+{\beta }_{2}M{\tau }_{12}-1,$$

$$B_2={\beta }_{1}M{\tau }_{21}+{\beta }_{2}M-1,$$

$$B_3={\beta }_{1}M{\tau }_{31}+{\beta }_{2}M{\tau }_{32}-1,$$

$${\beta }_1=\mathrm{SNR}+\frac{M{(\mathrm{SNR})}^2[M(\mathrm{SNR})({|{\tau }_{12}|}^2-1)-(1+{\tau }_{12})]}{{[M(\mathrm{SNR})+1]}^2-{[M(\mathrm{SNR})|{\tau }_{12}|]}^2},$$

$${\beta }_2=\mathrm{SNR}+\frac{M{(\mathrm{SNR})}^2[M(\mathrm{SNR})({|{\tau }_{12}|}^2-1)-(1+{\tau }_{21})]}{{[M(\mathrm{SNR})+1]}^2-{[M(\mathrm{SNR})|{\tau }_{12}|]}^2}.$$

This particular case is similar to the work reported in Ref. 17. Solving this nonlinear equation provides the resolution. The results are shown in Fig. 3. Thus, the resolution increases with the model order and the SNR. However, this relation is not clear for a low SNR ($<0\mathrm{dB}$). This means better resolution may not be achieved for a very low SNR. In practical applications, one does not consider an SNR $<0\mathrm{dB}$.

Fig. 3.

Signal-to-noise ratio (SNR) versus resolution and order for an ideal point spread function (PSF). The SNR goes from 25 dB at the top to $-30\mathrm{dB}$ for the bottom line.

2.3.2. Gaussian envelope

Let us now consider the PSF as $g(u)=e^{-(u^2/2{\chi }^2)}\cos (2\pi f_{0}u)$. We define $B=(1/2\pi \chi )$. The relative (frequency) bandwidth of $g$ is then defined by $B/f_0$, with $f_0$ the center frequency of the transducer. In ultrasound imaging, this is an important parameter that is used to characterize the transducers.

We consider here relative bandwidths of 30, 50, 100, and 150%. As can be seen in Figs. 4–7, the resolution is directly related to the bandwidth and the model order.

Fig. 4.

SNR versus resolution and order for a Gaussian PSF of 30% relative bandwidth.

Fig. 5.

SNR versus resolution and order for a Gaussian PSF of 50% relative bandwidth.

Fig. 6.

SNR versus resolution and order for a Gaussian PSF of 100% relative bandwidth.

Fig. 7.

SNR versus resolution and order for a Gaussian PSF of 150% relative bandwidth.

The curves in Fig. 4 have the same shape as in the ideal case, but with a lower resolution for a given SNR.

As expected, the resolution increases when the AR model order increases. Compared to the ideal case, the resolution is globally worse when the band is taken into account.

From this, and by interpolation, we can give a simple empirical expression for the resolution distance $d$:

$$d=\frac{K_B{(1+1/B)}^{{\theta }_B}}{{\mathrm{SNR}}^{{\theta }_S}{(1+M)}^{{\theta }_M}},$$ (25)

with $K_B=0.29$; ${\theta }_B=0.25$; ${\theta }_S=0.31$; ${\theta }_M=0.52$. This expression shows the relation among resolution, SNR, model order, and bandwidth. Typically, for ultrasound systems, the bandwidth is constant. So the higher the order, the better is the resolution. However, an optimal order should be chosen to avoid artifacts.

2.3.3. Nonsymmetric ultrasound echo

In ultrasound imaging, ultrasound is emitted through a pulse which is not symmetrical. In this section, we, hence, use a nonsymmetric PSF in order to more closely approximate the medical ultrasound imaging conditions. In this case, based on the example in Sec. 2.3.2, the PSF is assumed to be $g(u)=u^3{e}^{-(u^2/2{\chi }^2)}\cos (2\pi f_{0}u)$, see Fig. 8.

Fig. 8.

Example of an asymmetric echo function for $g$.

The results in Fig. 9 are similar to the Gaussian envelope case. This confirms the relationship among the resolution, model order, and bandwidth of the system (PSF).

Fig. 9.

SNR versus resolution and order for symmetrical and nonsymmetrical PSF.

The relationship between AR model order and the characteristics of the transducer PSF is direct but nonlinear. This relation suggests that finding the order is an important issue for the resolution improvement of an ultrasound image processed using an AR model. A solution is provided in the following sections with the practical application to ultrasound images.

3. Ultrasound Radio Frequency Signal Processing

3.1. New Multidimensional Instrumental Matrix Estimation Technique

As stated above, the B-mode ultrasound image consists of RF line envelopes. In order to increase the resolution of B-mode images, these envelopes have to be improved. In this study, instead of directly estimating the envelopes of the RF signals, we used specific AR modeling to achieve this via the inverse Fourier transform of analytical signal $g_{I/Q}(z)$ introduced in Eq. (3). In the following, we will dentote it by $y$.

Let us consider a second-order stationary multidimensional complex AR process (ND-AR) defined as follows:

$$\begin{gathered} y(n_1,n_2,\dots ,n_N)=\sum _{(k_1,k_2,\dots ,k_N)\in I}\sum a(k_1,k_2,\dots ,k_N) \\ \times y(n_1-k_1,n_2-k_2,\dots ,n_N-k_N)+w(n_1,n_2,\dots ,n_N), \end{gathered}$$ (26)

where $w(n_1,n_2,\dots ,n_N)$ is a field of random colored or correlated noise, and the parameters $a(k_1,k_2,\dots ,k_N)$ are complex numbers and provide a stable system. We consider here an ND-model to make our method general and applicable to three-dimensional (3-D) or four-dimensional (4-D) images. As in Ref. 18, we focus on the first hyperplane model without loss of generality. The methodology may be applied to other hyperplanes, i.e., the set of neighbors is $I=\{(k_1,k_2,\dots ,k_N)|k_i=\mathrm{1,2},\dots ,p_i,i=\mathrm{1,2},\dots ,N\}$. For convenience, we assume that $p_1=p_2=\dots p_N=m$, meaning that the model orders are identical in all directions. The well known one-dimensional (1-D) instrumental variable technique is a simple and efficient way to overcome basic least-square estimations, which are biased in the presence of colored noise or correlated data.¹³ We propose here a generalization of the 1-D case to a multidimensional structure based on ${\mathrm{UDV}}^H$ factorization, allowing access to recursive estimates of parameters of orders 0 to $m$. The approach is based on investigating multiple-dimensional signals through long vectors, which makes it very simple. In addition, the algorithm is defined for complex numbers. First, we define the following vectors where elements of $y$ and $a$ are stacked:

$$\begin{gathered} {\varphi }_m^T(n_1,n_2,\dots ,n_N)= \\ [y(n_1,n_2,\dots ,n_N-1)\dots y(n_1,n_2,\dots ,n_N-m)\dots \\ y(n_1,n_2-1,\dots ,n_N)\dots y(n_1,n_2-m,\dots ,n_N)\dots \\ y(n_1-m,n_2-m,\dots ,n_N-m)\dots y(n_1,n_2,\dots ,n_N)], \end{gathered}$$ (27)

$$\begin{gathered} {\theta }_m^T(n_1,n_2,\dots ,n_N)=[a(\mathrm{0,0},\dots 1,)\dots a(\mathrm{0,0},\dots m)a(\mathrm{0,1},\dots 0,)\dots \\ a(0,m,\dots 0,)\dots a(m,0,\dots ,0)\dots a(m,m,m\dots m)\dots 1]. \end{gathered}$$ (28)

Note that $y(n_1,n_2,\dots ,n_N)$ and 1 are parts of these vectors. Defining

$$\begin{gathered} x_m^T(n_1,n_2,\dots ,n_N)=[y(n_1,n_2,\dots ,n_N-1)\dots y(n_1,n_2,\dots ,n_N-m)\dots \\ y(n_1,n_2-1,\dots ,n_N)\dots y(n_1,n_2-m,\dots ,n_N)\dots \\ y(n_1-m,n_2-m,\dots ,n_N-m)] \end{gathered}$$ (29)

means that

$${\varphi }_m^T(n_1,n_2,\dots ,n_N)=[x_m^T(n_1,n_2,\dots ,n_N)y(n_1,n_2,\dots ,n_N)].$$ (30)

The instrumental data structure can be defined as

$$\begin{gathered} {\zeta }_m^T(n_1,n_2,\dots ,n_N)= \\ [z(n_1,n_2,\dots ,n_N-1)\dots z(n_1,n_2,\dots ,n_N-m)\dots \\ z(n_1,n_2-1,\dots ,n_N)\dots z(n_1,n_2-m,\dots ,n_N)\dots \\ z(n_1-m,n_2-m,\dots ,n_N-m)\dots z(n_1,n_2,\dots ,n_N)]. \end{gathered}$$ (31)

Different approaches can be used to select the instrument $z$, see, e.g., Ref. 19, for example, a solution consists of introducing a delay for each component of $x$.

$${\psi }_m^T(n_1,n_2,\dots ,n_N)=[{\zeta }_m^T(n_1,n_2,\dots ,n_N)z(n_1,n_2,\dots ,n_N)].$$ (32)

Defining the data regression matrix as

$$\begin{gathered} P_m(n_1,n_2,\dots ,n_N)= \\ {[\sum _{i_1=1}^{n_1}\dots \sum _{i_N=1}^{n_N}{\psi }_m(n_1,n_2,\dots ,n_N){\varphi }_m^H(n_1,n_2,\dots ,n_N)]}^{-1} \end{gathered}$$ (33)

and assuming that $p={(m+1)}^N$, the size of this matrix is $p\times p$. For convenience, the following notations can be used when no confusion is possible:

• •$\mathbf{n}=(n_1,n_2,\dots ,n_N)$
• •$\mathbf{k}=(k_1,k_2,\dots ,k_N)$
• •and more generally, for any index variable $(t_1,t_2,\dots ,t_N)$: $\mathbf{t}=(t_1,t_2,\dots ,t_N)$.

In addition, we denote

• •${\sum }_{\mathit{i}=1}^{\mathbf{n}}$ instead of ${\sum }_{i_1=1}^{n_1}\dots {\sum }_{i_N=1}^{n_N}$
• •${\sum }_{\mathbf{i}=1}^{\mathbf{n}-i}$ instead of ${\sum }_{i_1=1}^{n_1-i}\dots {\sum }_{i_N=1}^{n_N-i}$.

We can then write $P_m$ in its factorized form as follows:

$$P_m(\mathit{n})=U_m(\mathit{n})D_m(\mathit{n})V_m^H(\mathit{n}),$$ (34)

where ${()}^H$ denotes the Hermitian matrix transpose. The difference between this and the case presented in Ref. 18 is that the noise considered in Eq. (26) is a colored or correlated noise. Thus, through instrumental data as in Eq. (31), we used a matrix decomposition such as in Eq. (34) in which $U\ne V$ (taking $U=V$ would have resulted in biased parameter estimates, as in the case of classical estimation theory). As in Eq. (34), $U$ is an upper triangular matrix with all its diagonal elements equal to one. The elements of this upper triangular matrix are column vectors of dimensions 1 to $p$ defined as follows:

$$\begin{gathered} U_m(\mathit{n})=[\begin{array}{ccccc}1& \mathrm{col}\{\begin{array}{cc}{\vartheta }_{0,p}(\mathit{n})& 1\end{array}\}& \dots & \mathrm{col}\{\begin{array}{cc}{\vartheta }_{p-i,i}(\mathit{n})& 1\end{array}\}& \dots \end{array} \\ \begin{array}{ccc}\mathrm{col}\{\begin{array}{cc}{\vartheta }_{p-\mathrm{1,1}}(\mathit{n})& 1\end{array}\}& \dots & \mathrm{col}\{\begin{array}{cc}{\vartheta }_{p,0}(\mathit{n})& 1\end{array}\}\end{array}]. \end{gathered}$$ (35)

Remark:

• •${\vartheta }_{p-i,i}(\mathit{n})$ is a column vector of dimension $p-i$.
• •$\mathrm{col}\{\begin{array}{cc}{\vartheta }_{p-i,i}(\mathit{n})& 1\end{array}\}=\left[\begin{array}{c}{\vartheta }_{p-i,i}(\mathit{n})\\ 1\end{array}\right]$ is the $(p-i+1)$’th column.
• •Due to the model structure in Eq. (28), ${\vartheta }_{p-i,i}(\mathit{n})$ consists of parts or all of the instrumental variable estimates of the true parameters of the model, depending on whether the model order is smaller or larger than the dimension of ${\vartheta }_{p-i,i}(\mathit{n})$.

Similarly, $V$ is an upper triangular matrix with all its diagonal elements equal to one. The other elements of $V$ are different from those of $U$. These elements are the intermediate variables necessary to obtain the matrix decomposition. They also consist of column vectors of dimensions 1 to $p$, defined as follows:

$$\begin{gathered} V_m(\mathit{n})=[\begin{array}{ccccc}1& \mathrm{col}\{\begin{array}{cc}{\chi }_{0,p}(\mathit{n})& 1\end{array}\}& \dots & \{\begin{array}{cc}{\chi }_{p-i,i}(\mathit{n})& 1\end{array}\}& \dots \end{array} \\ \begin{array}{ccc}\mathrm{col}\{\begin{array}{cc}{\chi }_{p-\mathrm{1,1}}(\mathit{n})& 1\end{array}\}& \dots & \{\begin{array}{cc}{\chi }_{p,0}(\mathit{n})& 1\end{array}\}\end{array}]. \end{gathered}$$ (36)

$D_m(\mathit{n})$ in Eq. (34) is a diagonal matrix. Equation (34) is obtained from successive decompositions. Using Eq. (30) in Eq. (33), we obtain

$$P_m^{-1}(\mathit{n})=\left[\begin{array}{cc}\sum _{j=1}^{\mathit{n}}{\zeta }_m(\mathit{j})x_m^H(\mathit{j})& \sum _{j=1}^{\mathit{n}}{\zeta }_m(\mathit{j})y(\mathit{j})\\ \sum _{j=1}^{\mathit{n}}z(\mathit{j})x_m^H(\mathit{j})\sum _{j=1}^{\mathit{n}}& \sum _{j=1}^{\mathit{n}}y(\mathit{j})z(\mathit{j})\end{array}\right].$$ (37)

The methodology developed in Ref. 18 may be used to estimate the matrices $U$, $V$, and $D$, and select the estimated parameters and orders of the model presented in Eq. (26). The difference here is that the elements of $D$ are generalized loss functions for orders 1 to $m$. The first component of $D$ is then:

$$J_{p,0}(\mathit{n})=\sum _{j=1}^{\mathit{n}}[y(\mathit{j})-\widehat{y}(\mathit{j})][z(\mathit{j})-\widehat{z}(\mathit{j})],$$ (38)

$$\begin{gathered} \text{where}\widehat{y}(\mathit{n})=\sum _{\mathbf{j}=1}^{\mathit{n}}{x}_m^H(\mathit{j}){\vartheta }_{p,0}(\mathit{n}) \\ \widehat{z}(\mathit{n})=\sum _{\mathbf{j}=1}^{\mathit{n}}{z}_m^H(\mathit{j}){\chi }_{p,0}(\mathit{n}) \end{gathered}$$

and its $p-i$ column is:

$$J_{p-i,i}(\mathit{n})=\sum _{j=1}^{\mathit{n}}[y(\mathit{j})-\widehat{y}(j)][z(\mathit{j})-\widehat{z}(\mathit{j})]\mathrm{.}$$

Note that the result of this last equation is a complex number. By taking its magnitude value, a cost function is provided. Also, note that all elements of the matrices are stacked and $D_m$ has ${(m+1)}^N\times {(m+1)}^N$ elements. Thus, to find the true order of the model, the following steps may be used:

• 1.Split elements of $D_m$ into successive segments of $m+1$ elements, and create a new vector, $M_1$, consisting of the minima of the segments.
• 2.Repeat step 1 using the above set of minima.
• 3.Stop when the size of the vector of minima is $m+1$. This vector is called $M_N$. This procedure needs $N$ steps.
• 4.Finally, the minimum of $M_N$ gives the true order $m_0$.

Depending on the organization of $x_m^H(\mathit{n})$ in Eq. (29), the true parameters may be accessed following the methodology developed in Ref. 18. The decomposition of Eq. (34) may be performed by batch, or recursively. In this last case, the main step of the recursion is defined as follows. From Eq. (33), it can be written as

$$P_m(\mathit{n})={[P_m^{-1}(\mathit{n}-1)+{\psi }_m(\mathit{n}){\varphi }_m^H(\mathit{n})]}^{-1}.$$ (39)

The variables can be defined as follows:

$$f_1=U_m^T(\mathit{n}-1){\varphi }_m(\mathit{n})$$

$$f_2=V_m^T(\mathit{n}-1){\psi }_m(\mathit{n})$$

$$g_1=D_m(\mathit{n}-1)f_1^{*}$$

$$g_2=D_m(\mathit{n}-1)f_2^{*}$$

$$\beta (\mathit{n})=1+f_1^T{g}_2,$$

where the asterisk denotes the complex conjugate. $P_m(\mathit{n})$ can now be expressed by:

$$P_m(\mathit{n})=U_m(\mathit{n})D_m(\mathit{n})V_m^H(\mathit{n})=U_m(\mathit{n}-1)[D_m(\mathit{n}-1)-\frac{g_2{g}_1^H}{\beta (\mathit{n})}]V_m^H(\mathit{n}-1).$$ (40)

From these recursions, only elements of $U_m$ with physical meanings are retained.

The choice of model order is highly important: the higher the order, the better is the resolution; this is also the case for noise artifacts. Thus, this methodology makes it possible to simultaneously access the order and the parameters of the $N-D$ AR model.

3.2. Improvement of Ultrasound Image Resolution

3.2.1. Power spatial density

We consider the model described by Eq. (26) with $N=2$. We also recall^13,14 that the PSD is defined in the first quadrant by

$$\begin{gathered} P_{\mathrm{ar}1}(f_1,f_2)=\frac{{\sigma }_1^2(f_1,f_2)}{{|A_1(f_1,f_2)|}^2}, \\ \text{where}{A}_1(f_1,f_2)=\sum _{k_1=0}^{p_1}\sum _{k_2=0}^{p_2}{a}_1[k_1,k_2]e^{-j2\pi (f_1{k}_1+f_2{k}_2)}, \end{gathered}$$ (41)

and in the fourth quadrant by

$$\begin{gathered} P_{\mathrm{ar}4}(f_1,f_2)=\frac{{\sigma }_4^2(f_1,f_2)}{{|A_4(f_1,f_2)|}^2} \\ \text{where}{A}_4(f_1,f_2)=\sum _{k_1=0}^{p_1}\sum _{k_2=-p_2}^0{a}_4[k_1,k_2]e^{-j2\pi (f_1{k}_1+f_2{k}_2)}. \end{gathered}$$ (42)

To account for the causality and due to the equivalence between planes,¹³ the overall PSD is:

$$P_{\mathrm{ar}\mathrm{1,4}}(f_1,f_2)=\frac{{\sigma }^2(f_1,f_2)}{\frac{1}{2}[{|A_1(f_1,f_2)|}^2+{|A_4(f_1,f_2)|}^2]},$$ (43)

where ${|A_1(f_1,f_2)|}^2$ and ${|A_4(f_1,f_2)|}^2$ are the denominators in Eqs. (41) and (43). We used the parametric modeling developed in Sec. 3.1 to estimate the model parameter. Instead of the PSD, we obtained an image envelope estimate which is equivalent to PSD. Thus, to improve the spatial resolution, we applied this technique to the inverse Fourier transform of the demodulated I/Q signal given by the RF signals. Our methodology can, thus, be summarized as follows:

• 1.Obtain the Fourier transform of the nondemodulated I/Q signal in order to be in the Fourier domain.
• 2.Perform parametric modeling (using the approach proposed in Sec. 3.1).
• 3.Then estimate PSD and get an envelope of the demodulated I/Q signal with better resolution.

4. Results

4.1. Example of N-D Model Order and Parameters Estimation

To illustrate the method presented through Eqs. (26) to (40), we set $N=2$, since our aim is to deal with images, $m=4$ and $m_0=2$. We consider the following two-dimensional (2-D) complex AR model as defined in Eq. (28), where the model parameters $a(k_1,k_2)$, $k_1=\mathrm{0,1},2$; $k_2=\mathrm{0,1},2$ are defined as follows. $a(\mathrm{0,0})=1$ is not estimated.

$$y(n_1,n_2)=\sum _{k_1=0}^2\sum _{k_2=0;(k_1,k_2)\ne (\mathrm{0,0})}^{2}a(k_1,k_2)y(n_1-k_1,n_2-k_2)+w(n_1,n_2),$$

with $w(n_1,n_2)=e(n_1,n_2)+(1+3i)e(k_1-1,k_2)$, where $y$ is a $64\times 64$ complex field driven by a complex colored noise $w(n_1,n_2)$ obtained via $e(n_1,n_2)$, a Gaussian random field with variance 0.1. Following the same order determination as in Ref. 18, we find an order of 2, as can be seen in Fig. 10. The parameter estimates are shown in Table 1.

Fig. 10.

Cost function: the curve shows the loss function after extraction of the minima. As can be seen, the minimum is achieved for $m_0=2$.

Table 1.

Theoretical parameters (top) and estimated parameters (bottom) $a(k_1,k_2)$.

1	$-1.0000-1.0000i$	$0.0625+0.5000i$
$-1.0000-1.0000i$	$0+2.0000i$	$0.4375-0.5625i$
$0.0625+0.5000i$	$0.4375-0.5625i$	$-0.2461+0.0625i$
1	$-1.0049-1.0022\mathrm{i}$	$0.0632+0.4833\mathrm{i}$
$-1.0477-1.0386i$	$0.0031+2.0308i$	$0.4373-0.5749i$
$0.0528+0.5580i$	$0.4524-0.5728i$	$-0.2447+0.0699i$

4.1.1. Application to ultrasound images

The proposed method was applied to simulation (Fig. 11), in vitro (Fig. 13) and in vivo images (Fig. 14). For our simulation, we first generate a synthetic ultrasound image, representing a homogeneous medium crossed by a vessel, using the Field II simulation program.²⁰ The simulation parameters were as follows: central frequency = 3 MHz, number of RF lines simulated = 256, number of scatterers = 10,000. Then we generate the same image at a central frequency = 6 MHz, in order to double the image spatial resolution. The 3 MHz image was processed using our approach and the result was compared with the 6 MHz image. As can be seen qualitatively, the results are close to Fig. 11, last row (from top to bottom). Moreover, a profile of the envelope is extracted from the same RF lines of these images. As can be seen in Fig. 12, the profile obtained from our approach is close to that of the 6 MHz image. Figure 13 shows, for illustration purposes, the results on an in vitro image using a phantom made of a thread embedded in a gel. To give an indication of what an expected synthetic image should look like, the theoretical image is arbitrarily drawn and shown at the top of Fig. 13. The bottom-left image is the original 20 MHz ultrasound image, and the bottom right shows the resulting image after using the proposed method. The processed image is closer to the expected theoretical image. We also show, in Fig. 14, the results on an in vivo ultrasound image of, from top to bottom, the bladder of a mouse (first row), a rabbit eye (second row), the uterus of a pregnant mouse (third row), and a simulation image (fourth row). Qualitatively, as can be seen in each processed image, the edges are sharpened by the processing technique, which indicates that the spatial resolution has been improved. For spatial resolution estimation, we use Eq. (25). From this equation, we can compute the improvement in resolution. Given an image and device characteristics, this equation empirically indicates the theoretical minimal achievable resolution. This resolution depends on the SNR, model order, and bandwidth of the ultrasound probe. From images in Fig. 14, we performed 1-D analysis on individual RF lines of the images. The estimated axial bandwidth of the PSF of the probe used to obtain images in Fig. 14 (first three rows) was 60%. In Table 2, for these images, we evaluated the improvement in resolution defined by the ratio

$$R_{\mathrm{est}}/R_{a\mathrm{ct}},$$

where $R_{\mathrm{act}}$ is the actual resolution [Eq. (1)] for the ultrasound frequency $f_0$ and $R_{\mathrm{est}}$ is the $a$ posteriori resolution estimated by our approach. Note that this ratio should be $<1$ to show resolution enhancement. The theoretical axial improvement in resolution obtained for Fig. 14, from top to bottom, first row, second row, third row, and fourth row, is, respectively, 0.9, 0.7, 0.6, and 0.5, as is shown in Table 2. To evaluate the improvement in lateral resolution, we also performed a 2-D analysis. For the images in Fig. 13, the estimated bandwith in the axial direction was 60% and that in the lateral direction was 11%. The orders found in axial and lateral directions were respectiveley 5 and 5. The axial and lateral resolution improvements were 0.6 and 0.3, respectively. We also compared our technique to classical ones, namely, homomorphic filering, e.g., Ref. 7, and parametric inverse filtering via hybrid approach (HYPIF) filtering.⁸ For this comparison, we used the classical gain in resolution parameter, e.g., Refs. ⁸ and ⁹, since these methods do not allow the estimation of the improvement in resolution introduced here. The results are shown in Table 3. As can be seen, our technique outperforms these methods. Moreover, although the primary goal of the proposed method was not to enhance the contrast (but the resolution), we also compare our method with the existing methods in terms of contrast-to-noise ratio (CNR), see, e.g., Ref. 21. The CNR is defined as

$$\mathrm{CNR}=\frac{|{\mu }_1-{\mu }_2|}{\sqrt{{\sigma }_1^2+{\sigma }_2^2}},$$

where ${\mu }_1$, ${\mu }_2$, ${\sigma }_1$, and ${\sigma }_2$ are, respectively, the mean and the variance of the pixels in two chosen regions of interest highlighted by the square boxes in Fig. 14. The higher the CNR, the better is the contrast. As can be seen in Table 4, even if the CNR is not high for all the images, our method outperforms the existing ones in all the cases. Finally, Table 5 shows the computation times of the different methods for an image of size $1024\times 256$, using a 2.1 MHz dual core Xeon PC. As can be seen, the computation time of our method is reasonably low (although higher than the one of homomorphic filtering).

Fig. 11.

Comparison between synthetic original and processed images: (a) original 3 MHz image, (b) 6 MHz image, which is the ground truth, and (c) the processed image obtained from the original 3 MHz image, with our method.

Fig. 13.

Comparison between the expected theoretical image (a), the original ultrasound image of a phantom consisting of a thread embedded in a gel (b), and the processed image (c) obtained from the original ultrasound image. The thread edge is sharper on the processed image.

Fig. 14.

Comparison of visual appreciation of different methods on different images. From left to right, first column: the original images, second column: the results for homorphic filtering, third column: the results for HYPIF, and fourth column: the results of the proposed method. From top to bottom, first row: a mouse bladder image, second row: rabbit eye image, third row: uterus of a pregnant mouse, and fourth row: a synthetic image.

Fig. 12.

Comparison between profiles: profiles of the log-envelope of an extracted radio frequency line located in the top of Fig. 11 (solid line) from the synthetic 3 MHz image (solid), the synthetic 6 MHz image (dash dot), and processed images (dash).

Table 2.

Improvement in resolution. The bandwidth of the used transducer was 60% except for the simulation one, which was 90%.

	Mouse bladder	Pregnant mouse	Rabbit eye	Synthetic image
	$f_0=20\mathrm{MHz}$	$f_0=20\mathrm{MHz}$	$f_0=20\mathrm{MHz}$	$f_0=3.5\mathrm{MHz}$
Found order	79	82	70	70
$R_{\mathrm{est}}/R_{\mathrm{act}}$	0.9	0.6	0.7	0.5

Table 3.

Empirical gain in resolution.

	Mouse bladder	Pregnant mouse	Rabbit eye	Synthetic image
$G$ (proposed)	11.97	2.76	4.20	3.71
$G$ (HYPIF)	1.1	1.26	1.16	0.49
$G$ (Homorphic)	1	0.99	1.00	0.36
$G$ 3.5 MHz  versus 7 MHz	—	—	—	1.5

Table 4.

Contrast-to-noise ratio in decibels. The higher, the better.

	Mouse bladder	Pregnant mouse	Rabbit eye	Synthetic image
$G$ (proposed)	9.2	3.6	$-4.9$	9.9
$G$ (HYPIF)	$-15.3$	$-26.6$	$-12.8$	9.6
$G$ (Homorphic)	07.8	2.9	$-5.2$	3.8
Original	$-20.3$	$-35$	$-14.3$	8.7

Table 5.

Computation time (in seconds) for an image of size $1024\times 256$.

Method	Homomorphic	HYPIF	Proposed
Computation time (s)	0.04	185.40	0.71

5. Conclusion and General Remarks

In this work, we first revisited super-resolution theory in the framework of ultrasound imaging. The improvement of resolution can be quantified, given the PSF shape, the SNR, and the AR model order. For better readability, all results and derivations were presented in 1-D. Then we presented a way to improve ultrasound image resolution. The methodology developed is based on specific spectral analysis like modeling, which fits ultrasound images. Applied to synthetic, in vitro, and in vivo images, this methodology shows a qualitative and quantitative resolution improvement. An important characteristic of the proposed approach is its ability to estimate a priori the improvement in resolution. The improvement in resolution achieved here is up to a ratio of two. To our knowledge, this work is the first to deal with the a priori estimation of the gain in resolution in ultrasound imaging. Finally, comparisons with existing techniques were provided. The results show that the proposed method outperforms existing techniques, both in terms of subjective visual quality and in terms of quantitative evaluation using the classical gain in resolution. Although we focus here on the 1-D and 2-D cases and show improvement of the resolution of ultrasound images using the proposed method, the methodology proposed may be applied whatever be the imaging processing dimension (3-D or 4-D).

Biographies

Marie Ploquin received her MS in 2007 from the University of Tours, Tours, France, and PhD degrees in 2011, in signal and image processing, from the Universities of Tours and Toulouse, France. She is currently a senior engineer at Solent France.

Adrian Basarab received MS and PhD degrees in signal and image processing from the National Institute for Applied Sciences of Lyon, France, in 2005 and 2008. Since 2009, he has been an assistant professor at the University Paul Sabatier Toulouse 3 and a member of IRIT Laboratory (UMR CNRS 5505). His research interests include medical imaging, and more particularly motion estimation, inverse problems, and ultrasound image formation.

Denis Kouamé is with University Paul Sabatier, Toulouse, France. His research interests are focused on signal and image processing with applications to medical imaging and particularly ultrasound imaging, including high resolution imaging, image resolution enhancement, doppler signal processing, detection and estimation with application to cerebral emboli detection, multidimensional parametric modeling, spectral analysis, Inverse problems related to compressed sensing, and restoration.