UltraFlex: Iterative Model-Based Ultrasonic Flexible-Array Shape Calibration

Abstract

UltraFlex is an iterative model-based ultrasonic flexible-array shape calibration framework that uses automatic differentiation. This work evaluates array shape calibration model performance while examining multiple image quality metrics: speckle brightness, envelope entropy, coherence factor, lag-one coherence, common-midpoint correlation coefficient, and common-midpoint phase error. The accuracy of these image quality metrics was evaluated on simulated phantoms using a variety of array shapes. Experimental phantom and in vivo liver datasets were also investigated using transducers with known geometries. While speckle brightness, envelope entropy, and coherence factor enable model convergence under many conditions, lag-one coherence, common-midpoint correlation coefficient, and common-midpoint phase error enable more accurate element position estimations and improved visual ultrasound image focusing quality. Furthermore, the models based on the common-midpoint correlation coefficient and phase-error quality metrics are the most robust against additive white noise while achieving median mean Euclidean errors (MEEs) of 3.7 μm for simulation, 29.7 μm for phantom, and 69.0 μm for in vivo liver data. These array shape calibration results show promise for future development of experimental flexible- and wearable-ultrasonic arrays.

Graphical Abstract

I. Introduction

For decades, medical ultrasound imaging has been confined to conventional rigid-array transducers. While these conventional transducers have provided immeasurable benefit to ultrasound imaging, the recent development of flexible-array ultrasound transducers opens the door to new applications for ultrasound, including continuous monitoring and wearable ultrasound technologies [1]–[5]. Flexible-array ultrasound systems provide a conformable aperture that can accommodate the contoured surface of an imaging subject, and some flexible-array ultrasound systems can be used in a wearable configuration to alleviate the need for continuous ultrasound navigation, reducing operator dependency [1].

For example, rodent-wearable flexible-array transducers have previously been investigated with the goal of neural activity monitoring [2]. Human-wearable flexible-array device studies have been designed for assessing cardiac dysfunction via cardiac output, ejection fraction, and stroke volume [3]. Another application area of wearable flexible-array ultrasound systems can be found in therapeutics. Wound healing with focused ultrasound has demonstrated protein stimulation in dermal and epidermal layers of diabetic rats [4]. Neuromodulation using a prototype wearable flexible-array has also been proposed [5]. Despite the benefits that flexible-array ultrasound systems provide, image reconstruction and therapeutic focusing are compromised without knowing the pose of each transducer element (that is, orientation and position in ${\mathbb{R}}^3$)

Two primary approaches have been used to estimate unknown element poses. The first class of approaches involves external mechanical (e.g., fiber-optic strain [6], resistive-strain, etc.) or optical [7], [8] tracking-based analysis of array deformation. Chen et al. [6] used a near-infrared fiber-optic reflectometer device embedded within a custom flexible-array transducer to achieve maximum position errors of 400 μm and 421 μm in the $y-z$ (lateral) plane for a convex and sinusoid array shape, respectively. China et al. [8] passively tracked infrared-reflecting spheres affixed to a flexible array to achieve position errors of 500 ± 290 μm (CIRS phantom), 540 ± 350 μm (deformable phantom), and 360 ± 240 μm (cadaveric specimen). These approaches require external equipment that adds bulk to the transducer, additional complexity to clinical translation, or both.

The second class of approaches involves intrinsic determination of element pose via algorithms that rely only on the channel data collected from the imaging system. These approaches include methods based on assumed array geometries [9]–[11], derivative-free black-box optimization (e.g. simulated annealing [12], [13] or exhaustive sampling [14], [15]), deep learning methods [16], [17], and direct gradient-based optimization [7], [18]. Some approaches within this class assess an image quality metric such as the maximum lag-one spatial coherence [11], image contrast [12], envelope amplitude-variance region-of-interest sharpness [13], phase variance [14], short-lag spatial coherence [15], and image entropy [7], [18]. Ingram et al. [14] achieved root mean square errors in element positions of 0.18 $\lambda$ (110.9 μm) and 0.4 $\lambda$ (246.4 μm) for simulated and experimental data, respectively. Omidvar et al. [15] achieved mean Euclidean errors (MEEs) lower than 0.1 $\lambda$ (43.1 μm) and 1.4 $\lambda$ (603.9 μm) for simulated and experimental data, respectively. Noda et al. [16] achieved an average MEE of 860 μm and 1,110 μm for simulation and in vivo test data, respectively, using a deep learning approach. Noda et al. [18] reported average MEEs between 40 and 54 μm across three array geometries for simulation data, and between 8.6 and 37 μm across three array geometries for phantom data while using a direct gradient-based approach with an aperture size of only 20 elements in each case. Huang et al. [7] built upon [18] with coarse initialization from an optical tracker while achieving a post-iterative-model MEE of 148.8 μm.

As mentioned previously, methods within the second class of approaches may be derivative-free or involve gradient backpropagation via automatic differentiation frameworks (e.g., JAX [19], PyTorch, and TensorFlow). Conventional deep learning-based methods utilize gradient backpropagation to update the weights and biases of a multilayer neural network. A direct gradient-based optimization approach may similarly utilize an automatic differentiation framework with a differentiable objective function to iteratively update model assumptions (e.g., array element poses) [18].

In Noda et al. [18], an array shape estimation is made using a linear combination of orthogonal functions. Based on this array shape, time-of-flight values are calculated, and image focus is evaluated by calculating the beamformed envelope entropy. The errors are backpropagated through the beamformer to update the combination of orthogonal functions. Our method differs from Noda et al. [18] in that (a) we do not constrain our estimated shape to a combination of orthogonal functions (our only assumption is that the number of elements and the pitch of the array are known), (b) we preferentially use a differentiable objective function that is based on common-midpoint signals and is not subject to errors resulting from natural signal decorrelation or variations in signal amplitude, (c) we account for the orientation of individual elements, and (d) we validate our model on in vivo liver data. Here, we extend our previous work in Hyun et al. [20] to rigorously evaluate our UltraFlex framework across various quality metric-based objective functions, update the reconstruction model to account for curved surfaces, and examine the performance on phantoms and in vivo data.

II. UltraFlex Framework

A. Differentiable Image Reconstruction

Pulse-echo ultrasound imaging utilizes transmit and receive beamforming to focus pressure signals (i.e., to spatially localize them) [21]. Let ${\mathit{r}}_a$ denote the spatial position of the $a$-th array element in the $x-z$ (elevational) plane and $c$ denote a constant speed of sound. The time-of-flight from the element to the coordinate ${\mathit{r}}_{\mathrm{\Omega }}$ in the target domain $\mathrm{\Omega }$ is

$$t_d\left({\mathit{r}}_a;{\mathit{r}}_{\mathrm{\Omega }},c\right)=\frac{1}{c}{‖{\mathit{r}}_{\mathrm{\Omega }}-{\mathit{r}}_a‖}_2,$$ (1)

where ${\mathit{r}}_{\mathrm{\Omega }}$ and $c$ are static (fixed) parameters that are not updated via backpropagation. Elements with spatial extent exhibit directivity, which can be modeled using an acceptance cone mask based on $f$-number. For convenience in calculating a reconstruction region using a constant $f$-number, we first transform ${\mathit{r}}_{\mathrm{\Omega }}$ into the element’s frame of reference using a translation and rotation:

$${\mathit{r}}_{\mathrm{\Omega }}^{\prime }=\left[\begin{array}{c}{x}_{\mathrm{\Omega }}^{\prime }\\ z_{\mathrm{\Omega }}^{\prime }\end{array}\right]=\left[\begin{array}{cc}\mathrm{c}\mathrm{o}\mathrm{s}\theta \left({\mathit{r}}_a\right)& -\mathrm{s}\mathrm{i}\mathrm{n}\theta \left({\mathit{r}}_a\right)\\ \mathrm{s}\mathrm{i}\mathrm{n}\theta \left({\mathit{r}}_a\right)& \mathrm{c}\mathrm{o}\mathrm{s}\theta \left({\mathit{r}}_a\right)\end{array}\right]\left[\begin{array}{c}{x}_{\mathrm{\Omega }}-x_a\\ z_{\mathrm{\Omega }}-z_a\end{array}\right],$$ (2)

where the element orientation ${\theta }_a$ is inferred from neighboring elements as

$$\theta \left({\mathit{r}}_a\right)=-{\mathrm{t}\mathrm{a}\mathrm{n}}^{-1}\left(\left(z_{a+1}-z_{a-1}\right)/\left(x_{a+1}-x_{a-1}\right)\right).$$ (3)

Then, the acceptance cone mask (apodization) of the $a$-th element using a constant $f$-number (denoted by $F$) is

$$h_a\left({\mathit{r}}_{\mathrm{\Omega }}\right)=\left|x_{\mathrm{\Omega }}^{\prime }\right|\le \frac{F}{2}{z}_{\mathrm{\Omega }}^{\prime }.$$ (4)

Given the set of all element positions ${\overrightarrow{\mathit{r}}}_a$, the round-trip time-of-flight from the $t$-th transmit element to ${\mathit{r}}_{\mathrm{\Omega }}$ and back to the $r$-th receive element is simply ${\tau }_{t,r}({\overrightarrow{\mathit{r}}}_a)=t_d\left({\mathit{r}}_t\right)+t_d\left({\mathit{r}}_r\right)$.

Transmit and receive beamforming are applied as follows. Let $𝒜$ be the aperture element indices $\{1,2,\dots ,N\}$. For a transmit subaperture $𝒯\subseteq 𝒜$ and receive subaperture $ℛ\subseteq 𝒜$, the beamformed signal is

$$B_{𝒯,ℛ}({\overrightarrow{\mathit{r}}}_a;{\mathit{r}}_{\mathrm{\Omega }},\mathbf{\Psi })=\sum _{t\in 𝒯}\sum _{r\in ℛ}{\mathbf{\Psi }}_{t,r}\left({\tau }_{t,r}\right)\mathrm{e}\mathrm{x}\mathrm{p}\left(j2\pi f_d{\tau }_{t,r}\right)h_t{h}_r,$$ (5)

where $\mathbf{\Psi }$ is the full synthetic aperture (FSA) data represented as baseband in-phase and quadrature (IQ) data (hereafter referred to as the demodulated FSA data), ${\tau }_{t,r}({\overrightarrow{\mathit{r}}}_a)$ is abbreviated as ${\tau }_{t,r}$, the exponential term represents a phase rotation to compensate for the demodulation-induced phase offset at the demodulation frequency, $f_d$, and $h_t$ and $h_r$ are the transmit and receive apodizations, respectively. For notational simplification, $B$ will often be expressed without a position ${\mathit{r}}_{\mathrm{\Omega }}$ in the target domain or the demodulated FSA data $\mathbf{\Psi }$ because these arguments are constant across all model iterations. Because each step (i.e., Eqs. 1–5) is differentiable, the entire image reconstruction process is differentiable.

B. Array Shape Calibration

Automatic differentiation frameworks such as JAX [19], PyTorch, and Tensorflow can offer model representation and optimization by storing differentiable operations in a computational graph. Similarly to training neural networks, we use automatic differentiation to minimize an objective function, where gradient backpropagation is used to update unknown model parameters (in this case, the element positions of the array). The objective function errors are backpropagated as gradients through a physical model (in this case, the differentiable image reconstruction process) to directly update the estimated element positions.

The entire iterative flexible array shape estimation process is shown in Fig. 1. The flexible array will be referred to as an imaging aperture. First (Fig. 1a), an FSA transmission sequence is utilized to collect FSA data from an unknown aperture shape. During an FSA transmission sequence, only one element is activated during a transmission event while all the other elements receive the back-scattered echoes. This transmission event is repeated for each element of the aperture, forming an FSA dataset (i.e., a tensor with dimensions of time by number of transmit events by number of receive channels). The acquired radio-frequency (RF) data is demodulated to extract the in-phase and quadrature (IQ) baseband components such that $\mathbf{\Psi }\in {\mathbb{C}}^{N_s\times \mathrm{T}\mathrm{x}\times \mathrm{R}\mathrm{x}}$. Next (Fig. 1b), an initial assumption is made about the aperture geometry (e.g., the aperture shape is a linear, flat array) given a known pitch. From this aperture shape, the element time-of-flight ${\tau }_{t,r}({\overrightarrow{\mathit{r}}}_a)$ values are calculated given geometric target domain positions ${\overrightarrow{\mathit{r}}}_{\mathrm{\Omega }}$ and an assumed sound speed $c$ using Eq. 1. Finally, the demodulated FSA data $\mathbf{\Psi }$ is beamformed (Eq. 5) over a set of target domain positions ${\overrightarrow{\mathit{r}}}_{\mathrm{\Omega }}$, and a focusing or image quality metric $Q$ is calculated to assess the accuracy of the estimated element positions ${\overrightarrow{\mathit{r}}}_a$. The metric is used within an optimizable objective function $ℒ$ (Eq. 14, see Sec. II-D), and the objective function error is backpropagated through the quality calculation, the beamforming process, and the time-of-flight calculation to update ${\overrightarrow{\mathit{r}}}_a$. This process is repeated iteratively (Fig. 1c), where ${\overrightarrow{\mathit{r}}}_a$ from the current iteration $i$ is used to initialize the next iteration $i+1$:

$${\overrightarrow{\mathit{r}}}_a^{(i+1)}={\overrightarrow{\mathit{r}}}_a^{\left(i\right)}-\gamma \frac{\partial ℒ}{\partial {\overrightarrow{\mathit{r}}}_a}({\overrightarrow{\mathit{r}}}_a^{\left(i\right)}),$$ (6)

where $\gamma$ is a step size parameter, $\partial ℒ({\overrightarrow{\mathit{r}}}_a^{\left(i\right)})/\partial {\overrightarrow{\mathit{r}}}_a$ denotes the gradient of the objective function $ℒ$ with respect to the element positions ${\overrightarrow{\mathit{r}}}_a$ evaluated at iteration $i$, and the element positions are updated in the direction of the negative gradient of the objective function. In this paper, we used a fixed number of iterations to observe model error for different quality metrics. In practice, a convergence criterion could be applied to stop the optimization process once a stable error has been reached.

Fig. 1.

(a) Acquisition of the FSA data. (b) The iterative aperture shape estimation process begins by computing the bi-directional time-of-flight ${\mathit{\tau }}_{\mathit{t},\mathit{r}}$. From these time-of-flight calculations, the demodulated FSA data $\mathbf{\Psi }$ is beamformed using a synthetic aperture approach to produce a subaperture image $\mathit{B}$ (the image above has been envelope-detected and log-compressed). An image quality $\mathit{Q}$ is evaluated, which is then used as part of an objective function $ℒ$. The objective function is differentiated with respect to the element positions ${\overrightarrow{\mathit{r}}}_{\mathit{a}}$, and the red arrows represent the error that is backpropagated through the model to update the array element positions at iteration $\mathit{i}$. (c) Visualization of B-mode reconstruction over iterations, starting with the aperture shape before the first iteration and after one thousand iterations. The color of the aperture loosely represents the deviation of the aperture shape from the ground truth, where red and green represent more and less deviation, respectively.

C. Quality Metrics

Seven image quality metrics from the literature are adapted to the UltraFlex framework for evaluation. For the speckle brightness, envelope entropy, coherence factor, and lag-one coherence metrics below, each metric expression is evaluated using the full transmit $(𝒯=𝒜)$ and receive $(ℛ=𝒜)$ apertures (i.e., $B_{𝒯,ℛ}=B_{𝒜,𝒜}$). For the common-midpoint correlation coefficient and common-midpoint phase-error metrics, each metric expression is evaluated using a pair of transmit (${𝒯}_{\alpha }$, ${𝒯}_{\beta }\subset 𝒜$) and receiver (${ℛ}_{\alpha }$, ${ℛ}_{\beta }\subset 𝒜$) subapertures.

1). Speckle Brightness:

Image speckle brightness is defined as the magnitude of the synthetic focused images within a region of interest. Nock et al. [22] introduced the speckle brightness quality metric as part of a method for correcting unknown phase aberrations. The criterion can be expressed as

$$Q_{\mathrm{S}\mathrm{B}}({\overrightarrow{\mathit{r}}}_a)=\left|B_{𝒜,𝒜}({\overrightarrow{\mathit{r}}}_a)\right|.$$ (7)

2). Envelope Entropy:

Envelope entropy is a measure of image disorder or defocusing. When the envelope of a beamformed signal has high entropy, the image has more distortion due to malformed PSF functions. Noda et al. [18] adapted this focusing criterion for ultrasonic flexible-array shape estimation from applications in inverse synthetic-aperture radar [23]. The metric can be expressed as

$$Q_{\mathrm{E}\mathrm{E}}({\overrightarrow{\mathit{r}}}_a)=-B_{\mathrm{e}\mathrm{n}\mathrm{v}}({\overrightarrow{\mathit{r}}}_a){\mathrm{l}\mathrm{o}\mathrm{g}}_2{B}_{\mathrm{e}\mathrm{n}\mathrm{v}}({\overrightarrow{\mathit{r}}}_a),$$ (8)

where $B_{\text{env}}$ represents the normalized envelope-detected signal over the sum of all $N_{\mathrm{\Omega }}$ domain pixels:

$$B_{\mathrm{e}\mathrm{n}\mathrm{v}}({\overrightarrow{\mathit{r}}}_a)=\frac{\left|B_{𝒜,𝒜}({\overrightarrow{\mathit{r}}}_a)\right|}{\sum _{v=1}^{N_{\mathrm{\Omega }}}\left|B_{𝒜,𝒜}({\overrightarrow{\mathit{r}}}_a;{\mathit{r}}_v)\right|}.$$

3). Coherence Factor:

Mallart and Fink [24] introduced a focusing criterion, also known as the coherence factor, as a measure of focusing quality that is optimized with perfect focusing and decreases under increasing aberration. The coherence factor is a ratio between the coherent and incoherent beamformed signal sums across the receive aperture. Normalization was not applied because, by definition, coherence factor is normalized by the incoherent sum to be on the range [0,1]. The metric can be expressed as

$$Q_{\mathrm{C}\mathrm{F}}({\overrightarrow{\mathit{r}}}_a)=\frac{\left|\sum _{r=1}^{N_r}{B}_{𝒜,r}\right|}{\sum _{r=1}^{N_r}\left|B_{𝒜,r}\right|},$$ (9)

where the abbreviation $B_{𝒜,r}=B_{𝒜,r}({\overrightarrow{\mathit{r}}}_a)$ refers to the transmit-beamformed receive-channel signals that have been time-delayed, and $N_r$ is the number of receive channels.

4). Lag-One Coherence:

Lag-based spatial coherence describes the covariance between receiver-domain signals received at an element and its neighbor $m$ elements (lags) away [24]. Lag-one coherence is a specific case of lag-based spatial coherence that only includes the first lag ($m=1$) and was introduced as a quality metric by Long et. al [25]. The metric can be expressed as

$$Q_{\mathrm{L}\mathrm{C}}({\overrightarrow{\mathit{r}}}_a)=\frac{\mathfrak{R}\left\{\sum _{r=1}^{N_r-1}{B}_{𝒜,r}{B}_{𝒜,r+1}^{\mathrm{*}}\right\}}{\sqrt{\sum _{r=1}^{N_r-1}{\left|B_{𝒜,r}\right|}^2{\left|B_{𝒜,r+1}\right|}^2}},$$ (10)

where $\mathfrak{R}\{\cdot \}$ is the real part of the enclosed complex value.

5). Common-Midpoint Correlation Coefficient (CMCC):

The common-midpoint correlation coefficient (CMCC) is similar to the lag-one coherence quality metric but is applied in the common-midpoint domain. Signals correctly beamformed from common midpoint apertures should theoretically be identical [26], [27]. The CMCC metric describes the correlation between two pairs of transmit and receive subapertures that share a common midpoint in the aperture domain. For example, apertures ${𝒯}_{\alpha }\subset 𝒯$ and ${ℛ}_{\alpha }\subset ℛ$ may represent transmit and receive subapertures, respectively, each composed of $N$ elements. Suppose that subapertures ${𝒯}_{\beta }\subset 𝒯$ and ${ℛ}_{\beta }\subset ℛ$ are another pair of transmit and receive apertures, respectively, that are lag-one neighbors (one element separation) with ${𝒯}_{\alpha }$ and ${ℛ}_{\alpha }$ and share the same physical midpoint in the aperture domain as shown in Fig. 2. The correlation between $B_{\alpha }=B_{{𝒯}_{\alpha },{ℛ}_{\alpha }}({\overrightarrow{\mathit{r}}}_a)$ and $B_{\beta }=B_{{𝒯}_{\beta },{ℛ}_{\beta }}({\overrightarrow{\mathit{r}}}_a)$ is an example of the CMCC focusing criterion and is expressed as

$$Q_{\mathrm{C}\mathrm{C}}({\overrightarrow{\mathit{r}}}_a)=\frac{\mathfrak{R}\left\{⟨B_{\alpha }{B}_{\beta }^{\mathrm{*}}⟩\right\}}{\sqrt{{⟨|B_{\alpha }|}^2⟩{\left.⟨|B_{\beta }\right|}^2⟩}}.$$ (11)

Fig. 2.

Illustration of common-midpoint subaperture pairs with subapertures composed of three elements. Pair $\mathit{\alpha }$ and $\mathit{\beta }$ are lag-one neighbors. ${\mathit{r}}_{\mathrm{C}\mathrm{M}}$ and ${\mathit{r}}_{\mathbf{\Omega }}$ are the common-midpoint position in the aperture domain and the image focusing position in the target domain, respectively. The element orientation $\mathit{\theta }\left({\mathit{r}}_{\mathit{a}}\right)$ is drawn for reference.

Here, the numerator is the real part of the cross-correlation, while the denominator provides normalization by the square root of the product of the individual auto-correlations. Angle brackets represent the correlation over a spatial kernel.

6). Common-Midpoint Phase Error (CMPE):

Common-midpoint phase error was introduced as an objective function by Simson et al. [28] and describes the phase error between two lag-one pairs of transmit and receive subapertures that share a common midpoint in the aperture domain. Similar to the CMCC metric, the phase error between $B_{\alpha }=B_{{𝒯}_{\alpha },{ℛ}_{\alpha }}({\overrightarrow{\mathit{r}}}_a)$ and $B_{\beta }=B_{{𝒯}_{\beta },{ℛ}_{\beta }}({\overrightarrow{\mathit{r}}}_a)$ can be expressed as

$$\mathrm{\Delta }{\varphi }_{\alpha ,\beta }({\overrightarrow{\mathit{r}}}_a)=\mathrm{\angle }[B_{\alpha }{B}_{\beta }^{\mathrm{*}}],$$

and the quality metric for phase error is

$$Q_{\mathrm{P}\mathrm{E}}({\overrightarrow{\mathit{r}}}_a)=\left|\mathrm{\Delta }{\varphi }_{\alpha ,\beta }({\overrightarrow{\mathit{r}}}_a)\right|.$$ (12)

The CMPE metric is the complex angle of the CMCC. The CMPE is computed from all lag-one pairs of transmit and receive subapertures, where $\alpha \subset A$ and $\beta \subset B$. Furthermore, a filtered version of the CMPE metric can be introduced to improve robustness against noise by selecting CMPE values at pixels with CMCC values $\ge {\rho }_{\text{thresh}}$. This ensures that only regions with sufficiently high cross-correlation contribute to the final metric. The filtered CMPE is defined as

$$Q_{\mathrm{P}\mathrm{E}\mathrm{F}}({\overrightarrow{\mathit{r}}}_a)=\left|\mathrm{\Delta }{\varphi }_{\alpha ,\beta }({\overrightarrow{\mathit{r}}}_a)\cdot \mathbb{1}(Q_{\mathrm{C}\mathrm{C}}({\overrightarrow{\mathit{r}}}_a)\ge {\rho }_{\text{thresh}})\right|,$$ (13)

where $\mathbb{1}(\cdot )$ is the indicator function and selects a subset of values returned by $\mathrm{\Delta }{\varphi }_{\alpha ,\beta }({\overrightarrow{\mathit{r}}}_a)$. For the remainder of the paper, the CMPE and CMCC quality metrics will be referred to as phase error and correlation coefficient, respectively.

D. Objective Function

The objective function used for optimization incorporates one of the previously defined image quality metrics with regularization. The general expression for the objective function is given as

$$\begin{array}{l}ℒ({\overrightarrow{\mathit{r}}}_a)=w_1\cdot \mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}[{ℒ}_1(Q({\overrightarrow{\mathit{r}}}_a))]\\ +w_2\cdot R_{\mathrm{T}\mathrm{V}}({\overrightarrow{\mathit{r}}}_a)\\ +w_3\cdot R_{\mathrm{d}z/\mathrm{d}x}({\overrightarrow{\mathit{r}}}_a),\end{array}$$ (14)

where $w_j$ are weights applied to each term, ${ℒ}_1$ is the reduction (transformation) of a quality metric $Q$, $R_{\mathrm{T}\mathrm{V}}$ is the total variation regularization term, and $R_{\mathrm{d}z/\mathrm{d}x}$ is the shape differential regularization term. Learning rates and reduction expressions for each quality metric were empirically tuned for (a) optimal model performance within 1,000 iterations (as detailed in Table I) and (b) to achieve a consistent metric output range, enabling the use of a consistent step size and regularization constant across different metric-based objectives. The expectation operator denotes the mean across all pixels and, in the case of the correlation coefficient and phase-error metrics, across all subaperture pairs. Speckle brightness and envelope entropy were linearly scaled before averaging. Lagone coherence and common-midpoint correlation coefficient metrics used a inverse hyperbolic tangent transform (i.e., the Fisher transformation, commonly used to expand the tails of correlation coefficients to infinity for statistical purposes) to emphasize values near the boundaries of (−1,1). Phase-error metrics were squared to penalize large errors and then passed through the natural logarithm of one plus the input for numerical stability. Negative signs in reduction expressions indicate metrics maximized by correct element positioning, and unsigned expressions indicate metrics minimized by correct element positioning. A hyperbolic tangent function is applied to ${ℒ}_1$ to confine its range to (−1, 1), preventing the need for dynamic rescaling and enabling standardized regularization across different metrics.

TABLE I.

Learning Rates (LR) and Reductions

Quality Metric	LR	Reduction Expression
Envelope Entropy	1 · 10−7	${ℒ}_1=𝔼\{5\cdot {10}^{-2}\cdot Q_{\mathrm{E}\mathrm{E}}\}$
Coherence Factor	1 · 10−7	${ℒ}_1=-𝔼\{Q_{\mathrm{C}\mathrm{F}}\}$
Speckle Brightness	1 · 10−7	${ℒ}_1=-𝔼\{2\cdot {10}^{-5}\cdot Q_{\mathrm{S}\mathrm{B}}\}$
Correlation Coefficient	1 · 10−7	${ℒ}_1=-𝔼\{{10}^{-2}\cdot {\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}}^{-1}(Q_{\mathrm{C}\mathrm{C}})\}$
Lag-One Coherence	7 · 10−7	${ℒ}_1=-𝔼\{{\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}}^{-1}(Q_{\mathrm{L}\mathrm{C}})\}$
Phase Error (Filtered)	1 · 10−7	${ℒ}_1=𝔼\{\mathrm{l}\mathrm{n}(1+{({10}^2\cdot Q_{\mathrm{P}\mathrm{E}\mathrm{F}})}^2)\}$
Phase Error	1 · 10−7	${ℒ}_1=𝔼\{\mathrm{l}\mathrm{n}(1+{({10}^2\cdot Q_{\mathrm{P}\mathrm{E}})}^2)\}$

The purpose of the total variation regularization term $R_{\mathrm{T}\mathrm{V}}$ is to penalize variations between model-estimated inter-element Euclidean distances $d$ and multiples of the element pitch $p$. This square difference is summed over the first $M=10$ lags. Here, the lag index $m$ enables the calculation of the distance between an element and another element $m$ neighbors away. The number of lags was empirically chosen and can be increased to de-emphasize aperture locality (i.e., increasing the lag number will increase the contribution of elements further away from each other in the calculation of regularization terms $R_{\mathrm{T}\mathrm{V}}$ and $R_{\mathrm{d}z/\mathrm{d}x}$). The expression for $R_{\mathrm{T}\mathrm{V}}$ is

$$R_{\mathrm{T}\mathrm{V}}({\overrightarrow{\mathit{r}}}_a)=\sum _{m=1}^M\sum _{n=1}^{N-m}{\left(d_{n,m}-m\cdot p\right)}^2,$$ (15)

where $d_{n,m}$ is the distance between element positions:

$$d_{n,m}={‖{\mathit{r}}_{n+m}-{\mathit{r}}_n‖}_2.$$

The purpose of the shape differential regularization term $R_{\mathrm{d}z/\mathrm{d}x}$ is to penalize the first derivative (i.e., the rate of change) of the $z$ components with respect to the $x$ components of the element positions in the aperture, effectively penalizing the local curvature of the estimated array shape. This is done by calculating the mean squared error difference in surface normals of each element over the first $M=10$ lags using the index $m$. The same number of lags is used in both $R_{\text{TV}}$ and $R_{\mathrm{d}z/\mathrm{d}x}$ to promote the same aperture locality. The expression for $R_{\mathrm{d}z/\mathrm{d}x}$ is

$$R_{\mathrm{d}z/\mathrm{d}x}({\overrightarrow{\mathit{r}}}_a)=\sum _{m=1}^M\frac{1}{N-m}\sum _{n=1}^{N-m}{\left({\theta }_{n+m}-{\theta }_n\right)}^2,$$ (16)

where ${\theta }_n$ is the finite difference in neighboring element surface normals given by Eq. 3.

III. Methods

A. Simulation

Nine unique transducer configurations were simulated to evaluate the performance of the proposed method: two convex, two concave, four sinusoidal, and one linear array. The transducer configurations are shown in Table II. For arrays 1–4 (convex and concave), the defined radius of curvature was used along with sine and cosine functions to parameterize the curved surface of the array. For arrays 5–8 (sinusoidal), the sinusoid frequency was determined such that the array size fits within one period. The sinusoid was laterally shifted using spatial-phase offsets to manipulate its concavity. Array 9 was initialized as a line in the lateral dimension. For each array geometry, the arc length of the parametric curve was defined as $\ell =(N-1)\cdot p$, where $N$ is the number of elements and $p$ is the pitch of the array. This variable arc length accommodates both simulation and experimental (e.g., the linear L12–3v and curvilinear C5–2) array geometries with varying numbers of elements and pitch such that a sinusoid, for example, completes a full cycle within the bounds of the array. The simulated array configurations utilized a center frequency of 7.5 MHz, 128 elements, and a pitch of 200.0 μm.

TABLE II.

Simulated Array Geometries

Array	Details
1	Convex, Radius: +50 mm
2	Concave, Radius: −50 mm
3	Convex, Radius: +10 mm
4	Concave, Radius: −10 mm
5	Sinusoid (1 cycle)
6	Sinusoid (1 cycle), +π phase shift
7	Sinusoid (1 cycle), +π/2 phase shift
8	Sinusoid (1 cycle), −π/2 phase shift
9	Linear

Ten in silico phantoms were produced and are shown in Fig. 3: one point grid target (point spacing 2.0 mm), one speckle phantom, one multi-target phantom with various echoic inclusions and point targets, one Stanford logo-based phantom, and six ImageNet-based speckle phantoms [29].

Fig. 3.

Resulting B-mode images for custom (first row) point grid, speckle, multi-target, Stanford logo-based, and (second row) ImageNet-based Field II target simulations. The red rectangular region represents the areas used to compute the quality metrics and B-mode reconstructions to prevent sampling outside Field II simulation boundaries.

Field II simulations of RF channel signals were produced using the calc_scat_multi function with an FSA transmit sequence to parallelize the simulations across a CPU cluster. A sampling frequency of 160 MHz and a center frequency of 7.5 MHz were used. The ImageNet-based in silico phantoms were created using a methodology similar to Hyun et al. [30] and Brickson et al. [31]. These natural images provide a diverse set of patterns, contrasts, and heterogeneity ideal for model validation before moving to experimental phantom and in vivo data. Each in silico phantom was defined to have a sound speed of 1540 m/s and no attenuation. First, the natural images were loaded into MATLAB, converted to grayscale, normalized to the range (0,1], and interpolated onto a grid. A scatter density of 10 scatterers per resolution cell was used, where a resolution cell was assumed to have a volume ${\lambda }^3$ [32], resulting in a scatterer density of 1,155 scatterers/mm³. Transmit events and channel data collection were time-synchronized such that the start time $t_0$ was identical for all transmit events, regardless of aperture shape. Field II simulations were produced for all 10 phantom layouts using each of the nine array geometries for a total of 90 simulations.

B. Additive Noise

The impact of white noise on shape estimation model performance was investigated using an additive noise model applied to simulation data. Here, white noise represents varying levels of thermal noise present in ultrasound imaging systems. Let ${\mathbf{\Psi }}_n\left(t\right)$ be the normalized signal representing the demodulated FSA data. Let $n\left(t\right)$ be the additive Gaussian white noise expressed as

$$n\left(t\right)=n_I\left(t\right)+j\cdot n_Q\left(t\right),$$ (17)

where $n_I\left(t\right)$ and $n_Q\left(t\right)$ are the in-phase and quadrature components, respectively, and are independent Gaussian random variables with zero means and variances ${\sigma }^2$ (i.e., $n_I\left(t\right)$, $n_Q\left(t\right)\sim 𝒩\left(0,{\sigma }^2\right)$). The noise, $n\left(t\right)$, is scaled by $\alpha ={\mathbf{\Psi }}_{\text{rms}}\cdot {10}^{(\eta /20)}$, where ${\mathbf{\Psi }}_{\text{rms}}$ is the root-mean-square of the signal (which is 1 after the normalization process described in Sec. III-D), and $\eta$ is the noise level in decibels (dB). The final expression for the signal with the additive noise is

$${\mathbf{\Psi }}_{\eta }(t)={\mathbf{\Psi }}_n(t)+\frac{1}{\sqrt{2}}n(t)\cdot \alpha .$$ (18)

While the shape estimation model configuration remained the same for each noise level, demodulated FSA data ${\mathbf{\Psi }}_{\eta }(t)$ with varying levels of noise ($\eta =[-\mathrm{\infty },0,6.0,12.0,18.0]$ dB) resulted in signal-to-noise ratios (SNR) of ${\mathrm{S}\mathrm{N}\mathrm{R}}_{{\mathbf{\Psi }}_{\eta }}=[\mathrm{\infty },0,-6.0,-12.0,-18.0]$ dB.

C. Experimental Data Acquisition

Experimental data was obtained from three sources using a Vantage 256 research ultrasound system (Verasonics Inc., Kirkland, WA, USA). First, RF channel data from an ATS 549 phantom (Sun Nuclear, Norfolk, VA) with a previously-calibrated ground truth sound speed of 1460 m/s [33] was acquired for this work using linear L12–3v and curvilinear C5–2v rigid-array transducers. Second, RF channel data from three in vivo rat livers using an L12–3v transducer was obtained from a previously acquired dataset [34]. Third, in vivo human liver data was acquired using a C5–2v array under Protocol IRB-56630 and informed consent was obtained. The simplified assumption of a homogeneous sound speed distribution of 1540 m/s was made for both in vivo datasets. In total, six samples were selected from the L12–3v datasets (three phantom and three in vivo rat liver), and six samples were selected from the C5–2v datasets (three phantom and three in vivo human liver). While the ground truth array shape is known for both transducers, the model can be initialized with any array geometry to emulate a flexible array and study the impact of unknown model initialization on the subsequent shape estimation capability and convergence of the model. Table III summarizes the different acquisition configurations for each setup. In all cases, an FSA transmission sequence was used. The RF channel data was demodulated and low-pass filtered using the bandwidth $f_d$.

TABLE III.

Summary of Experimental Data Acquisition

Trans.	Target	c (m/s)	fc (MHz)	fd (MHz)	Elem.*
L12–3v	ATS phantom	1460†	6.00	6.25	128
L12–3v	Rat liver	1537‡	7.81	7.81	128
C5–2v	ATS phantom	1460†	4.00	3.91	64
C5–2v	Human liver	1540§	4.00	3.91	64

^*The L12–3v and C5–2v arrays have 192 and 128 elements, respectively; only the center N elements were selected.

^†The ATS phantom sound speed was previously calibrated in [33].

^‡The rat liver sound speed was estimated by averaging ground truth values from corresponding specimens in [34].

^§The sound speed in human liver was assumed for image reconstruction.

For both the L12–3v phantom and rat liver images, only data from the center 128 elements were used to reduce the runtime and memory usage of the shape estimation model. For both the C5–2v phantom and human liver images, only data from the center 64 elements were used to avoid model errors due to poor contact between the outer elements and the target surface.

D. Model Implementation

The UltraFlex framework described in Sec. II, including the differentiable image reconstruction, quality metrics, and objective function, was implemented using JAX [19]. After simulation or experimental acquisition, the RF data is demodulated into baseband in-phase and quadrature (IQ) components, low-pass filtered, and decimated by a factor of eight (only in the case of simulation data). The resulting demodulated FSA data, array element positions ${\overrightarrow{\mathit{r}}}_a$, downsampled sampling frequency $f_s$, demodulation frequency $f_d$, and the initial time $t_0$ are loaded into the UltraFlex framework.

Once in the framework, the demodulated FSA data (IQ data) is first normalized by the maximum IQ magnitude so that all entries have a magnitude in the range [−1,1] and then normalized by the root mean square (RMS) magnitude to promote numerical floating-point stability. After this normalization process, the additive noise is introduced. For array shape estimation model optimization, the Adam optimizer [35] was used within the JAX-based Optax library. In the current framework configuration, we have chosen 1,000 optimization iterations for all quality metric-based models.

1). Array Initialization:

For simulations, the shape estimation model was initialized with a flat array geometry (Array 9) in all cases. For experimental (phantom and in vivo) datasets, the shape estimation model was initialized with each of the nine array geometries from Table II as a “reciprocal” to the simulation experiments. Initialization geometries that matched the ground truth geometry were included, as this was a useful test to observe the stability of the UltraFlex model. The same array initialization code structure was used for Field II simulations in MATLAB and the UltraFlex framework in Python to minimize numerical differences between programming languages.

2). Reconstruction Grid Implementation:

For image reconstruction, a delay-and-sum beamformer (Eq. 5) is used; however, different reconstruction grids are used for imaging and quality metric calculation. The quality metrics described previously are evaluated on patches within the target domain. Each patch is centered at a specific 2D position ${\mathit{r}}_p$ and includes a local 3 × 3 (lateral by axial) kernel of relative offsets ${\overrightarrow{\mathit{\delta }}}_k$ spanning the range $[-\lambda /2,+\lambda /2]$. The set of absolute positions corresponding to one patch is given by ${\overrightarrow{\mathit{r}}}_{\text{patch}}={\mathit{r}}_p+{\overrightarrow{\mathit{\delta }}}_k$, where ${\overrightarrow{\mathit{r}}}_{\text{patch}}\subset {\overrightarrow{\mathit{r}}}_{\text{patches}}\in \mathrm{\Omega }$. The full collection of patch-sampled positions across the domain is denoted by ${\overrightarrow{\mathit{r}}}_{\text{patches}}$, and is distinct from the higher-resolution imaging grid ${\overrightarrow{\mathit{r}}}_{\text{ing}}\in \mathrm{\Omega }$ used for B-mode reconstruction (grid spacing of $\lambda /3$). Pseudo-code for initializing the B-mode imaging and patch-level grids is provided in Alg. 1. An 18 × 21 (lateral by axial) grid of patch centers ${\overrightarrow{\mathit{r}}}_p$ was used, with spacings of 1.67 mm × 1.90 mm for regular field-of-view apertures (simulations and L12–3v) and 11.11 mm × 2.48 mm for larger apertures (C5–2v). For metrics employing synthetic subapertures in common-midpoint analysis, a subaperture size of 17 elements was used.

Algorithm 1.

Reconstruction Grid Definitions

1:	Define image and patch limits (mm):
2:	if array is “C5–2v” then
3:	${\mathit{x}}_{\text{img\_range}}\in [-100,100]$, ${\mathit{z}}_{\text{img\_range}}\in [0,140]$
4:	${\mathit{x}}_{\text{patch\_range}}\in [-100,100]$, ${\mathit{z}}_{\text{patch\_range}}\in [20,72]$
5:	else
6:	${\mathit{x}}_{\text{img\_range}}\in [-15,15]$, ${\mathit{z}}_{\text{img\_range}}\in [0,40]$
7:	${\mathit{x}}_{\text{patch\_range}}\in [-15,15]$, ${\mathit{z}}_{\text{patch\_range}}\in [2,42]$
8:	end if
9:	Compute B-mode image reconstruction grid:
10:	${\mathit{x}}_{\text{img}}\leftarrow$ Linspace in ${\mathit{x}}_{\text{img\_range}}$ with spacing $\lambda /3$
11:	${\mathit{z}}_{\text{img}}\leftarrow$ Linspace in ${\mathit{z}}_{\text{img\_range}}$ with spacing $\lambda /3$
12:	${\overrightarrow{\mathit{r}}}_{\text{img}}\leftarrow$ Cartesian product of ${\mathit{x}}_{\text{img}}$ and ${\mathit{z}}_{\text{img}}$
13:	Compute patch center grid:
14:	${\mathit{x}}_{\text{p}}\leftarrow$ Linspace in ${\mathit{x}}_{\text{patch\_range}}$ with 18 points
15:	${\mathit{z}}_{\mathrm{p}}\leftarrow$ Linspace in ${\mathit{z}}_{\text{patch\_range}}$ with 21 points
16:	${\overrightarrow{\mathit{r}}}_{\mathrm{p}}\leftarrow$ Cartesian product of ${\mathit{x}}_{\mathrm{p}}$ and ${\mathit{z}}_{\mathrm{p}}$
17:	Define local kernel offsets:
18:	${\mathit{\delta }}_k\leftarrow$ Cartesian product of lateral and axial linspaces in [$-\lambda /2$, $\lambda /2$]
19:	Compute full patch sampling grid:
20:	${\overrightarrow{\mathit{r}}}_{\text{patches}}\leftarrow {\overrightarrow{\mathit{r}}}_p+{\overrightarrow{\mathit{\delta }}}_k$ (vector sum of patch centers and kernel)

3). Regularization:

The impact of the two regularization terms was examined by a 2D parametric sweep of term weightings, where $w_2=\left[{10}^{-4},{10}^9\right]$ and $w_3=\left[{10}^{-9},{10}^1\right]$ in steps of powers of ten. The evaluation was performed across the speckle simulations produced using the nine array geometries to determine the ideal weightings for each regularization term of Eq. 14. The model was initialized with the linear (flat) array (Array 9) for each speckle simulation dataset.

E. Model Evaluation

1). Model Evaluation Criterion:

The mean Euclidean error (MEE) $ℰ$ between the estimated element positions and ground truth element positions was used as the main evaluation criterion across all experiments and is defined as

$$\begin{gathered} {ℰ}^{(i)}=\frac{1}{N_a}\sum _{a=1}^{N_a}{‖{\mathit{r}}_a^{(i)}-{\mathit{r}}_a^{(\mathrm{G}\mathrm{T})}‖}_2 \\ =\frac{1}{N_a}\sum _{a=1}^{N_a}\sqrt{{\left(x_a^{(i)}-x_a^{(\mathrm{G}\mathrm{T})}\right)}^2+{\left(z_a^{(i)}-z_a^{(\mathrm{G}\mathrm{T})}\right)}^2} \end{gathered}$$ (19)

where $N_a$ is the total number of elements in the aperture, ${\mathit{r}}_a^{(i)}$ represents a model-estimated element position at iteration $i$, and ${\mathit{r}}_a^{(\text{GT})}$ represents the corresponding ground truth element position. MEE is used to evaluate model performance for different quality metrics and SNRs.

2). Model Convergence Validation:

The convergence of the iterative model over $N_i=\mathrm{1,000}$ iterations is examined using the following stability error criterion: During the last 100 iterations, the mean displacement of the element positions must be less than or equal to 100 μm. The mean magnitude of the displacement of element positions over the last 100 iterations and all $N_a$ elements is

$$\stackrel{\mathrm{‾}}{\mathrm{\Delta }}=\frac{1}{N_i{N}_a}\sum _{i=N_i-100}^{N_i-1}\sum _{a=1}^{N_a}{‖{\mathit{r}}_a^{(i+1)}-{\mathit{r}}_a^{(i)}‖}_2$$ (20)

given iteration $i$ and element index $a$. Notably, this convergence validation is independent of the true element positions.

3). Speed of Sound Error:

Using all simulated array geometries and the speckle target, the beamforming speed of sound was varied from 1440 m/s to 1640 m/s to assess the effect of sound speed error on model performance.

4). Computational Runtime:

Model development was explored on an NVIDIA RTX 3090 with 24 GB of VRAM (NVIDIA, Santa Clara, CA) and then deployed on a SLURM cluster comprised of NVIDIA RTX A6000 GPUs with 48 GB of VRAM. Finally, to demonstrate the computational accessibility of this work, model runtime results for an NVIDIA RTX 3060 with 12 GB of VRAM are shown. In all cases, JAX was used with jaxlib built for CUDA 12.2.

IV. Results

A. Simulation

1). Regularization:

The parametric sweep results of regularization weights are visualized in Fig. 4. For the particular model configuration described in the caption of Fig. 4, shape-differential regularization ($R_{\mathrm{d}z/\mathrm{d}x}$) had no impact on MEE results for $w_3<{10}^{-1}$. At this weighting and above ($w_3\ge {10}^{-1}$), shape differential regularization adversely affected MEE results. Additionally, TV regularization ($R_{\text{TV}}$) did not affect the MEE results for $w_2<{10}^3$, followed by an improvement in estimation on weighting range $w_2=\left[{10}^3,{10}^6\right)$. For $w_2\ge {10}^6$, MEE results for Arrays 5–8 (i.e., sinusoidal geometries) were adversely affected, MEE results for Arrays 1–4 (convex and concave geometries were either adversely effected or exhibited little change, and MEE results for Array 9 (linear array) did not change. Based on these results, the remaining experiments presented in this paper utilize a model configuration with TV regularization only and a term weighting of $w_2={10}^4$.

Fig. 4.

Comparison of regularization method weightings for each of the nine array geometries. The shape estimation model was initialized with the linear (flat) array (Array 9) for a speckle target (top row, second column, Fig. 3) while using the filtered common-midpoint phase-error model. No additive noise was applied (i.e., ${\mathrm{S}\mathrm{N}\mathrm{R}}_{{\mathrm{\Psi }}_{\eta }}=\mathrm{\infty }$).

2). Metric Comparison:

In Fig. 5, average MEE across the simulated data for various metric-based objectives are plotted against model iteration. The envelope entropy metric results in the worst shape estimation model performance, characterized by the greatest final average MEE. Coherence factor and speckle brightness exhibit similar model performance with final average MEEs of 318 μm and 236 μm, respectively. The correlation coefficient, lag-one coherence, and phase-error metric-based models result in the best shape estimation performance, with final average MEEs ≤ 54 μm.

Fig. 5.

Average MEE vs. model iteration results for various metric-based objective functions. For each quality metric, the MEE results are averaged across the nine array configurations and the four simulation targets (i.e., the top row of Fig. 3). The model was initialized with the linear (flat) array geometry, and a regularization weighting of ${\mathit{w}}_2={10}^4$ was used in all cases. No additive noise was applied.

An example of B-mode reconstructions at various iterations using the filtered phase-error metric-based model is shown in Fig. 6. Between iterations 70 and 130, image structures come into focus and shape estimation becomes more accurate. Between iterations 230 and 1,000, smaller improvements in shape estimation occur. Improvements in image focusing and quality are harder to identify visually. Very small improvements may be observed between iteration 230 and higher, although almost no visually detectable changes are observed after iteration 250 despite the reduction in MEE. Additionally, MEE decreases from 62.8 μm to 19.3 μm.

Fig. 6.

Comparison of focusing at various iterations. A pink curve represents the ground-truth array shape, and the overlaid curve represents the model-estimated array shape with a variable color map from green to red representing lower and higher element-wise Euclidean errors, respectively. The array-wise MEE, in μm, is displayed above each array.

Fig. 7 exhibits B-mode focusing across different quality metrics for selected simulations. A supplementary video, simulation.mp4, corresponds to Fig. 7. Compared with the quantitative results averaged across all nine array geometries shown in Fig. 5, the results shown in Fig. 7 correspond to Array 5 only. The envelope entropy metric-based model fails to estimate the array shape for all of the phantoms. The correlation coefficient and phase-error metric-based models achieved the best focusing of the point grid phantom target, evident in the sharpness of the point-spread function on and near the point targets.

Fig. 7.

B-mode images based on model convergence using different image quality metrics. Each column represents a different image quality metric used in the objective function of the iterative model. The columns are organized from left to right based on the ascending order of the final average MEEs presented in Fig. 5. From top to bottom, a point grid, speckle, multi-target, and Stanford logo-based phantom are visualized in each row. A pink curve represents the ground-truth array shape, and the overlaid curve represents the model-estimated array shape with a variable color map from green to red representing lower and higher element-wise Euclidean errors, respectively. The array MEE, in μm, is displayed above each array.

3). Additive Noise:

Fig. 8 exhibits the impact of additive white noise on the final-iteration MEE model performance across 270 simulation configurations (including six ImageNet-based simulations, nine array geometries, and five noise values). Before plotting, the model results were validated by the procedure described in Sec. III-E.2, and only the results that pass validation are included. The first two columns of Table IV summarize the number of invalid model results.

Fig. 8.

Quantitative comparison of final-iteration shape estimation model performance across different image quality metrics and ${\mathrm{S}\mathrm{N}\mathrm{R}}_{{\mathrm{\Psi }}_{\eta }}=[\mathbf{\infty },0,-6.0,-12.0,-18.0]$ dB. Each boxplot (a,c) and partial boxplot (b,d) contains all validated results averaged across the nine array geometries and the six ImageNet-based speckle phantoms. The initial average MEE when using the linear array (Array 9) for model initialization is shown as a dotted red line. (a) A boxplot of model results for each quality metric is shown without model regularization, representing the first group in the (b) partial boxplot (median and IRQ) comparison of different ${\mathrm{S}\mathrm{N}\mathrm{R}}_{{\mathbf{\Psi }}_{\mathit{\eta }}}$ values. (c) A boxplot of model results for each quality metric is shown with model regularization $\left({\mathit{w}}_2={10}^4\right)$, representing the first group in the (d) partial boxplot comparison of various ${\mathrm{S}\mathrm{N}\mathrm{R}}_{{\mathbf{\Psi }}_{\mathit{\eta }}}$ values.

TABLE IV.

Model Validation Results Based on Sec. III-E.2

Quality Metric	Invalid Sim. Results (Fig. 8)		Invalid Exp. Results (Fig. 10)
	No Reg.	Reg.	ATS	Liver
Envelope Entropy	146/270	0/270	6/54	25/54
Coherence Factor	7/270	0/270	0/54	0/54
Speckle Brightness	9/270	0/270	0/54	0/54
CMCC	2/270	0/270	1/54	0/54
Lag-One Coherence	58/270	3/270	0/54	0/54
CMPE (Filtered)	6/270	0/270	0/54	0/54
CMPE	2/270	1/270	0/54	0/54

Starting with the case of no model regularization, Fig. 8a shows boxplots of the MEE for the final iteration, arranged in descending order of median value from left to right. Notably, the boxplot includes model results above the initial average MEE. This means that in some cases, the shape estimation model produced MEEs that were further from the ground truth than if no shape estimation model had been used. In Fig. 8b, a partial boxplot is shown for the median and IQR of each metric group. The same metric order is maintained for all SNR groupings established in Fig. 8a. As the additive noise level increases, all median MEEs monotonically increase with the exception of envelope entropy and lag-one coherence. The lagone coherence metric-based model produces no valid results at ${\mathrm{S}\mathrm{N}\mathrm{R}}_{{\mathbf{\Psi }}_{\eta }}=-18.0\mathrm{d}\mathrm{B}$.

Fig. 8c shows the same information as Fig. 8a but with model regularization $\left(w_2={10}^4\right)$. With regularization, the lagone coherence, correlation coefficient, and phase-error metrics have a much smaller IQR compared to the other metrics. The coherence factor results have the largest IQR despite having the fourth lowest median value.

B. Experimental

The proposed shape estimation models were validated using experimental phantom and in vivo datasets. For Fig. 9a–b, the first three columns show various shape initialization approaches for data from the calibrated ATS 549 phantom. Columns 4–6 show various shape initialization approaches for in vivo liver data from a rat (L12–3v only) and a human (C5–2v only). In each case, the filtered phase-error metric-based model was used. The model converges to the “correct” image in all cases despite sound-speed assumption errors and different initialization configurations. Supplementary videos experimental_linear.mp4 and experimental_curvilinear.mp4 show the convergence process corresponding to Fig. 9a and Fig. 9b, respectively. However, for the ground-truth-initialized linear and curvilinear examples, the post-convergence shapes contain non-zero MEEs.

Fig. 9.

B-mode images using the UltraFlex framework and filtered phase-error metric for (a) L12–3v and (b) C5–2v transducer data. Three array shape configurations are used to initialize the shape estimation model for both phantom and in vivo data: Array 1 (curvilinear; first column), Array 6 (sinusoidal; second column), and Array 9 (linear; third column). In both panels (a) and (b), the top row depicts the model initialization and resulting B-mode, whereas the bottom row depicts the final-iteration shape estimation and resulting B-mode. A pink curve represents the ground-truth array shape, and the overlaid curve represents the model-estimated array shape with a variable color map from green to red representing lower and higher element-wise Euclidean errors, respectively. The array MEE, in μm, is displayed above each array.

In all results shown in Fig. 9, the qualitative focusing of the resulting B-modes improves. For the L12–3v phantom data results, speckle becomes brighter and all cyst targets become visible. Furthermore, for the L12–3v in vivo rat liver data, the muscle fascia between the depths of 1 and 5 mm straightens and becomes brighter, the specular tissue at $x=-5.0\mathrm{m}\mathrm{m}$, $z=16.0\mathrm{m}\mathrm{m}$ becomes brighter, and features below 20 mm increase in visual clarity. In the phantom and in vivo results, the speed of sound of the medium may be different than the assumed sound speed for beamforming. An analysis of beamforming sound speed error showed that MEE increased at a rate of 4.1 μm per 1 m/s error.

Fig. 10 shows a quantitative comparison of experimental results from different metric-based models. Again, before plotting, the model results were validated by the procedure described in Sec. III-E.2, and only the results that pass validation are included. The last two columns of Table IV summarize the number of invalid model results for the experimental data across the 54 phantom and 54 in vivo configurations (two transducers, nine initialization shapes, and three different images) for each quality metric.

Fig. 10.

Quantitative comparison of MEE across image quality metrics. Results are grouped by (a) ATS phantom and (b) in vivo liver data. For each grouping, three L12–3v datasets and three C5–2v datasets are utilized. The model is initialized with each of the nine array geometries using the indicated beamforming sound speed. Thus, each boxplot consists of nearly 54 results (depending on the criterion validation process).

In the idealized homogeneous phantom results, most of the metric-based models (except speckle brightness and envelope entropy) have similar final MEEs, which are larger than the noise-free simulation results. In vivo results yielded good performance with increased error compared to the simulation and phantom results.

C. Computational Runtime

Table V provides a runtime comparison (1,000 iterations) between the various metric-based models and NVIDIA RTX GPUs. Each entry in the table includes the average runtime of one simulation, one L12–3v, and one C5–2v result.

TABLE V.

UltraFlex Model Runtime

Quality Metric	Nvidia RTX GPU
	3060 (s)	A6000 (s)	3090 (s)
Envelope Entropy	316 ± 11	232 ± 19	147 ± 5
Coherence Factor	331 ± 23	223 ± 23	152 ± 5
Speckle Brightness	316 ± 12	231 ± 19	146 ± 4
Correlation Coefficient	344 ± 34	231 ± 20	155 ± 6
Lag-One Coherence	342 ± 29	227 ± 21	156 ± 5
Phase Error (Filtered)	346 ± 34	231 ± 20	156 ± 7
Phase Error	345 ± 34	231 ± 19	156 ± 6

V. Discussion

The main objective of this work was to introduce UltraFlex, an iterative optimization framework for array shape estimation, and to explore optimizable models based on various quality metrics. Recent flexible-array shape calibration methods in the literature achieve an average MEE in position as low as 40 μm [18] for noise-free simulation data and as high as 1,100 μm [16] for in vivo data. With the proposed method, a median MEE as low as 3.7 μm was achieved for noise-free simulations while using the phase-error quality metric. For intuitive understanding of these errors, visual inspection of the images in Fig. 6 and 7 shows that errors in the vicinity of 40 μm MEE and below do not appear to produce visibly detectable changes in image quality. For example, in Fig. 7, the point targets and the borders of the “S” in the Stanford logo appear nearly identical to the ground truth for the speckle brightness images, which have MEEs of 41.2 μm and 40.1 μm for the point target and Stanford logo images, respectively. Visibly apparent degradations begin to appear at 55.5 μm MEE in Fig. 7 (e.g., the vertical borders of the Stanford logo in the lag-one coherence image) and at 62.8 μm MEE in Fig. 6. Note that these errors are under the assumption of a correct and constant speed of sound. Also, because the optimization process is based on stochastic gradient descent, MEE values have small variations in each run.

A. Effect of Sound Speed Errors

Fig. 9a shows all the results for the L12–3v linear array, both phantom and in vivo rat liver. For the phantom results, the final shape estimations have a small symmetric convex bias. However, in the case of the in vivo data, the final shape estimations have a small asymmetric convex bias. These errors are likely the result of the lens sound speed and an incorrect sound speed for beamforming in the model. Given an error of 4.1 μm/m/s, the convex bias is consistent with a 24 m/s error due to the lens and beamforming speed of sound. For in vivo cases, an asymmetric convex bias was observed, indicating tissue heterogeneity, lens sound speed, and beamforming sound speed affected the estimated shape. Because both sound speed errors and element position errors result in phase error during reconstruction, the shape estimation model incorporates the sound speed error into the shape, similar to the way a near-field phase-screen model is used for aberration correction.

Fig. 9b shows all the results for the C5–2v convex array, both phantom and in vivo human liver. A similar deviation from the ground truth array shape can be observed in the phantom results. In addition to sound speed error, experimental data has noise, so the error is a combination of both sound speed error and SNR. For in vivo cases, there is also potentially diffuse reverberation noise that will contribute to increased error. Although the in vivo shape estimations show relatively large errors compared to simulation results, the image quality remains high because the model is compensating for both the shape and the sound speed error with a phase screen. Thus, the MEE shown in Fig. 9 cannot be directly compared to the MEE of the simulations (Fig. 7) because some of the MEE is due to corrections for sound speed errors appearing as position errors. However, in many applications of flexible arrays, the higher-quality image may be preferable to knowing the exact position of the array elements.

B. Comparison of Quality Metrics

The performance of the speckle brightness-based model on experimental data is significantly worse compared to its performance in the simulation and additive noise studies. This is likely due to the metric reductions in Table I being empirically tuned for simulation datasets. Thus, the tuning for speckle brightness may not be best suited for phantom and in vivo datasets because speckle brightness is inherently sensitive to amplitude distribution variations. This is a fundamental limitation of speckle brightness and entropy: optimization objective metrics with fewer local minimums or maximums will lead to better final performance than speckle brightness and envelope entropy because of their lack of sensitivity to the sampling of the amplitude distribution.

The performance of the envelope entropy-based model across all experiments in this work was substantially worse than previous reports in the literature [7], [18]. This is likely due to differences in the implementation of envelope entropy and region-of-interest (ROI) windowing. In Noda et al. [18], model evaluation was performed on very small aperture sizes of only 20 elements, ranging between 4 and 16 mm in length, which is a favorable condition for good convergence because the initialization errors are small. In this work, the larger aperture sizes inherently introduced larger initialization errors, leading to envelope entropy having the worst convergence performance in terms of the number of invalid results. Furthermore, this work used a fixed grid to compute the quality metric at target domain positions ${\overrightarrow{\mathit{r}}}_{\text{patches}}$. The spacing between the coordinates on this grid was greater than or equal to 2 mm. Therefore, localized entropy variations were not considered. For envelope entropy results presented in Fig. 8c and Fig. 8d, some of the final average MEEs were greater than the initial average MEE. These images appear to have better qualitative or visual focusing than expected, but this is usually exhibited as rotated image features that appear in focus (e.g., the coherence factor result for the Stanford logo shown in Fig. 7). This optimization state is due to the optimizer converging to a local minimum.

Overall, the correlation coefficient and phase-error metric-based models (regular and filtered) had the best performance compared to the other models, and are therefore the preferred quality metrics. In Fig. 5, the correlation coefficient and phase-error models were the only results that had their average MEE vs. iteration curves intersect (around iteration 50) all other average MEE vs. iteration curves. This suggests that in a stepwise comparison, the correlation coefficient and phase-error-based objectives could more efficiently optimize the array shape on the iteration range [50, 150] than the other objectives, despite having the same or lower learning rate. While lag-one coherence was the most efficient on the iteration range [0, 50], this is likely due to the higher learning rate used in the lag-one coherence-based model, suggesting an opportunity to further adjust learning rates to improve convergence. The sudden decrease in average MEE for all metrics (except lag-one coherence) around iteration 50 is due to the chosen optimization configuration (i.e., using the Adam optimizer with a specific learning rate and the Optax-default exponential decay rates for the first and second moments). All metrics except lag-one coherence, which has a higher learning rate (see Table I), share the same optimization configuration.

Different simulation targets presented unique focusing results across the various quality metrics. The point target grid sometimes had the best and the worst reconstruction compared to the other phantom targets in Fig. 7. The periodic structure of the point target grid and points in the Stanford logo presented a greater number of local minima in the objective function, thereby hampering performance. This is also potentially why there is a difference between the average MEE results shown in Fig. 5 and the median MEE results shown in Fig. 8a and Fig. 8c. While these imaging targets may not be representative of real tissue, they are useful for assessing image quality.

C. Influence of Array Geometry and Regularization

Rigid-array transducers were chosen to provide a known ground truth for performance evaluation. Data from the C5–2v array was limited to the central 64 elements due to poor element contact between the convex array and the flat phantom surface; it was not due to model failure with larger arrays.

Regarding array shape regularization, our goal was to select regularization weightings that provided reasonable performance for a flat array initialization and the filtered phase-error metric, which may have resulted in non-optimal weightings for all initialization shapes and quality metrics. While we anticipate that the framework can accommodate very small bending radii (i.e., highly deformed shapes), most body surfaces of interest will not have such aggressive contours. This was the motivation for introducing the shape differential regularization term $R_{\mathrm{d}z/\mathrm{d}x}$. In our experiments, we used mild-to-moderate array deformations, with a peak axial deviation of approximately 2 mm (Arrays 5–8), resulting in initialization MEEs between 1,250 μm and 1,300 μm when compared to the flat geometry (Array 9). Increasing the amount of deformation or the bending radius would likely increase the initialization error, which could lead to larger final MEE values, and would likely require re-evaluation of the $R_{\mathrm{d}z/\mathrm{d}x}$ term weight. Furthermore, with the assumption that the pitch of the array is fixed during fabrication (while flexibility is still enabled), we introduced the total variation regularization term $R_{\text{TV}}$.

D. Resource Usage

Regarding the computational runtime of the current UltraFlex framework implementation, each model currently takes less than 2.6 minutes to achieve convergence, but a significant amount of this time is for B-mode reconstruction, which was used to visually observe model performance over itrations. It is likely that implementation in practice would not require the B-mode reconstruction step. Three potential approaches can significantly improve runtime. The first approach is to apply one or more of the following: reduce the number of iterations, further tune the optimizer, or modify the reduction expressions presented in Table I. The second approach is to investigate alternative auto-differentiation frameworks, directly implement UltraFlex in a compiled language, or acquire different GPU hardware that is faster or has more VRAM, or any combination of these. The third approach involves increasing the patch spacing ${\overrightarrow{\mathit{r}}}_{\text{patches}}$, which would decrease model runtime, but it would also worsen quality assessment. Outside of these three approaches for decreasing runtime during each iteration, another method may involve using UltraFlex for only an initial shape calibration; for some applications, the body surfaces do not change significantly over time.

Computational memory usage was an important consideration. The simulation, experimental L12–3v, and experimental C5–2v datasets presented in this work utilized 128, 128, and 64 elements, respectively. Generally, more elements require more memory or a longer model runtime; a similar tradeoff can be found in increasing reconstruction grid resolution.

VI. Conclusion

UltraFlex, an iterative model-based ultrasonic flexible-array shape calibration framework based on automatic differentiation, was presented. Model performance was quantitatively evaluated while examining multiple image quality metrics: envelope entropy, speckle brightness, lag-one coherence, coherence factor, common-midpoint correlation coefficient, and common-midpoint phase error. These image quality metrics were evaluated on simulated phantoms using a variety of array shapes. Experimental phantom and in vivo liver datasets were also investigated using transducers with known geometries. Speckle brightness, envelope entropy, and coherence factor enabled model convergence under many conditions. Lagone coherence, common-midpoint correlation coefficient, and common-midpoint phase error enabled more accurate element position estimations and improved visual ultrasound image focusing quality. Results indicate that common-midpoint correlation coefficient and phase-error quality metrics were the most robust against additive white noise while achieving median MEEs of 3.7 μm for simulation, 29.7 μm for phantom, and 69.0 μm for in vivo liver data. These array shape calibration results show promise for future development of experimental flexible- and wearable-ultrasonic arrays.

Highlights.

• UltraFlex is a model-based flexible-array shape calibration framework that uses iterative optimization updates through automatic differentiation to enable lower position estimation error compared to previous models.

• Model performance is evaluated using envelope entropy, speckle brightness, lag-one coherence, coherence factor, correlation coefficient, and phase-error focusing quality metrics.

• These iterative model results demonstrate promise for future hardware development of flexible- and wearable-ultrasonic arrays that utilize ultrasound-autofocusing software.