Quantitative Viscoelastic Response (QVisR): Direct Estimation of Viscoelasticity with Neural Networks

Abstract

We present a machine learning method to directly estimate viscoelastic moduli from displacement time-series profiles generated by Viscoelastic Response (VisR) ultrasound excitations. VisR uses two colocalized acoustic radiation force pushes to approximate tissue viscoelastic creep response and tracks displacements on-axis to measure the material relaxation. A fully-connected neural network is trained to learn a nonlinear mapping from VisR displacements, the push focal depth, and the measurement axial depth to the material elastic and viscous moduli. In this work, we assess the validity of Quantitative Viscoelastic Response (QVisR) in simulated materials, propose a method of domain adaption to phantom VisR displacements, and show in vivo estimates from a clinically acquired dataset.

I. Introduction

TISSUE characterization is studied to identify measurable properties that can be correlated to clinical outcomes [1]. Considerable effort has been spent developing quantitative, as opposed to qualitative and semi-quantitative, metrics of characterization because they are better suited for cross-sectional and longitudinal clinical studies [2], [3]. Tissue viscoelasticity is a quantitative parameter with an intuitive link to clinical outcomes since many pathologies present measurable changes in mechanical property (e.g., fibrosis, inflammation, etc.) [4]–[6].

The subfield of ultrasound elastography attempts to measure changes in tissue viscoelasticity. The majority of elastography methods involve tracking the physical displacement of speckle-generating scatterers constituting the interrogated tissue. The magnitude and/or rate of displacement is then associated to mechanical property through model-based or data driven approaches [7], [8]. Physical displacement of tissue is generated through external indentation or vibration, passive stresses such as breathing, or application of acoustic radiation force (ARF). Reduction to clinical practice can be challenging for techniques that use external indentation or vibration due to the need for extra equipment and/or non-standardized application of stress [9]. Passive stress methods generally require boundary condition measurements or mathematical models that are often unknown for in vivo imaging contexts [10].

Most elastography techniques using ARF pushes track shear wave propagation off-axis laterally of the applied ARF push. These methods typically relate shear wave measurements (phase or group velocity, dispersion, etc.) to tissue mechanical property [7], [11]–[13]. Shear wave based methods generally assume mechanical property homogeneity over the wave propagation region. This assumption reduces the resolution of shear wave based methods compared to on-axis methods and can result in spatial averaging of mechanical property estimation if heterogeneities exist. Shear waves are also prone to reflection off out-of-plane targets, such as bone or connective tissue, which can distort the measured wave properties. Additionally, shear wave based methods are depth limited since the ARF pushes need to generate enough displacement to support shear wave propagation over a measurable distance [7], [11].

Our lab developed Viscoelastic Response (VisR) ultrasound to evaluate the viscoelastic properties of tissue by approximating a creep response [14]. VisR uses a colocalized, double-push ARF excitation to minimize acquisition time when compared with similar methods (KAVE [15], MSSR [16]). Additionally, VisR tracks the induced displacements on-axis with the ARF excitations to avoid the aforementioned challenges of methods that monitor shear-wave propagation. VisR displacements can be fit to a 1D mass-spring-damper model with fit parameters rearranged to derive semi-quantitative measures of elasticity and viscosity (RE = Relative Elasticity and RV = Relative Viscosity) [17] relative to the applied force amplitude, which can be assumed constant over a small, local region [11], [18]. Additional corrections have been studied to improve comparisons of RE and RV over a wider region of interest in an image [19]–[21]; however, all VisR sequence derived viscoelasticity measures have been semi-quantitative.

Model-based elastography approaches become more ill-posed when a model parameter or boundary condition is unknown/immeasurable, such as the applied force amplitude for VisR. Data driven statistical approaches, such as neural networks, often require large sets of labeled data for training that, ideally, represent all data variations present in the final test application of the network. Although experimental and clinical elastography datasets are usually small, large simulation datasets can be generated to capture variations in displacement profiles due to imaging configuration (focal depth, transducer parameters, etc.) and tissue property (viscoelasticity, attenuation, etc.).

In this study, we regress multi-layer perceptron neural networks on simulated, ultrasonically tracked VisR displacement time series to jointly estimate elastic and viscous moduli. The predicted moduli are direct estimates of the underlying mechanical properties coining the term Quantitative Viscoelastic Response (QVisR) ultrasound. This work is an extension of methods, results, and discussion presented in [22], [23], and [24]. In [22], we evaluated bagged trees regressors models against neural network architectures on mechanically homogeneous simulations without noise and concluded that neural networks provide lower test set error. In [23], we varied the added noise levels in homogeneous materials to test QVisR performance degradation. In [24], we trained QVisR on both homogeneous and heterogeneous simulations with noise and evaluated QVisR generalizability by estimating the elasticity scans acquired in a CIRS elastic tissue-mimicking phantom. This manuscript includes extended analysis of QVisR results in homogeneous and heterogeneous simulations with noise, provides metrics related to model evaluation in CIRS phantom, and shows both simulation-trained and phantom fine-tuned QVisR estimates in the muscles of boys with Duchenne muscular dystrophy not present in any previous work. We also provide comparisons of QVisR with conventional VisR RE and RV semi-quantitative images in the phantom and clinical examples to show the benefits of quantitative viscoelasticity estimation.

II. Methods

A. Simulations

Mechanically homogeneous and heterogeneous viscoelastic materials interrogated through a VisR beam sequence were simulated using methods adapted from Palmeri et al [25]. Fixed ARF excitation and tracking parameters across all simulations are listed in Table I. Major material and finite element parameters shared by all simulations and unique to the homogeneous and heterogeneous simulations are shown in Table II. For each set of simulations, on-axis displacement through time data was paired with the respective axial depth, focal depth, elasticity, and viscosity of the simulation and then saved for further processing. Viscoelastic material properties were chosen to represent realistic ranges found in soft tissues [4], [5] for all simulations.

TABLE I.

ARF Excitation and Tracking Parameters for Simulations and Phantom Acquisitions

Name	Value
Beam Sequence Parameters
Transducer	VF7-3
Elevational Lens Focus (mm)	37.5
Sampling Frequency (MHz)	40
ARF Center Frequency (MHz)	4.21
ARF Duration (μs, cycles)	70, 300
ARF F/#	3.0
Tracking Center Frequency (MHz)	6.15
Tracking Tx F/#	3.0
Tracking Rx F/#*	0.75
Tracking Tx Focus (mm)	20
Tracking PRF (kHz)	10
Normalized Cross Correlation Parameters
Kernel Length (wavelengths, μm)	3, 751
Search Window (μm)	50

^*Aperture growth and dynamic Rx focusing enabled

TABLE II.

Simulation Parameters

Name	Value
All Simulations
Material Model	Kelvin-Maxwell
Poisson’s Ratio	0.499
Speed of Sound (m/s)	1540
Acoustic Attenuation (dB/cm/MHz)	1.0
Density (kg/m3)	1.0
Homogeneous Simulations Only
Elasticity (kPa)	5–100 step 10.56
Viscosity (Pa.s)	0.01–7.01 step 0.78
Focal Depth (mm)	15, 17, 20, 23, 25, 27, 30, 32, 35, 38, 40
Heterogeneous Simulations Only
Inclusion Elasticity (kPa)	15.56, 22, 28, 36.67
Inclusion Viscosity (Pa.s)	0.01, 1.57, 2.6, 3.9
Background Elasticity (kPa)	26.11
Background Viscosity (Pa.s)	2.34
Focal Depth (mm)	15–35 step 5
Inclusion Diameter (mm)	10
Inclusion Depth (mm)	25
Inclusion Lateral Offset (mm)	0, 2.5, 4, 7

1). Homogeneous Dataset:

A Siemens VF7-3 transducer (Siemens Healthineers, Issaquah, WA) was parameterized to be modeled with Field II [26]. Transducer point spread functions (PSFs) for ARF excitations and tracks were modeled using the beam sequence parameters in Table I and material properties in Table II. ARF excitation PSFs were calculated for each of the 11 focal depths enumerated in Table II.

A rectangular finite element mesh was defined using LS-DYNA (Ansys Inc., Canonsburg, PA) with 280,440 elements of volume 250 μm³ spanning a 1×1×4 cm (lateral x elevation x axial) region with axial quarter symmetry. Simulated ARF PSF intensities were converted to body forces and mapped to nodes on the mesh. LS-DYNA material files were created on a grid of 10 elasticities (5–100 kPa in 10.56 kPa steps) and 10 viscosities (0.01–7.01 Pa.s in 0.78 Pa.s steps) for a total of 100 material combinations. Each material was defined with the Kelvin-Maxwell Viscoelastic material model, a Poisson’s Ratio of 0.499, and an equivalent elastic perfectly matched layer (PML). Finite element simulations were performed for every material combination and focal depth. Each simulation applied nodal forces with a VisR beam sequence and evaluated the resulting nodal displacements through time by solving the dynamic equations of motion between nodes. The simulations were carried out over 4.36 ms with a control time step of 1 μs, a sampling period of 25 μs, and two successive 70μs (300 cycle) ARF pushes applied at 5 μs and 442 μs respectively. Finite element hourglassing control type (ihq) of 4 and hourglassing coefficient (qh) of 0.10 were set for all elements in the mesh.

Nodal displacements for each time step were translated to scatterer displacements in 5 uniform scattering phantoms with randomly generated initial scatterer positions and 11 scatterers per resolution cell. Scatterers were tracked in Field II with the previously generated track PSFs in each phantom with a pulse repetition frequency (PRF) of 10 kHz. White Gaussian noise from 30-50dB signal-to-noise ratio (SNR) was added in steps of 10dB to the received RF data before scatterer displacement was estimated with 1D axial normalized cross correlation (3λ kernel, 50μm search window) [27] generating 1559 displacement through time profiles from 10–40 mm axially. Each displacement profile was treated as an independent sample resulting in 34,298,000 simulated displacement profiles (focal depths x elasticities x viscosities x scatterer realizations x SNRs x axial depths).

2). Heterogeneous Dataset:

Heterogeneous simulations were carried out in a similar manner to the homogeneous simulations aside from the focal depths and material properties. Beginning with the finite element mesh formed for the homogeneous simulations, a 10mm diameter spherical inclusion was placed with centroid at an axial depth of 25mm. LS-DYNA material files were generated on a grid of 4 inclusion elasticities (15.56, 22, 28, 36.67 kPa), 4 inclusion viscosities (0.01, 1.57, 2.6, 3.9 Pa.s), and a single background viscoelasticity (26.11 kPa elasticity, 2.34 Pa.s viscosity). The background viscoelasticity was selected to match a set of homogeneous simulations, and the inclusion viscoelasticities were varied around the background values to provide both lower and higher contrast sets of simulations. The ARF excitation PSFs were calculated at 5 focal depths (15–35 mm in 5 mm steps) to interrogate the medium above, below, within, and at the boundary of the inclusion. Additionally, ARF excitations were applied at 4 lateral offsets from the center of the inclusion (0, 2.5, 4, 7 mm) to model the effects of pushing closer to the lateral inclusion boundary and, in the extreme case of a 7 mm lateral offset, pushing outside of the inclusion. In total, the heterogeneous dataset consisted of 9,977,600 simulated displacement profiles (focal depths x inclusion elasticities x inclusion viscosities x scatterer realizations x SNRs x axial depths x lateral offsets).

B. Scanner Acquisitions

1). Phantom Acquisitions:

Imaging of a CIRS Cylindrical Elasticity QA Phantom (CIRS, Norfolk, VA) was performed with a Siemens Acuson Antares scanner and a VF7-3 linear array transducer (Siemens Healthineers, Issaquah, WA). Custom VisR beam sequences were created through the Axius Direct Ultrasound Research Interface (URI) allowing access to the raw radio frequency data. The imaging sequence executed a raster scan focused B-Mode acquisition followed by a VisR sequence at each of 40 lateral locations evenly spaced over 20mm for 2D VisR imaging. The VisR sequences used the beam sequence parameters listed in Table I and consisted of 2 reference tracks, 2 ARF excitations, 4 tracks between the ARF excitations, and 40 tracks after the second ARF excitation.

The imaging sequence was applied to the transverse cross-section of the 30 mm deep cylindrical inclusions forming 2D images with circular inclusions. Data was acquired for each inclusion type (4 inclusion elasticities) and the largest 3 circular cross-section diameters (6.49, 10.41, 16.67 mm) at 3 focal depths (25, 30, 35 mm). Each acquisition was repeated twice; however, only the first acquisition was used unless an error occurred in the data collection routine. True elastic modulus measurements calibrated by CIRS were reported as follows: Background (26 kPa), Type 1 Inclusion (6.5 kPa), Type 2 Inclusion (15.3 kPa), Type 3 Inclusion (49 kPa), and Type 4 Inclusion (88 kPa).

2). Clinical Acquisitions:

Samples from a previously acquired clinical dataset were processed similarly to those from the CIRS phantom. The clinical data set included VisR data acquired in the lower limb muscles of male patients, aged 5 to 10 years, with Duchenne muscular dystrophy and in age-match control boys with no known neuromuscular disorders. Data acquisitions were performed using the same scanner, transducer, and sequencing as listed in the phantom acquisitions section. All acquisition procedures were approved by the Institutional Review Board (IRB) of the University of North Carolina at Chapel Hill. Further information on the collection and composition of this dataset are reported in [28].

3). Scanner Processing:

Raw acquisition data from the scanners was transferred to workstations for custom processing. B-mode images were created through the absolute value of the log-compressed, Hilbert-transformed RF data. VisR displacements were estimated with 1D axial normalized cross correlation with parameters listed in Table I. A linear motion filter was applied to each displacement to remove any low-frequency bulk displacements from the acquisition process [29]. For each phantom acquisition, on-axis displacement through time data were paired with the respective axial depth, focal depth, calibrated elasticity, and assumed viscosity and saved for use in model fitting. Clinical acquisitions were similarly saved aside from pairing with the true elasticity and viscosity, which is unknown. The clinical acquisition system SNR is estimated to be roughly 40dB given that the muscle tissue were echogenic, stationary through the acquisition cycle, and did not contain any large structures with blood flow.

C. Semi-quantitative Viscoelastic Response (VisR) Model Fit Procedure

Ultrasonically tracked VisR displacements are fit to a mass-spring-damper (MSD) 2nd order non-homogeneous differential equation [17], [21]. The MSD model is equivalent to a Kelvin-Voigt viscoelastic model with the addition of an acceleration term with mass coefficient which captures some of the complex inertial effects of tissue deformation induced by transient ARF pushes. The base form of the MSD model is as follows:

$$m\ddot{z}\left(t\right)+\eta \dot{z}\left(t\right)+\mu z\left(t\right)=Au\left(t\right)$$ (1)

where m [Kg] is the mass, η [Nm⁻¹s] is the damper viscosity, μ [Nm⁻¹] is the spring stiffness, A [N] is the applied force magnitude, z(t) [m] is the time-dependent axial displacement, and u(t) is the time-dependent applied VisR force profile. The VisR force profile can be modeled as a series of time-delayed ARF pushes through the following equation:

$$u\left(t,t_{ARF},t_s\right)=H\left(t\right)-H\left(t-t_{ARF}\right)\hspace{0.33em}+\hspace{0.33em}H\left(t-t_{ARF}-t_s\right)-H\left(t-2t_{ARF}-t_s\right)$$ (2)

where H denotes a heaviside step function, t_ARF [s] is the ARF push duration, and t_s [s] is the time between the two ARF pushes. This equation generates two square impulses that approximate the applied forcing function of a double-push VisR beam sequence. Combining 1 and 2 and rearranging coefficients leads to the following form:

$$\begin{array}{l}\ddot{z}\left(t\right)+{\omega }^2\tau \dot{z}\left(t\right)+{\omega }^{2}z\left(t\right)=\hspace{0.33em}S{\omega }^2\left[H\left(t\right)-H\left(t-t_{ARF}\right)\right.\end{array}\hspace{0.33em}\left.+\hspace{0.33em}H\left(t-t_{ARF}-t_s\right)-H\left(t-2t_{ARF}-t_s\right)\right]$$ (3)

where ω [s⁻¹] is the natural frequency, τ [s] is the material relaxation time constant, and S [m] is the system static sensitivity. These coefficients are defined as follows:

$$\omega =\sqrt{\frac{\mu }{m}}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\tau =\frac{\eta }{\mu }\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}S=\frac{A}{\mu }$$ (4)

An analytic expression for z(t) in equation 3 is found in the Laplace domain. Non-linear optimization (Nelder-Mead algorithm) is then used to estimate the coefficients (ω, τ, and S) that minimize the difference between the analytic displacement z(t) and ultrasonically tracked displacement measurements. Once the fit procedure is complete, the coefficients are rearranged into the semi-quantitative estimates of elasticity and viscosity, Relative Elasticity (RE, [m⁻¹]) and Relative Viscosity (RV, [m⁻¹s]), respectively:

$$RE=\frac{1}{S}=\frac{\mu }{A}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}RV=\frac{\tau }{S}=\frac{\eta }{A}$$ (5)

These parameters are considered semi-quantitative measures of elasticity and viscosity because each is estimated relative to the unknown applied force amplitude, A. Considering A varies over axial depth due to the focusing effects of the applied ARF push point-spread-function, a depth-compensation factor is applied to RE and RV. Additionally, since fitting displacements to a series of 1D MSD models does not capture the complex system inertia that develops in an actual tissue environment, a simulation-derived elasticity compensation factor is applied to the RV estimates. Briefly, in a continuum environment, the displacement recovery measured on-axis to the ARF push is due to both the viscoelastic recovery effects of the media as well as the spatially averaged measure of viscoelastic shearing effect [30]. The shearing effect is measured as an artificially prolonged displacement recovery, which results in an overestimation of viscosity when using the 1D MSD model. Further details for the derivation and application of the depth and elasticity compensation factors are documented in [21].

D. Quantitative Viscoelastic Response (QVisR) Machine Learning Model Training and Evaluation

1). Preprocessing:

Max normalization was applied to each displacement profile to remove amplitude differences caused primarily by the unknown applied force distribution (Figure 1). Min-max normalization over the respective global parameter range was used for axial depth, focal depth, elasticity, and viscosity to homogenize the data scales from 0 to 1. The focal depth variable refers to the ARF push focus and the axial depth is the displacement estimation kernel location. Displacements were concatenated with the associated axial depth and focal depth of each profile to create the model inputs, and elasticity and viscosity were grouped as the outputs. Dataset generation for each acquisition type is described in greater detail below:

Fig. 1.

Simulated VisR displacement profiles before (top row) and after (bottom row) max normalization as elasticity (a,e), viscosity (b,f), focal depth (c,g), and axial depth (d,h) vary. When not varying, the following fixed parameters were used: elasticity = 36.67kPa, viscosity = 2.34Pa.s, focal depth = 25mm, axial depth = 25mm, SNR = 40 dB. Note that the green displacements in each subplot have equivalent parameters.

Simulation datasets were grouped into homogeneous simulations only (Train_Hom, Test_Hom), heterogeneous simulations only (Train_Het, Test_Het), and a mixture of the two (Train_Mix, Test_Mix). The primary difference between the training/validation and testing sets for simulations was the random scatterer realization since this variable would model noisy differences in the posterior distribution of the viscoelasticity estimates. Samples from each dataset were assigned a draw probability to balance classes during training. For example, the background samples of the Train_Het dataset were assigned a lower draw probability than the inclusion samples because the background for all heterogeneous simulations was the same viscoelastic material combination, whereas the inclusion material combinations varied. Validation sets were generated as 350,000 sample random splits from the training sets using the aforementioned draw probabilities. To account for different training set sizes for the Train_Hom, Train_Het, and Train_Mix datasets, an epoch was defined as a random weighted draw of 2 million total samples, e.g. an epoch of Train_Hom draws 2 million homogeneous samples and an epoch of Train_Mix draws 1 million homogeneous samples and 1 million heterogeneous samples.

Phantom images were cropped axially to only include data between 22 and 38 mm to reduce oversampling of the single background elasticity. Phantom displacements were linearly interpolated to match the simulation time grid so that model inputs were the same length (43 samples).

2). Model Selection:

A Bayesian hyperparameter search was executed for each of the training datasets to minimize the combined Huber loss of the elasticity and viscosity estimates on the validation sets [31]. Huber loss acts as L2, or Mean Square Error, loss within a delta range and L1, or Mean Absolute Error, outside of this range, providing regularization. Given the elasticities and viscosities were min-max normalized between 0 and 1, the delta range was set to 1, so the Huber loss acted as L2 when the summed estimate errors were within normalization ranges and L1 otherwise. Additionally, combining losses with this normalization equally weights penalties for elasticity and viscosity estimates. This search found similar optimal hyperparameters for each training dataset, and so the Train_Mix parameters were used for all cross-validation runs to standardize training. Using the random weighted sampling procedure described above, 11 cross-validation runs were performed for each train dataset resulting in 33 trained models.

For each cross-validation training run, a fully-connected, feed-forward, multi-layer perceptron neural network model was trained for 150 epochs with an ADAM optimizer [32], a learning rate λ of 0.0037, first moment β₁ of 0.9, second moment β₂ of 0.999, a batch size of 4096 samples, and early stopping. The model consisted of four hidden layers with the following neuron count connecting each layer: 128, 256, 256, 128, 64. The concatenated displacements, axial depth, and focal depth had an input size of 45, and jointly estimating elasticity and viscosity gives an output size of 2. Hidden layer neurons used Tanh activation functions and Softplus activation was used on the 2 output neurons to constrain the model to positive elastic and viscous moduli predictions. Model performance results are reported as the root mean square error.

3). Fine-Tuning:

Using model weights trained with simulated data, fine-tuning was performed by lowering the learning rate to 0.0001 and further training for 5 epochs on the phantom displacements. Early stopping was employed while monitoring the combined Huber loss on simulation and phantom displacement estimates on the validation sets. The fine-tuning training set contained phantom acquisitions from the 10.41 mm and 16.67 mm cross-sectional diameter inclusions with 25 mm and 35 mm focal depths. Performance was tested on the 6.49 mm cross-sectional diameter inclusion with a 30 mm focal depth.

E. Performance Metrics

Quantitative elasticity and viscosity estimations using QVisR are evaluated with the quantitative Root Mean Square Error (RMSE) metric [33]. Given the large sample sizes and approximately normal error distributions, RMSE is appropriate to give an average estimate of the error in the correct unit of the estimate (kPa for elasticity, Pa.s for viscosity). When comparing semi-quantitative estimates of elasticity and viscosity (VisR RE and RV) to each other and to the quantitative estimates, the semi-quantitative metric, Generalized Contrast to Noise Ratio (gCNR), is used [34]. This metric allows comparison of the ability to delineate the contrast between image patches independently of any dynamic range transformations to the data.

III. Results

Fig. 2 shows QVisR elasticity (panel a) and viscosity (panel b) estimation RMSE for each train and test material subset pairing cross-validation run. Neural networks are poor at extrapolation [35], so results for the Train_Het models were filtered to exclude estimates on Test_Hom samples that were outside of the simulated heterogeneous dataset parameter ranges (filtered subset RMSEs represented by square markers in Fig. 2).

Fig. 2.

Elasticity (a) and viscosity (b) RMSE grouped by train and test simulation datasets for each cross-validation training run. RMSEs displayed are over samples in the test sets (color) with an SNR of 50dB. Square marker points represent the subset of Test_Hom and Test_Mix with parameter ranges filtered to match Train_Het ranges (e.g., Test_Hom elasticities greater than 36.67 kPa were removed since 36.67 kPa is the max elasticity in Train_Het).

Fig. 3 depicts QVisR Train_Mix model RMSEs on the Test_Hom dataset grouped by axial distance to the focal depth. Due to displacement profiles being measured only from 10-40mm, some focal depths within the distance to focal depth ranges (colors) have sparse data as illustrated by truncated lines in Fig. 3c and 3f. Results are shown with varying test set elasticity, viscosity, and focal depth.

Fig. 3.

Elasticity (top row) and viscosity (bottom row) RMSE grouped by distance from displacement axial depth to ARF push focal depth with varying elasticity (left column), viscosity (middle column), and focal depth (right column). Points and shading represent the mean ± max and min RMSE over the 11 cross-validation Train_Mix models. RMSEs were calculated over all seed realizations with an SNR of 50dB in the Test_Hom dataset.

Fig. 4 compares QVisR Train_Mix model RMSEs on the Test_Mix set when grouped by SNR. Results are shown with varying test set elasticity, viscosity, focal depth, and scatterer realization. At 20dB system SNR, the estimate RMSEs are no longer viable representing a lower limit of noise where QVisR is no longer a good estimator. Significant spiked decreases in RMSE for lower SNRs (≤ 30dB) at elastic moduli of 22 kPa and 28 kPa as well as viscous modulus of 2.34 Pa.s indicate moduli that are present only in the heterogeneous dataset. Similar alternating spikes in the focal depth subplots are also caused by these moduli since the RMSE at each focal depth is averaging over all elasticities and viscosities and the lower RMSE spikes occur at the focal depths in the heterogeneous dataset.

Fig. 4.

Elasticity (top row) and viscosity (bottom row) RMSE grouped by SNR for varying elasticity (first column), viscosity (second column), focal depth (third column), and scatterer realization (fourth column). Points and shading represent the mean ± max and min RMSE over the 11 cross-validation Train_Mix models. RMSEs calculated over all samples in the Test_Mix set except those greater than 20mm below the focus.

Fig. 5 shows the point-wise axial QVisR estimates of elasticity and viscosity in heterogeneous inclusion simulations. Results are shown for heterogeneous materials with the following inclusion elastic and viscous moduli pairs: (15.56 kPa, 0.01 Pa.s), (15.56 kPa, 3.90 Pa.s), (36.66 kPa, 0.01 Pa.s), and (36.66 kPa, 3.90 Pa.s). The heterogeneous material background was held constant with 26.11 kPa elastic modulus and 2.34 Pa.s viscous modulus. Heterogeneous material axial estimation results are shown at 4 lateral offsets from the inclusion center (0 mm, 2.5 mm, 4.0 mm, and 7.0 mm). The 7.0 mm lateral offset measurements are completely outside the inclusion boundary so the material is homogeneous over the axial extent (Fig. 5 d, i). Results in a separate homogeneous simulation matching the heterogeneous simulation background viscoelasticity are also plotted (Fig. 5 e, j).

Fig. 5.

Point-wise axial estimates of elasticity (top row) and viscosity (bottom row) for heterogeneous and homogeneous simulation samples. Four heterogeneous inclusion viscoelasticity combinations (colors) are shown for each inclusion center lateral offset (a-d and f-i). The homogeneous sample matched the background viscoelasticity of the heterogeneous samples (e, j). Estimates (solid lines) are plotted over true (dashed lines) elasticities and viscosities. Each sample was drawn from a random seed realization in the Test_Mix set and have a fixed focal depth of 25mm and 40dB SNR. Estimate results are plotted with a ~ 1 mm sliding axial median filter. Estimates shown for the Train_Mix cross-validation model with the lowest Test_Mix RMSE.

Fig. 6 shows the VisR MSD relative elasticity as well as both simulation-trained and phantom fine-tuned QVisR elasticity estimation results in the CIRS cylindrical elasticity phantom test set. True elasticity masks created from manual B-Mode segmentation illustrate the target elasticity for each model (Fig 6 a–d). Model results are shown for each of the 4 calibrated inclusion elasticities (6.5, 15.3, 49.0, and 88.0 kPa). Overlaid on the figure are semi-quantitative (gCNR) and quantitative (RMSE) metrics with corresponding Region of Interest (ROI) patch sizes of 2 × 2 mm. The gCNR metric uses a patch within the center of the inclusion and one to the right of the inclusion while RMSE is measured per patch. Table III lists the changes in the gCNR metric as the image patch size increases to encompass most of the interior of the inclusion and more of the boundary pixels.

Fig. 6.

Calibrated cylindrical step phantom bmode-segmented true elasticity reference (1st row) compared with VisR MSD model fit relative elasticity (2nd row) and QVisR estimated elasticities before (3rd row) and after (4th row) fine-tuning. Each column is a different inclusion type with varying inclusion elasticity and a fixed background elasticity (calibrated inclusion and background elasticities marked on the elasticity colorbar). The segmentation mask inclusion boundary is overlaid in transparent white on the estimate images to mark the outer extent of the inclusion. VisR displacement acquisitions had a 30mm ARF push focal depth. Estimates are displayed with a 2D median filter with a ~ 1 × 0.5 mm kernel (axial x lateral). Overlaid statistics were calculated using 2 mm square image patches in the center of the inclusion and to the right of the inclusion. RMSE in the inclusion and background were calculated over the respective image patches with the ground truth from an equivalent size patch from the first row of images and gCNR was calculated using the inclusion and background patch of each image. Note that RMSE cannot be calculated for VisR MSD relative elasticity because RMSE is a quantitative metric and relative elasticity is a semi-quantitative parameter; however, gCNR, a semi-quantitative metric, can be applied to both semi-quantitative and quantitative images. RMSE is in units of kPa Young’s modulus.

TABLE III.

CIRS phantom gCNR values for varying ROI patch sizes

Parameter	ROI Size	Inc. Type 1	Inc. Type 2	Inc. Type 3	Inc. Type 4
VisR MSD Relative Elasticity	2 × 2 mm	0.79	0.93	0.29	0.92
	3 × 3 mm	0.69	0.95	0.36	0.69
	4 × 4 mm	0.70	0.73	0.28	0.77
Simulation Trained QVisR Elasticity	2 × 2 mm	1.00	0.95	0.98	0.40
	3 × 3 mm	1.00	0.97	0.95	0.55
	4 × 4 mm	0.97	0.90	0.78	0.55
Phantom Fine Tuned QVisR Elasticity	2 × 2 mm	1.00	1.00	1.00	1.00
	3 × 3 mm	1.00	1.00	1.00	1.00
	4 × 4 mm	1.00	0.97	0.98	1.00

Fig. 7 shows B-Mode, peak displacement, VisR MSD relative elasticity and relative viscosity, and both simulation-trained and phantom fine-tuned QVisR viscoelasticity estimates in the gastrocnemius and rectus femoris muscles of two boys with Duchenne Muscular Dystrophy. The simulation trained model, and base model used for fine-tuning, is the Train_Mix cross-validation model with the lowest Test_Mix RMSE. Displacements within key regions of interest, labeled in Fig. 7, are shown in Fig. 8.

Fig. 7.

B-Mode, VisR peak displacement, VisR MSD relative elasticity and relative viscosity, and QVisR elastic and viscous moduli estimates on in vivo gastrocnemius (left) and rectus femoris (right) muscles of boys with Duchenne Muscular Dystrophy. QVisR estimates are shown for the simulation trained model (3rd row) and the phantom fine-tuned model (4th row). Acquisitions had 20mm (left) and 17mm (right) ARF push focal depths and estimates are displayed with a 2D median filter with ~ 1 × 0.5 mm kernel (axial x lateral) on the image overlays. Peak displacement images (b, h) were depth normalized with a parabolic curve fit to minimize variations due to applied force amplitude around the focal depth.

Fig. 8.

VisR displacement profiles from gastrocnemius (left) and rectus femoris (right) muscle regions of interest. Shading shows the min and max displacement over a 0.5 mm axial kernel. Legend labels 1 and 2 correspond to regions 1 and 2 in each Fig. 7 muscle.

IV. Discussion

QVisR was found to work best when trained with a mixture of homogeneous and heterogeneous materials (Fig. 2). Although the Train_Mix model had the lowest RMSE for the Test_Mix dataset, the Train_Hom and Train_Het models had slightly better performance on their respectively material subset test datasets than the Train_Mix model. Additionally, each train/test set pairing had minimal bias (<0.1 kPa, <0.03 Pa.s), and errors were approximately normally distributed. RMSEs can therefore be interpreted as a standard deviation capturing 68% of the error variation, e.g. 68% of the Train_Mix model elasticity estimates on the Test_Mix dataset were within ±2.54 kPa of the true elasticity. Filtering by test subset brought RMSEs down to comparable levels with Train_Hom errors; however, training with both homogeneous and heterogeneous simulations still provided the best or comparable performance to models trained with each set individually. These results suggest that QVisR estimates depend on material geometry in cases where the boundary effects distort the displacement profiles. QVisR models designed to generalize to arbitrary material geometries will require training on a much greater array of simulated material geometries than presented in this study. For the remaining simulation analyses, Train_Mix models are used.

When isolated to estimating on homogeneous materials to study model performance over the axial extent (Fig. 3), QVisR elasticity and viscosity estimate RMSEs are consistent over all axial depth groups aside from greater than 20 mm below the focal depth (orange). In limited cases where the true elastic modulus is less than ~80 kPa, the elasticity estimation RMSE is also consistent >20 mm below the focal depth (Fig. 3a). This wide axial extent of stability is thought to be due to the combination two factors. First, the relatively high acoustic attenuation coefficient of 1 dB/cm/MHz spreads the force distribution over the axial extent [11], particularly in the near field, causing several micron displacements far from the focus. Second, peak-normalizing each displacement profile and pairing it with both the measurement axial depth and the ARF push focal depth acts as a type of depth normalization allowing QVisR to estimate viscoelasticity far from the focus. Training QVisR models on millions of displacements with varying ARF focal depth, measurement axial depth, and material property results in a network that can compensate for complex system inertia peak displacement time delays that appear similar to an increase in viscosity. The effect of force distribution on QVisR estimates, specifically the modification of the acoustic attenuation coefficient and focal configuration, is a topic of ongoing research. For example, using a lower acoustic attenuation will make the applied force more localized and may disrupt QVisR estimates far from the focal depth. For the remaining simulation analyses, estimates greater than 20mm below the focus are considered beyond the useful range of the model and excluded.

Estimates are also stable over a range of scanner-realistic system SNRs (Fig. 4). The homogeneous dataset elasticity and viscosity step sizes (10.56 kPa and 0.78 Pa.s, respectively) were considered acceptable RMSE cutoffs when estimating across large data subsets. QVisR meets this criteria for both elasticity and viscosity RMSE across all variations of elasticity, viscosity, focal depth, and scatterer realization when the SNR is at least 40dB; however, only elasticity RMSE is within the tolerance down to 30dB SNR. This suggests elasticity is easier for QVisR to estimate in noisier environments than viscosity. One possible explanation for this result could be determined by visually inspecting the peak normalized displacements as elasticity and viscosity vary (Fig. 1 e, f). Changes in displacement as viscosity varies are less dramatic than changes in displacement as elasticity varies, so it is intuitive that elasticity would be easier to estimate from the displacement profile at a lower SNR. Further analysis is necessary for proper characterization of QVisR’s performance in lower SNR environments.

Shifting from evaluation of error data subset groups to individual estimate error, QVisR estimates closely match the true viscoelastic moduli through axial depth (Fig. 5). Heterogeneous sample estimates delineate the viscoelastic inclusion boundary within 1mm of the true boundary despite focusing at the center of the inclusion (Figs. 5 a–c, f–h). Background viscoelasticity is estimated with negligible error aside from a large overestimate around 15mm for samples with lower inclusion elastic modulus (Figs. 5 a–b, f–g). These large overestimation errors may be caused by speckle bias, reflection artifacts off the top of the inclusion, or estimating close to the edge of simulated focal depths. When focusing with a lateral offset outside the inclusion boundary, estimate errors increase within the axial range of the inclusion projection (20-30mm, Figs. 5 d, i). When looking at estimates of a simulation without an inclusion and matching viscoelastic modulus to the heterogeneous sample background, a similar error profile is seen for the elastic modulus estimates but not for the viscous modulus estimates (Figs. 5 e, j), although the homogeneous viscous modulus estimates have a higher variance, which may obfuscate the viscous error profile. The repeated error profiles in both heterogeneous samples measured in the background and matched viscoelastic modulus homogeneous samples indicate a slight model bias (1–2 kPa elastic modulus, 0.5–1.0 Pa.s viscous modulus) based on the training set inclusion geometry. Additionally, the error profiles appear larger for the heterogeneous background cases implying a slight distortion of the displacement profiles due to reflections off the inclusion boundary. These insights are significant given that the model inputs are just the displacement profile, axial depth, and focal depth, so the Train_Mix models have no a priori information about the existence of an inclusion or information from neighboring displacement profiles. This suggests QVisR trained on simulations with more material geometries may still accurately estimate near the inclusion boundaries.

A. Phantom Acquisitions

Testing QVisR generalization on CIRS calibrated elastic phantom images (Fig. 6) shows that fine-tuning is necessary for quantitative domain adaption. Phantom image estimates for the model trained only on simulation data show the correct trends (i.e., estimates correctly identify when the inclusion is softer or stiffer than the background and generally indicate that stiffness increases with true inclusion elasticity), but the estimates have large errors and poor boundary delineation (Figs. 6 i–j). After fine tuning (Figs. 6 m–p), QVisR both delineates the inclusion boundary and estimates the elastic moduli of the inclusion and background with a lower RMSE; however, the softest and stiffest inclusions (Figs. 6 m, p) are still under and overestimated, respectively. Elasticity estimation with QVisR somewhat gradually transitions from inclusion to background values across the inclusion boundary leading to higher RMSE near the boundary than near the center of the inclusion. Viscosity estimates are not shown since the phantom only had elasticity variations and a negligible viscous moduli. Comparing QVisR elasticity estimation with VisR MSD relative elasticity estimation gCNR scores show that both simulation-trained and phantom fine-tuned QVisR better delineate an inclusion from the background. Additionally, the phantom fine-tuned QVisR model has perfect gCNR scores indicating that the 2mm patch in the center of the inclusion is perfectly separable from the 2mm patch outside the inclusion to the right. These results degrade only as the ROI patches approach the boundary of the inclusions and noted in Table III. Note that although it seems the VisR RE estimates (Figs. 6 e–h) visually appear to more closely match the ground truth of the background material in color than the simulation-trained QVisR model (Figs. 6 i–l), the VisR RE results are on a semi-quantitative scale which means an error metric for the background material can’t be calculated and, therefore, it can’t be evaluated if the VisR RE background is closer to the ground truth than the simulation-trained QVisR results. Since both the simulation-trained and phantom-fine tuned (Figs. 6 m–p) models are directly estimating the elastic modulus quantitatively, the RMSE of the background estimates can be compared to determine that fine-tuning improves the background and inclusion RMSE.

The ability of QVisR simulation-trained models to be domain adapted to scanner-acquired data in a calibrated phantom suggest model generalization would be feasible with relatively small labeled datasets. Additionally, QVisR domain adaption works best on materials in the interior range of samples, i.e., the medium stiffness materials rather than the softest or stiffest, suggesting an ideal labeled dataset would extend beyond the range of materials QVisR would ultimately be estimating in practice.

B. Clinical Acquisitions

As a secondary evaluation of QVisR model generalization, the simulation and phantom fine-tuned models estimated viscoelastic moduli in the gastrocnemius and rectus femoris muscles of two boys with DMD (Fig. 7). Since fine-tuning was necessary for adaption from simulations to phantoms, the DMD dataset estimates should not be considered quantitative; however, a few labeled regions of interest show promising trends.

The gastrocnemius peak displacement image (Fig. 7b) has a region of high displacement bisected laterally by a streak of low displacement near the ARF focal depth (region 1). Interpreting the blue low peak displacement streak in isolation would indicate high elasticity; however, it is not possible to separate the effects of elasticity and viscosity. QVisR estimates indicate this low displacement streak is dominated by a change in viscous modulus (Fig. 7f, h) since the elasticity is relatively constant above this region (Fig. 7 e, g). This result is supported visually by the VisR MSD RE and RV fits in Fig. 7 c and d; however, since the VisR results are semi-quantitative, the exact values cannot be quantitatively compared. The corresponding displacement in Fig. 8 has a delayed peak, long recovery time, and minimal recovery after the first peak which also tends to happen with highly viscous materials at the focus. Additionally, the peak image lower left corner (region 2) shows two small regions of high displacement which QVisR suggests are points with higher elasticity and lower viscosity than the surrounding tissue (Fig. 7 e, f, g, h). This could suggest a very stiff region that was displaced further by compressing the softer, more viscous tissue in neighboring regions. The displacements in this region are fairly typical of those with higher elasticity and low viscosity (i.e., rapid ascension to peak displacement, partial recovery between the two ARF pushes, and rapid peak displacement recovery as shown in Fig. 8). Comparing the simulation-trained and phantom fine-tuned model estimates shows an interplay between the elasticity and viscosity estimates. The phantom dataset had no viscosity, which biased the fine-tuned models towards estimating low viscosity. Region 1 still has a 2–3 Pa.s viscosity indicating strongly viscous features; however, the background of the phantom fine-tuned image is relatively homogeneous. Comparing the elasticity estimates pre and post fine-tuning shows an increase in detail and elasticity in region 2. This suggests the QvisR elasticity and viscosity estimates may be coupled and that a proper domain adaption dataset would need to include both elasticity and viscosity variations to reduce bias.

The rectus femoris peak displacement image (Fig. 7j) has a central area with two points of high peak displacement (region 1) and a third point of high displacement near the bottom of the peak overlay (region 2). Interpreting the peak displacement images alone would indicate both regions have low elasticity. Simulation-trained QVisR indicates that the central high peak points in region 1 are driven by decreases in both elastic and viscous moduli whereas the high peak in region 2 is primarily due to a change in viscosity (Fig. 7o, q). After fine-tuning, the viscosity estimate variability decreases and the regions of interest are only visible in the elasticity estimate image (Fig. 7p). VisR RE and RV show the elevated elasticity and viscosity in region 2; however, there is little change in contrast in either the RE or RV images near region 1 (Fig. 6 l, m). This could just be due to the lower RE sensitivity to changes in softer materials as seen in the similar RE values in the inclusions of Fig. 6 e and f. Displacements for each region (Fig. 8) are difficult to interpret since the measurement axial depth is different (13.8 mm for region 1 and 17.5 mm for region 2) and axial depth can fairly dramatically change the shape of peak normalized displacements as shown in Fig. 1h. Further analysis is necessary to determine cases where QVisR estimates are valid and a proper fine-tuning dataset for in vivo evaluations.

C. QVisR Outcomes and Study Limitations

QVisR is able to estimate elastic and viscous moduli well given in silico displacement profiles and shows promise for generalization to ex vivo/in vivo applications when a limited dataset can be used to tune and validate model performance. Since QVisR uses displacements measured in the region of excitation and only selectively uses limited spatial averaging as a post-processing procedure, the proposed model can estimate viscoelasticity with a finer resolution than methods that rely on measurement of a shear wave propagating over several millimeters.

A primary limitation of this study is the availability of quality, labeled displacement data. True point-wise viscoelastic moduli values are not currently measurable in clinical datasets. Additionally, calibrated phantoms with mechanical heterogeneities that vary both elastic and viscous moduli are an active area of research. Simulation material parameters were selected to cover a range of realistic values [4], [5]; however, simulating all variations of material viscoelasticities, geometries, and acoustic properties is not possible. Namely, this study is limited by use of a fixed, frequency-independent attenuation as well as the use of a single geometry and single background viscoelasticity for the heterogeneous simulations. The simulation imaging parameters were fixed to match those of the phantom and DMD dataset acquisitions with the VF7-3 transducer. QVisR is sensitive to variations in many of these imaging parameters, so separate datasets and models would need to be generated for imaging parameter combinations. For example, if the F/# or time between pushes in the VisR beam sequence were changed, a dataset with that imaging configuration would need to be simulated and a new QVisR model would need to be trained on the new dataset. Additionally, fine-tuning was necessary for translating a simulation-trained QVisR model to the calibrated CIRS phantom scanner acquisitions. This fine-tuning step is likely necessary due to a mismatch between simulated and scanner-acquired displacements with matched imaging configuration and material properties. Improving the match between simulations and scanner-acquisitions is a topic of ongoing research and, in theory, would eliminate the need for fine-tuning once the simulation error is sufficiently small. Alternatively, a composite model could be trained with data from multiple imaging configurations where the imaging parameters could be included as features, similarly to the focal depth parameter in current QVisR models; however, these composite models may be harder to train due to the large amount of data across varying imaging configurations.

Expanding the breadth of the simulation datasets to encompass varying attenuation, more heterogeneous structures, and different imaging parameters may improve model generalization. Any additional parameter which could be known a priori, such as the imaging configurations, could be directly incorporated to the models as an input feature, similar to how axial depth and focal depth were treated in this study. Unknown confounding parameters, such as attenuation, could be estimated with other methods before use as an input or could be jointly estimated with elastic and viscous moduli if labels are available for a training set.

This study used a relatively simple neural network model architecture with minimal preprocessing of displacements before use as a model input. Focus on improving the model’s understanding of the data and dataset information quality may yield estimate improvements. Using a model more appropriate for time-series analysis, such as a 1D Convolutional Neural Network (CNN) or Recurrent Neural Network (RNN), or a dimensionality reduction technique, such as principal component analysis (PCA), may help the model efficiently extract the necessary information content from displacement profiles. Training efficiency may also be increased by use of a displacement quality factor to remove erroneous displacements or samples with high correlation. Use of a Bayesian approach may additionally allow use of labeling data with uncertainty, e.g. clinical muscle data could be parameterized by a mean and standard deviation from literature. Viscosity bias to the phantom dataset may be reducible by decoupling the elasticity and viscosity estimation into separate networks and freezing the viscosity network weights; however, this would remove any benefits from joint estimation of elasticity and viscosity. Investigation of alternative QVisR methods is ongoing.

V. Conclusions

This study shows that there is a functional mapping, although potentially complex, between on-axis displacements and viscoelastic moduli that can be learned from data. Across several million simulated displacement samples in the test set, QVisR estimates have an elastic modulus RMSE less than 2 kPa and a viscous modulus RMSE less than 0.5 Pa.s. QVisR estimates are stable in regions far above and below the focal depth (10–15+ mm) and with realistic scanner system SNRs (30–50 dB). When training on a random sampling of both mechanically homogeneous and heterogeneous displacement profiles, QVisR is able to delineate viscoelastic inclusion boundaries while maintaining similar performance in mechanically homogeneous materials. Although the simulation trained models do not generalize perfectly to phantom and clinical datasets currently, fine-tuning methods show promise for future implementations, and alternative modeling to improve generalization are under investigation.

Highlights.

• QVisR uses a neural network trained on simulations to estimate the elastic and viscous moduli of materials given the displacement profile measured in the region of acoustic radiation force excitation.

• QVisR has low elastic and viscous moduli estimation RMSE in mechanically homogeneous and heterogeneous simulations and can be fine-tuned to estimate using displacements acquired with a real scanner.

• QVisR estimates both the elastic and viscous moduli without monitoring shear-wave propagation and shows promise for clinical application with improvements in model calibration.

Biographies

Joseph Richardson Joey Richardson is a graduate student research assistant in the Gallippi Lab. He graduated from the University of Alabama at Birmingham with a BS in Electrical Engineering in 2019 and is currently working on his PhD in Electrical and Computer Engineering at North Carolina State University in Raleigh, NC. His current research interests include developing quantitative viscoelastic measurements of anisotropic tissue using ARF-based imaging techniques and physics constrained machine learning model applications in ultrasound.

Christopher Moore Dr. Christopher Moore was born in Newark, Ohio in 1989. He completed his bachelors degrees in fine arts and electrical engineering in 2012 and 2013, respectively, at Ohio University in Athens OH, followed by his doctorate in electrical engineering at North Carolina State University in Raleigh, NC in 2019. As of 2023, Dr. Moore serves as an imaging specialist for Revvity, where he works with industry and academic partners to identify imaging-based solutions for drug discovery and biomarker tracking.

Caterina Gallippi Caterina M. Gallippi earned a B.S.E. in Electrical Engineering and a certificate in Engineering Biology from Princeton University in 1998. She completed her Ph.D. in Biomedical Engineering at Duke University in 2003 with a focus on ultrasonic imaging. She is currently a professor in the Joint Department of Biomedical Engineering at the University of North Carolina at Chapel Hill and North Carolina State University, NC, USA. She is also the Director of the Unified Medical Ultrasound Technology Development (UNMUTED) NIH T32 predoctoral training program. Her research interests include radiation force imaging, adaptive signal filtering, multidimensional motion tracking, and magneto-motive ultrasound. Her group works to apply these novel technologies to cardiovascular disease, cancer, kidney disease, and musculoskeletal disorders.