Can machine learning accelerate soft material parameter identification from complex mechanical test data?

Abstract

Identifying the constitutive parameters of soft materials often requires heterogeneous mechanical test modes, such as simple shear. In turn, interpreting the resulting complex deformations necessitates the use of inverse strategies that iteratively call forward finite element solutions. In the past, we have found that the cost of repeatedly solving non-trivial boundary value problems can be prohibitively expensive. In this current work, we leverage our prior experimentally derived mechanical test data to explore an alternative approach. Specifically, we investigate whether a ma-chine learning-based approach can accelerate the process of identifying material parameters based on our mechanical test data. Toward this end, we pursue two different strategies. In the first strategy, we replace the forward finite element simulations within an iterative optimization framework with a machine learning-based metamodel. Here, we explore both Gaussian process regression and neural network metamodels. In the second strategy, we forgo the iterative optimization framework and use a stand alone neural network to predict the entire material parameter set directly from experimental results. We first evaluate both approaches with simple shear experiments on blood clot, an isotropic, homogeneous material. Next, we evaluate both approaches against simple shear experiments on right ventricular myocardium, an anisotropic, heterogeneous material. We find that replacing the forward finite element simulations with metamodels significantly accelerates the parameter identification process with excellent results in the case of blood clot, and with satisfying results in the case of right ventricular myocardium. On the other hand, we find that replacing the entire optimization framework with a neural network yielded unsatisfying results, especially for right ventricular myocardium. Overall, the importance of our work stems from providing a baseline example showing how machine learning can accelerate the process of material parameter identification for soft materials from complex mechanical data, and from providing an open access experimental and simulation dataset that may serve as a benchmark dataset for others interested in applying machine learning techniques to soft tissue biomechanics.

1. Introduction

Material parameter identification is a critical step towards both determining soft biological tissues’ biomechanical phenotype [6,9,38] and establishing accurate mechanical simulations of soft materials [5,29]. These endeavors critically depend on well-informed constitutive laws that link the problem’s kinetic quantities to its kinematic quantities, e.g., stress to strain. This process of informing constitutive laws may be broken down into two steps in which the first step is to experimentally interrogate the material’s response to deformation. The second step is to inversely identify the material parameters that yield the best fit between an apriori assumed constitutive model and the experimental data. For all but a few simple experimental test modes, the inverse identification of the material parameters makes use of computational approaches [1,32,37]. Typically, this process requires repeatedly calling finite element simulations within an iterative optimization problem framework [31,17,34,13]. For soft materials, relevant deformations are often large, and materials laws are complex [11,8]. Thus, to identify one set of soft material parameters from complex mechanical test data requires calling a nonlinear finite element solver on the order of thousands to ten thousands of times. The cumulative computational cost of this process is very large – and in some cases may be prohibitively expensive. For example, we recently combined a least squares optimization approach with a nonlinear finite element scheme to identify the eight material parameters of a hyperelastic constitutive law for right ventricular myocardium [12,13]. For the total of 11 test samples, this process required approximately 20,000 nonlinear finite element solutions that each took 70s to run on a 32CPU work station, for a total run time of ≈ 40 hours. Additional complexities in the process arise from simulations that do not converge and return inadmissible values to the least squares solver, terminating the process and re-quiring repeated re-starts of this costly problem; thus, adding to the total cost of this process.

Naturally, we are not the first to identify this challenge; making inverse modeling approaches tractable is a rich area of research [30,2]. One approach is to ignore the complexity of the experimental test data and assume its deformations to be homogeneous [33, 21]. Thereby, we can replace finite element solutions with analytical solutions in our least squares optimization approach; thus, accelerating the inverse identification problem dramatically. While convenient when warranted, the resulting parameters can be less accurate and it’s typically not possible to account for material spatial heterogeneity. Additionally, other, more efficient methods to solving the inverse problem have been introduced. However, this increased efficiency often comes at a loss in flexibility. For example, others have used reduced order unscented Kalman filtering or direct inverse solvers; both require problem-specific custom code [20,22]. Similarly, adjoint methods have been applied to inverse problems. Rather than accelerating the forward computations, these methods accelerate the evaluation of the gradient of the optimization problem [2]. That is, they accelerate the least squares optimization without replacing the finite element simulation and without sacrificing accuracy. Unfortunately, they also come at the cost of flexibility as here, too, standard finite element solvers cannot be used without modification. Finally, machine learning approaches could be used to accelerate soft material parameter identification.

This latter approach has been applied to material characterization in multiple different ways [16,18]. This includes, but is certainly not limited to, physics-informed neural network approaches to inverse analysis [28], data driven constitutive modeling [35], and supervised learning approaches where large datasets are acquired and used to train a machine learning model [15,40]. Within the framework of supervised learning, where a model is trained to predict labeled data [4,10,14], there are still multiple ways in which an inverse analysis could be implemented. For example, one could train a neural network to predict constitutive model parameters directly from experimental data [36]. Alternatively, one could retain an iterative optimization framework and simply use supervised learning to replace the forward model [39].

The scope of our current work lies in using supervised learning to accelerate soft material parameter identification from complex mechanical test data. Through our effort, we leverage the speed of machine learning methods while retaining the general constitutive model framework that is readily implemented in most finite element solvers. To this end, we chose a two-step approach: In the first strategy, we investigate whether we can use machine learning-based metamodels to replace the finite element analysis component of the iterative optimization pipeline. In the second strategy, we investigate whether we can replace the entire identification pipeline with a machine learning approach. Critically, we see the approach presented in our manuscript as a baseline. In the future, others may improve upon our framework. Therefore, we make all experimental data, synthetic data, and code associated with this work available under open source licenses; thus providing an accessible point of entry for others.

2. Abbreviated Methods

2.1. Overview

Figure 1 provides a methodological overview of our ap-proach. Throughout this work, we used two open access benchmark experimental datasets. Dataset 1 is comprised of simple shear mechanical test data of blood clot, while Dataset 2 is comprised of simple shear and confined tension/compression mechanical test data of right ventricular myocardium. These datasets represent the breadth of soft tissue complexity: from homogeneous and isotropic, to heterogeneous and anisotropic, respectively. Because both simple shear and confined tension/compression yield heterogeneous deformations in our cuboid samples, we have previously combined a Least Squares solver with nonlinear Finite Element Method simulations (LS FEM) to identify material parameters based on these datasets, see Figure 1A. In our current work, we first tested whether we can accelerate the material parameter identification by replacing the forward finite element solutions with a machine learning-based metamodel. Specifically, we used Gaussian Process Regression and/or Neural Network regression (LS GPR/NN) as our metamodeling approach, see Figure 1B. Note, we trained these metamodels with finite element-based synthetic data of simple shear and confined tension/compression tests. Finally, we also replaced the iterative, least squares approach in its entirety and used Neural Network Regressors (NNR) to capture the entire parameter identification process, thus escaping the need for iteration, see Figure 1C. Note, extensive details on each aspect of our approach are provided in Supplement A.

Fig. 1.

Methodological overview. (A) Within our framework, we considered the Least Squares regression with forward Finite Element Method simulation (LS FEM) the gold-standard. (B) In our first attempt of accelerating material parameter identification we replaced the forward FE simulation with Gaussian Process Regression (LS GPR) and/or Neural Network (LS NN) metamodels. (C) In a second attempt, we replaced the entire identification process with a Neural Network Regressor (NNR) to estimate material parameters directly from the experimental stress-strain curves.

2.2. Experimental Dataset

We tested our ability to accelerate material parameter identification against two disparate experimental datasets. The first, less complex dataset contains the normal and shear forces of red blood clot in response to simple shear of up to 50% strain. A total of 27 datasets for as many individual samples are included. The experimental details are described in Supplement A and in our original work [34]. The second, more complex dataset contains the normal and shear forces of right ventricular myocardium in response to simple shear of up to 40% strain. Additionally, this second dataset contains normal forces of right ventricular myocardium in response to confined tension/compression of up to 15% strain. Note, when sheared, each sample was tested in two directions and three different orientations. Therefore, the total of 11 samples yielded 99 data normal force and shear force sets. Additionally, note that all blood clot samples had the same dimensions of 10 × 10 × 10mm, while the dimensions of the right ventricular myocardium varied for each sample.

2.3. Least squares regression with forward finite element simulations (LS FEM)

Here, we used our previously established finite element-based inverse pipeline as the gold-standard. In short, we used a least squares algorithm to call forward finite element simulations of the simple shear problem and the confined tension/compression problem to iteratively identify the material parameters given hyperelastic material laws for red blood clot and right ventricular myocardium. Specifically, we used the least squares regression implementation lsqnonlin in MATLAB (Mathworks, Version 2019) to iteratively call the implicit, nonlinear finite element solver FEBio [19]. For blood clot, we modeled the material response with the two-parameter Ogden model [24]. In contrast, for right ventricular myocardium, we modeled the material response with the eight parameter Holzapfel model, where we specifically modeled one dispersed fiber family to represent muscle fibers [12]. Note, we included sample-specific, histology-based spatial heterogeneity in the right ventricular myocardium by varying the mean fiber direction and its dispersion through the sample thickness.

2.4. Least squares regression with forward stress-strain metamodels (LS GPR/NN)

In our first machine learning-based strategy, we independently trained Gaussian process regression and neural network metamodels with synthetic data to replace the forward finite element simulations in our LS FEM approach. To this end, we first established two separate synthetic datasets. The first synthetic dataset contains 10,000 finite element simple shear simulations of blood clot, where each simulations differs only in the two parameters of the 1-term Ogden model. The second synthetic dataset contains 108,000 finite element simple shear and confined tension/compression simulations of right ventricular myocardium, where each simulation differs in the eight Holzapfel material parameters, the three sample dimension, and six anisotropy parameters, see Figure 3A-B. We conducted training, validation, and testing of the GPR metamodels in the Scikit-learn library (v0.24.2) with the function GaussianProcessRegressor using an anisotropic radial-basis function kernel (RBF) [26]. We also conducted detailed sensitivity analyses to choose the optimal hyperparameters, see Supplement B. Similarly, we conducted training, validation, and testing of the NN metamodels in the PyTorch framework (v1.9.0) [25]. Here, too, we conducted detailed sensitivity analyses to chose the best network architectures, see Supplement B and Table 1. Finally, we integrated the trained metamodels in our least squares regression pipeline using LMFIT [23].

Fig. 3.

Feature space (input) and targets (output) of our machine learning approaches. (A) Gaussian Process Regression (GPR) or Neural Network (NN) blood clot metamodels predicting the Ogden material response. (B) Gaussian Process Regression (GPR) or Neural network (NN) right ventricular myocardium metamodels predicting the Holzapfel material response. (C-D) Neural Network Regressor (NNR) to predict the Ogden and Holzapfel material response for blood clot and right ventricular myocardium, respectively.

Table 1.

Network architectures of Neural Network (NN) metamodels and the Neural Network Regressor (NNR). Additional details on hyperparameter tuning, namely the depth, width and number of training epochs for each neural network, are given in Supplement B.

		Input	Depth	Width	Output	Output Layer
Forward NN	Blood Clot	2	3	50	100	Linear
	Myocardium	17	4	100	100 / 50	Linear
NNR	Blood Clot	21	3	20	2	Sigmoid
	Myocardium	174	4	50	8	Sigmoid

2.5. Direct inverse approach with neural network regressor (NNR)

In our second machine learning-based strategy, we trained a neural network regressor (NNR) to replace the entire parameter identification process. That is, instead of using metamodels to accelerate the least squares-based iterative optimization, we trained the neural network regressor to directly provide the Ogden and Holzapfel parameters given the experimental dataset as inputs, see Figure 3C-D. Here we used the same synthetic data as above for training and validation. And here, too, we used the PyTorch framework (V1.9.0) to train, validate, and test the NNR and chose network parameters according to detailed sensitivity analyses, see Table 1 and Supplement B.

3. Results

We present our results by first reporting the synthetic data-based training and validation errors of the GPR, NN, and NNR models with respect to the number of training samples. After training, decisions about each model architecture were made by evaluating their performance on synthetic validation datasets; that is, datasets that were different from the training datasets. Then, we tested the performance of our trained metamodels on, yet separate, synthetic test datasets. To this end, we split our total synthetic datasets into 80% training data, 10% validation data, and 10% testing data. After demonstrating the efficacy of our approach on synthetic test data, we used the LS GPR/NN and NNR approaches independently to identify the material parameters of our experimental datasets and then compared the results against the gold-standard LS FEM solutions.

3.1. Training and Validation

Figure 4A,B illustrates the training and validation error of the GPR metamodels, where we used target noise hyperparameter of α = 1e — 8 and α = 0.1 for the blood clot (Ogden) and right ventricular myocardium (Holzapfel) case, respectively. The training and validation errors for the blood clot GPR metamodel converged for less than 3,000 training samples to a mean absolute error (MAE) of less than 1e — 4 kPa. In contrast, the training and validation errors for the myocardium GPR metamodel remained large for 3,000 training samples. Additionally, both models differed significantly in the necessary training time. While the blood clot GPR metamodel required 35 minutes for training with 3,000 samples, the myocardium GPR metamodel required 166 minutes; both on a 36-core CPU at 2.20 GHz. Given the significant cost of training the myocardium GPR metamodel and the resulting impracticability of the approach, we did not expand our training set for the Holzapfel material model and right ventricular myocardium dataset.

Fig. 4.

Learning curves and training time of our machine learning approaches. (A-B) Forward Gaussian Process Regression (GPR) metamodels of blood clot and right ventricular myocardium in simple shear. (C-D) Corresponding curves for the forward Neural Network (NN) metamodels. (E-F) Neural Network Regressor (NNR) of blood clot and right ventricular myocardium in simple shear, respectively. On the right vertical axis we show the total training time, with all computations performed on 36-core CPU at 2.20 GHz. Please note that the y-axis of these plots are not identical, and that the performance shown here corresponds to synthetic data exclusively.

Figure 4C,D illustrates the training and validation error of the NN metamodels. For the blood clot NN metamodel, we chose a fully connected NN with ELU activation function on the input and the two hidden layers (width of 50 nodes/layer), followed by a linear output layer (see Table 1). In contrast, for the myocardium NN metamodel, we used 100 nodes/layer and three hidden layers. Given the significantly lower cost of training NNs over GPR models, we trained the blood clot and myocardium NN metamodels with 8,000 and 9,600 samples, respectively. In contrast to the GPR metamodels, both the NN metamodels show satisfying predictive accuracy with an MAE of approximately 1e — 2 kPa for the blood clot NN metamodel and an MAE of approximately 1e — 1 kPa for the myocardium NN metamodel. Note, that the improved predictive accuracy of the myocardium NN metamodel over the GPR metamodel stems from a larger training set of up to 9,600 samples over the previously 3,000 samples, which, in turn, was possible because of the lower training cost of the NN metamodels over the GPR metamodels.

Figure 4E,F illustrates the training and validation error for the NNR models. For the blood clot NNR model, we used a fully connected NN with two hidden layers and 20 nodes per layer (depth = 3, width = 20). In contrast, for the myocardium NNR model, we used a fully connected NN with three hidden layers and 50 nodes per layer (depth = 4, width = 50). For a summary of all neural network architectures see Table 1. Here, as in the case of the NN metamodels, we trained the blood clot NNR model with n = 8, 000 samples and the myocardium NNR model with n = 9, 600 samples. Please note that a direct comparison between MAEs between the metamodels and NNR models would not be sensible, as the former are trained and validated on the forward problem, i.e., predicting stress and strain from material parameters, while the latter are trained and validated against the inverse problem, i.e., predicting the material parameters from stress and strain. With that being said, the blood clot NNR appears to be well-trained after few thousand samples. While showing overall a comparable predictive accuracy as the blood clot NNR the myocardium NNR appears not yet fully trained even with 9,600 samples.

3.2. Testing

Figure 5 shows the test results for the blood clot GPR, NN, and NNR. The test results correspond well with the validation results. That is, the GPR and the NN metamodels accurately predict the blood clot material response. Specifically, they accurately predict the shear and normal stress under simple shear. This match is reflected in the one-to-one correspondence of the predicted stress and the known target stress of the synthetic data in Figure 5A,B. The NNR also performs well in predicting the material parameters of the Ogden material model with normalized root mean squared error (NMSE) between the predicted parameters and the ground truth parameters of 99.96% and 99.94% for the parameters a and b, respectively.

Fig. 5.

Performance of the two machine learning approaches on n = 1,000 synthetic blood clot test samples.(A) Forward Gaussian Process Regression (GPR) metamodel trained with n = 3, 000 samples. (B) Forward Neural Network (NN) metamodel trained with n = 8, 000 samples. (C) Neural Network Regressor (NNR) framework also trained with n = 8, 000 samples. Please note that the performance shown here corresponds to synthetic data exclusively.

Figure 6 shows the test results for the myocardium NN metamodel. Note that we excluded the myocardium GPR metamodel from further consideration for its prohibitive training computational cost. Using the NN metamodel, we see generally good prediction of the right ventricular myocardial material response, with some discrepancies in quality between different modes. That is, there is generally good correspondence between the predicted shear/normal stresses under simple shear and the known target stresses of the synthetic data. However, some modes, such as the FSz mode, i.e., the normal stress in response to simple shear in the FS-plane, show some scatter away from a one-to-one correspondence. Overall, it appears that those modes that activate fibers and therefore result in higher stresses show larger scatter. Figure 7 shows the test results for the myocardium NNR. Here we note that the NNR. performs well in predicting the isotropic parameters of the Holzapfel model, i.e., a and b, that yield an NMSE of 99.72% and 99.51%, respectively. However, the NNR does not perform well on the anisotropic terms a_f, a_s, b_f, b_s, or the fiber coupling terms a_fs, b_fs. The latter show significant deviations from one-to-one correspondence with b_fs presenting a nearly random correlation between the predicted parameter and the target parameters yielding an NMSE of 27.81%.

Fig. 6.

Performance of the trained Neural Networks (NN) on n = 1,200 synthetic right ventricular myocardium test samples. The Neural Network (NN) metamodel was trained with n = 9, 600 synthetic samples. Note that we trained a separate NN for each testing mode for a total of 86, 400 finite element simulations. See Figure 2 for an explanation on the mode nomenclature. Please also note that the performance shown here corresponds to synthetic data exclusively.

Fig. 7.

Performance of the trained Neural Network Regressor (NNR) framework on n = 1, 200 synthetic right ventricular myocardium test samples. The Neural Network Regressor (NNR) framework was trained with n = 9, 600 synthetic samples. Note that we trained the NNR using simultaneously all 9, 600 samples per mode, for a total of 86, 400 finite element simulations. Please also note that the performance shown here corresponds to synthetic data exclusively.

3.3. Application to the Experimental Datasets

Table 2 summarizes the performances of the LS GPR, LS NN, and NNR approach to identifying the material parameters of blood clot from simple shear data. That is, we used the trained, validated, and tested blood clot GPR metamodel and NN metamodel, as well as the blood clot NNR to identify the material parameters of the Ogden material model from our experimental data. To test the accuracy of these parameters, we, in turn, applied the same parameters to forward finite element simulations of the simple shear problem that yield predicted stress-strain curves. We then compared those stress-strain curves to those measured in our experiments. We applied our three strategies to all 27 samples in our experimental dataset and representatively show the best, median, and worst results among those samples. For each case, we reported the NMSE against the experimental data and the accuracy loss in comparison to the gold-standard LS FEM approach. Note, we define accuracy loss as the relative NMSE change between the LS GPR/NN and NNR approach relative to the gold standard LS FEM approach. These data reflect our findings from the validation and testing steps against synthetic data. Specifically, the least squares approach making use of the GPR and NN metamodels performs nearly perfect when compared to the gold-standard with accuracy losses around zero even for the worst case. On the other hand, the NNR shows significant accuracy losses. Figure 8 visually compares the LS NN, LS GRP, and NNR-based stress-strain curves against the LS FEM-based stress-strain curves and the actual experimental data for a median fit. From these curves it becomes evident that our first strategy of replacing the finite element method in the least squares pipeline with GPR and NN metamodels works very well. The LS GPR and LS NN predicted material parameters yield highly accurate predictions for the stress-strain behavior of blood clot under simple shear. On the other hand, the NNR approach yields stress-strain curves that significantly deviate from the experimental data and the LS FEM-based stress-strain curves, even for the simpler blood clot application example.

Table 2.

Validation of Ogden parameter estimation against the blood clot experimental dataset for the best, typical (median), and worst overall fit samples. We compare the two Ogden parameters, a and b, as estimated with Least Squares regression with forward Finite Element Method solutions (LS FEM), Gaussian Process Regression (LS GPR), Neural Networks (LS NN), and the direct Neural Network Regressor (NNR). The normalized mean square error (NMSE) was calculated against the experimental data (please recall that a perfect fit yields an NMSE of 1). On the other hand, the accuracy loss was calculated against the LS FEM approach. See Supplement C for the full result table of all samples.

Sample	Method	a (Pa)	b (-)	NMSE (-)	Acc. Loss (%)
Best	LS FEM	657.78	16.17	0.981	0.00
	LS GPR	627.25	16.49	0.980	0.01
	LS NN	656.99	16.24	0.980	0.01
	NNR	91.94	26.35	0.904	7.86
Median	LS FEM	530.39	16.32	0.989	0.00
	LS GPR	527.16	16.36	0.989	0.00
	LS NN	558.05	16.03	0.989	0.01
	NNR	194.67	26.21	-0.272	127.47
Worst	LS FEM	847.24	15.38	0.988	0.00
	LS GPR	845.42	15.39	0.988	0.00
	LS NN	881.57	15.14	0.988	0.01
	NNR	398.96	29.56	-23.212	2449.85

Fig. 8.

Visual comparison of our two acceleration strategies for identifying Ogden model material parameters from experimental blood clot data. Performance of the Least Squares regression with Gaussian Process Regression metamodels (LS GPR), Neural Networks (LS NN), and direct Neural Network Regression (NNR) compared to the conventional Least Squares Finite Element Method (LS FEM). (A) Shear Stress and (B) Normal Stress. Note, these curves were created by first using the LS GPR/NN and NNR approaches to identify the optimal material parameters from our blood clot experimental dataset, and then using the identified parameters in forward finite element simulations of the simple shear problem of blood clot. Please see Fig. 3 for a schematic of this procedure.

Table 3 summarizes the performances of the LS NN and NNR approach to identifying the material parameters of right ventricular myocardium from simple shear data and confined tension/compression data. That is, we used the trained, validated, and tested myocardium NN metamodel and NNR to identify the material parameters of the Holzapfel material model from our experimental data. Then, we tested the accuracy of these parameters by applying the same parameters to forward finite element simulations of the simple shear/confined tension/compression problem that yield predicted stress-strain curves. We then compared those stress-strain curves to those measured in our experiments. We applied these strategies to all 11 samples in our experimental dataset and, again, representatively show the best, median, and worst results among those samples. Similarly to the blood clot data data, these results reflect our findings from the validation and testing steps against synthetic data. Specifically, the least squares approach making use of the NN metamodel performs well, albeit not as well as it performed on the blood clot data with accuracy losses as high as 49.5%. Moreover, the NNR shows unacceptable accuracy losses. Figure 9 visually compares the LS NN and NNR-based stress-strain curves against the LS FEM-based stress-strain curves and the actual experimental data for a median fit. From these curves it becomes evident again that our first strategy of replacing the finite element method in the least squares pipeline with the NN metamodel works well; in other words, only few modes show significant deviations between the LS FEM-based predictions and the LS NN-based predictions. However, here, similar to the blood clot experimental data, the NNR approach yields stress-strain curves that significantly deviate from the experimental data and the LS FEM-based stress-strain curves.

Table 3.

Validation of Holzapfel parameter estimation against the right ventricular myocardium experimental dataset for the best, typical (median), and worst overall fit samples. We compare the eight Holzapfel parameters as estimated with Least Squares regression using forward Finite Element Method simulations (LS FEM), Gaussian Process Regression (LS GPR), Neural Networks (LS NN), and the direct Neural Network Regressor (NNR). The normalized mean square error (NMSE) was calculated against the experimental data (please recall that a perfect fit yields an NMSE of 1). On the other hand, the accuracy loss was calculated against the LS FEM approach. See Supplement C for the full result table of all samples.

Subject	Method	a (Pa)	b (-)	af (Pa)	bf (-)	as (Pa)	bs (-)	afs (Pa)	bfs (-)	NMSE (-)	Acc. Loss (%)
Best	LS FEM	1928.4	9.29	3925.4	19.42	1592.0	0.00	1587.8	0.00	0.878	0.0
	LS NN	2065.4	11.04	11580.1	8.72	780.1	0.03	0.1	18.59	0.758	13.7
	NNR	2319.3	18.88	3215.9	27.24	410.0	24.20	162.8	29.96	0.275	68.7
Median	LS FEM	1238.8	10.28	487.6	29.14	610.2	0.00	0.0	0.00	0.781	0.0
	LS NN	1259.7	11.50	2418.6	15.31	31.7	16.72	102.2	9.39	0.701	10.3
	NNR	1121.8	16.64	2787.2	27.06	794.0	20.96	1445.4	29.29	-8.349	1168.4
Worst	LS FEM	726.6	7.80	17707.5	0.00	0.2	0.12	0.0	0.00	0.713	0.0
	LS NN	765.8	10.89	15542.2	0.03	219.3	11.65	0.3	10.41	0.360	49.5
	NNR	1835.2	14.49	13346.3	27.00	7997.8	17.04	680.2	19.04	-Inf	Inf

Fig. 9.

Visual comparison of our two acceleration strategies for identifying Holzapfel model material parameters from experimental right ventricular myocardium data. Performance of the Least Squares regression with forward Neural Networks (LS NN) and direct Neural Network Regression (NNR) compared to the conventional Least Squares Finite Element Method (LS FEM). Note, these curves were created by first using the LS NN and NNR approaches to identify the optimal material parameters from our right ventricular myocardium experimental dataset, and then using the identified parameters in forward finite element simulations of the simple shear problem of right ventricular myocardium. Please see Fig. 3 for a schematic of this procedure.

Finally, Table 4 presents our findings on the computational cost of our two strategies, including time required for training dataset generation, machine learning model training, and constitutive model parameter identification. For context, we note that a single forward finite element simulation for blood clot with the Ogden model requires 39.6 sec to complete, while running a full set of simulation of right ventricular myocardium with the Holzapfel model requires 70.3 seconds. Note, a full set of simulations of right ventricular myocardium comprises running 9 simulations in parallel, one for each mode. All times were clocked on our workstation with a 36-core CPU at 2.20 GHz. Given an average of twenty iterations to reach convergence, the LS FEM approach leads total execution times of 40 min and 211 min for the blood clot and right ventricular myocardium cases, respectively. Relative to this significant cost, the cost of our LS NN strategy is negligible. That is, after training the LS NN approach requires 0.1 sec and 3.8 sec for the blood clot and right ventricular myocardium samples, respectively, Similarly, the NNR requires 0.005 sec and 0.01 sec, for each model, respectively. All speeds were clocked on the same computer.

Table 4.

Typical computational cost of our strategies. All times were clocked on our workstation with a 36-core CPU at 2.20 GHz. The “Single Run” times are per data set; for the Ogden model that is one simulation, for the Holzapfel model that are 9 simulation for the nine different test modes. The “Data Generation” comprises 10,000 runs for the Ogden model and 12,000 runs for the Holzapfel model. Least Squares (LS), Finite Element Method (FEM), Neural Network (NN), Neural Network Regressor (NNR).

Method		Material model
		Ogden	Holzapfel
Forward FEM	Single Run	5.0 sec	70.3 sec
	Data Generation	13 hrs	234 hrs
LS FEM	Param. Ident.	40 min	211 min
LS NN	Training	4 min	45.2 min
	Param. Ident.	0.1 sec	3.8 sec
NNR	Training	4.5 min	13.6 min
	Param. Ident.	≪ 1 sec	≪ 1 sec

4. Discussion

We set out to answer the question whether machine learning can accelerate soft material parameter identification from complex mechanical test data. We were motivated by our recent experience that parameter identification via least squares-based inverse analysis with the finite element method can be very computationally expensive, particularly when applied to a large number of experimental samples.

We note briefly that many previous studies showcasing the efficacy of machine learning for material characterization have validated their results with synthetic data alone [3]. Though these studies are important first steps, we hold that it is critical to also evaluate the efficacy of these methods on experimental datasets because, unlike synthetic data generated via finite element analysis, experimental data will deviate from the assumptions inherent to the ultimately phenomenological constitutive law. Because we do not have a good statistical model for these deviations, the best approach for understanding how they will influence the results of our approach is to evaluate efficacy on experimental data directly – in particular it is important to evaluate on the specific type of experimental data that we are interested in, i.e., soft tissue.

To answer the question we posed in our title, we tested two fundamentally different strategies. The first strategy replaced the forward finite element simulations in our least squares-based inverse analysis with GPR and NN metamodels. Our second strategy replaced the entire inverse pipeline with a direct NNR-based framework. We tested both strategies against two experimental datasets of which one was isotropic and homogeneous – simple shear of blood clot – and the other of which was anisotropic and heterogeneous – simple shear and confined tension/compression of right ventricular myocardium. For these datasets, we set out to identify the parameters for the Ogden material model and the Holzapfel material model, respectively.

In short, we found that our first strategy yielded excellent results for the relatively simple blood clot problem. Specifically, we tested GPR and NN metamodels to replace the finite element simulations in our least squares-based identification of Ogden material parameters from simple shear testing data. Both metamodels resulted in highly accurate material parameters when compared to our gold-standard least squares-based inverse analysis using the finite element method. While our second strategy was less accurate than the first strategy, it too resulted in material parameters that compared favorably with the gold-standard. Given that the evaluation cost of the GPR and NN metamodels is minimal, the NNR approach provides marginal time savings over the LS GPR or LS NN approach. Hence, given its higher accuracy, we propose using our first strategy for similar problems.

When we applied our first strategy to the more complex problem of the right ventricular myocardium, we achieved less satisfying results. First, we found the GPR metamodeling approach to be prohibitively expensive given the high dimensional feature space of this problem (17 versus 2 in the case of the blood clot dataset). This finding may not be surprising given the known limitations of memory requirements and cost of training GPRs [7]. While we tried to overcome this limitation via a Bayesian optimization approach, the resulting small gains did not make up for the GPRs high cost in comparison to the NN approach [4,27] (see Supplement C). We therefore abandoned the GPR approach and only tested the NN metamodel. While this approach yielded reasonable results with errors on the order of 10% in the best case, it also performed poorly on some data with errors as large as 49.5%. In those cases where errors were large, we noticed that they arose primarily from the mechanical modes that activated the exponential fiber behaviors. On the other hand, the isotropic terms were well represented by the NN metamodel. Whether this error is prohibits the use of this strategy remains to be studied in that its errors may wane in the light of other errors, such as experimental errors, numerical errors, discretization errors etc. In contrast, we can un-equivocally say that our second strategy failed for right ventricular myocardium. Here, errors in the best case scenario were as large as 68.7% and in the worst case as large as thousands of percent. It is possible that our second approach failed because of the non-uniqueness, i.e., ill-posedness, of the inverse problem. That is, that multiple material/sample parameters may yield the same stress-strain curves.

We do not believe that our failed attempt of using the NNR approach to learning the entire inverse pipeline is proof that this inverse pipeline couldn’t be learned without iteration in general. Rather, it is evidence that this straightforward first attempt is not effective for these data. Critically, future attempts that either use physics-informed implementations with additional constraints or better address the discrepancy between the experimental and synthetic data may overcome the poor performance as found in our work.

Our work is naturally subject to limitations. Some were mentioned above, such as our somewhat naive NNR approach to learning the inverse pipeline for complex materials. Additionally, it should be noted that our specific implementation and trained networks are somewhat specific to our particular dataset. For example, our parameter estimation pipeline assumes the existence of a complete simple shear and confined tension-compression dataset up to fixed strain magnitudes. There-fore, any change either to maximum testing strain or the loading modes necessitates corresponding changes to the pipeline. It should also be noted that, when evaluating the cost of machine learning-based approaches, the time for synthetic data generation must be considered. Indeed, the cost of generating synthetic data for training, validation, and testing was very expensive and – at this point in time – has far exceeded our potential time-savings during the actual parameter identification. In the future, this initial investment will be off-set only through repeated use of our trained models, which, given the high specificity to our particular datasets, is not necessarily given. With that being said, our work is explorative in nature and hopefully contributes to fu-ture, more efficient, more accurate, and more versatile approaches that truly accelerate soft material parameter identification from complex mechanical data.

In conclusion, we tested whether machine learning may accelerate soft tissue material parameter identification from complex mechanical data. Our answer to this question is: probably. For the simple case of the isotropic, homogeneous blood clot under simple shear, we succeeded in providing an accurate, machine learning accelerated identification of the Ogden parameters. Similarly, for the complex case of the anisotropic, heterogeneous right ventricular myocardium, we provided mostly accurate, machine learning accelerated identification of the Holzapfel parameters. In both cases, we recommend using a least squares-based inverse approach in which metamodels replace finite element solutions. Whether to chose this strategy over the classic finite element-based one should depend on the frequency with which the user likely conducts such analysis given the high initial investment into the machine learning approaches. Finally, and most importantly, we have provided our vast experimental and synthetic dataset for others to take advantage of our initial investment and to improve upon our admittedly rudimentary first attempt. We look forward to future advances that either incrementally improve upon these methods or take entirely different approaches to working with these data.

Fig. 2.

Test protocols of our two experimental datasets. (A) Simple shear data of cuboid blood clot samples that were created by (i) coagulating blood in cuboid molds, (ii) mounting the samples to pin stubs via hook-and-loop material, and (iii) shearing the test samples in one direction. (B) Simple shear data of cuboid right ventricular myocardial samples that were created by (i) excising samples from ovine right ventricles, (ii) mounting the samples on pin stubs, (iii) loading the test samples in multiple directions and in multiple orientations. (C) Note, simple shear is not so simple and results in heterogeneous deformation fields as illustrated in a representative finite element simulation of the simple shear problem.

Data Availability

All experimental and synthetic data, as well as all Python code is available for open use under: https://github.com/SoftTissueBiomechanicsLab/ML-soft-material-parameters.