Democratizing biomedical simulation through automated model discovery and a universal material subroutine

Abstract

Personalized computational simulations have emerged as a vital tool to understand the biomechanical factors of a disease, predict disease progression, and design personalized intervention. Material modeling is critical for realistic biomedical simulations, and poor model selection can have life-threatening consequences for the patient. However, selecting the best model requires a profound domain knowledge and is limited to a few highly specialized experts in the field. Here we explore the feasibility of eliminating user involvement and automate the process of material modeling in finite element analyses. We leverage recent developments in constitutive neural networks, machine learning, and artificial intelligence to discover the best constitutive model from thousands of possible combinations of a few functional building blocks. We integrate all discoverable models into the finite element workflow by creating a universal material subroutine that contains more than 60,000 models, made up of 16 individual terms. We prototype this workflow using biaxial extension tests from healthy human arteries as input and stress and stretch profiles across the human aortic arch as output. Our results suggest that constitutive neural networks can robustly discover various flavors of arterial models from data, feed these models directly into a finite element simulation, and predict stress and strain profiles that compare favorably to the classical Holzapfel model. Replacing dozens of individual material subroutines by a single universal material subroutine—populated directly via automated model discovery—will make finite element simulations more user-friendly, more robust, and less vulnerable to human error. Democratizing finite element simulation by automating model selection could induce a paradigm shift in physics-based modeling, broaden access to simulation technologies, and empower individuals with varying levels of expertise and diverse backgrounds to actively participate in scientific discovery and push the boundaries of biomedical simulation.

Motivation

Computational simulations play a pivotal role in understanding and predicting the biomechanical factors of a wide variety of cardiovascular diseases [7, 63, 64]. In vascular medicine, knowing the precise stress and strain fields across the vascular wall is critical for understanding the formation, growth, and rupture of aneurysms [27]; for identifying high-risk regions of plaque formation, rupture, and thrombosis [49]; and for optimizing stent materials, structure, and deployment in aortic stenosis [30]. The accurate simulation of cardiovascular disease is a complex challenge that requires collective efforts across a multitude of disciplines including cardiovascular medicine, applied mathematics, biomechanics, and computer science [44]. Clearly, it is impossible that everyone has a specialized training in material modeling and an in-depth knowledge in finite element simulation [9]. However, selecting a poor material model does not only jeopardize the success of the entire simulation, but can have life-threatening consequences for the patient. The objective of this manuscript is to explore whether and how we can automate the process of material modeling and its integration into a finite element analysis.

Constitutive neural networks autonomously discover material models from data Throughout the past couple of years, two alternative strategies have emerged to discover models directly from data: non-interpretable and interpretable approaches. Non-interpretable approaches closely follow traditional neural networks and typically discover functions of rectified linear unit, softplus, or hyperbolic tangent type [17]. Representatives of this category are tensor basis Gaussian process regression [13, 14], plain constitutive artificial neural networks [25, 33], and neural ordinary differential equations [57, 59]. These approaches are straightforward to implement, provide an excellent approximation of the data, and can be integrated manually within finite element software packages [17, 58]. However, the models and parameters that these methods learn are non-interpretable, meaning they provide little insight into the underlying material behavior [47]. Interpretable approaches discover models that are made up of a library of functional building blocks that resemble traditional constitutive models. Representatives of this category are sparse regression [10, 11], symbolic regression [2], and custom-designed constitutive neural networks [34, 54], the method we adopt here. These approaches a priori satisfy material objectivity, material symmetry, thermodynamic consistency, and polyconvexity [29], and autonomously discover free energy functions that feature popular constitutive terms and parameters with a clear physical interpretation. By design, all three translate smoothly into user material subroutines for a finite element analysis [1], and we could adopt any of these interpretable approaches. Here, for illustrative purposes, we use a custom-designed constitutive neural network to discover the best constitutive model for aortic tissue from thousands of possible combinations of a few functional building blocks [31]. We integrate all discoverable models into the finite element workflow by creating a universal material subroutine that contains 2$_{}^{16}$ = 65,536 constitutive models, made up of 16 individual terms [35]. We train and test our network with biaxial extension tests of the medial and adventitial layers of a human aorta, and discover various flavors of arterial models from the experimental data [24, 40].

Model discovery is a non-convex optimization problem with multiple local minima Unfortunately, in practice, the sixteen terms of the network tend to span a parameter space with multiple local minima, the network often discovers non-sparse solutions, and model discovery can become non-unique [38]. A successful strategy to address these limitations is $L_p$ regularization [10], a powerful method to shrink the parameter space by penalizing the loss function with a penalty term that consists of the $L_p$ norm of the parameter vector, weighted by a penalty parameter [6]. To illustrate the potential of $L_p$ regularization, we first use $L_0$ regularization, or discrete combinatorics [12], to discover the best-in-class one- and two-term models [32], and then use $L_1$ regularization, or lasso [61], to systematically reduce the number of terms. This allows us to discover a suite of different models for the media and for the adventitia, and learn about their structural and mechanical differences [22].

Mechanical differences in media and adventitia modulate the pathogenesis of cardiovascular disease Understanding the subtle structural and mechanical distinctions between the media and adventitia layers of the aorta is crucial for comprehending vascular health and disease [27]. The media is rich in smooth muscle cells and elastin fibers to provide elasticity and contractility, and facilitate hemodynamic function, while the adventitia is made up primarily of fibroblasts and collagen fibers to provide structural support [23]. Disruptions in the delicate structural and mechanical balance between the media and the adventitia contribute to pathological conditions such as aortic aneurysms, thrombosis, or stenosis [21]. Mechanical heterogeneity plays a pivotal role in the pathogenesis of these conditions: Alterations in the isotropic extracellular matrix can lead to vessel dilation, while changes in the anisotropic collagen content can affect overall integrity. Finite element models that account for layer-specific structural and mechanical properties are critical to accurately simulate disease progression, assess rupture risk, and develop targeted interventions [15]. A comprehensive understanding of the interplay between the layers of the aorta can inform strategies for early detection, risk stratification, and tailored therapeutic approaches in the benefit of cardiovascular health.

Automated model discovery does democratize finite element simulations For more than half a century, scientists have developed constitutive models for biological tissues [20] and today’s finite element packages offer large libraries of material models to choose from [1, 3, 36, 60, 63]. However, the scientific criteria for appropriate model selection remain highly subjective and prone to user bias. Importantly, the objective of our study is not to discover yet another marginally better constitutive model. Instead, our goal is to prototype an intelligent and automated workflow-from experiment to simulation- to robustly discover constitutive models from data [31], feed these models directly into a finite element simulation [46], and reliably predict physically meaningful stress and strain profiles. If successful, this new technology could make physics-based simulation more user-friendly, more accessible, and less vulnerable to human error.

Experiment

We begin by briefly describing our experimental data from the healthy human aorta of a 56-year-old male [40], collected as an intact tube within 24 h of death, and stored in saline solution [24]. The sample was cleared, dehydrated in ethanol, and stored in benzyl alcohol-benzyl benzoate, all at room temperature. For the structural characterization, second harmonic generation imaging was used to quantify two microstructural parameters: the collagen fiber angle $\alpha$ with respect to the circumferential direction, and the fiber dispersion $\kappa$ [50]. In the circumferential-axial plane, the median collagen fiber angle was $\alpha =\pm 7.{00}^{\circ }$ with a fiber dispersion of $\kappa =0.0737$ for the media and $\alpha =\pm 66.{78}^{\circ }$ with a fiber dispersion of $\kappa =0.0909$ for the adventitia [24, 41].

For the mechanical characterization, a squared $20\times 20$ mm sample of the media and a cruciform-shaped $35\times 35$ mm sample with a squared $5\times 5$ mm center testing region of the adventitia were manually separated from the remaining tissue and tested in biaxial extension while submerged in saline solution at 37 $_{}^{\circ }$C. To ensure a homogeneous deformation state, both samples were mounted with the collagen fibers oriented symmetrically with respect to the two loading directions, and loaded at five different stretch ratios, ${\lambda }_{\text{cir}}:{\lambda }_{\text{axl}}=\{1.20:1.10,1.20:1.15,1.20:1.20,1.15:1.20,1.10:1.20\}$. Tables 1 and 2 summarize the resulting five pairs of datasets, $\{{\lambda }_{\text{cir}},{\sigma }_{\text{cir}}\}$ and $\{{\lambda }_{\text{axl}},{\sigma }_{\text{axl}}\}$, for the media and for the adventitia [40]. Figure 1 illustrates the circumferential and axial stress-stretch relations of the media, left, and of the adventitia, right, of the 56-year-old healthy human aorta.

Table 1.

Biaxial testing of human aortic media. Samples are stretched biaxially in the circumferential and axial directions at ${\lambda }_{\text{cir}}$ and ${\lambda }_{\text{axl}}$ at five different stretch ratios. The mean fiber angle is $\pm 7.{00}^{\circ }$ against the circumferential direction. Stresses are reported as ${\sigma }_{\text{cir}}$ and ${\sigma }_{\text{axl}}$, see Fig. 1 [24, 40]

Table 2.

Biaxial testing of human aortic adventitia. Samples are stretched biaxially in the circumferential and axial directions at ${\lambda }_{\text{cir}}$ and ${\lambda }_{\mathrm{axl}}$ at five different stretch ratios. The mean fiber angle is $\pm 66.{78}^{\circ }$ against the circumferential direction. Stresses are reported as ${\sigma }_{\text{cir}}$ and ${\sigma }_{\text{axl}}$, see Fig. 1 [24, 40]

Fig. 1.

Biaxial testing of human aortic media and adventitia. Samples are stretched biaxially in the circumferential and axial directions at ${\lambda }_{\text{cir}}$ and ${\lambda }_{\text{axl}}$ at five different stretch ratios, from dark red to dark blue. The mean fiber angles of the media and adventitia are $\pm 7.{00}^{\circ }$ and $\pm 66.{78}^{\circ }$ against the circumferential direction. Stresses are reported as ${\sigma }_{\text{cir}}$ and ${\sigma }_{\text{axl}}$, see Tables 1 and 2 [24, 40]. (Color figure online)

Model

Kinematics During testing, particles $\mathit{X}$ of the undeformed sample map to particles $\mathit{x}=\mathit{\phi }(\mathit{X})$ of the deformed sample via the deformation map $\mathit{\phi }$. Its gradient with respect to the undeformed coordinates $\mathit{X}$ is the deformation gradient, $\mathit{F}={\mathrm{\nabla }}_{\mathit{X}}\mathit{\phi }$. Its spectral representation introduces the principal stretches ${\lambda }_i$ and the principal directions ${\mathit{N}}_i$ and ${\mathit{n}}_i$ in the undeformed and deformed configurations, where $\mathit{F}·{\mathit{N}}_i={\lambda }_i{\mathit{n}}_i$ , and

$$\begin{array}{c}\hfill \mathit{F}={\mathrm{\nabla }}_{\mathit{X}}\mathit{\phi }=\sum _{i=1}^3{\lambda }_i{\mathit{n}}_i\otimes {\mathit{N}}_i.\end{array}$$ 1

We assume that the vascular tissue has two pronounced fiber directions [22], ${\mathit{n}}_{01}$ and ${\mathit{n}}_{02}$, with unit length, $||{\mathit{n}}_{01}||=1$ and $||{\mathit{n}}_{02}||=1$, in the undeformed configuration, and assume that these directions map affinely onto the pronounced directions [56], ${\mathit{n}}_1=\mathit{F}·{\mathit{n}}_{01}$ and ${\mathit{n}}_2=\mathit{F}·{\mathit{n}}_{02}$, with fiber stretches, $||{\mathit{n}}_1||={\lambda }_{n1}$ and $||{\mathit{n}}_2||={\lambda }_{n2}$, in the deformed configuration. We characterize its deformation state through the three principal invariants $I_1$, $I_2$, $I_3$, and six additional invariants $I_4$, $I_5$, $I_6$, $I_7$, $I_8$, $I_9$ [52],

2

A perfectly incompressible material has a constant Jacobian equal to one, $I_3=J^2=1$, the ninth invariant is constant by definition, $I_9=$ const., and the set of independent invariants reduces to seven, $I_1$, $I_2$, $I_4$, $I_5$, $I_6$, $I_7$, $I_8$.

Biaxial extension For the special homogeneous deformation of biaxial extension, we apply stretches ${\lambda }_1\ge 1$ and ${\lambda }_2\ge 1$ in the circumferential and longitudinal directions, and adopt the incompressibility condition, $I_3={\lambda }_1^2{\lambda }_2^2{\lambda }_3^2=1$, to express the stretch in the radial direction, ${\lambda }_3={({\lambda }_1{\lambda }_2)}^{-1}\le 1$. We assume that the fiber pairs, initially oriented at an angle $\pm \alpha$ to the circumferential direction, ${\mathit{n}}_0={[cos(\alpha ),\pm sin(\alpha ),0]}^{\text{t}}$, remain symmetric with respect to the stretch directions, such that the deformation remains homogeneous and shear free, and the deformation gradient,

$$\begin{array}{c}\hfill \mathit{F}=\text{diag}\{{\lambda }_1,{\lambda }_2,{({\lambda }_1{\lambda }_2)}^{-1}\}\end{array}$$ 3

remains diagonal at all times. We now use the the principal stretches ${\lambda }_1$ and ${\lambda }_2$ to express the invariants (2),

$$\begin{array}{cc}\hfill I_1& ={\lambda }_1^{+2}+{\lambda }_2^{+2}+{({\lambda }_1{\lambda }_2)}^{-2}\hfill \\ \hfill I_2& ={\lambda }_1^{-2}+{\lambda }_2^{-2}+{({\lambda }_1{\lambda }_2)}^{+2}\hfill \\ \hfill I_4& ={\lambda }_1^2{\text{cos}}^2\alpha +{\lambda }_2^2{\text{sin}}^2\alpha =I_6\hfill \\ \hfill I_5& ={\lambda }_1^4{\text{cos}}^2\alpha +{\lambda }_2^4{\text{sin}}^2\alpha =I_7\hfill \\ \hfill I_8& =({\lambda }_1^2{\text{cos}}^2\alpha -{\lambda }_2^2{\text{sin}}^2\alpha )\text{cos}(2\alpha )\hfill \end{array}$$ 4

and their derivatives,

$$\begin{array}{cc}\hfill {\partial }_{\mathit{F}}{I}_1& =2\text{diag}\{{\lambda }_1,{\lambda }_2,{({\lambda }_1{\lambda }_2)}^{-1}\}\hfill \\ \hfill {\partial }_{\mathit{F}}{I}_2& =2\text{diag}\{({\lambda }_1{\lambda }_2^2+{\lambda }_1^{-1}{\lambda }_2^{-2}),\hfill \\ \hfill & ({\lambda }_1^2{\lambda }_2+{\lambda }_1^{-2}{\lambda }_2^{-1}),({\lambda }_1{\lambda }_2^{-1}+{\lambda }_1^{-1}{\lambda }_2)\}\hfill \\ \hfill {\partial }_{\mathit{F}}{I}_4& =2\text{diag}\{{\lambda }_1{cos}^2\alpha ,{\lambda }_2{sin}^2\alpha ,0\}={\partial }_{\mathit{F}}{I}_6\hfill \\ \hfill {\partial }_{\mathit{F}}{I}_5& =2\text{diag}\{2{\lambda }_1^3{cos}^2\alpha ,2{\lambda }_2^3{sin}^2\alpha ,0\}={\partial }_{\mathit{F}}{I}_7\hfill \\ \hfill {\partial }_{\mathit{F}}{I}_8& =2\text{diag}\{{\lambda }_1{cos}^2\alpha ,-{\lambda }_2{sin}^2\alpha ,0\}cos(2\alpha ).\hfill \end{array}$$ 5

We conclude that the case of biaxial extension probes both fiber directions equally, $I_4=I_6$ and $I_5=I_7$.

Constitutive equations A hyperelastic material satisfies the second law of thermodynamics, and its Piola stress $\mathit{P}=\partial \psi (\mathit{F})/\partial \mathit{F}$ is the derivative of the free energy $\psi (\mathit{F})$ with respect to the deformation gradient $\mathit{F}$. A perfectly incompressible hyperelastic material uses this stress definition modified by a pressure term, $-p{\mathit{F}}^{\text{-t}}$ [37],

$$\begin{array}{c}\hfill \mathit{P}=\frac{\partial \psi }{\partial \mathit{F}}-p{\mathit{F}}^{-\text{t}},\end{array}$$ 6

where the hydrostatic pressure, $p=-\frac{1}{3}\mathit{P}:\mathit{F}$, acts as a Lagrange multiplier that we determine from the boundary conditions. We express the free energy function in terms of the seven invariants, $\psi (I_1,I_2,I_4,I_5,I_6,I_7,I_8)$, and obtain the following explicit expression for the Piola stress,

$$\begin{array}{cc}\hfill \mathit{P}& {\displaystyle =\frac{\partial \psi }{\partial I_1}\frac{\partial I_1}{\partial \mathit{F}}+\frac{\partial \psi }{\partial I_2}\frac{\partial I_2}{\partial \mathit{F}}+\frac{\partial \psi }{\partial I_4}\frac{\partial I_4}{\partial \mathit{F}}+\frac{\partial \psi }{\partial I_5}\frac{\partial I_5}{\partial \mathit{F}}}\hfill \\ \hfill & {\displaystyle +\frac{\partial \psi }{\partial I_6}\frac{\partial I_6}{\partial \mathit{F}}+\frac{\partial \psi }{\partial I_7}\frac{\partial I_7}{\partial \mathit{F}}+\frac{\partial \psi }{\partial I_8}\frac{\partial I_8}{\partial \mathit{F}}-p{\mathit{F}}^{-\text{t}}.}\hfill \end{array}$$ 7

Biaxial extension For homogeneous and shear free biaxial extension, the Piola stress $\mathit{P}$ remains diagonal at all times,

$$\begin{array}{c}\hfill \mathit{P}=\text{diag}\{P_{11},P_{22},0\},\end{array}$$ 8

and we can use the zero-normal-stress condition, $P_{33}=0$, to determine the pressure p,

$$\begin{array}{c}\hfill p=\frac{2}{{\lambda }_1^2{\lambda }_2^2}\frac{\partial \psi }{\partial I_1}+\left[\frac{2}{{\lambda }_1^2},+,\frac{2}{{\lambda }_2^2}\right]\frac{\partial \psi }{\partial I_2}.\end{array}$$ 9

Equation (7) then provides explicit analytical expressions for the Piola stresses $P_1$ and $P_2$ in terms of the stretches ${\lambda }_1$ and ${\lambda }_2$, In what follows, we assume that the mechanical behavior of the two fiber families is identical and combine their effects in the fourth and fifth invariants, $I_4$ and $I_5$. In addition, we assume that the two fiber families do not interact and drop the eighth invariant $I_8$ [39]. This results in the following expressions,

$$\begin{array}{cc}\hfill P_1& {\displaystyle =2\left[{\lambda }_1,-,\frac{1}{{\lambda }_1^3{\lambda }_2^2}\right]\frac{\partial \psi }{\partial I_1}+2\left[{\lambda }_1,{\lambda }_2^2,-,\frac{1}{{\lambda }_1^3}\right]\frac{\partial \psi }{\partial I_2}}\hfill \\ \hfill & {\displaystyle +4{\lambda }_1{cos}^2\alpha \frac{\partial \psi }{\partial I_4}+8{\lambda }_1^3{cos}^2\alpha \frac{\partial \psi }{\partial I_5}}\hfill \\ \hfill P_2& {\displaystyle =2\left[{\lambda }_2,-,\frac{1}{{\lambda }_1^2{\lambda }_2^2}\right]\frac{\partial \psi }{\partial I_1}+2\left[{\lambda }_1^2,{\lambda }_2,-,\frac{1}{{\lambda }_2^3}\right]\frac{\partial \psi }{\partial I_2}}\hfill \\ \hfill & {\displaystyle +4{\lambda }_2{sin}^2\alpha \frac{\partial \psi }{\partial I_4}+8{\lambda }_2^3{sin}^2\alpha \frac{\partial \psi }{\partial I_5}}\hfill \end{array}$$ 10

Finally, we translate these nominal stresses $P_1$ and $P_2$ into the true stress ${\sigma }_1$ and ${\sigma }_2$,

$$\begin{array}{cc}\hfill {\sigma }_1& {\displaystyle =2\left[{\lambda }_1^2,-,\frac{1}{{\lambda }_1^2{\lambda }_2^2}\right]\frac{\partial \psi }{\partial I_1}+2\left[{\lambda }_1^2,{\lambda }_2^2,+,\frac{1}{{\lambda }_1^2}\right]\frac{\partial \psi }{\partial I_2}}\hfill \\ \hfill & {\displaystyle +4{\lambda }_1^2{cos}^2\alpha \frac{\partial \psi }{\partial I_4}+8{\lambda }_1^4{cos}^2\alpha \frac{\partial \psi }{\partial I_5}}\hfill \\ \hfill {\sigma }_2& {\displaystyle =2\left[{\lambda }_2^2,-,\frac{1}{{\lambda }_1^2{\lambda }_2^2}\right]\frac{\partial \psi }{\partial I_1}+2\left[{\lambda }_1^2,{\lambda }_2^2,+,\frac{1}{{\lambda }_2^2}\right]\frac{\partial \psi }{\partial I_2}}\hfill \\ \hfill & {\displaystyle +4{\lambda }_2^2{sin}^2\alpha \frac{\partial \psi }{\partial I_4}+8{\lambda }_2^4{sin}^2\alpha \frac{\partial \psi }{\partial I_5},}\hfill \end{array}$$ 11

that are reported in the experiment [40].

Constitutive neural network To discover the best model and parameters to explain the biaxial testing data, we adopt the concept of constitutive neural networks, a special class of neural networks that satisfy the conditions of thermodynamic consistency, material objectivity, material symmetry, perfect incompressibility, polyconvexity, and physical constraints by design [31].

Figure 2 illustrates our neural network with two hidden layers and eight and sixteen nodes [35]. The first layer generates powers $(\circ )$ and ${(\circ )}^2$ of the network input, the four invariants $I_1$, $I_2$, $I_4$, $I_5$, and the second layer applies the identity, $(\circ )$ and the exponential function $(\text{exp}(\circ ))$ to these powers. The free energy function of this networks takes the following explicit form,

$$\begin{array}{cc}\hfill \psi & =w_{2,1}{w}_{1,1}[I_1-3]+w_{2,2}[exp(w_{1,2}[I_1-3])-1]\hfill \\ \hfill & +w_{2,3}{w}_{1,3}{[I_1-3]}^2+w_{2,4}[exp(w_{1,4}{[I_1-3]}^2)-1]\hfill \\ \hfill & +w_{2,5}{w}_{1,5}[I_2-3]+w_{2,6}[exp(w_{1,6}[I_2-3])-1]\hfill \\ \hfill & +w_{2,7}{w}_{1,7}{[I_2-3]}^2+w_{2,8}[exp(w_{1,8}{[I_2-3]}^2)-1]\hfill \\ \hfill & +w_{2,9}{w}_{1,9}[I_4-1]+w_{2,10}[exp(w_{1,10}[I_4-1])-1]\hfill \\ \hfill & +w_{2,11}{w}_{1,11}{[I_4-1]}^2+w_{2,12}[exp(w_{1,12}{[I_4-1]}^2)-1]\hfill \\ \hfill & +w_{2,13}{w}_{1,13}[I_5-1]+w_{2,14}[exp(w_{1,14}[I_5-1])-1]\hfill \\ \hfill & +w_{2,15}{w}_{1,15}{[I_5-1]}^2+w_{2,16}[exp(w_{1,16}{[I_5-1]}^2)-1],\hfill \end{array}$$ 12

corrected by the pressure term $\psi =\psi -p[J-1]$. Its derivatives with respect to the four invariants,

13

complete the definition of the principal Cauchy stresses in equations (11). The network has two times sixteen weights $\mathbf{w}$, which we constraint to always remain non-negative, $\mathbf{w}\ge \mathbf{0}$. We learn the network weights $\mathbf{w}$ by minimizing a loss function L that penalizes the error between model and data. We characterize this error as the mean squared error, the $L_2$-norm of the difference between the stresses predicted by the network model, ${\sigma }_1$, ${\sigma }_2$, and the experimentally measured stresses, ${\widehat{\sigma }}_{1,i}$, ${\widehat{\sigma }}_{2,i}$, divided by the number of training points $n_{\text{trn}}$, and add a penalty term, $\alpha ||\mathbf{w}||_p^p$, to allow for $L_p$ regularization,

$$\begin{array}{cc}\hfill L& {\displaystyle =\frac{1}{n_{\text{trn}}}\sum _{i=1}^{n_{\text{trn}}}||{\sigma }_1({\lambda }_{1,i},{\lambda }_{2,i})-{\widehat{\sigma }}_{1,i}{||}^2}\hfill \\ \hfill & {\displaystyle +\frac{1}{n_{\text{trn}}}\sum _{i=1}^{n_{\text{trn}}}||{\sigma }_2({\lambda }_{1,i},{\lambda }_{2,i})-{\widehat{\sigma }}_{2,i}{||}^2+\alpha ||\mathbf{w}||_p^p\to \text{min}.}\hfill \end{array}$$ 14

Here $\alpha \ge 0$ is a non-negative penalty parameter and $||\mathbf{w}||_p^p={\sum }_{i=1}^{\text{par}}{|w_i|}^p$ is the $L_p$ norm of the vector of the network weights $\mathbf{w}$. We train the network by minimizing the loss function (14) using the ADAM optimizer, a robust adaptive algorithm for gradient-based first-order optimization.

Fig. 2.

Constitutive neural network. Perfectly incompressible hyperelastic constitutive neural network with two hidden layers to approximate the free-energy function $\psi (I_1,I_2,I_4,I_5)$ as a function of the invariants of the deformation gradient $\mathit{F}$ using sixteen terms. The first layer generates powers $(\circ )$ and ${(\circ )}^2$ of the network input and the second layer applies the identity $(\circ )$ and exponential function $(\text{exp}(\circ ))$ to these powers. (Color figure online)

Universal material subroutine To seamlessly integrate our discovered model and parameters into a simulation, we create a universal material subroutine [46]. This subroutine operates on the integration point level of the finite element analysis and translates the local deformation, for example in the form of the deformation gradient $\mathit{F}$, into the current stress, for example the Piola stress $\mathit{P}$ [1]. We reformulate the free energy function $\psi$ from equation (12) as the sum of all k nodes of the final hidden layer,

$$\begin{array}{c}\hfill \psi =f_2\circ f_1\circ f_0(\mathit{F})=\sum _{k=1}^n{w}_{2,k}{f}_{2,k}(f_{1,j}(f_{0,i});w_{1,k}),\end{array}$$ 15

where $f_2$, $f_1$, $f_0$ are the nested activation functions associated with the second, first, and zeroth layers,

$$\begin{array}{c}\hfill f_2=\left\{\begin{array}{c}{w}_1(\circ )\\ \text{exp}(w_1(\circ ))-1\\ -\text{ln}(1-w_1(\circ ))\\ ⋮\end{array}\right)f_1=\left\{\begin{array}{c}{(\circ )}^1\\ {(\circ )}^2\\ {(\circ )}^3\\ ⋮\end{array}\right)f_0=\left\{\begin{array}{c}[I_1-3]\\ [I_2-3]\\ [I_3-1]\\ [I_4-1]\\ [I_5-1]\end{array}\right).\end{array}$$ 16

Here $f_0$ maps the deformation gradient $\mathit{F}$ onto a set of invariants, $[I_1-3]$, $[I_2-3]$, $[I_3-1]$, $[I_4-1]$, $[I_5-1]$, $f_1$ raises these invariants to the first, second, or any higher order powers, ${(\circ )}^1$, ${(\circ )}^2$, ${(\circ )}^3$, and $f_2$ applies the identity, exponential, or natural logarithm, $(\circ )$, $(\text{exp}(\circ )-1)$, $(-\text{ln}(1-(\circ )))$, or any other thermodynamically admissible function to these powers. The material subroutine calculates the Piola stress following equation (13),

$$\begin{array}{c}\hfill \mathit{P}=\sum _{k=1}^n{w}_{2,k}\frac{\partial f_{2,k}}{\partial (\circ )}\frac{\partial f_{1,k}}{\partial (\circ )}\frac{\partial f_{0,k}}{\partial \mathit{F}}\end{array}$$ 17

in terms of the first derivatives of the activation functions $f_2$ and $f_1$,

$$\begin{array}{c}\hfill \frac{\partial f_2}{\partial (\circ )}=\left\{\begin{array}{c}{w}_1\\ w_1\text{exp}(w_1(\circ ))\\ w_1/(1-w_1(\circ ))\\ ⋮\end{array}\right)\text{and}\frac{\partial f_1}{\partial (\circ )}=\left\{\begin{array}{c}1{(\circ )}^0\\ 2{(\circ )}^1\\ 3{(\circ )}^2\\ ⋮\end{array}\right)\end{array}$$ 18

and the tensor basis, $\partial f_{0,k}/\partial \mathit{F}=\partial I_k/\partial \mathit{F}$. In implicit finite element algorithms with a global Newton Raphson iteration, the material subroutine also calculates the tangent moduli,

$$\begin{array}{cc}\hfill \frac{\text{d}\mathit{P}}{\text{d}\mathit{F}}& =\sum _{k=1}^n{w}_{2,k}\left[\left[\frac{{\partial }^2{f}_{2,k}}{\partial {(\circ )}^2},{\left[\frac{\partial f_{1,k}}{\partial (\circ )}\right]}^2,+,\frac{\partial f_{2,k}}{\partial (\circ )},\frac{{\partial }^2{f}_{1,k}}{\partial {(\circ )}^2}\right]\right)\hfill \\ \hfill & \left(\frac{\partial f_{0,k}}{\partial \mathit{F}},\otimes ,\frac{\partial f_{0,k}}{\partial \mathit{F}},+,\left[\frac{\partial f_{2,k}}{\partial (\circ )},\frac{\partial f_{1,k}}{\partial (\circ )}\right],\frac{{\partial }^2{f}_{0,k}}{\partial {\mathit{F}}^2}\right]\hfill \end{array}$$ 19

in terms of the second derivatives of the activation functions $f_2$ and $f_1$,

$$\begin{array}{c}\hfill \frac{{\partial }^2{f}_2}{\partial {(\circ )}^2}=\left\{\begin{array}{c}0\\ w_1^2\text{exp}(w_1(\circ ))\\ w_1^2/{(1-w_1(\circ ))}^2\\ ⋮\end{array}\right)\text{and}\frac{{\partial }^2{f}_1}{\partial {(\circ )}^2}=\left\{\begin{array}{c}0\hfill \\ 2\hfill \\ 6(\circ )\hfill \\ ⋮\hfill \end{array}\right)\end{array}$$ 20

and the tensor basis, ${\partial }^2{f}_{0,k}/\partial {\mathit{F}}^2={\partial }^2{I}_k/\partial \mathit{F}\otimes \partial \mathit{F}$. We translate our discovered model into a modular universal material subroutine within the Abaqus finite element analysis software suite [1]. We leverage the UANISOHYPER_INV subroutine to introduce our strain energy function (12) or (16) in terms of the discovered pairs of network weights and activation functions. Our universal material subroutine uses the strain energy density, $\mathtt{UA}(\mathtt{1})=\psi$, and its first and second derivatives, $\mathtt{UI}\mathtt{1}(\mathtt{NINV})=\partial \psi /\partial I_i$, and $\mathtt{UI}\mathtt{2}(\mathtt{NINV}_{}^{\ast }(\mathtt{NINV}+\mathtt{1})/\mathtt{2})={\partial }^2\psi /\partial I_i\partial I_j$, with respect to the invariants. Following the Abaqus convention, we introduce an array of generalized invariants, $\mathtt{aInv}(\mathtt{NINV})=I_i^{\ast }$ with $i=1,\dots ,\mathtt{NINV}$, where NINV is the total number of isotropic and anisotropic invariants. In our case, for a material with two fiber families, ${\mathit{n}}_0={[cos(\alpha ),\pm sin(\alpha ),0]}^{\text{t}}$, we introduce four additional invariants, $I_4,I_5,I_6,I_7$, where $I_4,I_6$ and $I_5,I_7$ share the same parameters [1].

Algorithm 1 illustrates the UANISOHYPER_INV pseudocode to compute the arrays, UA(1), UI1(NINV), UI2(NINV $_{}^{\ast }$(NINV+1)/2), at the integration point level during a finite element analysis. First, we initialize all relevant arrays and read the activation functions $k{f}_{1,k}$ and $k{f}_{2,k}$ and weights $w_{1,k}$ and $w_{2,k}$ of the n color-coded nodes of our constitutive neural network in Fig. 2 from our user-defined parameter table UNIVERSAL_TAB. Then, for each node, we evaluate its row in the parameter table UNIVERSAL_TAB and additively update the strain energy density function and its first and second derivatives, UA, UI1, UI2.

Algorithm 1.

Pseudocode for universal material subroutine UANISOHYPER_INV

Algorithm 2.

Pseudocode to update energy and its derivatives UA, UI1, UI2

Algorithm 3.

Pseudocode to evaluate output of first network layer f,df,ddf

Algorithm 4.

Pseudocode to evaluate output of second network layer f,df,ddf

Algorithm 2 summarizes the additive update of the free energy and its first and second derivatives, UA, UI1, UI2, within the universal material subroutine uCANN. Algorithms 3 and 4 provide the pseudocode for the two subroutines uCANN_h1 and uCANN_h2 that evaluate the first and second network layers for each network node with its discovered activation functions and weights.

Finite element simulation We implement the universal material subroutine in Abaqus FEA, and make it publicly available on Github. To integrate it into a finite element simulation, we need to define our discovered model and parameters in a parameter table [1]. Each row of this table represents one of the color-coded nodes in Fig. 1 and consists of five terms: an integer kf0 that defines the index of the pseudo-invariant xInv, two integers kf1 and kf2 that define the indices of the first- and second-layer activation functions, and two float values w1 and w2 that define the weights of the first and second layers. We declare this input format using the parameter table type definition in the UNIVERSAL_PARAM_TYPES.INC file.

Within Abaqus FEA, we include the parameter table type definition using

at the beginning of the input file. We activate our user-defined material model through the command

followed by the discovered parameters. From the constitutive neural network in Fig. 2, we obtain sixteen entries for the parameter table, four for each isotropic invariant, $I_1$ and $I_2$, and four for each anisotropic invariant, $I_4$ and $I_5$, associated with the first fiber family, ${\mathit{n}}_0={[cos(\alpha ),+sin(\alpha ),0]}^{\text{t}}$. We add eight entries, four for each anisotropic invariant, $I_6$ and $I_7$, indexed in Abaqus as invariants 8 and 9, associated with the second fiber family, ${\mathit{n}}_0={[cos(\alpha ),-sin(\alpha ),0]}^{\text{t}}$, with the same parameters as $I_4$ and $I_5$. The header and the twenty-four lines of our parameter table take the following format,

The first index of each row selects between the first, second, fourth, fifth, sixth, and seventh invariants, $I_1$, $I_2$, $I_4$, $I_5$, $I_6$, $I_7$, the second index raises them to linear or quadratic powers, ${(\circ )}^1$, ${(\circ )}^2$, and the third index selects between the identity or the exponential function, $(\circ )$, $(\text{exp}(\circ )-1)$. For brevity, we can simply exclude terms with zero weights from the list.

Results

To demonstrate how we can translate information seamlessly from experiment to simulation, we perform three types of examples: First, we discover the best model and parameters to explain the experimental data with a limited number of model terms from our library of sixteen commonly used terms. We discover the best-in-class one- and two-term models, interpret their terms, and discuss their model parameters. For the four best-in-class two-term models, we illustrate the fit to the data, and perform a direct comparison with the widely used classical Holzapfel model. Second, we discover the best model and parameters to explain the data, but now without restricting the number of terms. We demonstrate how to embed the model into our universal material subroutine, and validate its implementation by comparing its finite element simulations against the experimental data and against the stress plots from our initial model discovery. Third, we predict the diastolic and systolic wall stretches and stresses across a human aortic arch, and compare the simulations with our newly discovered model against the classical Holzapfel model. We illustrate how to parameterize the two models and discuss their similarities and differences, locally at the integration point level and globally at the structural level.

Discovering the best-in-class models First, to gain a better intuition of our data, we discover the best families of models with a limited number of terms [32]. In the most general sense, our sixteen-node network in Fig. 2 introduces the sixteen-term model in equation (12) parameterized in terms of sixteen pairs of weights, $\{w_{1,\circ },w_{2,\circ }\}$. In the most naive approach, we could test all possible models. From combinatorics, we know that this is a total of $2^{16}-1=65,535$ models, 16 with a single term, 120 with two, 560 with three, 1820 with four, 4368 with five, 8008 with six, 11,440 with seven, 12,870 with eight, 11,440 with nine, 8008 with ten, 4368 with eleven, 1820 with twelve, 560 with thirteen, 120 with fourteen, 16 with fifteen, and 1 with all sixteen terms. To understand the relevance of these sixteen terms, we begin with a simplified analysis that constrains the number of non-zero terms to either one or two [38]. We train the network in Fig. 2 by minimizing the loss function (14) with the stress definitions (11) using the biaxial test data of the media and adventitia in Tables 1 and 2, and explicitly set the weights of the remaining terms to zero [55].

Figure 3 summarizes the discovery of the best-in-class one- and two-term models for the human aortic media and adventitia in two $16\times 16$ heat maps. Terms 1 through 8 are associated with the isotropic invariants $I_1$ and $I_2$, terms 9 through 16 are associated with the anisotropic invariants $I_4$ and $I_5$. The squares on the diagonale indicate the goodness of fit of the 16 one-term models for the media and the adventitia. All other squares indicate the goodness of fit of the 120 two-term models. The color code represents the remaining loss after training, and is a measure for the goodness of fit of each model. The best-in-class models are the models with the lowest remaining loss, highlighted in dark blue. At first glance, we observe four distinct blocks, the iso–iso block in the upper left, the aniso–aniso block in the lower right, and the iso–aniso blocks in the upper right and lower left. The color code confirms our intuition, that a combination of two isotropic or two anisotropic terms does not provide a good explanation of the data. Instead, the best-in-class models with the lowest remaining loss and the dark blue colors are all located in the iso-aniso blocks.

Fig. 3.

Discovering the best-in-class models. Best-in-class one- and two-term models of the media and adventitia. Remaining loss of the 16 one-term models and 120 two-term models of the constitutive neural network from Fig. 2, trained with all ten datasets from Table 1 for the media, left, and Table 2 for the adventitia, right. Terms 1 through 8 are associated with the isotropic invariants $I_1$ and $I_2$, terms 9 through 16 are associated with the anisotropic invariants $I_4$ and $I_5$. Squares on the diagonale indicate the losses of the 16 one-term models, all other squares indicate the losses of the 120 two-term models. Best-in-class models are the models with the lowest remaining loss, highlighted in dark blue. (Color figure online)

Best-in-class one-term models The squares on the diagonales of Fig. 3 indicate the goodness of fit of the 16 one-term models for the media and the adventitia.

Table 3 summarizes the four best-in-class one-term models: the exponential linear first invariant Demiray model [8], the linear second invariant Blatz Ko model [4], the exponential linear second invariant model, and the linear first invariant neo Hooke model [62]. Each block summarizes the constitutive model, the input to the universal material subroutine, their parameterizations for the media, top, and adventita, bottom, and their overall ranking, right. Since our constitutive neural network uses parameters with a clear physical interpretation, we can translate the network weights into the classical shear modulus $\mu$, the stiffness-like parameter a, and the unitless exponential weighting factor b. From comparing the discovered parameters for both tissue types across all four models, we conclude that the media, in each top row, is about three to four times stiffer than the adventitia, in each bottom row. Interestingly, the exponential first invariant Demiray model [8] is the best of all sixteen models, both for the media and adventitia. The linear second invariant Blatz Ko model [4] is the second best model for the media, and the third best for the adventitia. Strikingly, the widely used linear first invariant neo Hooke model [62] is not among the three best-in-class one-term models, neither for the media nor for the adventitia.

Table 3.

Best-in-class one-term models. Models and parameters of the constitutive neural network from Fig. 2, trained with data from Table 1 for the media and Table 2 for the adventita. The four models are the best-in-class one-term models from Fig. 2. Each block summarizes the constitutive model, the input to the universal material subroutine, their parameterizations for the media, top, and adventita, bottom, and their overall ranking

Best-in-class two-term models of the media All squares that are not located on the diagonale of Fig. 3 illustrate the goodness of fit of the 120 two-term models. Notably, for the media, the four best-in-class two-term models are all located in the second row and column of Fig. 3, left.

Table 4 summarizes the four best-in-class two-term models for the media. They all contain the isotropic exponential linear first invariant Demiray term [8] from the best-in-class one-term model, combined with an anisotropic term: the quadratic fifth invariant term, the quadratic fourth invariant term, the exponential quadratic fourth invariant term, or the exponential quadratic fifth invariant term. For comparison, Table 4 also reports the classical two-term Holzapfel model [22] that contains the isotropic linear first invariant term and the anisotropic exponential quadratic fourth invariant term. Each block of the table summarizes the constitutive model, the input to the universal material subroutine, and their parameterizations.

Table 4.

Best-in-class two-term models of the media. Models and parameters of the constitutive neural network from Fig. 2, trained with all ten datasets from Table 1 simultaneously. The first four models are the best-in-class two-term models from Fig. 2, left; the fifth model is the classical Holzapfel model. Each block summarizes the constitutive model, the input to the material subroutine, and their parameters

Figure 4 illustrates the performance of the four best-in-class two-term models for the media from Fig. 2, left, summarized in Table 4, and for comparison, the classical Holzapfel model [22]. The circles represent the equibiaxial testing data from Table 1. The reported loss quantifies the goodness of fit for a simultaneous training with all all ten stress-stretch pairs. The color coded regions highlight the contributions of the individual model terms to the circumferential and axial stresses, ${\sigma }_{\text{cir}}$ and ${\sigma }_{\text{axl}}$, as functions of stretches ${\lambda }_{\text{cir}}$ and ${\lambda }_{\text{axl}}$. The red regions represent the isotropic exponential linear first invariant term. The blue, green, turquoise, and dark blue regions represent the anisotropic fourth and fifth invariant terms. With a median collagen fiber orientation of $7.{00}^{\circ }$, the fibers in the media are almost aligned with the circumferential direction. This implies that the axial direction, bottom, only sees the red isotropic response, while the circumferential direction, top, sees a superposition of both, the red isotropic and the green-to-blue anisotropic responses. For the sake of compactness, we only display the equibiaxial response, but note that the other four curves provide an equally good fit to the experimental data. The classical Holzapfel model [22] in Fig. 4, right, combines the isotropic linear first invariant term in dark red and the anisotropic exponential quadratic fourth invariant term in turquoise. While it performs well in the circumferential direction, top right, its linear isotropic term is incapable of capturing the nonlinear isotropic matrix behavior in the axial direction, bottom right. Its loss is about three times higher than the loss of the discovered best-in-class two-term-model, Fig. 4, left.

Fig. 4.

Best-in-class two-term models of the media. True stresses ${\sigma }_{\text{cir}}$ and ${\sigma }_{\text{axl}}$ as functions of stretches ${\lambda }_{\text{cir}}$ and ${\lambda }_{\text{axl}}$ for the constitutive neural network from Fig. 2, trained with all ten datasets from Table 1 simultaneously. The first four columns illustrate the best-in-class two-term models from Fig. 2, left; the right column illustrates the Holzapfel model [22] for comparison. Circles represent the equibiaxial testing data from Table 1. Color-coded regions represent the discovered model terms. The remaining loss indicates the quality of the overall fit. (Color figure online)

Best-in-class two-term models of the adventitia All squares that are not located on the diagonale of Fig. 3 illustrate the goodness of fit of the 120 two-term models. Interestingly, for the adventitia, the four best-in-class two-term models are located in the second and sixth rows and columns of Fig. 3, right. Table 5 summarizes the four best-in-class two-term models for the adventitia. They contain the isotropic exponential linear first or second invariant term from the best-in-class one-term models, combined with the anisotropic exponential linear or quadratic fifth invariant term. For comparison, Table 5 also reports the classical two-term Holzapfel model [22]. Figure 5 illustrates the performance of the four best-in-class two-term models for the adventitia from Fig. 2, right, summarized in Table 5, and for comparison, the classical Holzapfel model [22]. The circles represent the equibiaxial testing data from Table 2. The red and light orange regions represent the isotropic exponential linear first and second invariant terms. The blue and dark blue regions represent the anisotropic exponential linear and quadratic fifth invariant terms. With a median collagen fiber orientation of $66.{78}^{\circ }$, the fibers in the adventitia are much closer to the axial direction than for the media. As a result, both circumferential and axial directions see the red and and orange isotropic response and the blue anisotropic response, with a pronounced anisotropy in the axial stresses, bottom. Similar to the media, Fig. 4, the classical Holzapfel model [22] for the adventitia, Fig. 5, right, has a loss that is about three times higher than the loss of the discovered best-in-class two-term-model, Fig. 5, left.

Table 5.

Best-in-class two-term models of the adventitia. Models and parameters of the constitutive neural network from Fig. 2, trained with all ten datasets from Table 2 simultaneously. The first four models are the best-in-class two-term models from Fig. 2, right; the fifth model is the classical Holzapfel model. Each block summarizes the constitutive model, the input to the material subroutine, and their parameters

Fig. 5.

Best-in-class two-term models of the adventitia. True stresses ${\sigma }_{\text{cir}}$ and ${\sigma }_{\text{axl}}$ as functions of stretches ${\lambda }_{\text{cir}}$ and ${\lambda }_{\text{axl}}$ of the constitutive neural network from Fig. 2, trained with all ten datasets from Table 2 simultaneously. The first four columns illustrate the best-in-class two-term models from Fig. 2, right; the right column illustrates the Holzapfel model [22] for comparison. Circles represent the equibiaxial testing data from Table 2. Color-coded regions represent the discovered model terms. The remaining loss indicates the quality of the overall fit. (Color figure online)

Discovering the best model and parameters Next, we discover the best model and parameters–but now without prescribing the number of terms–and use the model to validate simulations with our universal material subroutine against the experimental data and against the stress plots from our model discovery. Model discovery is a sophisticated trade-off between the number of discovered terms and the accuracy of the fit [38]. Fortunately, we can fine-tune this trade-off by adding an $L_p$ regularization term to the loss function in Eq. (14). Specifically, with $L_1$ regularization and a penalty parameter $\alpha$ varying between $\alpha =[0.000,0.001,0.010,0.100]$, we observe that we can tune the number of discovered model terms between five and one. For our example, a penalty parameter of $\alpha =0.001$ provides a good balance between the number of terms and the accuracy of the fit. Strikingly, for this penalty parameter, the network discovers exactly the same model for the media and the adventitia: a three-term model with the isotropic linear and exponential first invariant terms and the anisotropic quadratic fifth invariant term,

$$\begin{array}{c}\hfill \psi =\frac{1}{2}{\mu }_1[I_1-3]+\frac{1}{2}a/b[exp(b[I_1-3])-1]+\frac{1}{2}{\mu }_5{[I_5-1]}^2.\end{array}$$

While the discovered model is the same for both tissue types, the discovered parameters are different, with ${\mu }_1=33.45$ kPa, $a=3.74$ kPa, $b=6.66$, ${\mu }_5=2.17$ kPa for the media and ${\mu }_1=8.30$ kPa, $a=1.42$ kPa, $b=6.34$, ${\mu }_5=0.49$ kPa for the adventitia. These parameters reflects the different tissue compositions [23], with the media about three to four times stiffer than the adventitia. The discovered model translates into the following four-line parameter table for our universal material subroutine,

with w$_{\mathtt{1},\mathtt{1}}^{}=38.01$, w$_{\mathtt{2},\mathtt{1}}^{}=0.44$ kPa, w$_{\mathtt{1},\mathtt{2}}^{}=6.66$, w$_{\mathtt{2},\mathtt{2}}^{}=0.28$ kPa, w$_{\mathtt{1},\mathtt{15}}^{}=24.68$, w$_{\mathtt{2},\mathtt{15}}^{}$ = 0.04 kPa for the media and w$_{\mathtt{1},\mathtt{1}}^{}=34.28$, w$_{\mathtt{2},\mathtt{1}}^{}=0.12$ kPa, w$_{\mathtt{1},\mathtt{2}}^{}=6.34$, w$_{\mathtt{2},\mathtt{2}}^{}=0.11$ kPa, w$_{\mathtt{1},\mathtt{15}}^{}=15.32$, w$_{\mathtt{2},\mathtt{15}}^{}=0.02$ kPa for the adventitia.

Figures 6 and 7 illustrate the discovered model for the media and the adventita, top, and, for validation, the finite element simulations with our universal material subroutine, bottom. The circles illustrate the biaxial testing data from Tables 1 and 2. The color coded regions highlight the contributions of the individual model terms to the circumferential and axial stresses, ${\sigma }_{\text{cir}}$ and ${\sigma }_{\text{axl}}$, as functions of stretches, ${\lambda }_{\text{cir}}$ and ${\lambda }_{\text{axl}}$. The dark red and red regions represent the isotropic linear first invariant neo Hooke term [62] and the exponential first invariant Demiray term [8]. The blue regions represent the anisotropic quadratic fifth invariant term. Overall, the discovered model provides an excellent fit to the data, both for the media and the adventitia. In both examples, in Figs. 6 and 7, the finite element simulations with our universal material subroutine, bottom, agree well with the experimental data and with the model discovery plots, top.

Fig. 6.

Discovered model and finite element simulation of the media. True stresses ${\sigma }_{\text{cir}}$ and ${\sigma }_{\text{axl}}$ as functions of stretches ${\lambda }_{\text{cir}}$ and ${\lambda }_{\text{axl}}$ of the constitutive neural network from Fig. 2, trained with all ten datasets from Table 1 simultaneously. Circles illustrate the biaxial testing data from Table 1. Top graphs display the discovered model, $\psi =\frac{1}{2}{\mu }_1[I_1-3]+\frac{1}{2}a/b[exp(b[I_1-3])-1]+\frac{1}{2}{\mu }_5{[I_5-1]}^2$; bottom graphs display the finite element simulation with the discovered parameters ${\mu }_1=33.45$ kPa, $a=3.74$ kPa, $b=6.66$, ${\mu }_5=2.17$ kPa. (Color figure online)

Fig. 7.

Discovered model and finite element simulation of the adventitia. True stresses ${\sigma }_{\text{cir}}$ and ${\sigma }_{\text{axl}}$ as functions of stretches ${\lambda }_{\text{cir}}$ and ${\lambda }_{\text{axl}}$ of the constitutive neural network from Fig. 2, trained with all ten datasets from Table 2 simultaneously. Circles illustrate the biaxial testing data from Table 2. Top graphs display the discovered model, $\psi =\frac{1}{2}{\mu }_1[I_1-3]+\frac{1}{2}a/b[exp(b[I_1-3])-1]+\frac{1}{2}{\mu }_5{[I_5-1]}^2$; bottom graphs display the finite element simulation with the discovered parameters ${\mu }_1=8.30$ kPa, $a=1.42$ kPa, $b=6.34$, ${\mu }_5=0.49$ kPa. (Color figure online)

Predicting wall stresses in the human aortic arch To explore whether our finite element simulations generalize robustly, from the material point level to the structural level, we now use our universal material subroutine to predict the wall stresses across the human aortic arch and compare our results against the Holzapfel model [16]. We explore the aortic arch during diastole, at a blood pressure of 80 mmHg, and during systole, at 120 mmHg, both applied quasi-statically, and visualize the predicted stresses and stretches in the media, in the adventitia, and in selected cross sections.

Figure 8 shows our finite element model of the aortic arch, created from high-resolution magnetic resonance images of a healthy, 50th percentile U.S. male [43, 44]. We assume an average aortic wall thickness of 3.0 mm, where the inner 75% of the wall make up the media and the outer 25% make up the adventitia. The finite element discretization uses 60,684 linear tetrahedral elements for the media and 30,342 linear tetrahedral elements for the adventitia, and has a total of 61,692 degrees of freedom. The local collagen fiber angles against the circumferential direction are $\pm 7.{00}^{\circ }$ in the media and $\pm 66.{78}^{\circ }$ in the adventitia. The simulation in Fig. 8, left, uses our newly discovered three-term model with an isotropic linear first invariant neo Hooke term, an isotropic exponential first invariant Demiray term, and an anisotropic quadratic fifth invariant term, $\psi =\frac{1}{2}{\mu }_1[I_1-3]+\frac{1}{2}a/b[exp(b[I_1-3])-1]+\frac{1}{2}{\mu }_5{[I_5-1]}^2$, with the discovered parameters ${\mu }_1=33.45$ kPa, $a=3.74$ kPa, $b=6.66$, ${\mu }_5=2.17$ kPa for the media and ${\mu }_1=8.30$ kPa, $a=1.42$ kPa, $b=6.34$, ${\mu }_5=0.49$ kPa for the adventitia. The simulation in Fig. 8, right, uses the Holzapfel model with an isotropic linear first invariant neo Hooke term and an anisotropic exponential term that couples the first and fourth invariants through the dispersion parameter $\kappa$, $\psi =\frac{1}{2}\mu [I_1-3]+\frac{1}{2}a/b[exp(b[\kappa [I_1-3]+[1-3\kappa ]{[I_4-1]}^2])-1]$ with the best-fit parameters $\mu =48.68$ kPa, $a=6.67$ kPa, $b=23.17$, $\kappa =0.074$ for the media and $\mu =13.22$ kPa, $a=0.93$ kPa, $b=12.06$, $\kappa =0.091$ for the adventitia. We follow the common practice and only activate the fourth and fifth invariant terms in tension, but not in compression.

Fig. 8.

Human aortic arch model and wall stresses in the media and adventitia. The finite element discretization uses linear tetrahedral elements, 60,684 for the media and 30,342 for the adventitia, and has a total of 61,692 degrees of freedom. The color code highlights the maximum principal stresses in the media and adventitia of the human aortic arch predicted by the newly discovered three-term model, left, and by the Holzapfel model, right, both trained with the media and adventitia datasets from Tables 1 and 2. (Color figure online)

Figure 8 illustrates four stress profiles that provide a first glance at the performance of both models: First, we emphasize that all large scale structural simulations with our universal material subroutine run and converge robustly, and predict physically reasonable, smooth stress profiles across the entire aortic arch. Second, we observe that, for both constitutive models, the maximum principal stresses in the media are about three times higher than in the adventitia, which agrees well with the recorded stresses in the experiment in Tables 1 and 2 and with the discovered stiffness-like parameters in Tables 3 through 5. Third, and most interestingly, the direct side-by-side comparison of the two different models reveals an excellent agreement of the stress profiles in the low-stress regimes of the adventitia, and a very good agreement in the high-stress regimes of the media, with only a few minor local discrepancies. Overall, we conclude that our universal material subroutine generalizes well from the local material point level to the global structural level and that the simulations with our newly discovered three-term model perform similar to the widely used Holzapfel model [16].

Predicting aortic arch mechanics with the newly discovered model Figure 9 illustrates the circumferential and radial stresses and stretches in the media, in the adventita, and in selected cross sections, during diastole, top, and systole, bottom. All simulations use the newly discovered model, $\psi =\frac{1}{2}{\mu }_1[I_1-3]+\frac{1}{2}a/b[exp(b[I_1-3])-1]+\frac{1}{2}{\mu }_5{[I_5-1]}^2$, with the discovered parameters ${\mu }_1=33.45$ kPa, $a=3.74$ kPa, $b=6.66$, ${\mu }_5=2.17$ kPa for the media and ${\mu }_1=8.30$ kPa, $a=1.42$ kPa, $b=6.34$, ${\mu }_5=0.49$ kPa for the adventitia. The simulations provide a nuanced perspective of the mechanics of the aortic arch and detailed insights into the performance of the new three-term model: First, we note that both, stresses and stretches, are larger during systole than during diastole, larger in the the media than in the adventitia, and larger circumferentially than axially. Second, in the stresses profiles, we observe a significant jump between the media and adventita layers, which is most visible in the cross sectional view, and most pronounced during systole. These intra-layer stress discontinuities could play a critical role in the pathogenesis of aortic dissection and aortic aneurysm formation. Third, in the stretch profiles, we observe regional peaks beyond the experimental testing and network training regime of $1.0\le \lambda \le 1.2$, which are highlighted in bright yellow and most prominent in the circumferential stretch during systole. The smooth stress and stretch profiles beyond the training regime suggest that the discovered model generalizes well to larger stretch regimes, $1.2\le \lambda$, and to higher blood pressures. Overall, we conclude that our newly discovered model can predict physically meaningful stretch and stress profiles in complex biological structures and accurately capture the local and global mechanics of the aortic wall.

Fig. 9.

Diastolic and systolic stresses and stretches in the human aortic arch predicted by the newly discovered model. Circumferential and radial stresses, ${\sigma }_{\text{cir}}$ and ${\sigma }_{\text{axl}}$, and stretches, ${\lambda }_{\text{cir}}$ and ${\lambda }_{\text{axl}}$, in the media, in the adventita, and in selected cross sections, during diastole, top, and systole, bottom. Simulations use the discovered model, $\psi =\frac{1}{2}{\mu }_1[I_1-3]+\frac{1}{2}a/b[exp(b[I_1-3])-1]+\frac{1}{2}{\mu }_5{[I_5-1]}^2$, with the discovered parameters ${\mu }_1=33.45$ kPa, $a=3.74$ kPa, $b=6.66$, ${\mu }_5=2.17$ kPa for the media and ${\mu }_1=8.30$ kPa, $a=1.42$ kPa, $b=6.34$, ${\mu }_5=0.49$ kPa for the adventitia

Predicting aortic arch mechanics with the Holzapfel model Figure 10 illustrates the stresses and stretches in the in the media, in the adventita, and in selected cross sections, during diastole and systole similar to Fig. 9, but now using the Holzapfel model [16], $\psi =\frac{1}{2}\mu [I_1-3]+\frac{1}{2}a/b[exp(b[\kappa [I_1-3]+[1-3\kappa ]{[I_4-1]}^2])-1]$, with the best-fit parameters, $\mu =48.68$ kPa, $a=6.67$ kPa, $b=23.17$, $\kappa =0.074$ for the media and $\mu =13.22$ kPa, $a=0.93$ kPa, $b=12.06$, $\kappa =0.091$ for the adventitia. The simulations provide additional insight into the similarities and differences of both constitutive models: First, within the experimental testing and network training regime, $1.0\le \lambda \le 1.2$, the predictions with the Holzapfel model in Fig. 10 are virtually identical to the predictions with our new three-term model in Fig. 9. This is particularly evident during diastole, and across the entire adventita during both diastole and systole. Second, beyond the experimental testing and network training regime, $1.2\le \lambda$, we observe small discrepancies between both models, which are located primarily in the bright yellow regions of the high-stretch regime. This agrees with our intuition that the exponential term of the Holzapfel model introduces a more pronounced stiffening than the quadratic term of our newly discovered model, especially in the high-stretch regime. Overall, we conclude that both models perform almost identically during diastole, within their training regime, and very similarly during systole, beyond their training regime, where the stresses of the Holzapfel model are locally slightly higher than those of the new three-term model, while its stretches are locally slightly lower.

Fig. 10.

Diastolic and systolic stresses and stretches in the human aortic arch predicted by the Holzapfel model. Circumferential and radial stresses, ${\sigma }_{\text{cir}}$ and ${\sigma }_{\text{axl}}$, and stretches, ${\lambda }_{\text{cir}}$ and ${\lambda }_{\text{axl}}$, in the media, in the adventita, and in selected cross sections, during diastole, top, and systole, bottom. Simulations use the Holzapfel model, $\psi =\frac{1}{2}\mu [I_1-3]+\frac{1}{2}a/b[exp(b[\kappa [I_1-3]+[1-3\kappa ]{[I_4-1]}^2])-1]$, with the best-fit parameters, $\mu =48.68$ kPa, $a=6.67$ kPa, $b=23.17$, $\kappa =0.074$ for the media and $\mu =13.22$ kPa, $a=0.93$ kPa, $b=12.06$, $\kappa =0.091$ for the adventitia

Discussion

Computational modeling is vital for unraveling the biomechanics of the aorta and offering insights into its disease mechanisms. Finite element analyses enable a precise identification of regions of non-physiological deformations or stresses, which could indicate the onset of vascular diseases such as aortic dissection or aneurysm formation. Constitutive modeling lies at the heart of any finite element analysis, and selecting the best model and parameters is crucial for its success. Common finite element analysis tools offer a wide variety of constitutive models to choose from, but selecting the best model remains a matter of user experience and personal preference. The objective of this study is to eliminate user bias and fully automate the process of model selection using constitutive neural networks. We train these networks with experimental data from biaxial extension tests on the human aortic media and adventita, and discover the best models and parameters to explain the data. Our model discovery workflow automatically generates an input file for a universal material subroutine that can represent more than 60,000 different constitutive models and is seamlessly embedded in the finite element analysis pipeline. In this manuscript, we rationalize the process of model discovery, discuss models of different complexity, demonstrate their performance on the experimental data, compare them against the current gold standard model, and use the best model and parameters to predict stress and stretch profiles across the aortic arch during diastole and systole.

Model discovery is a balance between complexity and accuracy The universal approximation theorem states that a neural network with a single hidden layer—with a sufficient number of nodes and appropriate activation functions—can approximate any continuous function on a compact subset of its domain to arbitrary precision [26]. This implies that, with a sufficient number of nodes, a neural network should be able to approximate any of the stretch-stress pairs of our biaxial tests. However, in constitutive modeling, we are not interested in learning just any function. Instead, we seek to discover the best function that not only approximates the data, but also satisfies common thermodynamic principles and physical constraints [31]. These include material objectivity, material symmetry, incompressibility, polyconvexity [29, 57], and thermodynamic consistency [33, 59]. Conveniently, we hardwire these principles into our constitutive neural network in Fig. 2 to ensure that our discovered functions satisfy these constraints a priori. Specifically, our network has two hidden layers and represents the free energy function as the sum of the contributions of the sixteen nodes of its second layer [35]. Naturally, activating all sixteen nodes is the best strategy to fine-tune the fit to the data and achieve the highest level of accuracy. At the same time, the resulting sixteen-term model is inherently complex and difficult to interpret [53]. Nonetheless, if we are only interested in finding the best-fit model and parameters for a finite element analysis, this is probably just fine. We can feed all sixteen terms directly into our universal material subroutine and perform our engineering analysis. Undoubtedly, this will make the best and most explicit use of the available data.

Model discovery can be non-robust and non-interpretable In many practical applications, we are not just interested in finding the best-fit model with an arbitrarily large number of terms. Instead, we want to discover the most relevant terms to best describe experimental data. This can have multiple reasons: First, minimizing the loss function (14) with 16 terms and 24 independent weights translates into a complex non-convex optimization problem with flat gradients and multiple local minima [38]. It is computationally expensive, if not impossible, to find its global minimum. Second, with so many degrees of freedom, there is a risk of overfitting. Even if we found the global minimum, it might be highly sensitive to outliers or measurement errors [5]. In other words, we might find the best-fit model for a specific data set, but this model tends to be non-robust and non-generalizable to unseen data. Third, and probably most importantly for our purposes, a sixteen-term model is virtually non-interpretable [10]. We cannot interpret the relevance of its terms, compare the meaning of its parameters, and identify the underlying mechanisms associated with individual terms. A fully activated model provides virtually no microstructural insights into the material response. This raises the holy grail question in model discovery: How can we fine tune the number of terms?

Lp regularization promotes robust and interpretable models The concept of $L_p$ regularization or bridge regression dates back more than three decades and was introduced to shrink the parameter space in a data analysis [12]. It has re-gained attention as a powerful tool to promote sparsity in system identification [6], and, most recently, in discovering constitutive models from data [10, 38]. $L_p$ regularization adds a penalty term, ${\alpha ||\mathit{w}||}_p^p$, to the loss function (14), where $\alpha \ge 0$ is a non-negative penalty parameter and ${||\mathit{w}||}_p^p=[{\sum }_{i=1}^{\text{para}}|w_i{{|}^p]}^{1/p}$ is the $L_p$ norm of the vector of network weights $\mathit{w}$. $L_p$ regularization introduces two hyperparameters, the power p by which it penalizes the individual model parameters, and the penalty parameter $\alpha$ by which it scales the relative importance of the regularization term compared to the network loss [38]. Both parameters enable a precise control of model discovery and it is crucial to understand their mathematical subtleties, computational implications, and physical effects: $L_0$ regularization or subset selection directly penalizes the number of non-zero terms by solving the discrete combinatorial problem, which is a simple and unbiased method to explicitly prescribe the number of terms [28]. $L_1$ regularization or lasso enables feature selection and induces sparsity by reducing some weights exactly to zero, which effectively reduces model complexity and improves interpretability [61]. $L_2$ regularization or ridge regression seeks to reduce outliers and improve predictability by reducing absolute values while maintaining all parameters [18], which essentially does the opposite of what we seek to accomplish here.

L0 regularization identifies the best-in-class models $L_0$ regularization or subset selection turns the continuous model selection problem into a discrete combinatorial problem with ($2^n$-1) possible combinations of terms [28]. This makes this type of regularization computationally intractable for models with a large number of terms. However, instead of performing a subset selection from all possible 65,535 models, we use it to discover the best-in-class one- and two-term models—out of subsets of 16 and 120 possible models—and gain insight into the relevant terms and model parameters [38]. Interestingly, in our example, for both media and adventitia, the best-in-class one-term model in Table 3, with the lowest remaining loss of 84.49 for the media and 3.29 for the adventita, is the classical exponential linear first invariant Demiray model [8]. The best-in-class two-term models in Table 4 for the media and Table 5 for the adventitia expand this term by either the quadratic fifth invariant or the exponential quadratic fifth invariant, with remaining losses of 8.84 for the media and 0.79 for the adventitia. Since $L_0$ regularization explicitly penalizes every additional term in the loss function (14) by $\alpha$, it favors the two-term model for the media for $\alpha \le 84.49/8.84=9.56$ and for the adventitia for $\alpha \le 3.29/0.79=4.17$, and only selects the purely isotropic one-term Demiray model [8] for penalty parameters larger than these values. This simple example illustrates the role of the penalty parameter $\alpha$ as a hyperparamter to fine-tune subset selection by modulating the number of non-zero terms.

L1 regularization induces sparsity and improves interpretability A less invasive approach to regularize the loss function without having to explicitly probe all combinations of terms is $L_1$ regularization or lasso [61]. Here we apply $L_1$ regularization with varying penalty parameters $\alpha$ and monitor the remaining loss. We find a reasonable balance of complexity and accuracy for a penalty parameter of $\alpha =0.001$. Strikingly, for this parameterization, out of all 65,535 combinations of terms, our network discovers exactly the same model for the media and the adventitia: a three-term model with an isotropic linear first invariant neo Hooke term [62], an exponential linear first invariant Demiray term [8], and the anisotropic quadratic fifth invariant term. Its non-zero network weights translate into interpretable material parameters in the form of a shear modulus, a stiffness-like parameter, an exponential coefficient, and a shear-type modulus, all with physically meaningful units. Our newly discovered model is sparse, robust, and interpretable, its contains terms of popular constitutive models, and is a natural generalization of our discovered best-in-class one- and two-term models. We feel that it strikes an excellent balance between complexity and accuracy. From Figs. 6 and 7, we conclude that it approximates our experimental data well and integrates seamlessly into our universal material subroutine.

Our discovered model generalizes from the material point level to structural analysis One of the main reasons to develop constitutive models for biological tissues is to perform realistic biomedical simulations [42, 44]. The ultimate test of our discovered model is to probe its performance in realistic finite simulations, beyond the material point level. Here we use the example of stress analysis in the aortic arch. Understanding the structural and mechanical distinctions between the media and adventitia layers of the aorta is crucial for comprehending vascular health and disease [23, 27]: The media is rich in elastin fibers and smooth muscle cells, it provides elasticity and contractility, and enables hemodynamic function. The adventitia consists primarily of collagen fibers and fibroblasts and provides structural support and integrity. By building our model directly from data—without user bias through model selection—we can precisely capture the nuances between the load carrying capacity of the media and the adventitia [40, 41]. Disruptions in the delicate balance between these layers contribute to pathological conditions such as aortic dissection or aneurysms formation [48, 51]. Mechanical heterogeneity and regional stress variations play a pivotal role at the onset of these conditions. Our finite element model is built around our new universal material subroutine [46] that can account for these layer-specific properties and aid in predicting disease progression, assessing rupture risk, and developing targeted interventions. This subroutine not only includes our discovered three-term model, but all 65,535 possible models of our constitutive neural network in Fig. 2 [35], simply by twenty-four lines of its input file. A side-by-side comparison with the popular Holzapfel model in Figs. 9 and 10 suggests that our discovered model not only performs identically during diastole, within the stretch range of the training regime, and but also performs nearly similarly during systole, beyond the initial training regime. The small local discrepancies between both models are not a flaw of our new model, but rather a result of the limited experimental test range within stretches of only 1.0 to 1.2. From the experimental stretch-stress curves in Fig. 1, we conclude that within this range, the stress response of the fibers is neither fully quadratic as in our discovered model, nor fully exponential as in the Holzapfel model [16]. Overall, we believe that newly discovered model performs well in realistic structural simulations and can provide a comprehensive understanding of the interplay between the layers of the aorta to informs strategies for early disease detection, risk stratification, and tailored therapeutic approaches in the benefit of cardiovascular health.

Limitations While our results solidly suggest that we can discover interpretable models with physically meaningful parameters from data and integrate these models into a finite element simulation via our new universal material subroutine, a few limitations remain: First, here we have prototyped our approach for discovering a personalized arterial model of a healthy 56-year-old male. It would be interesting to expand our method to include all $n=17$ healthy and $n=11$ aneurysmatic aortas of the initial study [40]. Second, while our study shows that $L_p$ regularization is a robust method to control the number of model terms through the penalty parameter $\alpha$, especially the low-penalty models with a large number of discovered terms remain sensitive to the initial conditions. If the goal is to discover the best model with a small number of interpretable terms, we recommend to always perform an $L_0$ regularization first, and solve for the discrete combinatorics problem—at least for the best-in-class one- and two-term models—to gain a feeling for the relevant terms [38]. If the goal is to discover a viable model for a finite element simulation, the sparseness of the solution is less relevant, since we can feed any discovered model into our universal material subroutine and obtain comparable results. Third, while different discovered models perform similarly within the training range, they may deviate outside the training regime. For finite element simulations, this may occur in regions of local stress concentrations, where the simulated stretches and stresses exceed the experimental measurement range. This is not a flaw of the model discovery itself, but rather a limitation of the available training data, which, in our example, did not properly tease out the stretch-stiffening regime. As a result, the discovered anisotropic term that best explains our available data turns out to be quadratic, and not exponential like in the classical Holzapfel model [16, 22]. Fourth, for illustrative purposes, the neural network and the material subroutine we propose here are intentionally invariant-decoupled. Our recent study includes an advanced material subroutine that can handle coupled invariants like $I_1$ and $I_4$, selective activation under tension only, and quasi-incompressibility through the third invariant $I_3$ [45]. Fifth, we use fiber orientations from structural imaging. We could potentially learn these orientations as network weights in a zeroth layer that calculates the fourth and fifth invariants. We have tried this approach, and, while it works well for clean synthetic data, it is highly sensitive to noise in real-world experimental data, which is why we decided to assume fixed fiber orientations here. Finally, to address the current limitation to hyperelastic materials, we have recently expanded the concept of constitutive neural networks to viscoelasticity [65] and to general inelasticity [19] to address potential effects of time-dependency, growth, and remodeling.

Conclusion

Personalized computational simulations can help us understand the biomechanics of cardiovascular disease, predict patient-specific disease progression, and personalize treatment and intervention. Material modeling is critical to realistic physics-based simulations, but selecting the best model is limited to a few highly trained specialists in the field. In biomedical applications, poor model selection does not only jeopardize the success of the entire simulation, but can have life-threatening consequences for the patient. Here we explore the feasibility of removing user involvement and automating material modeling in finite element analyses. We leverage recent developments in constitutive neural networks, machine learning, and artificial intelligence to discover the best constitutive model from thousands of possible combinations of a few functional building blocks. We seamlessly integrate all discoverable models into the finite element workflow by creating a universal material subroutine that contains more than 60,000 models, made up of 16 individual terms. Our results suggest that constitutive neural networks can robustly discover various flavors of arterial models from data, feed these models directly into a finite element simulation, and predict stress and strain profiles that compare favorably to the classical Holzapfel model. Replacing dozens of individual material subroutines by a single universal material subroutine will make finite element simulations more accessible and user-friendly, more robust and reliable, and less vulnerable to human error. Democratizing biomedical simulation by automating model selection could induce a paradigm shift in physics-based simulation, broaden access to simulation technologies, and empower individuals with varying levels of expertise and diverse backgrounds to actively participate in scientific discovery in the benefit of human health.