Graph Neural Network Surrogates for Contacting Deformable Bodies with Necessary and Sufficient Contact Detection

Abstract

Surrogate models for the rapid inference of nonlinear boundary value problems in mechanics are helpful in a broad range of engineering applications. However, effective surrogate modeling of applications involving the contact of deformable bodies, especially in the context of varying geometries, is still an open issue. In particular, existing methods are confined to rigid body contact or, at best, contact between rigid and soft objects with well-defined contact planes. Furthermore, they employ contact or collision detection filters that serve as a rapid test but use only the necessary and not sufficient conditions for detection. In this work, we present a graph neural network architecture that utilizes continuous collision detection and, for the first time, incorporates sufficient conditions designed for contact between soft deformable bodies. We test its performance on two benchmarks, including a problem in soft tissue mechanics of predicting the closed state of a bioprosthetic aortic valve. We find a regularizing effect on adding additional contact terms to the loss function, leading to better generalization of the network. These benefits hold for simple contact at similar planes and element normal angles, and complex contact at differing planes and element normal angles. We also demonstrate that the framework can handle varying reference geometries. However, such benefits come with high computational costs during training, resulting in a trade-off that may not always be favorable. We quantify the training cost and the resulting inference speedups on various hardware architectures. Importantly, our graph neural network implementation results in up to a hundred- to thousand-fold speedup on GPU, and twenty- to two hundred-fold speedup on CPU for our benchmark problems at inference.

Graphical Abstract

1. Introduction

The use of finite element methods (FEM) for high-fidelity modeling is ubiquitous in continuum mechanics. With decades of development, the FEM can handle both geometric and material non-linearities [1], and incorporate multiple contacting bodies [2]. However, their iterative solution process incurs high computational and time costs that often grow super-linearly with the size of the problem. This computational cost is a bottleneck in the use of high-fidelity FEM models in applications that require near-real-time predictions. Surrogate modeling can help alleviate this issue [3] through its ability to learn and accelerate simulations, whether in solid or fluid mechanics, and has shown varying degrees of success in the areas of computational solid mechanics [4]. However, due to its complexity, effective surrogate modeling of applications involving contact of deformable bodies, especially in the context of varying geometries, is still an open issue [5].

Generally, we can distinguish between intrusive and non-intrusive surrogate models [6]. Intrusive methods directly modify or access the internal structure of the high-fidelity model, i.e., the partial differential equation (PDE), while non-intrusive methods define a surrogate without having access to the full-order operators or to their action on a vector [7]. Both archetypes have recently been employed for surrogate modeling of contact mechanics problems. However, since contact is a complex problem driven by PDEs under constraints, surrogate modeling remains an area of active research.

Intrusive Surrogate Models for Contact.

The most commonly employed intrusive surrogate modeling technique in computational mechanics is based on using singular value decomposition (SVD) on snapshot data for generating a reduced-order basis [8]. To the best of the authors’ knowledge, this approach was used for the first time on contact mechanics problems by Krenciszek et al. [9]. Balajewicz et al. [10] propose to use different approaches to build the primal reduced basis and the dual reduced basis. In particular, the primal reduced basis is obtained through standard SVD, while the dual reduced basis is built using nonnegative matrix factorization (NNMF) on the Lagrange multiplier snapshots. Thereby, the method guarantees positive basis vectors. This method is extended further using hyper-reduction techniques such as the discrete version of the empirical interpolation method (DEIM) by Fauque, et al., [11] and others [12, 13]. However, [14] have pointed out that low-rank model order reduction techniques, which are effective for many physical systems, struggle to accurately represent the contact pressure field in contact mechanics due to its inherently high-dimensional and spatially localized nature. They show that the dual space remains high-dimensional, even under aggressive truncation.

Recently, physics-informed neural network models have also been employed as intrusive surrogates for contact mechanics. These weakly enforce the physical laws, evaluated at a set of collocation points, in the loss function. They have been used in the context of energy minimizations, see [5], or by employing the strong form of the PDE [15, 16]. Intrusive surrogate modeling has successfully accelerated certain problems in contact mechanics. However, it is still not fully equipped to address some inherent challenges. These challenges include non-smoothness, inequality constraints, and complementarity conditions, especially when friction, multiple deformable bodies, or self-contact are involved. These complicate reduced formulations and require direct access to full solvers, including robust contact search algorithms.

Non-intrusive Surrogate Models for Contact.

Non-intrusive surrogate models rely on access to a dataset from a high-fidelity solver. They focus on minimizing data fit errors against the ground-truth values. Using this concept, non-intrusive surrogate models for selected parts of the contact response (such as surface topography to friction coefficient maps) between deformable bodies have been developed in recent years that use feedforward neural networks [17, 18, 19], or Gaussian process regression [20]. Graph-based surrogates are ideal for data-driven methods relying on ground truth generated using the mesh-based solution due to the natural representation of meshes as graphs. Since contact mechanics problems are commonly solved using the finite element method, meshed geometries and their deformations are usually available from the solver. Therefore, there is an obvious link to surrogate models based on graph representations. Graph representations allow for relational inductive biases that are invariant on the node and edge permutation [21, 22]. Furthermore, they can learn arbitrary relations, which makes them ideal for dynamical predictions in complex systems [23, 24, 25, 26]. Similar graph frameworks have been adapted for both data-driven and physics-informed approaches for cardiac mechanics [27, 28].

In the context of contacting bodies, one of the biggest challenges for graph neural networks (GNNs) is the consistent detection and prevention of contact penetration. GNNs have been applied for the deformation of a hyper-elastic plate deformed by a rigid actuator and flag dynamics with additional specialized structure to learn relations based on proximity in physical space, which the authors use to handle collision or contact [29]. This approach which the authors termed MeshGraphNets relies on changes to the architecture and not additional loss terms. FIGNet [30] is a similar approach that recognizes the complexity of collision detection and adds representative face-face edge features to the structure, based solely on the bounding volume hierarchy (BVH). Others have used fast contact filters to regularize data-driven training for a case of rigid tool and deformable body contact [31]; these filters utilize the necessary but not sufficient conditions for contact or collision. This approach builds on past work that suggests improved learning outcomes by adding specialized physics-based loss terms [32, 33].

However, to the best of the authors’ knowledge, no studies to date have implemented a full-fledged contact algorithm (bounding volumes and necessary as well as sufficient conditions) to regularize graph-based data-driven learning for contact between soft bodies. The current work is the first to present such a framework and to discuss its implications. Table 1 compares the current work with similar or related works in the past.

Table 1:

Comparison of the current framework with existing frameworks. †: FIGNet generalizes the notion of world-space edge from nodes to faces as well. #: Works suggest potential application to contact between soft bodies. ‡: However, these utilize pairwise distance and collision radius to build edge relations. *: Both necessary and sufficient conditions for contact or collision detection.

	Mesh Space Edge	World Space Edge	Contact between Soft Bodies	Contact Detection Filter	Precise Contact Detection*	Contact Loss Term	Code Availability	Dataset Availability	Hierarchical Graph
Dalton, et al. [27]	✓	X	#	X	X	X	✓	✓	✓
Pfaff, et al. [29]	✓	✓	X	X‡	X	X	✓	✓	X
Allen, et al. [30]	✓	†	#	X‡	X	X	✓	✓	X
Zhu, et al. [31]	✓	✓	X	✓	X	✓	X	✓	X
This work	✓	✓	✓	✓	✓	✓	✓	✓	X

In the current work, we extend a graph neural network framework that has continuous collision detection to incorporate sufficient conditions for contact. This collision detection is used to compute a penalty term representative of the extent of the contact violation. Due to its continuous nature, it is robust enough to handle cases where the contact constraint violation occurs at any time between a given time step (and not just at the end) and cases where the elements completely go through each other during a time step. This algorithm is general enough to allow contact between multiple soft deformable bodies, unlike others in the past who have applied it to rigid-soft body interactions [31] where the contact interface is known in advance or to rigid-rigid body interactions [30]. We are aware that this entails a significant increase in training time compared to existing methods. However, from a computational contact mechanics point of view, this is a natural step forward. Our intention is to provide a proof-of-concept for the community and to gauge whether this additional training time is necessary.

We apply the framework to benchmark problems and show that the developed algorithm regularizes the training, leading to better generalization. Furthermore, we show that these aspects are true even for a complex contact case wherein the contact planes move and the element normal angles are not uniform. Additionally, we demonstrate more than a thousand-fold speedup in inference time. To provide a complete picture, we provide an inference speedup on multiple hardware architectures and training time for a single epoch on one of these platforms. In the interest of open science, we share our complete datasets and code base on the Texas Data Repository and Github, respectively.

The work is arranged as follows: first, Section 2 describes the framework. This includes the architecture of the graph neural network (Section 2.1), the implemented contact algorithm (Section 2.2), the type of losses used (Section 2.3), and additional details on the implementation and the training process (Section 2.4). Second, we describe the two benchmark problem datasets used to test the framework (Section 3.1). Finally, we define performance metrics (Section 4.1), present the results based on these metrics (Section 4) and discuss them.

2. Methods

2.1. Architecture

2.1.1. Graphs and Notation

In the following, we use uppercase and lowercase to refer to graph and vertex indices, respectively. We use Graph Neural Networks (GNNs) that operate on an input graph (G_I). A graph consists of objects that form its vertices (x) and their relationships, which form the edges (e). In general, depending on the object(s) and relationship(s) chosen, there can be more than one vertex type and/or edge type. The information is organized at the vertex-level, edge-level, and graph-level. We denote a vertex-feature using x_Ii for the i-th vertex in the I-th graph. Similarly, there is an edge-feature $e_{Iij}^k$ for a k-th type of edge between vertices (i, j) in the I-th graph. Finally, we denote the graph-level feature by g_I for the I-th graph. The different types of edges are differentiated using their superscripts.

2.1.2. Features

The ground truth data arising from the solution of a finite element simulation can be represented as graphs. Each time step in the solution data is one graph, each node is a vertex, and each element edge is an edge in a graph. Like Pfaff et al. [29], we define, in addition to existing ‘mesh-space’ edges, an edge-type based on positional proximity of nodes in the physical space referred to as ‘world-space’ connectivity. We use a similar notation to theirs and refer to these ‘mesh-space’ and ‘world-space’ edges using superscripts M and W respectively.

Graph Feature (g_I).

Simulation parameters that are fixed for all vertices of a graph form its graph features. In our case, this is the absolute time in the solution t, the time step size δt, the material constants, and other information such as applied pressure.

Node Features (x_Ii, y_Ii).

The state of each node consists of its current position r_Ii, current velocity v_Ii and current acceleration a_Ii. This together forms input nodal features x_Ii = [r_Ii, v_Ii, a_Ii] at current time t. The goal is to predict the nodal accelerations at time t + δt denoted by y_Ii.

Edge Features ($e_{Iij}^M$, $e_{Ikl}^W$).

The mesh-space edge is defined only between vertices that are connected by a finite element mesh, irrespective of their position; it is denoted by $e_{Iij}^M$ (see Equation (1)). Similarly, we have a world-space edge denoted by $e_{Ikl}^W$ between all nodes closer than a collision radius R (see Equation (2)):

$$e_{Iij}^M=\left[r_{Ii}-r_{Ij},{\Vert r_{Ii}-r_{Ij}\Vert }_{L^2}\right]\forall (i,j)\in \text{element connectivity},$$ (1)

$$e_{Ikl}^W=\left[r_{Ik}-r_{Il},{\Vert r_{Ik}-r_{Il}\Vert }_{L^2}\right]\forall (k,l)\text{such that}{\Vert r_{Ik}-r_{Il}\Vert }_{L^2}\le R.$$ (2)

The graph G_I consists of all its graph features, input-nodal features, and edge features. Thus, $G_I=(g_I,x_{Ii},e_{Iij}^M,e_{Ikl}^W)$ where the indices run as stated previously. For each graph, we have a set of ground truth target values (y_Ii).

2.1.3. Neural Network

We utilize an encode-process-decode architecture similar to those previously used in [27][29] with the message passing processor. Each part in this architecture utilizes a standard fully connected feed-forward neural network (simply MLPs hereon) with a nonlinear activation function. The rectified linear unit is used as an activation function. Skip connections are present in the hidden layers of the MLPs.

Encoders.

There are four types of encoders (see Equation (3)) each of which embeds a corresponding feature from the input graph ($g_I$, $x_{Ii}$, $e_{Iij}^M$, $e_{Ikl}^W$) to the corresponding encoded feature, denoted here by boldface version of the same symbol (${\mathbf{g}}_t$, ${\mathbf{x}}_{Ii}$, ${\mathbf{e}}_{Iij}^M$, ${\mathbf{e}}_{Ikl}^W$). Each of the encoders is an MLP denoted by ${ℰ}_g$, ${ℰ}_x$, ${ℰ}_e^M$, and ${ℰ}_e^W$ respectively:

$${\mathbf{g}}_I={ℰ}_g(g_I);{\mathbf{x}}_{Ii}={ℰ}_x(x_{Ii});{\mathbf{e}}_{Iij}^M={ℰ}_e^M(e_{Iij}^M);{\mathbf{e}}_{Iij}^W={ℰ}_e^W(e_{Iij}^W).$$ (3)

Message Passing (Processor).

The encoded nodal and edge features are used for message passing. They consist of three steps: message creation, aggregation, and update. The message creation is described in Equation (4). A distinct message is created for each type of edge. In particular, on the mesh-space edge denoted by $m_{Iij}^M$ and on the world-space edge denoted by $m_{Ikl}^W$. At the cost of additional computation, we enforce skew-symmetry of the generated messages to strengthen their interpretation as forces between the nodes [27] (see Equation (5)). For aggregation, we define a one-hop neighborhood of node Ii in the mesh-space denoted by ${𝒩}^M(Ii)$ (and ${𝒩}^W(Ii)$ in world-space) and aggregate all messages from edges within this neighborhood of the central node. The aggregation operator ⊕ is the sum of all messages that arrive at the central node. Finally, the aggregations from the mesh-space and world-space edges are used to update the embedded feature of the central node. The aggregation and update step is described in Equation (6). Finally, the generated messages are used to update the embedded edge-features (Equation (7)). The message generation function ϕ and the update function γ are both MLPs. The message passing process (Equations (4) to (7)) can be repeated using distinct generation and update functions in each round, referred to by the index n for up to k rounds. Therefore, the process reads:

$$m_{Iij}^{M(n)}={\varphi }^{M(n)}\left({\mathbf{x}}_{iI}^{(n)},{\mathbf{x}}_{Ij}^{(n)},{\mathbf{e}}_{Iij}^{M(n)}\right);m_{Iij}^{W(n)}={\varphi }^{W(n)}\left({\mathbf{x}}_{iI}^{(n)},{\mathbf{x}}_{Ij}^{(n)},{\mathbf{e}}_{Iij}^{W(n)}\right),$$ (4)

$${\tilde{m}}_{Iij}^{M(n)}=\frac{m_{Iij}^{M(n)}-m_{Iji}^{M(n)}}{2};{\tilde{m}}_{Iij}^{W(n)}=\frac{m_{Iij}^{W(n)}-m_{Iji}^{W(n)}}{2},$$ (5)

$${\mathbf{x}}_{Ii}^{(n+1)}={\gamma }^{(n)}\left({\mathbf{x}}_{Ii}^{(n)},\underset{j\in {𝒩}^M(Ii)}{\oplus }{\tilde{m}}_{Iij}^{M(n)},\underset{j\in {𝒩}^W(Ii)}{\oplus }{\tilde{m}}_{Iij}^{W(n)}\right),$$ (6)

$${\mathbf{e}}_{Iij}^{M(n+1)}={\mathbf{e}}_{Iij}^{M(n)}+{\tilde{m}}_{Iij}^{M(n)};{\mathbf{e}}_{Iij}^{W(n+1)}={\mathbf{e}}_{Iij}^{W(n)}+{\tilde{m}}_{Iij}^{W(n)}.$$ (7)

Decoder.

The dot product of the encoded graph-level features ${\mathbf{g}}_I$ with embedded nodal features ${\mathbf{x}}_{Ii}^{(k+1)}$ after k rounds of message passing is used for decoding. Three different decoders (${𝒟}_i$; i ∈ {1, 2, 3}), each of which is an MLP, are used to get each component of the nodal acceleration as shown in Equation (9):

$${𝒳}_{Ii}={\mathbf{x}}_{Ii}^{(k+1)}\cdot {\mathbf{g}}_I,$$ (8)

$${\widehat{y}}_{Ii}=[{𝒟}_1({𝒳}_{Ii}),{𝒟}_2({𝒳}_{Ii}),{𝒟}_3({𝒳}_{Ii})].$$ (9)

2.2. Contact Algorithm

2.2.1. Existing Algorithms

MeshGraphNets [29] take advantage of the relational inductive biases of graph neural networks [22] and incorporate world-space edges as a mechanism to establish a relationship between nodes that are close in physical space. This adds structure within the framework for learning relations arising from physical proximity, such as collision or contact. FIGNet[30] extends this idea by building face-face edges where the edges are built using bounding volume hierarchy [34]. Both of these utilize a fixed collision radius for building world-space edges and face-face edges, respectively. FIGNet[30] focuses on rigid body dynamics but suggests that the framework could work more generally. However, this is not explicitly demonstrated. Again, both of these rely on architectural changes to enable learning of contact behavior without enforcing or penalizing the violations. A natural next step is to use these in conjunction with a common method, like a contact penalty that is added to the loss function. Thereby minimizing the loss penalizes contact violations during training, which could potentially enable better contact behavior. This requires implementing a contact detection and response computation. Such a joint approach has been pursued in Ref. [31] wherein additional contact terms are utilized to regularize learning for the case of tool-object interactions. Filters based on geometric tests are used to rapidly determine contact pairs. However, these filters represent necessary but not sufficient conditions for contact detection. In the current work, we take this one step further and add sufficient conditions for contact to allow robust and precise detection and penalty computation. Furthermore, due to its robust nature of detection, the more general case of contact between deformable bodies can be handled.

2.2.2. Implemented Contact Algorithm

Overview.

The framework presented so far (Section 2.1) predicts the nodal accelerations, which are then integrated twice to compute the position for each node at the next time point. Here, the trapezoidal scheme is used for time integration, which is identical to that used in FEBio[35] for the generation of the FEM datasets (Section 3.2). A continuous collision detection algorithm is used to identify each colliding pair of triangles along their trajectory in the time step (t, t + δt). The uniform average velocity is used for the trajectory of each vertex between the time points. For each colliding pair, the response is computed at the end of the time interval, i.e., at (t + δt). Importantly, unlike FEM, there is no change or refinement of time-step size during training to avoid contact. Instead, our aim is to train the framework to learn to avoid contact by adding a penalty to the loss term.

Necessary Conditions

(see ‘Necessary’ blue box in Figure 1). First, axes-aligned bounding boxes (AABB) [36] are computed for each triangle enclosing its trajectory across the time-step. An overlap test between these bounding boxes [36] is used to quickly cull non-colliding pairs of triangles. Second, a check is implemented to explicitly remove neighboring triangles connected through mesh-space edges to reduce false positives. Here, we compute displacement vectors for centroids of triangles that share a mesh-space edge. Only pairs whose included angle is higher than a pre-defined fixed value are kept, and the rest are culled. This step is similar to the w_tol parameter used in FEBio’s contact formulation [35, 37] and mimics its generality for cases of self-contact. Third, vertex-face and edge-edge filters are used to identify all triangle pairs that are coplanar at any time during the trajectory in the given time step [31, 38]. These tests are a necessary but not a sufficient condition for collision. Only triangle pairs deemed to have become coplanar are processed in further steps.

Figure 1:

Schematic of the framework used in this work.Orange boxes are core components of the graph neural network. Blue boxes highlight the steps of the contact algorithm as described in the Section 2.2. This is split into steps for necessary conditions of collision followed by sufficient conditions of collision. Green boxes indicate the computed losses which are combined using weighted sum.

Sufficient Conditions

(see ‘Sufficient’ blue box in Figure 1). Unlike in Ref. [31], a sufficient condition for collision is implemented using the procedure described in [39]. For each vertex-face or edge-edge pair, a cubic polynomial is formed, the roots of which are the exact time points where the coplanarity occurs. Newton’s method is used to find each root of these polynomials. All roots outside of the time step are discarded. For those within the time step: (i) for each vertex-face pair, a system of linear inequalities is solved to determine whether the vertex lies within the face, and (ii) for each edge-edge pair, a check is implemented to determine whether the two line segments in three-dimensional space intersect with each other. The latter of the two conditions replaces equation (2) of [39] to avoid false positives where collinear edges are detected as colliding edges, illustrated in Figure 2.

Figure 2:

Changes due to modification of edge-edge sufficiency condition present in [39]. (a) Top and front view of two meshes, in black and orange, initially separated by a distance that move across each other. Snapshots at initial time, time of collision, and final time. (b) The number of collisions detected for each triangle. Note without the modification the smaller mesh incorrectly detects false positives. (c) The contact response (eq. (13)) for each triangle. Note previously detected false positives now return zero contact response.

Computing the Contact Response.

After ascertaining that a pair of triangles collide during a time step using the collision detection algorithm, we compute the contact response. Each triangle pair (i, j) has six (3x2) vertex-face and nine (3x3) edge-edge responses defined in the Equations (10) and (11). We use the maximum response value among these (Equation (12)). Furthermore, since a single triangle can potentially collide with multiple triangles, we take the maximum response across all such colliding pairs for each triangle (Equation (13)). This computed contact response for two test cases is shown in Figure 3. The key strength of the algorithm is that it is a continuous collision detection. It identifies collisions even for cases where the elements go past each other completely during a time step. This strength is observable in regions of peak contact response values in both Figure 3.a and 3.b. The contact response is zero when two triangles do not contact or collide during a time step.

Figure 3:

Computed contact response (Equation (13)) using the implemented contact algorithm. a. Two parabola-like meshes move toward each other across a given time step. Some elements pass each other completely during the step. b. Two rectangular meshes with imposed undulations (Equation (18)) have fixed boundaries and inflate towards each other. Unlike a, the contact is asymmetric. As in a., some elements move past each other completely.

$$r_{Iij-VF}^p=\{\begin{array}{c}\text{orthogonal distance for the}{p}^{th}\text{vertex-face test evaluated at time}(t+\delta t),\hfill \\ 0\text{otherwise}.\hfill \end{array}\phantom{\}}$$ (10)

$$r_{Iij-EE}^q=\{\begin{array}{c}\text{distance between mid-points of two edges for the}{q}^{th}\text{edge-edge test evaluated at time}(t+\delta t),\hfill \\ 0\text{otherwise}.\hfill \end{array}\phantom{\}}$$ (11)

$$r_{Iij}=\underset{\forall p,q}{\max }\left\{r_{Iij-VF}^p,r_{Iij-EE}^q\right\},$$ (12)

$$r_{Ii}=\underset{\forall j}{\max }{r}_{ij}.$$ (13)

Algorithmic optimization.

We give special attention to optimizing the algorithm for computational efficiency and memory usage. The collision pairs and the response are stored in sparse COO format tensors. The algorithm is vectorized with as few iterators as possible. Batch dimensions are added at specific places to establish an upper bound on memory requirements. For root finding, we exploit the structure of cubic polynomials [40]. Newton iterations are implemented, not as a while loop conditional on convergence, but rather as for loops wherein the loop body has been lifted to a compiled function. The compilation (via torch.compile) has been used to wrap methods where such usage does not lead to incorrect behavior. Finally, Horner’s scheme is utilized to evaluate polynomials whenever necessary.

2.3. Loss Functions

Dynamic Loss.

Given a set of graphs $𝒮$ (in a specific batch), where each graph is defined as in Sections 2.1.1 & 2.1.2, the dynamic loss ${ℒ}_d$ is defined as the mean square error in the predicted nodal accelerations (Equation (9)) and the ground-truth nodal acceleration (y_Ii) across all components, each node [1, n_nodes], and all graphs [1, n_batch] in $𝒮$ (Equation (14)):

$${ℒ}_d=\frac{{\sum }_{I=1}^{n_{batch}}{\sum }_{i=1}^{n_{nodes}}{\sum }_{m=1}^3{({\widehat{y}}_{Ii}-y_{Ii})}_{,m}^2}{3\cdot n_{batch}\cdot n_{nodes}},$$ (14)

where (·),_k refers to the k-th component of vector (·).

Contact Loss.

The contact response (Equation (13)) is normalized using the length-scale of the problem (l_c) (Equation (15)). We choose this for normalization since contact response has base dimensions of length and is approximately proportional to the length scale of the problem. The mean absolute error of this normalized response is the unsupervised term which defines the contact-loss ${ℒ}_c$ (Equation (16)):

$${\overline{r}}_{Ii}=\frac{r_{Ii}}{l_c},$$ (15)

$${ℒ}_c=\frac{{\sum }_{I=1}^{n_{batch}}{\sum }_i\mid {\overline{r}}_{Ii}\mid }{{\sum }_{I=1}^{n_{batch}}{\sum }_{i}1}.$$ (16)

Total Loss.

The total loss $ℒ$ is computed as the weighted sum of the dynamic loss (Equation (14)) and contact loss (Equation (16)):

$$ℒ=w_d{ℒ}_d+w_c{ℒ}_c.$$ (17)

2.4. Implementation and Training Process

The implementation is done in PyTorch [41] & PyTorch Geometric [42] and relies on the Adam optimizer [43]. Multiple networks are trained with just dynamic loss or both dynamic and contact loss included (see Section 4.2 for more details). All datasets are split in the ratio of 8:1:1 into training, validation, and test sets. We utilize a step function for the learning rate schedule. The matrix multiplication approach is disabled for torch.cdist() to favor a slower but more precise distance computation for proximity calculations for building world-space edges.

3. Numerical Experiments

3.1. Benchmark Problems

We apply the presented framework to two problems of different graph sizes, length scales, dataset sizes, and contact complexities (see Figure 4 and Table 2). For each problem, contact occurs between deformable soft bodies. The dataset is generated through finite element simulations using FEBio [35]. The contact algorithm presented in Section 2.2 is different from the contact algorithms used in the generation of ground-truth data for the following benchmark problems. For more details, please refer to the FEBio [35] input file in .feb format supplied with the dataset.

Figure 4:

Problems used for the numerical experiments. (a) Varying geometry membrane wherein material properties, pressure load, and geometry is varied, and (b) bioprosthetic aortic valve.

Table 2:

Details of the graph dataset created for both the benchmark problems.

	Symbol	Varying Geometry	Aortic Valve
Contact Plane Symmetry	–	No	Yes
Same Contact Plane Location	–	No	Yes
Num. of Simulations	ns	25	25
time steps per Simulation	nt	41	21
Num. of Graphs	ns · nt	1025	525
Num. of Nodes	nn, nnodes	1250	1014
Num. of Elements	nface	2304	1872
Num. of Mesh-space Edges	ne	7104	5772
Num. of Global Parameters	ng	2	4
time step Size	δt	0.025	0.025
Collision Radius (for ‘world-space’ edges)	R	1.45 × 10−1	6.32 × 10−4
Length Scale	lc	2.84	1.26 × 10−2

3.1.1. Problem 1: Inflating Membranes with Varying Reference Geometry

A gap separates two rectangular sheets with fixed edges that have a pressure load applied to their outer faces, pushing them towards each other as shown in Figure 4(a). The sheets have undulations in the reference configuration described by:

$$\begin{gathered} z_{[u∕l]}=A\sin k_{1[u∕l]}x\sin k_{2[u∕l]}y+c_3, \\ {k}_{1[u∕l]},k_{2[u∕l]}\in \{\pi ∕2,\pi ,2\pi ,4\pi \}\text{where}(\mathrm{u},\mathrm{l})\text{are (upper, lower) surfaces}. \end{gathered}$$ (18)

The constant c₃ is chosen to be half the distance between the mean plane of each sheet and the amplitude A to be smaller than half the gap distance. While the amplitude for undulation for each sheet is the same, the {k₁, k₂} can be different for the upper and lower membranes (denoted by subscripts (k_xu, k_xl) respectively) and across the simulations. The material is isotropic linear elastic. This problem dataset consists of 25 simulations in which Young’s modulus and the applied pressure load are sampled from a uniform random distribution. Note that, unlike the next benchmark problem, the contact is asymmetric, i.e., (a) the contact does not need to occur at the mid-plane, and (b) the contact occurs at a wide range of element normal angles. This is necessary to clearly establish the framework’s ability to learn ‘contact’ wherever it occurs instead of ‘learning to stop at the mid-plane’. A ‘facet-to-facet sliding’ contact algorithm built in FEBio [35] with a penalty contact formulation is used for finite-element analysis of this benchmark problem.

3.1.2. Problem 2: Bioprosthetic Aortic Valve

We build a bioprosthetic aortic valve model (Figure 4(c)) geometry using the procedure described by [44]. The valve thickness and diameter are taken from [44] and [45], respectively. The belly-edge is the only free edge, while other edges are fixed. The material is an isotropic linear-exponential model given by the strain energy function Ψ:

$$\Psi =\frac{c_0}{2}(I_1-3)+\frac{c_1}{2}\left(e^{c_2{(I_1-3)}^2}-1\right),$$ (19)

where I₁ is the first invariant of the Cauchy-Green deformation tensor. We use the baseline values provided in [46]. The mean blood pressure is set at 95 mmHg [47]. This dataset contains 25 simulations. To create the data, we vary the material parameters (c₀, c₁, c₂) and the applied pressure load P. This case represents a simpler case of contact where the surfaces meet at known planes and the element angles are symmetric about this plane. However, the model needs to find and detect the contact itself. A ‘contact potential’ algorithm built in FEBio [35, 37] with a penalty contact formulation is used for finite-element analysis of this benchmark problem.

3.2. Generated Datasets

The varying geometry (Section 3.1.1) and aortic valve (Section 3.1.2) problems have respectively six (E, p, k_1u, k_2u, k_1b, k_2b) and four (c₀, c₁, c₂, p) parameters that vary across the simulation. We add absolute time in the simulation (t) and the time step (Δt) to form graph-level features for all graphs in the dataset. For the varying geometry example, we exclude the shape parameters (k_1[u/l], k_2[u/l]) to test the network’s ability to learn complex and differing reference geometries for similar problems. Note that the time step size, time in simulation, and the shape parameters are discrete variables, while the rest are continuous variables. Uniform random sampling is done to generate datasets. Appendix B describes the value of parameters and their distribution for each problem in detail.

4. Results & Discussion

4.1. Performance Metrics

The nodal positions are computed for the FEM and for the framework developed herein using Equation (20). The mean absolute error of the displacement step prediction ($\Delta {\widehat{r}}_{Ni}$) against the ground truth displacement step ($\Delta r_{Ni}$) normalized by the problem length scale (l_c) serves as position loss (${ℒ}_p$) (Equation(21)). Unlike the previously discussed losses, the position loss is directly interpretable, and its value on the training and validation sets is monitored during the training process. During training, we compute position loss over batches and monitor the average of all batches in the epoch, while in inference, we do so over each graph and report the statistics. Similarly, we monitor the contact loss (Equation (16)) and report statistics on each graph after inference. We also report the quartile boundaries of the prediction error set E defined in Equation (22). Finally, we accumulate the one-step prediction errors across all time points of a simulation (Equations (24), (25)) and report their evolution.

$$\begin{gathered} r_{(N+1)i}=\text{FEM}(r_{Ni}){\widehat{r}}_{(N+1)i}=\text{GNN}(r_{Ni}) \\ \Delta r_{Ni}=r_{(N+1)i}-r_{Ni}\to {\xi }_{Ni}=\Delta r_{Ni}-\Delta {\widehat{r}}_{Ni}\leftarrow \Delta {\widehat{r}}_{Ni}={\widehat{r}}_{(N+1)i}-r_{Ni} \end{gathered}$$ (20)

$$\mathbf{Position}\mathrm{Loss}:{ℒ}_p=\frac{{\sum }_{I=1}^{n_{batch}}{\sum }_{i=1}^{n_{nodes}}{\sum }_{m=1}^3\mid {(\Delta {\widehat{r}}_{Ni}-\Delta r_{Ni})}_{,m}\mid }{n_{batch}\cdot n_{nodes}\cdot 3\cdot l_c},$$ (21)

$$\mathbf{Position}\mathbf{Error}\mathrm{Set}:E=\left\{e_{Ni}\mid e_{Ni}={\Vert {\xi }_{Ni}∕l_c\Vert }_{L^2},i\in [1,n_{nodes}]\&N\in [1,n_{graphs}]\right\},$$ (22)

$$\mathbf{Average}\mathbf{Position}\mathrm{Errors}:{\overline{e}}_N=\frac{{\sum }_{i=1}^{n_{nodes}}{e}_{Ni}}{n_{nodes}};\overline{e}=\frac{{\sum }_{N=1}^{n_{graphs}}{\sum }_{i=1}^{n_{nodes}}{e}_{Ni}}{n_{graphs}\cdot n_{nodes}},$$ (23)

$$\mathbf{Accumulated}\mathbf{Position}\mathrm{Error}:{\Xi }_{Mi}=\sum _{N\le M-1}{e}_{Ni}\text{where}M\ge 1;{\Xi }_M=1\frac{{\sum }_{i=1}^{n_{nodes}}{\Xi }_{Mi}}{{\sum }_{i=1}^{n_{nodes}}1},$$ (24)

$$\mathbf{Average}\mathbf{Acc}.\mathbf{Position}\mathrm{Errors}:{\overline{\Xi }}_N=\frac{{\sum }_{i=1}^{n_{nodes}}{\Xi }_{Mi}}{n_{nodes}};\overline{\Xi }=\frac{{\sum }_{N=1}^{n_{graphs}}{\sum }_{i=1}^{n_{nodes}}{\Xi }_{Mi}}{n_{graphs}\cdot n_{nodes}}.$$ (25)

4.2. Networks and Training

Multiple networks are trained for each of the datasets. These networks are either (relatively) small networks (denoted by the prefix S-) or large networks (denoted by the prefix L-). The size of the small (and large) networks is different for each of the problems and is a function of the hyperparameters. The smallest network has around 600k trainable parameters. The networks are trained in two modes. In ‘dynamic-only’ mode (denoted by suffix -D), the training is only dependent on the dynamic loss, i.e., (w_d, w_c) = (1, 0) in Equation (17). In ‘dynamic and contact’ mode (denoted by suffix -DC), training starts with a ‘dynamic-only ’ mode for 1000 epochs followed by the inclusion of contact loss afterward, i.e., (w_d, w_c), (0, 0) after the thousandth epoch. We do so to avoid an overemphasis on contact loss when the predictions from the surrogate model are bad in its initial training stage. We highlight that the inclusion of contact loss increases the training time significantly, more in Section 4.6. For this reason, it was not possible for us to perform ablation studies. However, since the S-D and S-DC networks are initialized identically, we believe that a fair comparison is possible.

The weight for w_d is set to unity, and w_c is chosen to bring the computed contact loss to the same scale as the dynamic loss at the thousandth epoch. In the following sections, we look at three networks based on the combination of size and training modes: (S-D, S-DC, L-D). The overall training process is shown in Figure 5, where the total loss ($ℒ$, Equation (17)) reduces several orders of magnitude. All hyperparameters of the networks, learning rate schedules, and evolution of each type of loss for all networks and problems are presented in detail in Appendix A.

Figure 5:

Total ($ℒ$) over training set for (a) varying geometry and (b) aortic valve problems. Note that the losses are average across all batches at the respective epoch.

4.3. Impact of Contact Loss Inclusion on Learning

4.3.1. During the Training Process

During the training process, the loss quantities at each epoch are available as an average of all batches in an epoch. To get finer insights, we use the checkpointed model at regular intervals during the training process and infer on training, validation, and test sets. This inference leads to one quantity per graph for position (${ℒ}_p$, Equation (21)) and contact (${ℒ}_c$, Equation (16)) loss. We obtain a vector of dimension n_nodes for position error (e_Ni, Equation (22)) at each graph. Figure 6 presents the quartile boundaries and averages of the these quantities at various epochs on the test set. Inference is done at two common epochs (100 & 500) for both problems, followed by regular sampling until the end of the training.

Figure 6:

Quartile boundaries (box and whisker) and mean values (circles) of position loss, contact loss, and position error when inference is done on test set at specified epochs during the training process. (a) is for varying geometry problem and (b) for aortic valve problem. Note that for position and contact loss, we evaluate over one graph each in batch dimension axes.

Aortic Valve (see Figure 6.b).

The contact losses for the S-DC network are similar to the S-D and L-D networks before the thousandth epoch, at which point it becomes part of the total training losses. Thereafter, the distribution of the contact loss shifts considerably to lower values for S-DC compared to the S-D and L-D networks. A similar trend is observed for the position loss and position error; however, unlike contact losses, the position losses and position error start out higher for the S-DC network than for the S-D or L-D networks before overtaking them.

Varying Geometry (see Figure 6.a).

The reduction in contact and position losses occur as they did in the previous problem. However, both quantities do not undergo an as pronounced reduction as compared to the aortic valve problem, this can be seen by the differences in the scale of the vertical axes.

Common.

Thus, across both benchmark problems, we note that adding a contact loss term aids in learning the contact behavior. Even though the improvements in the positional losses (and error) are marginal, we observe a substantial downward shift of the distribution (and maximum values) for contact loss. This indicates that the inclusion of contact loss has a regularizing effect as the network progressively trains.

4.3.2. On the Final Trained Model

To see the regularization and generalization effect more clearly, the same metrics are shown for the final epoch and across all dataset splits in Figure 7.

Figure 7:

Quartile boundaries (box and whisker) and mean values (circles) of position loss, contact loss, and position error when inference is done using the final trained model on each of training, validation, and test sets. (a) is for varying geometry problem and (b) for aortic valve problem. Note that for position and contact loss, we evaluate over one graph each in batch dimension axes.

Aortic Valve (see Figure 7.b).

For contact losses, the network S-D performs well on the training set. However, it does not generalize well to either the validation or the test set. This is seen as a shift of the contact loss distribution towards much higher values. While the network S-DC also shifts towards higher contact loss, the shift is not nearly as substantial as for the S-D case. A much larger network L-D shows similar trends as S-D. The increase in size of the L-D network is not sufficient to counter the improvements brought in by the inclusion of contact loss. The inclusion of contact loss leads to better generalization in this case.

Varying Geometry (see Figure 7.a).

Similar performance (as for the aortic valve) is observed across training, validation, and test sets for each network. The contact losses are much lower for the bulk of the distribution (in the interquartile range) for the S-DC networks than for the S-D network across all dataset splits. Similar to the aortic valve problem, the performance on position error for S-DC is only marginally better than S-D and L-D. However, unlike the previous problem, we note that increasing the size of the network (L-D) without the inclusion of contact losses also leads to similar, but not better, contact loss performance. Note that this is a case of complex contact, specifically, (i) the surfaces do not meet at the same plane in each simulation or the same plane within the same simulation, and (ii) the contact angles of the elements that form the contact pair are not uniform. The regularization effect and generalization, in a case of complex contact scenario, is observed to be less pronounced.

4.4. Temporal Performance and Instabilities

The final trained model is used to infer the position errors (e_Ni, Equation (22)) and accumulated position errors (Ξ_Mi, Equation (24)) across all the dataset splits. These quantities are scalars for each node of each graph. We compute the averages over each node of the graph and over all graphs at the same time step across all simulations in training, validation, and test sets. This leads to average position errors ($\overline{e}$, Equation (23)) and average accumulated position errors ($\overline{\Xi }$, Equation (25)). These metrics are plotted against the time step number normalized by the total number of time steps for both problems in Figure 8. Note that not all simulations contain an equal number of time steps, and we restrict the plot to the least common time slice across all simulations of the given dataset to allow for meaningful comparison. We also show the position error for each problem and network at one of the timesteps in Figure 9.

Figure 8:

Average position error (Equation (23)) and average accumulated position error (Equation (25)), both averaged across nodes and graphs of simulation at same time step. a: Varying Geometry for first three column, left to right are evaluations over training, validation, and test set. b: Aortic Valve for the last three columns, left to right are evaluations over training, validation, and test set.

Figure 9:

Position error (Equation (22)) for a randomly selected simulation from the test set for each of the problems at the penultimate time step.

Average Position Error (see top row of Figure 8).

At each time step, we note that S-D and S-DC networks result in similar average position error $\overline{e}$ on the training set. For the aortic valve problem, this trend extends to validation and test sets. However, for the varying geometry problem, we see that the S-DC network leads to lower $\overline{e}$ than the S-D network during the majority of the time steps. The L-D network performs similarly to S-DC at each time step of the varying geometry problem but does worse for the aortic valve problem. These observations are consistent with the average values of the position error (e_Ni) seen for all the networks and sets in Figure 7. Although the S-DC network shows marginal improvements in positional error, we see it generalizes better than network without contact (S-D), at least for the varying geometry case.

Average Accumulated Position Error (see bottom row of Figure 8).

This measure computes the accumulated position error norms over time. Thus, it establishes an upper bound on the position error if all displacements were summed to obtain the current position. We see an increase in this upper bound as time progresses, for all the networks. We recognize that this error must be reduced to make the current framework a viable alternative surrogate for nonlinear contact mechanics problems with contact present. Rolling out, i.e., using the predicted current step for the next step, leads to instabilities. This is expected and detailed in the previous studies using graph neural networks [24][27][29]. So far, we have not utilized strategies, such as training with injected noise, to make our networks more robust to these instabilities due to the computational cost (see Section 4.6), but this remains a future direction.

4.5. Speedups

The FEM software FEBio [35] that has been used to generate the dataset cannot utilize GPU and relies on CPU parallelization. On the other hand, machine learning models can run with CPU but can also take advantage of GPU acceleration. To (i) avoid unfair comparison, and (ii) provide realistic speedups with each hardware accessible to each framework, we provide the run times (inference times) and speedups for three platforms. Platform 1 relies exclusively on CPU and is the only one accessible to the FEM process. Platforms 2 and 3 are for GPU inference, with the former on an NVIDIA Quadro P4000 GPU and the latter on an NVIDIA A100 GPU. Table 3 summarizes the average time per simulation and speedups, all with respect to FEM run times on platform 1. Note that a twenty five fold speedup is achieved even with just the CPU and a problem as complex as the varying geometry. However, the inferences on platforms 2 and 3 provide over a thousand-fold speedups. On platform 3, five of the six networks provide an inference per simulation within a second. We observe similar inference times for networks S-D and S-DC which is lower than network L-D. This is expected since no contact losses are computed in inference and the compute times are proportional to problem and network sizes. Refer to Supplementary A for a detailed comparison of inference speeds with other GNN simulators.

Table 3:

Inference time per simulation in seconds for three platforms and speedups achieved with respect to FEM run on platform 1 for both benchmark problems and all networks. The network prefixes S- and L- indicate relatively smaller and larger networks, respectively. The network suffixes -D and -DC are networks trained on just dynamic loss or both dynamic & contact loss, respectively. Platform 1: Inference using CPU; Intel(R) Xeon(R) CPU E5-2697 v4 @ 2.30GHz with 16 cores at 1 thread/core and 128GB RAM. Platform 2: Inference using NVIDIA Quadro^® Pascal P4000 GPU with memory of 8GB. Platform 3: Inference using NVIDIA A100^®GPU with memory of 40GB. Note that due to the hardware differences, some of the JIT compilation that we could use for platform 3, could not be used on platform 2.

Problem	Network	FEM	Inference
		Platform 1	Platform 1		Platform 2		Platform 3
		Time [s]	Time [s]	Speedup	Time [s]	Speedup	Time [s]	Speedup
Varying Geometry	S-D	1583.7	27.80	57.0	1.36	1164.5	1.00	1583.7
	S-DC		23.32	67.9	1.36	1164.5	0.96	1649.7
	L-D		62.60	25.3	3.60	439.9	1.84	860.7
Aortic Valve	S-D	954.1	4.96	192.4	0.40	2385.3	0.64	1490.8
	S-DC		4.76	200.4	0.44	2168.4	0.60	1590.2
	L-D		7.20	132.5	0.88	1084.2	0.88	1084.2

Note that due to the hardware differences, some of the JIT compilation that we could use for platform 3, could not be used on platform 2.

4.6. Training Costs and Trade-offs

The Table 4 shows the time it takes to train one epoch of a given network on platform 3. For these time profiling, the batch sizes are chosen to make the most efficient use of the GPU memory and vary across the networks. Clearly, the inclusion of contact losses incurs significantly more computational costs (for network S-DC) than (i) training a similar-sized network without contact (S-D) or (ii) a larger network (L-D) with the number of trainable parameters at least three-fold higher than that in the small networks (S-D and S-DC, see Table A.5). There is a trade-off between the cost incurred in training and the benefits provided by the regularizing effect (and better generalization) owing to the inclusion of contact loss. Another consideration is the importance of adherence to contact constraints or violations thereof in applications where the current framework might be useful. More specifically, similar adherence when inferring from unseen data. We might be able to swing this trade-off more in favor of the presented framework by improving the computational efficiency of the contact algorithm, possibly through the following directions:

Table 4:

Total training times for each of the networks for the specified number of epochs, as well as total training times. *: for the S-DC network the first thousand epochs are as fast as the S-D network and differences occur only thereafter; in the table above, times for 1 and 1000 epoch are with contact present, and the total training time takes into consideration that first 1000 epochs happen without contact loss computation. Note that the S-DC networks are the rate-determining network if comparison is to be made for ‘equally-trained models’. This allowed us to run S-D and L-D networks on slower NVIDIA A40 GPUs to allow more economical use of our computational budget.

Problem	Network	Training Time on Platform 3
		1 epoch [in min.]	1000 epoch [in hr]	Total Epochs [-]	Total Training Time [in days]
Varying Geometry	S-D	0.21	3.48	3000	0.44
	S-DC	8.81	146.68	3000	12.66*
	L-D	0.61	10.32	3000	1.29
Aortic Valve	S-D	0.10	1.74	9400	0.68
	S-DC	2.99	49.84	9400	18.13*
	L-D	0.34	5.66	9400	2.22

• Just-in-time compilation: Currently, although some parts of the contact algorithm are compilable, a wider compilation is not achieved due to constraints that come with such compilation. Reworking the code to make this possible would allow for larger code compilation.

• Replacement of current subalgorithms with more efficient ones: (i) for solving linear inequality systems, (ii) for roots of cubic polynomials within a given interval robust enough to handle all root scenarios, and (iii) for axes-aligned bounding boxes, preferably hierarchical and reliant on tree structure.

• Reduction in decision-making statements and GPU memory footprint.

• Incorporation of multi-GPU distributed training.

In shared code base we provide detailed documentation with ample inline comments to make it easy for any reader interested in improving on the current framework. These avenues remain options for the future direction of our work.

4.7. Contact Algorithm

The contact algorithm in the current work improves upon the work done in Ref. [31] by incorporating sufficient conditions for contact detection. This enables precise contact detection. We add two measures to reduce false-positives: (i) the neighboring element checks in ‘necessary condition’ phase, and (ii) replacing equation (2) of [39] in the ‘sufficient condition’ phase (see Section 2.2.2 for both). Former is for increasing robustness in case of contact between elements that are connected directly and latter is to fix an incorrect condition. Importantly, the implementation is designed for use on GPU. The contact implementation, openly available with our shared codebase, can be used with other machine-learning algorithms as well.

We note that some aspects of the contact algorithm are translatable to an FEM algorithm and some are not. For example, the presented contact detection can be used in FEM analysis in the same way. Practically, our mechanism of handling contact is different from FEM. We minimize contact losses during the training to penalize contact violations. But we do not refine the time-step size to avoid contact. Rather, we expect the framework to learn to predict correct displacements.

We remark that we do not keep track of stresses in the current version of our framework. Furthermore, the current algorithm is restricted to frictionless contact between deformable bodies. However, since the contact detection checks all elements that are part of the same mesh object and thus the same mesh-space graph, the algorithm can handle self-contact. Additionally, the algorithm can also handle disconnected subgraphs, since these can be encoded into one graph representation. This, for example, occurs for the varying geometry problem where there is just one mesh-space graph even though there are disconnected subgraphs, i.e, separate upper and lower sheets in Section 3.1.1. The algorithm is also general enough to handle multi-element contact. However, we have only demonstrated this in a limited manner, as it occurs in the central region of the aortic valve problem. Our choice of benchmark problems is primarily driven by our intention to apply this framework in the area of cardiovascular biomechanics.

In future work, the presented framework will need to be more quantitatively compared to physics-informed neural network [16, 15] and deep energy structures [48] that have successfully incorporated contact loss terms for structural mechanics problems. However, these works have so far been restricted to elastic materials with fixed geometries, while our primary focus is on varying geometries with (potentially) inelastic material behaviors.

4.8. Physical and Kinematic Consistency

Unlike FEM, there is a possibility that displacement solutions are physically inconsistent, but this is a problem for all data-driven machine learning methods (also physics-informed methods during inference) and beyond the scope of the current work. Further, the Dirichlet boundary conditions are learned from the data and not encoded into the problem. Any displacement solution that is kinematically inconsistent with these boundary conditions is penalized automatically through the dynamic loss term. However, in the future, we plan to encode such boundary conditions explicitly for stricter enforcement, see e.g. [49]

4.9. Future Directions

Datasets.

The graph datasets utilized contain 1025 and 525 graphs for varying geometry and aortic valve problems, respectively. Each of these corresponds to 25 simulations. The size of our current datasets is limited by the time required for training and the memory on the available GPU hardware. In the future, we plan to expand these dataset sizes to increase the number of simulations and thus the number of graphs. Further, we will build the capability to handle more nodes and edges in graphs. We posit that this would lead to better performances for all networks. Currently, we have not used any data augmentation strategies. We intend to utilize these for future works, at least applying affine transformations and Gaussian noise injection specifically to improve spatial invariance and stability during ‘rollouts’ [29, 24], respectively.

Stresses and Traction.

The current work only focuses on developing better surrogates for nodal displacements. We neither compute nor predict stress or traction. However, in future works, these can be either computed or predicted. This will be important, especially in the context of frictional contact, where tangential and normal surface tractions play an essential role.

Other Physics-Informed Losses.

The dynamic loss in the current work is a type of data misfit error. The contact loss is a penalty for contact violation. We could extend the framework by adding other physics-informed losses to address some of the concerns arising in Section 4.8. Such adaptation of graph-neural networks is possible and has been shown Refs. [28, 50].

Boundary Conditions.

In the current work, there are no nodal features to indicate whether the nodes lie on the Dirichlet boundary condition regions. Instead of enforcing this condition, the minimization of dynamic loss that has contributions from these nodes leads to ‘soft’ enforcement [48]. Thus, they are learned through the training process. This can limit its usage in problems where some nodes have prescribed trajectories. In future works, we will add a one-hot encoding for boundary condition nodes and enforce this. Such enforcement would perform better, especially for problems where the Dirichlet boundary conditions change with time or loadstep.

Inference on Different Graph.

One of the powerful features of graph representation learning is its ability to infer on graphs that are different in the number of nodes and edges from the ones it is trained on [23]. This might be useful in cases when inferring on meshes that represent the same reference geometry but have different mesh sizes. We remark that our current implementation lacks the bookkeeping to do this. However, others have shown that this is possible, see for example Ref. [29]. Our implementation could be extended to do the same. Since this would allow for the incorporation of meshes of different fidelities, we could extend our framework for multi-fidelity learning [51]

5. Conclusions

We presented an extension of the data-driven graph neural network to learn contact between two deformable bodies (unlike rigid-soft interactions). A contact algorithm is implemented that relies not only on necessary but also on sufficient collision conditions. This algorithm is continuous; i.e., it detects interpenetrations at any point between the time steps. This enables calculation of the contact loss term, which penalizes contact violations during training. This framework will be particularly helpful in applications that require rapid machine learning-based surrogates for nonlinear mechanics involving contact between soft bodies. The implemented framework, at least on the benchmark problems considered, resulted in a regularizing effect and led to better generalization. Moreover, this held for simple contact (contact at similar planes and element normals) and complex contact (contact at differing planes and element normal angles). It should also be noted that, for the considered varying geometry benchmark problem, the current framework had no information on the shape parameters. Yet, benefits were observed despite the varying reference configurations in each simulation. These benefits come in addition to up to a thousand-fold speedups in inference, but also incur tremendous computational costs during training.

The idea of improving graph neural networks by utilizing robust contact detection algorithms and then penalizing contact violations is straightforward. However, this direction is relatively unexplored, likely due to the considerable efforts required to implement a comprehensive contact detection algorithm within an automatic differentiation framework with a large graph. We find that although there are benefits to moving in this direction, it comes at a computational cost that can be prohibitive. Through this work, we share with the community a practical benchmark for weighing its benefits against its costs.

Appendix A. Training Process

The step function learning rate schedule is used to train each network and is shown in the Figure A.10. The learning rate schedules are identical for the ‘−D’ mode training and slightly different (near the end) for the ‘−DC’ training mode. The extensive exploration of the hyperparameter space or more experiments with different learning rate schedules could not be carried out due to the high computational costs of the training. This training cost is higher for the varying geometry problem than for the aortic valve problem due to the larger dataset size. The training of networks in ‘−D’ is computationally cheaper. Still, we select the equally trained model at the last common checkpointed epoch to present a fair comparison across all networks. Due to this consideration, the varying geometry and aortic valve problems are trained to 3000 and 9400 epochs, respectively. For the optimizer, we utilize weight decay during the learning process. Numerical stability is improved by: (i) using available options in the Adam optimizer implementation in PyTorch, i.e. setting the ϵ parameter, and (ii) enabling gradient norm clipping.

Options to improve numerical stability available in Adam optimizer implementation of PyTorch are also used in addition to gradient norm clipping.

Figure A.10:

Learning rate schedule for all networks for both the problems.

The network parameters for each encoder, processor, and decoder MLPs are shown in the Table A.5. We also report the dimension of each feature space and its corresponding embedding space. Note that different MLPs of the same size are used for each round of message passing. The rounds of messaging passing for each network are reported, too. Finally, we compute the total number of trainable parameters for each of the networks and report those.

Table A.5:

Neural network size and total number of trainable parameters for each network. $𝒟(\cdot )$ is dimension along non-batch axis, $𝒲(\cdot )$ is width of MLP hidden layer, and $\mathbb{D}(\cdot )$ is depth of MLP where (·) or other symbols follows the notation in Section 2.1. Parameters that are changed for the large network compared to the small network are denoted using (.)*.

Parameter	Varying Geometry		Aortic Valve
	Small Network S-D/S-DC	Large Network L-D	Small Network S-D/S-DC	Large Network L-D
$𝒟(x_{Ii})$	9	9	9	9
$𝒟(e_{Iij}^M)$	4	4	4	4
$𝒟(e_{Iij}^W)$	4	4	4	4
$𝒟({\mathbf{x}}_{Ii})$	64	64	64	64
$𝒲({ℰ}_x)$	128	128	128	128
$\mathbb{D}({ℰ}_x)$	3	3	2	4*
$𝒟({\mathbf{e}}_{Iij}^M)$	64	64	64	64
$𝒲({ℰ}_e^M)$	128	128	128	128
$\mathbb{D}({ℰ}_e^M)$	3	3	2	4*
$𝒟({\mathbf{e}}_{Iij}^W)$	64	64	64	64
$𝒲({ℰ}_e^W)$	128	128	128	128
$\mathbb{D}({ℰ}_e^W)$	3	3	2	4*
$𝒲({\varphi }^{M(n)})$	128	128	128	128
$\mathbb{D}({\varphi }^{M(n)})$	3	3	2	2
$𝒲({\varphi }^{W(n)})$	128	128	128	128
$\mathbb{D}({\varphi }^{W(n)})$	3	3	2	2
$\mathbb{D}({\gamma }^{(n)})$	128	128	128	128
$\mathbb{D}({\gamma }^{(n)})$	3	3	2	2
k	5	20*	3	20*
$𝒲({𝒟}_i)$	128	128	128	128
$\mathbb{D}({𝒟}_i)$	2	3*	2	4*
$𝒟({\widehat{y}}_{Ii})$	3	3	3	3
$𝒟(g_I)$	4	4	6	6
$𝒲({ℰ}_g)$	128	128	128	128
$\mathbb{D}({ℰ}_g)$	2	3*	2	4*
Trainable Parameters	1,216,451	4,251,779	622,659	3,376,899

See Figure A.11 for the complete evolution of all losses through the training process. Note the characteristic jump in the total loss on the validation set for ‘−DC’ mode training for both problems at the 1000th epoch. This is because the weight (w_c) is chosen using the information in the training set and does not guarantee contact losses on the same scale as dynamic losses for the validation set. Since contact computation is expensive, we do not compute the contact loss for ‘−D’ mode learning. However, inference using checkpointed models at periodic epochs fills in for this information as described in Section 4.3.1.

Appendix B. Dataset Generation Details

Each data set is generated using the finite element method, where the solver is FEBio. Each mesh is a four-node quadrilateral shell in the upstream finite-element simulation. However, before being used in the current framework, they are all converted to a three-node triangular shell mesh. This is done using the developed continuous collision detection algorithm. The number of time steps per simulation is the maximum that occurs across all the simulations. The time step size is the same for all the graphs except the final time step, which is set to zero. The framework learns to predict no displacement for this final step. The collision radius R is used for proximity criteria to build world-space edges. The length scale l_c is used to normalize length-based quantities and is of the order of the geometric dimension in the reference state.

Figure A.11:

Total ($ℒ$), contact (${ℒ}_c$), and dynamic (${ℒ}_d$) loss over training and validation sets for (a) varying geometry and (b) aortic valve problems. Note that the losses are average across all batches at the respective epoch.

Figure B.12 shows the scatter plot of the parameters (super-set of graph-level features, mentioned in Section 4.2) against each other along with their grouped histograms. The group labels are the training, validation, and test sets.

Figure B.12:

Pairwise scatter plot of all parameters used to generate the dataset labelled by the set they belong to. Also shown are grouped histograms of each parameter. Each parameter has been min-max normalized to [0, 1] interval. Varying Geometry: Top right block Aortic Valve: Bottom left block.