Sparse Gaussian Process Regression-Based Machine Learned First-Principles Force-Fields for Saturated, Olefinic, and Aromatic Hydrocarbons

Abstract

Universal machine learning (ML) interatomic potentials (IAPs) for saturated, olefinic, and aromatic hydrocarbons are generated by using the Sparse Gaussian process regression algorithm. The universal potentials are obtained by combining the potentials for the previously trained alkane/polyene systems and the potentials generated with the presently trained cyclic/aromatic hydrocarbon systems, along with the newly trained cross-terms between the two systems. The ML-IAPs have been trained using the PBE + D3 level of density functional theory for the on-the-fly adaptive sampling of various hydrocarbon molecules and these clusters composed of small molecules. We tested the ML-IAPs and found that they correctly predicted the structures and energies of the β-carotene monomer and dimer. Also, the simulations of liquid ethylene reproduced the molecular volume and the simulations of toluene crystals reproduced higher stability of the α-phase over the β-phase. These ab initio-level force-fields could eventually evolve toward universal organic/polymeric/biomolecular systems.

Introduction

Conventional classical molecular force-fields¹⁻⁶ have been widely used for large-scale computational studies of organic/biomolecular and macromolecular systems including drugs and polymers which are not easily tractable with first-principles calculations. Nevertheless, their accuracy has been limited due to their deficiencies in describing various properties with correct structures simultaneously. In particular, the potential energy is expressed in terms of “descriptors” such as bonds, angles, and so forth which are topology-dependent. Therefore, their applicability has been limited to nonreactive systems.

Over the past decade, machine learning interatomic potentials (ML-IAPs) representing ab initio potential energy surfaces have been introduced based on neural networks (NNs) and kernel regression methods.⁷⁻²⁵ Most of these potential energies are dependent on the environments of each atom and so need to be trained at every situation and at every instant that the atomic environment changes significantly. Here, we utilize a one-to-one mapping from the coordinate space to the descriptor space as an only local configuration-dependent generalized form reflecting many-body correlations while being invariant with respect to translations/rotations of the physical system and permutations of identical atoms.²⁶

Sampling by active/on-the-fly learning is conducted by using a measure for the uncertainty of the model at given input points. The uncertainty prediction is straightforward with Bayesian inference methods. Here, the sparse Gaussian process regression (SGPR) algorithm^20,21,24,25 as a Bayesian approach to regression is implemented for representing the potential energy surface as well as uncertainty predictions. Using the SGPR algorithm, the models in this study are generated in total with around 1000 training samples which are much less than the initial data needed for generating the seed NNs. Among Bayesian methods, SGPR scales as , while retaining the important characteristics of Gaussian process regression (GPR)^8,27,28 having the computational cost of with the size of training data (n). Thus, SGPR is ideal for on-the-fly sampling of optimal data sets for generating ML-IAPs.

Hydrocarbons are a diverse family of organic molecules relevant to most of the macromolecules and biological complexes. Recently, the ML force fields based on GPR have been developed for predicting the energies and forces of alkanes and proteins and performing on-the-fly molecular dynamics (MD) simulations.^29,30 Previously, we considered ML-IAPs using the SGPR algorithm²⁵ for a large subcategory of alkane, alkene, and polyene hydrocarbons as an initiative toward a universal ab initio quality force-field for these molecules. Here, we consider not only saturated, olefinic, and aromatic hydrocarbons which include alkane, alkene, alkadiene, and alkatriene molecules with branches but also cyclic saturated/unsaturated and aromatic hydrocarbons and further their combined structures.

Methods

Machine Learning Potentials

A configuration x of N atoms is compiled to a list of descriptors x = {ρ_i}_i=1^N where ρ_i is a rotationally invariant descriptor for the local chemical environment of atom i, which depends only on the relative coordinates of N – 1 atoms within a cutoff radius. In kernel-based regression methods, the potential energy becomes

where z = {χ_j}_j=1^m is the set of inducing descriptors, K is a covariance kernel, and w = {w_j}_j=1 is the vector of weights for the inducing descriptors. The weights, which depend on the regression algorithm, are obtained such that the potential energy and forces are reproduced for a set of ab initio data X = {x_k}_k=1ⁿ. The inducing descriptors are often subset descriptors extracted from X.

In SGPR,¹⁸⁻²¹w = (σ²k_mm + k_nm^Tk_nm)⁻¹k_nmY where k_mm and k_nm are the inter-inducing (z–z) and data-inducing (X–z) covariance matrices, Y is the data of potential energies (and forces), and σ is the noise hyperparameter. The noise scale σ and other possible hyperparameters in kernel K are optimized to maximize the likelihood of the data energies. For the similarity kernel, we use a variation of the smooth overlap of atomic positions²⁶ which we have defined in refs (20) and (21). For the existing data, the inducing descriptors are sampled from the data; otherwise, with on-the-fly learning, both the data and inducing descriptors are sampled during MD. The uncertainty prediction and its utilization for the on-the-fly learning algorithm are discussed in detail in ref (25).

On-the-Fly MD Simulation

To obtain ML potentials, we carried out MD simulations with an adaptive sampling algorithm.²³ It learned the potential energy surfaces from density functional theory (DFT) calculations which were performed using the Vienna ab initio simulation package³¹ with Perdew–Burke–Ernzerhof (PBE) functionals³² and van der Waals interactions (D3).³³ The projector augmented wave³⁴ pseudopotentials with 400 eV energy cutoff were used. The convergence criterion for the electronic energy difference was set to 10^–4 eV. The molecules were placed in the center of a cubic cell in which the interlayer distance between the molecules and the image is 15 Å vacuum, and the Brillouin zone was sampled using the Γ-point. We considered various conformers of given molecules and performed NVT MD simulations using a Nosé–Hoover thermostat and Parrinello–Rahman dynamics as implemented in the atomic simulation environment package.³⁵ The MD simulations were run for 3–6 ps at 300 K with 0.5 fs time step. The coefficient of determination defined by R² = 1 – (f_i – f̃_i)²/(f_i – f̅)² is exploited for the test of reliability of the SGPR-based first-principles potential/forces, where {f_i} are ab initio (PBE + D3) forces, f̅ is their average, and {f̃_i} are ML forces.²⁵

Owing to the abundance of relevant molecules, trained molecules are split into several groups, and independent SGPR models are trained in parallel using the AUTOFORCE package.³⁶ In this study, we considered expert models for linear or branched systems (C1–C8 alkane, C3–C10 isoalkane, C2–C6 alkene, C3–C6 branched alkene, C3–C8 alkadiene, and C7–C9 alkatriene), cyclic systems (C3–C10 cycloalkane, C3–C8 cycloalkene, and C4–C10 methyl-/dimethyl-cycloalkane, bicycloalkane), and aromatic systems (benzene, toluene, xylene, trimethylbenzene, and mesitylene) to build the universal model. To consider the intermolecular interactions, expert models for liquid phases of alkane (methane, ethane, propane, and butane), alkene (ethylene), and aromatic (benzene) are also trained and denoted as alkane-inter, alkene-inter, and aromatic-inter, respectively. The learnt expert models are available in gitlab website.³⁷

Results

Training Expert Models

On-the-fly MD simulations are performed for 3–6 ps to train expert SGPR models for each molecule (see Method). Here, expert models are newly trained for cyclic hydrocarbons (cycloalkane, cycloalkene, and bicycloalkane) and aromatic systems and then are combined with the previously generated model for alkane, isoalkane, alkene, and alkadiene.²⁵ Each generated expert model is validated by comparing it with the DFT calculation results (Table 1).

Table 1. Testing the Expert Models in Their Domains^a.

group	Ntest	energy MAE (meV)	force MAE (eV/Å)	force R2 testing/training
alkaneb	448	4.9	0.076	0.978/0.993
isoalkaneb	308	3.0	0.080	0.977/0.989
alkeneb	142	8.1	0.110	0.966/0.985
alkadieneb	374	5.3	0.110	0.959/0.978
alkatriene	238	3.2	0.108	0.992/0.966
cycloalkane	253	2.3	0.074	0.983/0.990
cycloalkene	77	3.4	0.110	0.963/0.984
methyl/dimethyl-cycloalkane	946	5.2	0.092	0.970/0.989
bicycloalkane	594	2.1	0.075	0.980/0.987
aromatic	83	2.1	0.074	0.997/0.983

^aN_test is the number of test samples and energy is the potential energy per atom.

^bData from ref (25).

Rotation of CH₃ Group in Ethane

The accuracy of universal ML-IAPs generated by combining expert models is tested first. As the simplest case, nudged elastic band calculation is conducted to predict the rotation barrier of the CH₃ group in ethane. The barrier to rotate the CH₃ group by 120° is calculated using universal ML-IAPs and compared with the PBE + D3 results. As a result, the barrier of the reaction was predicted to be 0.12 eV which is almost the same as the PBE + D3 results showing 0.11 eV and with the barrier of 0.12 eV obtained from the conventional OPLS-AA³ force fields in which the rotational barriers were optimized to the ab initio computational value.

Dimerization Energy of β-Carotene

The universal ML-IAPs are used to predict the energies of molecules containing various hydrocarbon groups in their structure. Here, the structures and energies of the β-carotene monomer and dimer (Figure 1) are investigated using the universal potential. β-Carotene is an organic molecule having 8 isoprenes and 2 trimethyl-cyclohexenes in its structure. The expert models including C–C single bonds, C=C double bonds, and cyclic hydrocarbons are used to construct the universal model. The relative energies of 20 different conformers of β-carotene monomer are predicted to be close to the PBE + D3 results within a low root-mean-squared error of 0.12 eV. The interaction energy between the two monomers in the β-carotene dimer is predicted to be −1.9 eV showing a strong interaction, consistent with the strong interaction by PBE + D3 calculation (−1.1 eV). In contrast, the conventional OPLS-AA³ and AIREBO³⁸ force-fields give only insignificant interaction energies of −0.02 and −0.2 eV, respectively.

Figure 1.

Structures of β-carotene monomer (a) and dimer (b).

Liquid Ethylene

We predicted the molecular volume of liquid ethylene using ML-IAPs. The NPT MD simulations of 34 ethylene molecules are conducted for 30 ps with a constant pressure of 1 bar and a temperature of 169.5 K. In this case, the volume per molecule v_l is predicted to be 76 ± 6 ų in good agreement with the experimental value³⁹ of 81.5 ų. Some discrepancy from the experimental value seems to arise from the inherent discrepancy in the DFT-D3 calculation.

Toluene Crystal

The prediction of crystal structures is accelerated by using the ML potential trained with DFT potential energies/forces. Here, our universal model is employed to predict the stable structure of the toluene crystal (Figure 2). The α-phase is predicted to be the most stable state in the crystal, while the β-phase is the metastable state.⁴⁰ The predicted total energy of each system is within 0.5% from the PBE + D3 values, and the energy difference between the two phases is predicted as 1.89 eV, which is comparable with the PBE + D3 result showing 1.31 eV. The conventional AIREBO-M³⁸ force-field gives an energy difference of 2.26 eV.

Figure 2.

Crystal structures of (a) α-toluene and (b) β-toluene.

Discussion and Conclusions

In this work, molecular and crystal structures of hydrocarbons are predicted by the ML-IAPs obtained using the SGPR algorithm. The expert models are generated for different hydrocarbon groups from on-the-fly ML and then are combined into the universal ML-IAPs. The universal ML-IAPs which cover most of configurationally important potential energy surfaces are generated from the expert models which trained for ∼90 molecules (∼350 conformers) and 5 bulk phases. It showed excellent transferability for various hydrocarbon systems. The universal ML-IAPs predicted the structure of β-carotene dimer with reasonable accuracy in interaction energy between two monomers. Intermolecular interactions in universal models adequately described the molecular volume of liquid ethylene. They were extended to investigate the phase of the toluene crystal, which showed the stability of α-phase over β-phase. All these results indicate that the universal model is generated efficiently from separately trained atomic potentials for local configurations of various subgroups. Therefore, this method can be utilized to generate universal models for various systems consisting of multicomponents, which have not been possible so far for other ML methods.

This study concludes one of the most extensive explorations of the phase space of hydrocarbon molecules in which less than 1000 configurations are sampled as support for the SGPR potential. These configurations are sampled on-the-fly with MD using a Bayesian criterion, and therefore the configurations with redundant information are automatically discarded. It has been shown that, for transferable ML potentials, the quality of data in terms of diversity/completeness is the most important factor rather than quantity.⁴¹ Therefore, the data sampled here can be easily used with any regression algorithm (kernel- or NN-based) in order to produce transferable ML potentials. The small size of ab initio data makes it straightforward to use higher-quality (better than PBE + D3) first-principles methods in the future. At last, it has been shown that structural descriptors with n-body correlations are generally degenerate,⁴² and significantly higher accuracy can be achieved with a more complete descriptor scheme. Aside from including higher order correlations, an intriguing concept is the development of “recursively embedded atom NNs”⁹ in which the local descriptors are still based on three-body correlations, but nonlocal correlations (which lift the degeneracies) are incorporated using local parameters which are linked to the global structure by a message passing NN. A similar concept can be explored with kernel-based regression methods such as SGPR which is a promising direction for future studies.

Author Contributions

^† equal contribution.

The authors declare no competing financial interest.