Artificial Neural Networks as Mappings between Proton Potentials, Wave Functions, Densities, and Energy Levels

Abstract

Artificial neural networks (ANNs) have become important in quantum chemistry. Herein, applications to nuclear quantum effects, such as zero-point energy, vibrationally excited states, and hydrogen tunneling, are explored. ANNs are used to solve the time-independent Schrödinger equation for single- and double-well potentials representing hydrogen-bonded molecular systems capable of proton transfer. ANN mappings are trained to predict the lowest five proton vibrational energies, wave functions, and densities from the proton potentials and to predict the excited state proton vibrational energies and densities from the proton ground state density. For the inverse problem, ANN mappings are trained to predict the proton potential from the proton vibrational energy levels or the proton ground state density. This latter mapping is theoretically justified by the first Hohenberg-Kohn theorem establishing a one-to-one correspondence between the external potential and the ground state density. ANNs for two- and three-dimensional systems are also presented to illustrate the straightforward extension to higher dimensions.

Graphical Abstract

The accessibility, development, and application of deep artificial neural networks (ANNs) has grown significantly over the past decade. ANNs with at least a single hidden layer have been rigorously shown to be capable of approximating any continuous function with arbitrary accuracy by increasing the number of hidden neurons.^1–3 Their use in quantum mechanics and specifically computational chemistry is widespread,^4–16 with applications ranging from generating potential energy surfaces to developing density functionals. Harnessing ANNs to study nuclear quantum effects (NQEs), including zero-point energy, vibrationally excited states, and proton tunneling,^17–21 would have broad implications. NQEs are particularly important for the description of proton transfer, proton-coupled electron transfer, and hydrogen bonding in chemical and biological systems.²² A variety of computational methods have been used to incorporate NQEs into simulations, including path integral molecular dynamics,²³ the multiconfigurational time-dependent Hartree method,²⁴ and the nuclear electronic orbital approach.²⁵ In this Letter, we introduce ANN mappings as a promising avenue for the inclusion of NQEs in chemical simulations with the potential for enhanced accuracy and computational efficiency.

Herein, ANN mappings are trained to predict proton vibrational energies as well as proton vibrational wave functions, proton densities, and proton potentials in the discrete variable representation (DVR). The scopes of the datasets used for training and testing these mappings are representative of hydrogen-bonded systems, typically capable of proton transfer or proton-coupled electron transfer. The ANN mappings can be expressed as

$$F_{\text{NN}}\left[\left\{{\theta }_{AB}\right\}\right]:A\to B$$ (1)

where F_NN is the trained ANN with input layer A, output layer B, and trained hyperparameters {θ_AB}. In this implementation, the ANN input (output) layers directly receive (produce) the DVR of the potentials, wave functions, and densities. Specifically, proton vibrational energies, wave functions, and densities are predicted from proton potentials; proton vibrational energies and excited state densities are predicted from proton ground state densities; and proton potentials are predicted from proton vibrational energies and proton ground state densities. A new custom loss function was utilized to train ANNs capable of predicting proton wave functions with arbitrary phase from proton potentials.

These ANN mappings are based on solid theoretical foundations. According to the time-independent Schrödinger equation (TISE) for a single particle of mass m, the external potential v(x) contains all necessary information to obtain the energies and wave functions for this system. The wave function in turn determines the density. This relationship establishes that proton vibrational state energies, wave functions, and densities can be obtained from the potential and that an ANN can represent and approximate such a mapping, which may be regarded as a solver of the TISE. Moreover, the first Hohenberg-Kohn theorem^26–27 establishes a one-to-one correspondence between the external potential v(x) and the ground state density ρ(x). In other words, the ground state density determines the potential and hence all related properties.²⁷ This theorem establishes the theoretical foundation for ANN mappings from the proton ground state density to the proton potential, the proton vibrational energies, and the proton excited state densities. Finally, the inverse problem for the TISE involves obtaining the potential from the wave functions and/or energies. This problem has been addressed by mathematicians, who have developed sophisticated solutions by means of boundary control and Neumann-Dirichlet mappings.^28–30 According to these studies, the spectrum does not uniquely define the potential due to symmetry but does contain sufficient information to reconstruct an isometry of the potential (i.e., an equivalent potential given a translation, rotation, and/or reflection). This relationship establishes the theoretical foundation for an ANN mapping from the proton vibrational energies to the proton potential.

To implement these ANNs, we constructed the potentials in the dataset from a previously generated set of 110 one-dimensional proton potentials for the intramolecular hydrogen-bonded interfaces of various substituted benzimidazole-phenol molecules at different proton donor-acceptor distances.^31–33 These potentials are representative of a family of potentials typical for hydrogen-bonded interfaces in molecular systems that often exhibit proton transfer reactions.³⁴ The initial dataset of 110 proton potentials was modified and augmented to generate a dataset of 10,000 potentials containing symmetric and asymmetric double-well potentials with varying barrier heights and reaction energies, as well as symmetric and asymmetric single-well potentials. Details about the generation of this dataset are provided in the Supporting Information. The total dataset contains 28% single-well potentials and 72% double-well potentials. The distributions of various characteristics of the potentials in the dataset are shown in Figure S2. The potentials were preprocessed by translation of the global minimum to x = 0 and possible reflection to ensure that the other minimum in the case of double-well potentials was at x > 0. The proton vibrational energies and wave functions were computed for each potential using the Fourier grid Hamiltonian (FGH) method³⁵ with 1024 equally spaced grid points. Note that a significantly smaller number of grid points could have been used to produce ANNs of similar accuracy (Table S8), although the larger number of grid points provides higher resolution in the DVR. A random selection of 8,000 of these samples (i.e., potentials, wave functions, and energies) comprised the training set, 1,000 comprised the validation set, and the final 1,000 comprised the test set.

A multilayer perceptron (MLP)³⁶ architecture was used in all ANN models in this study, as schematically depicted in Figure 1. The ANNs were created in Keras with a Tensorflow backend.³⁷ A hyperparameter grid search was conducted for each mapping to determine the optimal number of hidden layers and neurons. Input layers and output layers were sized appropriately for each given mapping. The training set detailed above was used to train the ANNs, and the validation set was used to determine which set of hyperparameters produced the most accurate results for each mapping. The final mappings with optimized hyperparameters were evaluated on the test set. The standard (Rectified Linear Unit) ReLU activation function was used for all layers of all models with the exception of the output layer, for which a linear activation function was used. The mean squared error (MSE) loss function was used for training of the ANNs predicting energies, densities, and potentials, while the custom loss function described below was used for training the ANNs predicting wave functions. Information regarding the convergence criteria, learning rates, and hyperparameter grid search is provided in the Supporting Information.

Figure 1.

MLP architecture of the ANNs used in this study. The input layers and output layers of each ANN were sized appropriately, and the number and size of the hidden layers were varied depending on the desired mapping. In the ANN shown here, the proton potential on a grid of 1024 points dictates the size of the input layer, and the n^th proton vibrational wave function on a grid of 1024 points dictates the size of the output layer.

For the ANNs trained to predict wave functions, the arbitrary phase factor of the wave function had to be considered. The proton potential contains sufficient information to obtain the proton vibrational wave functions up to a phase factor. This arbitrary phase factor presents difficulties for machine learning methods attempting to predict wave functions. A solution is offered in the following physically-motivated custom loss function featuring the square values of the overlap integrals between the true and predicted wave functions:

$$J=\frac{1}{M}\sum _{i=1}^M\left[{\left(1-{\left|\langle {\psi }_{\text{pred}}^{(i)}|{\psi }_{\text{true}}^{(i)}\rangle \right|}^2\right)}^2+{\left(1-{\left|\langle {\psi }_{\text{pred}}^{(i)}|{\psi }_{\text{pred}}^{(i)}\rangle \right|}^2\right)}^2\right]$$ (2)

where the sum runs over the M samples in the training set, and ${\psi }_{\text{true}}^{(i)}$ and ${\psi }_{\text{pred}}^{(i)}$ are the true and predicted wave functions for sample i. Squaring the overlap integral in the first term removes the dependence on the arbitrary phase, and the second term imposes the constraint that the predicted wave function must be normalized. The use of this loss function makes it possible to predict the wave functions up to a phase factor by allowing the ANN to generate its own arbitrary phase.

In our first application, an ANN mapping was trained to predict the first five proton vibrational energies from the proton potential. The mean absolute error (MAE) for the first five proton vibrational energies was 0.018 kcal/mol, and the MAE for the ground state energy was 0.0078 kcal/mol. This result is encouraging considering the diversity of proton potentials within the dataset (Figure S2), where the energies of the first five vibrational states can span 50 kcal/mol. Figure S10 shows the correlation between the proton vibrational energies calculated using the FGH method and the trained ANN mapping for the test set of 1,000 proton potentials.

In the second application, five ANNs were trained to predict the first five proton vibrational wave functions from the proton potential (i.e., one ANN for each state) using the custom loss function described above. The wave functions of the test set were predicted with excellent accuracy, as shown in Figure 2. The ANNs easily map the single-well potentials, even those with prominent shoulders, to the wave functions (Figure 2A). For the symmetric double-well potentials, the ANN mappings correctly predict the bilobal, delocalized symmetric ground state and antisymmetric first excited state wave functions (Figure 2B). For asymmetric double-well potentials, the ANNs accurately determine the well in which each proton vibrational wave function should localize (Figure 2C). In the test set, the mean overlap integrals between the FGH and ANN wave functions is 0.9993 for the ground state wave functions and 0.9955 for all five predicted wave functions. Significant error occurs on the rare occasion when the ANN mapping predicts that the wave function is localized in the wrong well for an asymmetric double-well potential.

Figure 2.

ANN predictions of proton vibrational wave functions from the proton potentials (solid black lines). The ANN results (dashed blue lines) are compared to the reference FGH results (solid red lines).

Training ANNs to map from the potential to both the energies and the wave functions essentially provides a solution to the TISE within the scope of the potentials in the dataset. It is appropriate to consider the computational efficiency of these ANNs and their advantages over other methods. There are many numerical methods for solving the one-dimensional TISE.^{35, 38–39} One of these methods is the FGH method used to obtain the reference energies and wave functions in this study. The implementation involves the construction and diagonalization of a symmetrical [L×L] matrix, where L is the number of equally spaced grid points, which is not computationally expensive in one dimension but quickly becomes infeasible for higher dimensions. Solving the one-dimensional TISE using the FGH method on the dataset of 10,000 potentials requires ~18.7 minutes or ~0.11 seconds per potential on a single CPU. On the other hand, the prediction of wave functions and energy levels using trained ANN mappings on the dataset of 10,000 potentials requires 1.33 seconds or 13 microseconds per potential with a MAE of 0.0179 kcal/mol on the test set. Note that the ANN mappings require training prior to their use, and their scopes are limited to predictions within the functional space spanned by the training set. The computational advantages will become more meaningful for multidimensional systems, as discussed further below.

In the third application, five ANN mappings were trained to predict the proton densities for the lowest five states from the proton potentials. The accuracy of the ANN mappings that produce proton densities can be quantified in two ways. The first metric is the MAE of the density at every discretized position i, for every state j, for each potential k in the test set:

$${\text{MAE}}_{\rho }=\frac{1}{L\times M\times N}\sum _{i,j,k}\left|{\rho }_{\text{true},j}^{(k)}\left(x_i\right)-{\rho }_{\text{pred},j}^{(k)}\left(x_i\right)\right|$$ (3)

where there are L discretized positions, M states, and N potentials. The second metric is the MAE in position expectation value $x_j^{(k)}={\sum }_{i=1}^L{x}_i{\rho }_j^{(k)}\left(x_i\right)$:

$${\text{MAE}}_x=\frac{1}{M\times N}\sum _{j,k}\left|x_{\text{true},j}^{(k)}-x_{\text{pred},j}^{(k)}\right|$$ (4)

Note that the grid points are equally spaced, and the densities are normalized. For these ANN mappings, the MAE_ρ is 1.49×10⁻⁴, and the MAE_x is 2.57×10⁻³ Å. Note that the ANN mappings have output layers with linear activation functions that are not prevented from predicting negative densities. As a result, the ANN mappings on rare occasion predict very small negative densities. Given that no densities in the training set are ever negative, the ANNs essentially learn that densities are positive definite. To correct this occasional unphysical behavior, the rarely observed negative values of the predicted densities were set to zero. Note that the densities could be obtained from the squares of the wave functions produced by the ANN mappings of the second application, but this direct mapping from the potential to the density may be useful in certain circumstances. In the fourth application, four ANN mappings were trained to predict the four proton excited state densities from the proton ground state density. Figure 3 shows the results for these ANN mappings. In this case, the MAE_ρ is 1.12×10⁻⁴, and the MAE_x is 0.026 Å, which are not as accurate as the predictions from the potentials. However, it is remarkable that an ANN is able to make predictions from the proton ground state density considering the apparent uniformity of these densities (i.e., compare the proton ground state densities in Figures 3A and 3B). The ANN is able to interpret the subtle features of the proton ground state density to correctly predict the excited state densities and distinguish between very similar systems.

Figure 3.

ANN predictions of proton excited state densities from the proton ground state density (solid red line labeled ρ₀). The proton potentials are shown with solid black lines, and the ANN results (dashed blue lines) are compared to the reference FGH results (solid red lines).

In the fifth application, an ANN mapping was trained to predict the first five proton vibrational energies from the proton ground state density. These results are shown in Figure S11. The MAE of the proton vibrational ground state energy is 0.049 kcal/mol, and the MAE of the lowest five proton vibrational energies is 1.08 kcal/mol. The predictions of the ground state energy from the ground state density are nearly as accurate as the predictions from the potentials (Figure S10), but the predictions of the higher vibrational state energies are significantly worse. It is nonetheless impressive that the ANNs are able to extract the spectral information for single-well and asymmetric double-well potentials from nearly identical proton ground state densities.

In the sixth application, an ANN mapping was trained to predict proton potentials from the first five proton vibrational energies. The analysis of spectroscopic data is the motivation for such an ANN mapping. Infrared and Raman spectra provide valuable information about bond connectivity, functional group positioning, and molecular structure of chemical systems,⁴⁰ as well as information about the nuclear potential energy surface. Typically, the entire spectrum of the system, including the frequencies of all overtones, is required to perform such a reconstruction. The results for this mapping are shown in Figure 4 and can be evaluated in terms of the MAEs of different characteristic quantities used to describe potential energy landscapes. The MAE of the entire potential under 50 kcal/mol is 0.715 kcal/mol. For potentials with double well character, the MAE of the energy difference between minima is 2.28 kcal/mol, the MAE of the energy barriers is 1.03 kcal/mol, the MAE of the separation between the minima is 0.0387 Å, and the MAE of the position of the maximum between the two minima is 0.0211 Å. These results are encouraging considering the 80 kcal/mol range of the energy differences between minima and the 40 kcal/mol range of energy barriers. Note that this ANN mapping is not predicting these characteristics of the potentials directly, but rather is predicting the entire DVR of the potential, from which these characteristics are extracted. Presumably an ANN trained specifically for the purpose of predicting such characteristics would perform significantly better. This ANN mapping also benefited from the preprocessing of potentials that removed translational and reflection symmetry, given that the spectrum does not contain enough information to describe symmetry invariances.

Figure 4.

ANN predictions of proton potentials from the first five proton vibrational energies. The proton vibrational energy levels are shown with horizontal black lines. The ANN potential (blue dashed line) is compared to the true proton potential (red solid line).

In the seventh application, an ANN mapping was trained to predict the proton potential from the proton ground state density. The results for this mapping are shown in Figure 5 and can be evaluated in a similar fashion to the previous mapping. The MAE of the entire potential under 50 kcal/mol is 2.36 kcal/mol. For potentials with double well character, the MAE of the energy difference between minima is 6.02 kcal/mol, the MAE of the energy barriers is 1.28 kcal/mol, the MAE of the separation between the minima is 0.0844 Å, and the MAE of the position of the maximum between the two minima is 0.0245 Å. As discussed above, this mapping is a direct demonstration of the first Hohenberg-Kohn theorem, which states that the proton ground state density possesses the requisite information to reconstruct the proton potential. As expected, the accuracy of the mapping is greatest about the global minimum, where the density is greatest. On the other hand, Figure 5B illustrates the successful prediction of a second minimum (i.e., the correct location and energy), despite the nearly negligible ground state density in that region.

Figure 5.

ANN predictions of proton potentials from the proton ground state density (solid black line). The ANN potential (blue dashed line) is compared to the true proton potential (red solid line).

We trained two additional ANNs to further investigate the mapping from the proton density to the proton potential. The first ANN was trained to predict the proton potential from the ground and first excited state proton densities, and the second ANN was trained to predict the proton potential from the proton densities associated with the first five proton vibrational states. The MAEs across the entire potential under 50 kcal/mol were 1.22 kcal/mol and 0.265 kcal/mol, respectively, for these two additional ANNs, compared to the MAE of 2.36 kcal/mol for the ANN that used only the ground state proton density. Thus, including excited state densities as input enhances the accuracy of the ANN, even if the ground state density should in principle contain all of the necessary information, presumably due to limitations of the training set and numerical errors.

The extension of these mappings to higher dimensions is straightforward and requires no modification of the ANN architectures except for the expansion of the input/output layers to accommodate the increased number of grid points accompanying the DVR in higher dimensions. Two-dimensional ANNs corresponding to all applications presented in this Letter, as well as one three-dimensional ANN, are provided in the Supporting Information. As an example, Figure 6 depicts the results from the ANN mapping from the 2D proton ground state density to the 2D proton potential. As the dimensionality is increased, several technical challenges may arise. First, the greater diversity of potentials due to increased dimensionality will require larger training sets. Second, the generation of datasets in higher dimensions will involve more costly calculations. Third, the ANNs may need more hidden layers and nodes to support the increasing complexity of the problem in higher dimensions. However, the results in Figure 6 were straightforward to generate and illustrate the effectiveness of this ANN mapping across a diverse set of samples.

Figure 6.

These plots illustrate the 2D proton potentials predicted from the 2D proton ground state densities. The colored contour plots depict the proton ground state densities used as inputs to the ANN mapping. To assess the accuracy of the output, the white contour lines indicate the true proton potentials, and the cyan contour lines indicate the predicted proton potentials, where the energies are given in kcal/mol.

In this Letter, we demonstrated the ability of ANNs to predict properties of a one-dimensional quantum mechanical system using the time-independent Schrödinger equation and the first Hohenberg-Kohn theorem as theoretical foundations. In particular, the proton potentials, proton vibrational energies, proton vibrational wave functions, and proton densities can be used as input or output for these ANNs. A new loss function was proposed and implemented to allow reliable and accurate predictions of wave functions with arbitrary phases. All of these ANNs were trained and validated on data sets corresponding to hydrogen-bonded molecular systems capable of proton transfer or proton-coupled electron transfer reactions. We also presented ANNs for two- and three-dimensional systems, demonstrating the straightforward extension to higher dimensions, where ANNs are expected to be more useful. This work provides the foundation for approaches using ANNs to include nuclear quantum effects in simulations of chemical systems.