CONSTRUCTING SURROGATE MODELS OF COMPLEX SYSTEMS WITH ENHANCED SPARSITY: QUANTIFYING THE INFLUENCE OF CONFORMATIONAL UNCERTAINTY IN BIOMOLECULAR SOLVATION

Abstract

Biomolecules exhibit conformational fluctuations near equilibrium states, inducing uncertainty in various biological properties in a dynamic way. We have developed a general method to quantify the uncertainty of target properties induced by conformational fluctuations. Using a generalized polynomial chaos (gPC) expansion, we construct a surrogate model of the target property with respect to varying conformational states. To alleviate the high-dimensionality of the corresponding stochastic space, we propose a method to increase the sparsity of the gPC expansion by defining a set of conformational “active space” random variables. With the increased sparsity, we employ the compressive sensing method to accurately construct the surrogate model. We demonstrate the performance of the surrogate model by evaluating fluctuation-induced uncertainty in solvent-accessible surface area for the bovine trypsin inhibitor protein system and show that the new approach offers more accurate statistical information than standard Monte Carlo approaches. Furthermore, the constructed surrogate model also enables us to directly evaluate the target property under various conformational states, yielding a more accurate response surface than standard sparse grid collocation methods. In particular, the new method provides higher accuracy in high-dimensional systems, such as biomolecules, where sparse grid performance is limited by the accuracy of the computed quantity of interest. Our new framework is generalizable and can be used to investigate the uncertainty of a wide variety of target properties in biomolecular systems.

1. Introduction

Biomolecular structures are inherently uncertain due to thermal fluctuations and experimental limits in structural characterization. At equilibrium, a biomolecule samples an ensemble of states governed by an energy landscape. For a biomolecule with well-defined native structure at an energetic global minimum, these states are generally located in the neighborhood of the native structure. While the native equilibrium structure of a biomolecule provides essential insight, it is also important to understand conformational fluctuations of biomolecular systems and their impact on molecular properties. In particular, it is of great interest to accurately quantify the uncertainty in these properties caused by stochastic conformational fluctuations.

Molecular dynamics (MD) simulations offer a powerful tool for examining the influence of conformational uncertainty on biomolecular properties [16, 2]. Over the past few decades, this approach has made great progress in the development of accurate empirical force field as well as efficient simulation algorithms [38]. However, despite these advances, MD is still a very computationally expensive simulation approach, particularly for large biomolecular complexes. Moreover, the finite durations of MD simulations are plagued with uncertainty in calculated properties due to non-ergodic sampling. Many coarse-grained (CG) models and methods have been developed to facilitate molecular simulation at larger length scales and longer time scales. One popular approach is the elastic network model (ENM), which involves a harmonic approximation of molecular energy landscape. It has been observed that the low-frequency normal modes of a biomolecular system can be reproduced using a single-parameter Hookean potential between neighboring residues [49, 25, 47]. In particular, by only modeling interactions between the neighboring α-carbon (C_α), ENMs are able to predict structural fluctuations (e.g., Debye-Waller or B-factors) with surprising accuracy [25, 4].

The simplified potentials used by CG models such as ENM allow us to examine structural fluctuations in a semi-analytical manner. However, there does not exist an analytical formula that directly leads from the structural fluctuations to target biomolecular properties computed from the structure. Instead, given a specific biomolecular conformation (e.g., one snapshot of biomolecule structure under fluctuation), we still need further numerical computation to obtain the target properties. This leads to an important practical question: how do we utilize the stochastic information obtained from these models to efficiently quantify the uncertainty of the target property induced by the biomoleculular conformational (structural) fluctuation? In many applications, a single native conformation of a molecule is used when computing a properties such as molecular volume and area [27, 41, 12], electrostatic and solvation properties [40, 44], titration states [3], and other quantities. However, these quantities are all sensitive to the structure of the molecule and therefore subject to uncertainty induced by conformational fluctuations. Many studies neglect this uncertainty; those which attempt to assess it are forced to resort to time-consuming monte carlo sampling over the numerous biomolecular conformation states.

In the present work, we address this issue by providing a general framework to quantify conformation-induced uncertainty on various biomolecular properties. In particular, we construct a surrogate model of a target quantity in terms of the molecular conformational states. The constructed surrogate model enables us to efficiently evaluate the statistical information of the target property, e.g., probability density function. To the best of our knowledge, this is the first demonstration of how a target property response surface – including property uncertainty – can be directly evaluated from the biomolecular conformational distribution.

To construct the surrogate model, we adopt the generalized polynomial chaos (gPC) [21, 53] and formulate the target property as an expansion of a set of gPC basis functions determined by the specific conformation states, where the gPC coefficients are determined by the values of the target properties on a number of sampling conformation states. Within this framework, numerical quantification of the conformation-induced uncertainty is formulated as the following problem: how can we accurately and efficiently construct the gPC based surrogate model of the target property using limited sampling points within the high-dimensional conformational space? Several probabilistic collocation methods (PCM) such as ANOVA [31, 17, 58, 55] and sparse grid methods [52, 19, 18, 36, 30] have been proposed to accurately construct gPC expansions by selecting specific collocation points for sampling. However, there are two fundamental barriers when directly applying these approaches to high-dimensional biomolecular systems with hundreds to thousands of degrees of freedom in CG representations. The first barrier is the required number of sampling points, which can be too large for any gPC approach beyond a linear approximation. Moreover, empirical evidence indicates that sparse grid methods are often limited to dimensions less than ~ 40 (e.g., see [37]). The second barrier is the presence of limited accuracy in the calculation of target properties – even in the absence of structural uncertainty. For example, many calculations related to biomolecular solvation properties are subject to errors in the discretization and numerical solution of the associated partial differential equations [5, 26]. The error between the true values and the computed values of these target properties can lead to erroneous results due to inhomogeneous weight distribution over the sampling points, as illustrated in this paper. To circumvent these difficulties, we adopt an alternative noncollocation method based on compressive sensing [11, 14, 7] which reduces the influence of the limited accuracy of the target property while taking advantage of the sparsity of the gPC expansion. The compressive sensing method was initially proposed for signal processing and later applied to wide range of applications, including uncertainty quantification frameworks [29, 15, 54, 56].

2. Stochastic model

In this section, we briey introduce a semi-analytical stochastic model based on the elastic network model presented in [49, 25, 47]. The resulting harmonic system yields a Gaussian probability distribution for conformational states [4] that is straightforward to use in stochastic models for uncertainty quantification. In addition to the dimensionality reduction provided by the coarse-grained ENM, we note that further dimensionality reduction can be obtained for biomolecular target properties that have local dependence on structure; i.e., where the values associated with a particular property depend only on a subset of atoms in the molecule.

2.1. Full stochastic model of conformational fluctuation

We construct the stochastic conformation space of the biomolecular system based on the coarse-grained (CG) anisotropic network model (ANM) [4], a variant of the ENM where each amino acid residue is modeled as a single CG particle connected to neighboring residues by anisotropic harmonic potentials. ANM can be viewed as a simplified CG model of normal mode analysis [22, 6, 28, 32], where the model potential does not rely on the complex atomic-detail force field. Consider a biomolecule of N residues, we denote the 3N-dimensional equilibrium position vector by ${\overline{\mathbf{R}}}^T=[{\overline{\mathbf{r}}}_1^T{\overline{\mathbf{r}}}_2^T\cdots {\overline{\mathbf{r}}}_N^T]$, where ${\overline{\mathbf{r}}}_i$ is a 3-dimensional vector representing the equilibrium position of residue i. Similarly, we denote the 3N-dimensional instantaneous position vector ${\mathbf{R}}^T=[{\mathbf{r}}_1^T{\mathbf{r}}_2^T\cdots {\mathbf{r}}_N^T]$, where r_i represents the instantaneous position vector of residue i. The fluctuation vector can then be defined by $\mathrm{\nabla }\mathbf{R}=\mathbf{R}-\overline{\mathbf{R}}$. The harmonic approximation for the potential energy V with respect to the instantaneous position R is given by

$$V(\mathbf{R})=\frac{\gamma }{2}\sum _{i<j}{(r_{ij}-{\overline{r}}_{ij})}^{2}h(r_c-{\overline{r}}_{ij}),$$ (2.1)

where ${\overline{r}}_{ij}$ and r_ij represent the equilibrium and instantaneous distances between residue i and j, γ is a model parameter representing the elastic coefficient of the harmonic potential, r_c is the cut-off distance of the harmonic potential, and h is the Heaviside function.

Given the potential defined by Eq. (2.1), the 3N × 3N Hessian matrix has the form

$$\mathbf{H}=\left(\begin{array}{ccc}\hfill {\mathbf{H}}_{11}\hfill & \hfill {\mathbf{H}}_{12}\cdots \hfill & \hfill {\mathbf{H}}_{1N}\hfill \\ \hfill {\mathbf{H}}_{21}\hfill & \hfill {\mathbf{H}}_{22}\cdots \hfill & \hfill {\mathbf{H}}_{2N}\hfill \\ \hfill ⋮\hfill & \hfill \hfill & \hfill \hfill \\ \hfill {\mathbf{H}}_{N1}\hfill & \hfill {\mathbf{H}}_{N2}\cdots \hfill & \hfill {\mathbf{H}}_{NN}\hfill \end{array}\right)$$

with the element H_ij defined by

$${\mathbf{H}}_{ij}=\left(\begin{array}{ccc}\hfill {\partial }^{2}V/\partial X_i\partial X_j\hfill & \hfill {\partial }^{2}V/\partial X_i\partial Y_j\hfill & \hfill {\partial }^{2}V/\partial X_i\partial Z_j\hfill \\ \hfill {\partial }^{2}V/\partial Y_i\partial X_j\hfill & \hfill {\partial }^{2}V/\partial Y_i\partial Y_j\hfill & \hfill {\partial }^{2}V/\partial Y_i\partial Z_j\hfill \\ \hfill {\partial }^{2}V/\partial Z_i\partial X_j\hfill & \hfill {\partial }^{2}V/\partial Z_i\partial Y_j\hfill & \hfill {\partial }^{2}V/\partial Z_i\partial Z_j\hfill \end{array}\right),$$

where X_i, Y_i and Z_i represent the Cartesian coordinates of residues i. We note that the rank of H is 3N − 6 since V is translationally and rotationally invariant. This harmonic form for the potential leads to Gaussian statistics for the conformational probability distribution (e.g., individual residue position distribution). The correlation between individual residue fluctuation can be determined by the pseudo-inverse of the Hessian matrix H as [4]

$$\mathbf{C}=𝔼[\mathrm{\nabla }\mathbf{R}\mathrm{\nabla }{\mathbf{R}}^T]=\frac{k_{B}T}{\gamma }{\mathbf{H}}^{-1},$$ (2.2)

where 𝔼[·] denotes the expectation, k_B is the Boltzmann constant, T is the temperature.

We perform an eigendecomposition of H

$$\mathbf{H}=\mathbf{W}\mathbf{\Lambda }{\mathbf{W}}^T,\hspace{1em}\mathbf{\Lambda }=\text{diag}({\lambda }_1,\cdots ,{\lambda }_{3N-6}),$$ (2.3)

where λ_i is the i-th nonzero eigenvalue of H. W is a 3N × (3N − 6) matrix defined by

$$\mathbf{W}=[{\mathbf{w}}_1{\mathbf{w}}_2\cdots {\mathbf{w}}_{3N-6}],$$ (2.4)

where w_i is the corresponding i-th eigenvector of H. Then, the correlation matrix can be written as

$$\mathbf{C}=\frac{k_{B}T}{\gamma }\mathbf{W}{\mathbf{\Lambda }}^{-1}{\mathbf{W}}^T=\mathbf{U}{\mathbf{U}}^T,$$ (2.5)

where $\mathbf{U}={\left(\frac{k_{B}T}{\gamma }\right)}^{\frac{1}{2}}\mathbf{W}{\mathbf{\Lambda }}^{-\frac{1}{2}}$. The stochastic conformation space can be given by

$$\mathbf{R}(\mathit{\xi })=\overline{\mathbf{R}}+\mathrm{\nabla }\mathbf{R}(\mathit{\xi }),$$ (2.6a)

$$\mathrm{\nabla }\mathbf{R}(\mathit{\xi })=\mathbf{U}\mathit{\xi },$$ (2.6b)

where ξ = (ξ₁, ξ₂, ⋯, ξ_3N−6) is an independent and identically distributed (i.i.d.) Gaussian random vector. Given a value of ξ, the corresponding coarse-grained conformation is fully determined by Eq. (2.6), allowing us to calculate target properties, denoted by X(ξ).

2.2. Reduced-dimensionality stochastic model for conformational fluctuations

The dimension of the stochastic conformation space constructed by Eq. (2.6) is 3N − 6, which can still be high when N is large. However, we note that some target properties of interest, particularly those related to a specific residue p, may have only local dependence on the neighboring residues’ conformation rather than depending on all degrees of freedom in the biomolecule. For example, the solvent accessible surface area (SASA) of a specific residue p only depends on the positions of itself as well as its neighboring residues within a certain cutoff distance $r_c^{\{p\}}$. Under such circumstances, the full position fluctuation correlation matrix C can be replaced by a 3N × 3N matrix C′ of larger sparsity where the (i, j) element (a 3 × 3 matrix) is given by

$${\mathbf{C}}_{ij}^{\prime }={\mathbf{C}}_{ij}h(r_c^{\{p\}}-r_{ip})h(r_c^{\{p\}}-r_{jp}),$$ (2.7)

r_ip = |r_p − r_i|, r_jp = |r_p − r_j|, and $r_c^{\{p\}}$ is a cut-off distance of residue p such that X^{p} is independent of the residue i if $r_{ip}>r_c^{\{p\}}$.

Fig. 2.1 illustrates this dimensionality reduction procedure for local properties as discussed above. Similar to Eq. (2.6), we can construct the reduced stochastic conformation space by

$${\mathbf{C}}^{\{p\}}={\mathbf{U}}^{\{p\}}{\mathbf{U}}^{{\{p\}}^T},$$ (2.8a)

$${\mathbf{R}}^{\{p\}}({\mathit{\xi }}^{\{p\}})=\overline{{\mathbf{R}}^{\{p\}}}+{\mathbf{U}}^{\{\mathbf{p}\}}{\mathit{\xi }}^{\{p\}},$$ (2.8b)

where ξ^{p} is a d-dimensional i.i.d. normal random vector.

Fig. 2.1.

Sketch of a typical reduced property correlation matrix. The square on the left hand side represents the full correlation matrix. Each block represents 9 elements (in x, y and z directions) of a residue in the correlation matrix C. The blocks in blue color represent the matrix elements associated with some local target property X. The square on the right hand side represents the reduced correlation matrix C^{p} with lower dimensionality.

In summary, the value of a target property X is determined by the specific conformation state, corresponding to a point ξ (or ξ^{p}) in the full (or reduced) random space. Our goal is to systematically quantify the uncertainty in X with respect to the conformational fluctuations through gPC expansion, as introduced in the next section. The rest of the manuscript focuses on local properties so we will omit the superscript {p} in the following text, and use X(ξ) and ξ to represent the target property and the d-dimensional random vector, respectively.

3. Numerical methods

In this section, we first review the generalized polynomial chaos (gPC) expansion with a brief discussion on possible difficulties with probabilistic collocation methods. Next, we introduce a noncollocation method to construct the gPC expansion based on compressive sensing. We note that the sparsity of gPC coefficients will affect the performance of compressive sensing method (see [10]). Hence, we propose a method to elevate the sparsity of the gPC expansion by defining a new set of random variables according to the direction of variability in the target properties.

3.1. gPC expansion and collocation method

We use the gPC expansion to construct the surrogate model of the target property X with respect to the model parameter ξ, the molecular conformation by

$$X(\mathit{\xi })=\sum _{|\mathit{\alpha }|=0}^{\infty }{c}_{\mathit{\alpha }}{\psi }_{\mathit{\alpha }}(\mathit{\xi }),$$ (3.1a)

$${\psi }_{\mathit{\alpha }}(\mathit{\xi })={\psi }_{{\alpha }_1}({\xi }_1){\psi }_{{\alpha }_2}({\xi }_2)\cdots {\psi }_{{\alpha }_d}({\xi }_d),\hspace{1em}{\alpha }_i\in \mathbb{N}\cup \{0\},$$ (3.1b)

where d is the number of random variables, α = (α₁, α₂, ⋯, α_d) is a multi-index, and c_α is the gPC coefficient to be determined. ψ_αi (ξ_i) are univariate normalized Hermite polynomials, which satisfy the orthonormality condition:

$${\int }_{-\infty }^{\infty }{\psi }_k({\xi }_i){\psi }_l({\xi }_i)\rho ({\xi }_i)\mathrm{d}{\xi }_i={\delta }_{kl},\hspace{1em}k,l\in \mathbb{N}\cup \{0\},$$ (3.2)

where δ_kl is the Kronecker’s delta and $\rho ({\xi }_i)=\frac{1}{\sqrt{2\pi }}\text{exp}(-{\xi }_i^2/2)$ is the normal distribution function. The ENM described in Sec. 2.1 relies on d i.i.d. standard normal random variables, hence the gPC basis functions are constructed as the tensor products of univariate normalized Hermite polynomials as shown in Eq. (3.1b).

We truncate the expression (3.1) up to polynomial order P, hence X is approximated as:

$$X(\mathit{\xi })\approx \tilde{X}(\mathit{\xi })=\sum _{|\mathit{\alpha }|=0}^P{c}_{\mathit{\alpha }}{\psi }_{\mathit{\alpha }}(\mathit{\xi }),$$ (3.3)

using a total number of n gPC terms with n = (P + d)!/(P!d!).

Ideally, we would construct the truncated gPC expansion of X(ξ) by computing c_α using the orthonormality of ψ_α; e.g.,

$$c_{\alpha }=\int X(\mathit{\xi }){\psi }_{\mathit{\alpha }}(\mathit{\xi })\rho (\mathit{\xi })\mathrm{d}\mathit{\xi },$$ (3.4)

where ρ(ξ) is the probability density function (PDF) of ξ.

The integration can be accomplished by utilizing probabilistic collocation approaches such as tensor product [42, 43] or sparse grid [52, 19, 18] methods. Specifically, by evaluating X on specific collocation points ξ¹, ξ², ⋯, ξ^S; we have

$$c_{\mathbf{\alpha }}=\int X(\mathit{\xi }){\psi }_{\alpha }(\mathit{\xi })\rho (\mathit{\xi })\mathrm{d}\mathit{\xi }\approx \sum _{i=1}^{S}X({\mathit{\xi }}^i){\psi }_{\mathit{\alpha }}({\mathit{\xi }}^i)w^i,$$ (3.5)

where wⁱ is the corresponding weight associated with collocation point ξⁱ.

However, for the high-dimensional biomolecular systems considered the present work, the required number of collocation points S can be computationally intractable. For example, a small biomolecular system with a 27-dimensional reduced conformation random space would require S = 7.6 × 10¹² tensor product collocation points to construct a quadratic-order gPC expansion. Standard sparse grid method based on Gaussian quadrature and Smolyak construction reduces this to 1513 sampling points; however, the required number of sampling points is fixed for each order of the gPC approximation (e.g., the required number of sampling points for a 3rd-order approximation is 27829) which makes it difficult to incorporate adaptive sampling strategies.

Also we note that X is generally accompanied by numerical error; e.g.,

$$X(\mathit{\xi })=\overline{X}(\mathit{\xi })+\varphi ,$$ (3.6)

where $\overline{X}(\mathit{\xi })$ is the true value of the target property and ϕ represents the numerical error. In this work, we require ϕ to satisfy

$$|\varphi |\ll |X|,$$ (3.7)

so we can systematically study the accuracy of the constructed surrogate model using different numbers of sampling data. In general, the condition |ϕ| ≪ |X| is not essential to the application of gPC expansion to construct the surrogate model. ϕ provides a lower bound of the numerical error of the surrogate model; e.g., we should not expect the error of the surrogate model to be less than the error accompanied by the sampling data.

Moreover, as will be shown in Sec. 4, the aliasing error and the numerical error ϕ associated with X may lead to poor approximation of c_α by using probabilistic collocation method even if Eq. (3.7) is satisfied. To overcome the above difficulties, we compute the gPC expansion by applying compressive sensing as described in Sec. 3.2.

3.2. Compressive sensing method

To construct the gPC expansion in Eq. (3.3), we compute X(ξ) on M sampling points (ξ¹, ξ², ⋯, ξ^M), which are generated according to the distribution of random variables ξ. In the present work, ξ are d-dimensional i.i.d. standard normal random variables. We discretize Eq. (3.3) as a linear system

$$\left(\begin{array}{cccc}\hfill {\psi }_{{\mathit{\alpha }}_1}({\mathit{\xi }}^1)\hfill & \hfill {\psi }_{{\mathit{\alpha }}_2}({\mathit{\xi }}^1)\hfill & \hfill \cdots \hfill & \hfill {\psi }_{{\mathit{\alpha }}_n}({\mathit{\xi }}^1)\hfill \\ \hfill {\psi }_{{\mathit{\alpha }}_1}({\mathit{\xi }}^2)\hfill & \hfill {\psi }_{{\mathit{\alpha }}_2}({\mathit{\xi }}^2)\hfill & \hfill \cdots \hfill & \hfill {\psi }_{{\mathit{\alpha }}_n}({\mathit{\xi }}^2)\hfill \\ \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill \hfill & \hfill ⋮\hfill \\ \hfill {\psi }_{{\mathit{\alpha }}_1}({\mathit{\xi }}^M)\hfill & \hfill {\psi }_{{\mathit{\alpha }}_2}({\mathit{\xi }}^M)\hfill & \hfill \cdots \hfill & \hfill {\psi }_{{\mathit{\alpha }}_n}({\mathit{\xi }}^M)\hfill \end{array}\right)\left(\begin{array}{c}\hfill c_{{\mathit{\alpha }}_1}\hfill \\ \hfill c_{{\mathit{\alpha }}_2}\hfill \\ \hfill ⋮\hfill \\ \hfill c_{{\mathit{\alpha }}_n}\hfill \end{array}\right)=\left(\begin{array}{c}\hfill X({\mathit{\xi }}^1)\hfill \\ \hfill X({\mathit{\xi }}^2)\hfill \\ \hfill ⋮\hfill \\ \hfill X({\mathit{\xi }}^M)\hfill \end{array}\right)+\left(\begin{array}{c}\hfill {\epsilon }^1\hfill \\ \hfill {\epsilon }^2\hfill \\ \hfill ⋮\hfill \\ \hfill {\epsilon }^M\hfill \end{array}\right)$$

or equivalently,

$$\mathbf{\Psi }\mathit{c}=\mathit{X}+\mathit{\epsilon },$$ (3.8)

where Ψ is the “measurement matrix” with entries Ψ_i,j = ψ_αj (ξⁱ), c = (c_α1, c_α2, ⋯, c_αn)^T is the vector of the gPC coefficients, X = (X(ξ¹), X(ξ²), ⋯, X(ξ^M))^T is the vector consists of the outputs and ε = (ε¹, ε², ⋯, ε^M)^T is related to the truncation error.

Notice that Ψ is an M × n matrix, and we are interested in the case when M < n or even M ≪ n. It is presented in [10] that if Ψ satisfies the restricted isometry property (RIP), we can estimate c by solving the following optimization problem:

$$(P_{1,\epsilon }):\hspace{1em}\text{arg }\underset{{\mathit{c}}^{*}}{\text{min}}{\Vert {\mathit{c}}^{*}\Vert }_1\text{ subject to }{\Vert \mathbf{\Psi }{\mathit{c}}^{*}-\mathit{X}\Vert }_2\le \epsilon ,$$ (3.9)

where ε = ‖ε‖₂. The upper bound of the error ‖c − c*‖₂ is decided by ε and the sparsity of c:

$${\Vert \mathit{c}-{\mathit{c}}^{*}\Vert }_2\le C_1\epsilon +C_2\frac{{\Vert \mathit{c}-{\mathit{c}}_s\Vert }_1}{\sqrt{s}},$$ (3.10)

where C₁, C₂ are constants, s is a positive integer, and c_s is c with all but the s-largest entries set to zeros. For c in the present work, “sparse” means small ‖c − c_s‖₁ with s being smaller (or much smaller) than the length of c. The (P_{1, ε}) optimization problem can be solved using classical convex optimization solvers (e.g., CVX [24]), sparse recovery software packages (e.g., SPGL1 package [50], ℓ₁-MAGIC [1]), or the split Bregman method [23, 57, 9, 8]. In this paper, we use SPGL1.

To solve Eq. (3.9), we need the value of ε, which is generally not known a priori. In this work, we estimate ε using a cross-validation method [15, 56]. We first divide M sampling data into two parts denoted by M_r and M_υ. Second, c is computed with M_r sample points with a chosen series of tolerance error ε_r. Next, an optimized estimate ${\hat{\epsilon }}_r$ is determined such that ‖Ψ_υc − X_υ‖₂ is minimized, where Ψ_υ and X_υ represent the submatrix of Ψ and the subvector of X corresponding to validation portion of the sampling data. Finally, we repeat the above process for different replicas of the sample points and determine the optimal ε as $\epsilon =\sqrt{M/M_r}{\hat{\epsilon }}_r$. In this work, we set M_r = 2M/3 and performed the cross-validation for three replications. We note that verifying the RIP for a given matrix is a NP-hard problem. The aforementioned cross-validation procedure also serves as the verification for applying the ℓ₁ minimization method to approximate c as it estimates the error of ε.

3.3. Sparsity recovery via a “renormalized active” random space

The performance of the compressive sensing method introduced above is closely related to the ratio between the numbers of sampling points M and basis functions n, as well as the sparsity of the linear system in Eq. (3.8). In general, accuracy improves with either larger M/n ratios or sparser target vectors c. One way to increase M/n (for a given M) is to reduce the dimension of stochastic space hence n is reduced. Unfortunately, for biomolecular systems, the dimension of the stochastic conformation space is determined by the structure of the molecule and is not always amenable to direct reduction. Constantine et al. [13] have developed an alternative approach that can be used to increase sparsity by analysis of variability in the target properties. For the target X(ξ) with respect to PDF ρ(ξ), we define gradient matrix G by [13]

$$\mathbf{G}=𝔼[\nabla X(\mathit{\xi })\nabla X{(\mathit{\xi })}^T],$$ (3.11)

where ∇X(ξ) is the gradient vector defined by $\nabla X(\mathit{\xi })={(\frac{\partial X}{\partial {\xi }_1},\frac{\partial X}{\partial {\xi }_2},\cdots \frac{\partial X}{\partial {\xi }_d},)}^T$. We conduct the eigendecomposition

$$\mathbf{G}=\mathbf{Q}\mathbf{K}{\mathbf{Q}}^T,\hspace{1em}\mathbf{Q}=[{\mathbf{q}}_1{\mathbf{q}}_2\cdots {\mathbf{q}}_d],$$ (3.12a)

$$\mathbf{K}=\text{diag}(k_1,\cdots ,k_d),\hspace{1em}{k}_1\ge \cdots \ge k_d\ge 0,$$ (3.12b)

where q_i is the i-th eigenvector of G. Therefore, the target property X exhibits the largest variability along the direction q₁ while it exhibits the smallest variability along the direction q_d. This motivates the definition of a new random vector

$$\mathit{\chi }={\mathbf{Q}}^T\mathit{\xi },$$ (3.13)

where Q is unitary and χ = (χ₁, χ₂, ⋯, χ_d)^T are i.i.d. Gaussian variables since ξ are i.i.d. Gausian (also similar to Ref. [48]). Dependence of the target property X on χ_i decreases from χ₁ to χ_d.

Therefore, if we represent X by a gPC expansion with respect to χ, X may depend primarily on the first few random variables. Then the gPC coefficients associated with other variables exhibiting much smaller value (or even close to 0), yielding sparser c for the linear system defined in Eq. (3.8). Hence, if we recover the gPC coefficients with respect to χ in Eq. (3.8) by the compressive sensing method, we expect more accurate result than directly recovering gPC coefficients with respect to ξ.

Unfortunately, the gradient vector ∇X(ξ) is generally not known a priori. Direct evaluation of 𝔼[∇X(ξ)∇X(ξ)^T] is very computationally expensive: the cost of evaluation of ∇X(ξ) is proportional to the dimension of the ξ. Therefore, we evaluate ∇X(ξ) by approximating it via the gPC expansion recovered from ξ; e.g.,

$$\mathbf{G}\approx 𝔼[\nabla X^{\mathrm{g}\mathrm{P}\mathrm{C}}(\mathit{\xi })\nabla X^{\mathrm{g}\mathrm{P}\mathrm{C}}{(\mathit{\xi })}^T]$$ (3.14a)

$$X^{\mathrm{g}\mathrm{P}\mathrm{C}}(\mathit{\xi })=\sum _{|\mathit{\alpha }|=0}^P{c}_{\mathit{\alpha }}^{\{\mathit{\xi }\}}{\psi }_{\mathit{\alpha }}(\mathit{\xi }),$$ (3.14b)

where the superscript {ξ} represents gPC coefficients directly recovered from ξ. Evaluation of 𝔼 [∇X^gPC(ξ)∇X^gPC(ξ)] is straightforward with respect to PDF ρ(ξ), can be used to define Q and, therefore, the new random basis χ = Q^T ξ. Finally, new basis functions associated with new random variables χ can be used to reconstruct the gPC expansion of X with respect to the χ which, in general, yields greater sparsity.

Remark 3.1

We do not reduce the dimension of the conformational space in the above procedure. Instead, we define a new basis spanning the random space based on the variability direction of the target property. This set of basis functions is not universal, it depends on the specific target property X.

Remark 3.2

The gradient matrix G is approximated by Eq. (3.14). Therefore, eigenvectors [q₁ q₂ ⋯ q_d] may not correspond exactly to the steepest decay directions of variability for the target property X. Nevertheless, we adopt Eq. (3.14) to construct a “rotated” space that provides larger (if not optimal) sparsity.

Remark 3.3

Notice that ξ are i.i.d. Gaussian random variables and so as χ due to the matrix Q being unitary, the new basis functions associated with χ are still tensor product of Hermite polynomials, i.e., of the same form as in Eq. (3.1).

We summarize the entire procedure presented above (Sections 2 and 3) in Algorithm 1. In the next Section, we apply this framework to quantify uncertainty in biomolecular solvent accessible surface area properties in the presence of conformational fluctuations.

Algorithm 1

[Procedure to construct the gPC response surface of a given target quantity X with respect to a stochastic biomolecular conformation space.]

• Step 1. For a biomolecular system, we model the potential energy using the harmonic elastic network approach so the conformation fluctuation is Gaussian-distributed. We construct the full stochastic conformation space given in Eq. (2.6). For “local” target properties, we further reduce the dimension of the stochastic conformation space as in Eq. (2.8). We conduct eigenvalue decomposition of the correlation matrix and represent the fluctuation by a d-dimensional i.i.d. standard normal random vector denoted by ξ.

• Step 2. Generate M sampling points ξ¹, ξ², ⋯, ξ^M based on the distribution of ξ. Numerically compute X on ξ¹, ξ², ⋯, ξ^M to obtain M outputs X¹, X², ⋯, X^M (where X^q = X(ξ^q)). Denote X = (X¹, X², ⋯, X^M) as the “observation” in (P_1,ε). The “measurement matrix” Ψ is constructed as Ψ_i,j = ψ_αj (ξⁱ), where ψ_αj are the basis functions. The size of Ψ is M × n, where n is the total number of basis functions depending on P in (3.3).

• Step 3. Set the tolerance ε in (P_1,ε) by employing the cross-validation method.

• Step 4. Solve the ℓ₁ minimization problem

  $$\text{arg }\underset{{\mathit{c}}^{*}}{\text{min}}{\Vert {\mathit{c}}^{*}\Vert }_1\mathit{\text{ subject to }}{\Vert \mathbf{\Psi }{\mathit{c}}^{*}-\mathit{X}\Vert }_2\le \epsilon .$$

  to obtain the gPC coefficients c^{ξ}.

• Step 5. Evaluate the gradient matrix G = 𝔼[∇X^gPC(ξ)∇X^gPC(ξ)^T], given c^{ξ} and define the random vector χ by Eq. (3.13). Compute the sample of χ as χ^q = Q^T ξ^q, q = 1, ⋯, M.

• Step 6. Construct new “measurement matrix” $\tilde{\mathbf{\Psi }}$ by setting ${\tilde{\mathbf{\Psi }}}_{ij}={\psi }_{{\mathit{\alpha }}_j}({\chi }^i)$. Construct the gPC expansion of X(χ) by repeating steps 3–4 on random vector χ and using X = (X¹, X², ⋯, X^M)^T that have been determined in step 2.

4. Numerical Results

As an example, we apply our method to quantify the uncertainty in solvent-accessible surface area (SASA) caused by conformational fluctuations in the biomolecule bovine pancreatic trypsin inhibitor (PDB code: 5pti) [51], shown in Fig. 4.1. SASA is an essential element of numerous solvation models [5, 44, 40]. The SASA for the entire molecule can be decomposed into residue-specific contributions, allowing us to explore the influence of conformational fluctuations on local area uncertainty. SASA is calculated following Shrake et al. [45], setting N^p nearly equidistant probing points on the solvent particle and determining SASA value for each residue from the fraction of probing points that are not buried by any of the neighboring residues. In particular, we choose N^p ≈ 2.5 × 10⁵ such that the numerical error ϕ satisfies |ϕ| / |X| ≲ 1.0 × 10⁻⁴, see Sec. 4.2 for further discussion on sensitivity study on the accuracy of the constructed surrogate model by choosing different magnitudes of numerical error.

Fig. 4.1.

(a) Tube diagram of the equilibrium structure of bovine pancreatic trypsin inhibitor (PDB code: 5pti) with spheres denoting the residue C_α positions. (b) Tube diagrams of the molecule representing instantaneous conformational states under thermal fluctuation.

To demonstrate the applicability of our method in exploiting information from limited sampling data, we focus on the performance of our method when constructing a surrogate model using less than 2500 sample data. This performance is assessed relative to two reference systems: a direct Monte Carlo simulation of the conformational space with 10⁶ sampling data as well as a system constructed by the standard sparse-grid collocation method. We test our method by examining the L₂ error of the model as well as the Kullback-Leibler divergence between the probability density functions obtained from this new approach and the reference data.

4.1. Surrogate model for SASA of individual residues

Fig. 4.1 shows a sketch of the CG biomolecular model under equilibrium and thermal-fluctuation states. Following Ref. [4], each residue is modeled as a single α-carbon particle as shown in Fig. 4.1(a). Due to thermal fluctuations, the molecule exhibits a distribution of conformation states where individual residues may deviate from the equilibrium positions, as shown in Fig. 4.1(b). To model the fluctuation of individual residues, we construct the ANM correlation matrix C by Eq. (2.5) using a cut-off distance for the harmonic potential of r_c = 9.8Å. The radius values of the α-carbon residue and the solvent probe were set to 2.8 and 1.2 Å, respectively, for the SASA calculations.

We first consider local properties and study the SASA of residue P14. Starting with the full 168-dimensional random correlation matrix C, we construct the local correlation matrix C′ via Eq. (2.7) by setting the neighbor cut-off distance $r_c^p$ to be 9.5 Å. This cutoff value yields 8 neighboring residues and therefore a 27–dimensional random space ℝ²⁷(ξ) by Eq. (2.8). As shown in Fig. 4.2, the PDFs of the SASA of residue P14 extracted from the local and the full random conformation spaces agree well with each other, indicating that this particular property can be represented within a reduced space rather than the full 168-dimensional space. The dashed line in Fig. 4.2 represents the PDF extracted from the local random space by neglecting the fluctuation correlation between different residues (e.g., setting the off-diagonal blocks to zero). The resulting distribution is wider than that predicted by the full correlation matrix. This is not surprising since the off-diagonal elements represent the harmonic potential contribution of molecular deformation in Eq. (2.1). Neglecting the off-diagonal block elements results in a more “exible” molecule model which lacks the harmonic restraints and therefore exhibits a wider distribution of SASA values.

Fig. 4.2.

Probability density function of the SASA of the 14th residue obtained from the full correlation matrix C (solid line) and the local reduced correlation matrix C′ (“●” symbol). The dashed line represents the distribution obtained from the reduced correlation matrix where off-diagonal elements are set to zero.

Next, we construct the surrogate model by computing the gPC coefficients within the reduced random space ℝ²⁷(ξ) following the method presented in Sec. 3. First, we calculate the gPC coefficients $c_{\mathit{\alpha }}^{\{\mathit{\xi }\}}$ up to order P = 2 (406 basis functions) by setting M = 300 in Algorithm 1 and applying step 1 − 4. Given $c_{\mathit{\alpha }}^{\{\mathit{\xi }\}}$, we next construct the approximate gradient matrix G by Eq. (3.14). Eigendecomposition of this matrix provides a set of rotated random variables χ by Eq. (3.13), (Step 5 in Algorithm 1). Fig. 4.3 shows the resulting normalized eigenvalues of G and the reduced correlation matrix C′. We note that C′ is independent of the target quantity X; it is completely determined by the molecular structure. The eigenvalues of C′ decay slowly, at a rate similar to the full correlation matrix C (not shown in the plot), while the eigenvalues of the gradient matrix G decay much more quickly. This result indicates that, for a particular quantity X, the eigenvectors of C do not necessarily correspond to the directions with the steepest decay of variability in a target property.

Fig. 4.3.

Normalized eigenvalues of the gradient matrix G (“▼” symbol) and the correlation matrix C′ (“■” symbol).

Given the variables χ, we compute the corresponding gPC coefficients $c_{\mathit{\alpha }}^{\{\mathit{\chi }\}}$ with order P = 2 by applying Step 6 in Algorithm 1. The results are shown in Fig. 4.4. Compared with $c_{\mathit{\alpha }}^{\{\mathit{\chi }\}}$, the spectrum of $c_{\mathit{\alpha }}^{\{\mathit{\chi }\}}$ exhibits a higher degree of sparsity, as expected. This result indicates that, with the same polynomial order, the target quantity X can be approximated using fewer gPC terms with respect to the set of random variables χ than with ξ.

Fig. 4.4.

gPC coefficients (up to 2nd order) for the SASA value on the 14th residue obtained from CS method with respect to random vector ξ (red) and χ (blue) with dimension d = 27.

To examine the constructed surrogate model, we compute the relative L₂ error ε of the surrogate model by

$$\epsilon ={\left(\frac{\int |X(\mathit{\xi })-\tilde{X}(\mathit{\xi }){|}^2\rho (\mathit{\xi })\mathrm{d}\mathit{\xi }}{\int |X(\mathit{\xi }){|}^2\rho (\mathit{\xi })\mathrm{d}\mathit{\xi }}\right)}^{1/2},$$ (4.1)

where $\tilde{X}$ is the gPC expansion of X by Eq. (3.3) with $c_{\mathit{\alpha }}^{\{\mathit{\xi }\}}$ and $c_{\mathit{\alpha }}^{\{\mathit{\chi }\}}$, respectively. As X(ξ) is unknown in general, we use Monte Carlo sampling to approximate the integral in Eq. (4.2)

$$\epsilon \approx {\left(\frac{\sum _{i=1}^{N_s}|X({\mathit{\xi }}^i)-\tilde{X}({\mathit{\xi }}^i){|}^2}{\sum _{i=1}^{N_s}|X({\mathit{\xi }}^i){|}^2}\right)}^{1/2},$$ (4.2)

where N_s is the number of sampling data. In this work, we choose N_s = 10⁶.

Fig. 4.5 shows the relative L₂ error of the constructed surrogate model with gPC coefficients recovered from two independent sets of sample data. For each sample set, we use 200–400 points to construct the order P = 2 gPC expansion with 406 basis functions and 500–600 sample points to construct the order P = 3 of gPC expansion with 4060 basis functions. For each case, the L₂ error decreases as we increase the number of sampling points from 200 to 400. For the same number of sample points, the surrogate models constructed with respect to χ exhibit smaller L₂ error than those constructed with respect to ξ. In particular, given the same number of sampling points, sparser gPC coefficients c in Eq. (3.8) lead to more accurate recovery of c from the compressive sensing method by Eq. (3.9). The accuracies of the compressive sensing methods based on ξ and χ are comparable when the number of sampling points is close to the number of basis functions.

Fig. 4.5.

Relative L₂ error of the SASA value on residue 14 predicted by the gPC expansions $\tilde{X}(\mathit{\xi })$ and $\tilde{X}(\mathit{\chi })$, where the gPC coefficients are obtained from two separate sets of sampling data, represented by (a) and (b), respectively. The symbols “▼” and “■” denote the 2nd and 3rd order gPC expansion by ξ. The symbols “◆” and “●” denote the 2nd and 3rd order gPC expansion by χ. The dash-dot and dash-dot-dot lines represent the relative L₂ error of 1st and 2nd order gPC expansion obtained from level-1 and level-2 sparse grid points, using 55 and 1513 sample points respectively.

For random variables ξ, the L₂ error changes non-monotonically as we compute c_α at order P = 3 by increasing numbers of sampling points. The error increases as we increase the number of sample points to 500 and then decreases as we increase the number of sample point to 600 (although it remains larger than the 400-point error). This behavior is primarily due to the fact that the number of basis functions for P = 3 is much larger than the number of sample points and therefore $c_{\mathit{\alpha }}^{\{\mathit{\xi }\}}$ is poorly recovered due to insufficient sample points. However, for the transformed random variables χ, $c_{\mathit{\alpha }}^{\{\mathit{\chi }\}}$ can be accurately recovered due to the high sparsity of the gPC spectrum with a monotonic decrease in error with increasing numbers of sampling points.

We examined the surrogate model constructed by the sparse grid method based on Gaussian quadrature collocation points and Smolyak structure with gPC coefficients computed according to Eq. (3.5). Fig. 4.5 shows the relative L₂ error of the surrogate model constructed by approximating the integral in Eq. (3.4) with level-1 and level-2 sparse grid methods using 55 and 1513 sample points, respectively. Note that the algebraic accuracy of level-1 and level-2 sparse grid methods we use are 3 and 5, respectively. Therefore, we construct 1st-order and 2nd-order gPC expansions with level-1 and level-2 methods respectively. The sparse grid results show systematically larger L₂ errors than the compressive sensing approach. An unexpected phenomenon is that the error of 2nd-order expansion is larger than that of the 1st-order expansion. This behavior will be explained in Sec. 4.2.

The differences between the models constructed by $c_{\mathit{\alpha }}^{\{\mathit{\xi }\}}$ and $c_{\mathit{\alpha }}^{\{\mathit{\chi }\}}$ can be further illustrated by examining the response surfaces in the reduced random space shown in Fig. 4.6. This figure shows the response surfaces $\tilde{X}(\mathit{\xi })$ and $\tilde{X}(\mathit{\chi })$ with respect to two random variables with the remaining 25 random variables fixed. The gPC coefficients are computed using 300 sample points with the order P = 2 for both cases. For $\tilde{X}(\mathit{\chi })$, we only consider the first two random variables χ₁ and χ₂. For $\tilde{X}(\mathit{\xi })$, we consider the random variables ξ₂₁ and ξ₂₂ which are associated with the largest magnitudes of the first order gPC coefficients. For each case, we fixed the remaining variables as constant values extracted from an i.i.d. standard normal distribution 𝒩(0, 1).

Fig. 4.6.

(a) The reduced response surface constructed by $\tilde{X}({\chi }_1,{\chi }_2,{\chi }_3^0,\cdots ,{\chi }_{27}^0)$, where $({\chi }_3^0,\cdots ,{\chi }_{27}^0)$ are fixed values extracted from the i.i.d. normal distribution 𝒩(0, 1). The scattered symbols (blue points) are direct numerical simulation results on stochastic points (χ₁, χ₂, ⋯, χ₂₇) in ℝ²⁷ following an i.i.d. normal distribution 𝒩²⁷(0, 1). (b) The reduced response surface constructed by $\tilde{X}({\xi }_1^0,\cdots ,{\xi }_{21},{\xi }_{22},\cdots ,{\xi }_{27}^0)$ where $({\xi }_1^0,\dots ,{\xi }_{20}^0,{\xi }_{23}^0,\dots ,{\xi }_{27}^0)$ are fixed values extracted from the i.i.d. normal distribution 𝒩(0, 1). The scattered symbols (blue points) are direct numerical simulation results on points (ξ₁, ξ₂, ⋯, ξ₂₇) following an i.i.d. normal distribution 𝒩²⁷(0, 1).

The behavior of the Monte Carlo data around the response surfaces in Fig. 4.6 indicates that the variation of $\tilde{X}(\mathit{\chi })$ strongly depends on χ₁, χ₂ while the dependence is much weaker for ξ₂₁ and ξ₂₂. Furthermore, most of the symbols generated by $\tilde{X}(\mathit{\chi })$ fall near the reduced response surface $\tilde{X}(\mathit{\chi })$ with small deviation while the deviations for $\tilde{X}(\mathit{\xi })$ are much larger around the response surface $\tilde{X}(\mathit{\xi })$. As expected with rotation of the space by Eq. (3.13), this result indicates that $\tilde{X}(\mathit{\chi })$ can be fitted fairly well by only using two variables. However, if we use the original random variables, the reduced response surface can not be captured well even if we use the two most important variables associated with the first order gPC expansion. Fig. 4.6 clearly illustrates that the different sparsities of c result in different accuracies for the recovered response surfaces $\tilde{X}(\mathit{\chi })$ and $\tilde{X}(\mathit{\xi })$.

To evaluate the statistical information extracted from the surrogate model, we compute the SASA PDF for target residue P14 by evaluating 10⁶ sampling data points with the constructed surrogate model. These results are shown in Fig. 4.7(a) and compared with a reference solution based on the PDF computed from 10⁶ direct MC sample points. The compressive sensing method with 300 sample points yields the closest approximation of the reference solution. In contrast, the PDFs constructed by the direct Monte Carlo and sparse grid methods show significant deviation from the reference solution. To quantify the numerical error of the obtained PDFs, we computed the Kullback-Leibler divergence

$$D_{\mathrm{K}\mathrm{L}}={\int }_{-\infty }^{\infty }\text{ln}\left(\frac{f^N(X)}{f^0(X)}\right)f^N(X)\mathrm{d}X$$ (4.3)

with the discrete form where f^N(X) and f⁰(X) represent the PDFs of the numerical and reference solution, respectively. For the compressive sensing method, D_KL decreases as we increase the number of sampling points, which is consistent with the L₂ error of the surrogate model (Fig. 4.5). The plateau value at 500–600 sampling points is primarily due to the finite resolution of the PDF: a sensitivity study shows that D_KL between two i.i.d. sets of 10⁶ MC sample points is on the order of 10⁻⁴. In contrast, D_KL values of the PDFs obtained by the level-1 and level-2 sparse grid methods are about 20 and 450 times larger (respectively) than the results of the compressive sensing method.

Fig. 4.7.

(a) Probability density function (PDF) of the SASA values on residue P14 obtained from the gPC expansion $\tilde{X}(\mathit{\chi })$ using 300 sampling points (“●” symbol). Reference solution (solid line) is obtained from MC sampling using 10⁶ sample points. Results of 1st and 2nd gPC expansion obtained from level-1 (dash-dot line, 55 sample points) and level-2 (dash-dot-dot line, 1513 sample points) sparse grid method, and result from, and direct MC sampling methods (dashed line, 300 sample points) are also presented for comparison. (b) Kullback-Leibler divergence between the PDF of the reference solution and the PDFs obtained from gPC expansion $\tilde{X}(\mathit{\chi })$ (“■” symbol) with varying numbers of sample points. Level-1 sparse grid (dash-dot line) and direct Monte Carlo (dashed line, 300 sample points) results are presented for comparison.

4.2. Error sources and sensitivity analysis

To further investigate the applicability of the numerical methods for biomolecular systems, we quantified the SASA uncertainty for two other residues P11 and P20, which have 13 and 20 neighboring residues and correspond to random conformation spaces ℝ⁴² and ℝ⁶³, respectively. For each case, we constructed the surrogate model by the compressive sensing method with respect to both ξ and χ, as well as by the level-1 (1st-order gPC expansion) and level-2 (2nd-order gPC expansion) sparse grid methods. Fig. 4.8 shows the relative L₂ error of the surrogate models, the PDFs of the SASA values, and the K-L divergence with respect to the reference solution.

Fig. 4.8.

(a–b) Relative L₂ error of the SASA value on residue 11 (a) and 20 (b) predicted by the gPC expansions $\tilde{X}(\mathit{\xi })$ and $\tilde{X}(\mathit{\chi })$. The symbols “▼” and “■” denote the 2nd and 3rd order gPC expansion by ξ. The symbols “◆” and “●” denote the 2nd and 3rd order gPC expansion by χ. The dash-dot lines represent the relative L₂ error of 1st order gPC expansion obtained from level-1 sparse grid methods using 85 and 127 sample points. The dash-dot-dot lines represent the relative L₂ error of 2nd order gPC expansion obtained from level-2 sparse grid methods using 3613 and 8065 sample points. (c–d) Kullback-Leibler divergence between the PDF of the reference solution and the PDFs obtained from the constructed surrogate models (“■” symbol) for residues 11 (c) and 20 (d). Level-1 sparse grid (dash-dot line) and direct Monte Carlo (dashed line, 300 sample points) results are presented for comparison.

Similar to the results for residue P14, the surrogate models constructed with respect to χ yield smaller error than the ones constructed with respect to ξ. The accuracy of the ξ and χ compressive sensing methods is comparable when the number of sampling points is close to the number of basis functions. However, the surrogate model constructed with respect to χ is more accurate than ξ when the number of sampling points is much less than the number of basis functions; e.g., when the third-order gPC terms are incorporated. In particular, the surrogate model (2nd-order gPC expansion) for residue 20 constructed by the level-2 sparse grid in random space ℝ⁶³ yields the largest deviation from the reference solution.

For the present system, the relatively large surrogate model error constructed by the sparse grid method (e.g., Eq. (3.5)) can be explained as follows. Given the target quantity X computed at collocation points, the gPC coefficient c_α is computed by

$$c_{\mathit{\alpha }}=\sum _{i=1}^{N^{\mathrm{s}\mathrm{p}}}{w}^i(\overline{X}({\mathit{\xi }}^i)+\varphi ({\mathit{\xi }}^i)){\psi }_{\mathit{\alpha }}({\mathit{\xi }}^i)=\sum _{i=1}^{N^{\mathrm{s}\mathrm{p}}}{w}^i({\overline{X}}^i+{\varphi }^i){\psi }_{\mathit{\alpha }}^i,$$ (4.4)

where ${\overline{X}}^i=\overline{X}({\mathit{\xi }}^i)$, ϕⁱ = ϕ(ξⁱ), ${\psi }_{\mathit{\alpha }}^i={\psi }_{\mathit{\alpha }}({\mathit{\xi }}^i)$ represent the true solutions, numerical error, and Hermite basis function evaluated at the sparse grid collocation point ξⁱ, respectively; N^sp is the required number of sampling point with integral accuracy up to order 2P +1; and ϕ is the associated numerical error accompanied with the computed value of $\overline{X}$. We assume that

$$|{\varphi }^i|\ll |{\overline{X}}^i|$$ (4.5)

and that c_α can be approximated by

$$c_{\mathit{\alpha }}=\sum _{i=1}^{N^{\mathrm{s}\mathrm{p}}}{w}^i{\overline{X}}^i{\psi }_{\mathit{\alpha }}^i+\sum _{i=1}^{N^{\mathrm{s}\mathrm{p}}}{w}^i{\varphi }^i{\psi }_{\mathit{\alpha }}^i={\overline{c}}_{\mathit{\alpha }}+\sum _{\begin{array}{c}\hfill |\mathit{\alpha }+\mathit{\beta }|\hfill \\ \hfill >2P+1\hfill \end{array}}\sum _{i=1}^{N^{\mathrm{s}\mathrm{p}}}{\overline{c}}_{\mathit{\beta }}{w}^i{\psi }_{\mathit{\alpha }}^i{\psi }_{\mathit{\beta }}^i+\sum _{i=1}^{N^{\mathrm{s}\mathrm{p}}}{w}^i{\varphi }^i{\psi }_{\mathit{\alpha }}^i,$$ (4.6)

where ${\overline{c}}_{\mathit{\alpha }}=\int \overline{X}(\mathit{\xi }){\psi }_{\mathit{\alpha }}(\mathit{\xi })\rho (\mathit{\xi })\mathrm{d}\mathit{\xi }$ represents the true value of the gPC coefficient of index α and ${\overline{c}}_{\mathit{\beta }}$ represents the gPC coefficients with order |α + β| > 2P +1. The second term on the righthand side of Eq. (4.6) represents aliasing error due to the sparse grid approximation. The third term $w^i{\varphi }^i{\psi }_{\mathit{\alpha }}^i$ represents the error due to the numerical error ϕ accompanied with the numerical computation of $\overline{X}$.

For system of high dimensionality, both the aliasing error $\sum _{\begin{array}{c}\hfill |\mathit{\alpha }+\mathit{\beta }|\hfill \\ \hfill >2P+1\hfill \end{array}}\sum _{i=1}^{N^{\mathrm{s}\mathrm{p}}}{\overline{c}}_{\mathit{\beta }}{w}^i{\psi }_{\mathit{\alpha }}^i{\psi }_{\mathit{\beta }}^i$ and the numerical error $\sum _{i=1}^{N^{\mathrm{s}\mathrm{p}}}{w}^i{\varphi }^i{\psi }_{\mathit{\alpha }}^i$ may induce pronounced error to the numerical computation of c_α. Specifically, we assume that the numerical error ϕ_i superimposed on each collocation point is i.i.d. with zero mean and small variance $|{\sigma }_{\varphi }^2|\ll {|X|}^2$. Given this assumption, the term ${\sum }_{i=1}^{N^{\mathrm{s}\mathrm{p}}}{w}^i{\varphi }^i{\psi }_{\mathit{\alpha }}^i$ is zero mean with variance

$$\text{Var }\left(\sum _{i=1}^{N^{\mathrm{s}\mathrm{p}}}{w}^i{\varphi }^i{\psi }_{\mathit{\alpha }}^i\right)=\sum _{i=1}^{N^{\mathrm{s}\mathrm{p}}}{(w^i)}^2{({\psi }_{\mathit{\alpha }}^i)}^2{\sigma }_{\varphi }^2.$$ (4.7)

When the dimension of ξ is large, we note that the weight distribution on sparse grid points is inhomogeneous; i.e.,

$$\sum _i{w}^i=1,\hspace{1em}\exists k,|w^k|\gg 1.$$ (4.8)

Fig. 4.9 plots the variance of the term ${\sum }_{i=1}^{N^{\mathrm{s}\mathrm{p}}}{\varphi }^i{w}^i$ (normalized by ${\sigma }_{\varphi }^2$) for c₀ (i.e., the coefficient of basis function ψ₀(ξ) ≡ 1) computed at different levels of sparse grid points. As the dimension increases, the variance of the error term increases rapidly, leading to non-negligible errors in the computation of c_α.

Fig. 4.9.

Variance of the numerical error term ${\sum }_{i=1}^{N^{\mathrm{s}\mathrm{p}}}{\varphi }_i{w}_i$ (normalized by ${\sigma }_{\varphi }^2$) for c₀ computed by level-1 (“■”), level-2 (“▲”) and level-3 (“●”) sparse grid points.

For illustration purposes, we consider the following 27-dimensional function:

$$f(\mathit{\xi })=305+\sum _{k=1}^{27}{\xi }_k-0.01{\left(\sum _{k=1}^{27}{\xi }_k^2\right)}^2.$$ (4.9)

We first construct a 1st-order gPC expansion f₁ by using the level-1 sparse grid method (algebraic accuracy 3) to compute the coefficients:

$$c_{\mathit{\alpha }}={\int }_{{ℝ}^{27}}f(\mathit{\xi }){\psi }_{\mathit{\alpha }}(\mathit{\xi })\rho (\mathit{\xi })\mathrm{d}\mathit{\xi }\approx \sum _{i=1}^{N_1^{\mathrm{s}\mathrm{p}}}f({\mathit{\xi }}^i){\psi }_{\mathit{\alpha }}({\mathit{\xi }}^i)w^i,$$ (4.10)

where ξⁱ, wⁱ are sparse grid points and corresponding weights and $N_1^{\mathrm{s}\mathrm{p}}$ is the total number of the level-1 sparse grid points. Next we construct a 2nd-order gPC expansion f₂ by using the level-2 sparse grid method (algebraic accuracy 5) to compute the coefficients with the same manner. We compute the relative L₂ error of f₁ and f₂ as:

$${\epsilon }_k={\Vert f_k-f\Vert }_2/{\Vert f\Vert }_2={\left(\sum _{q=1}^{N_4^{\mathrm{s}\mathrm{p}}}{(f_k({\mathit{\xi }}^q)-f({\mathit{\xi }}^q))}^2{w}^q\right)}^{1/2}/{\left(\sum _{q=1}^{N_4^{\mathrm{s}\mathrm{p}}}f{({\mathit{\xi }}^q)}^2{w}^q\right)}^{1/2},k=1,2,$$ (4.11)

where we use level-4 sparse grid method (algebraic accuracy 9) so that the numerical integral gives accurate result. The 2nd-order expansion yields larger L₂ error due to the aliasing error in numerical integration. In particular, ε₁ = 0.029 while ε₂ = 0.100.

We also constructed the gPC expansion of f by sample points superimposed with numerical error:

$$c_{\mathit{\alpha }}={\int }_{{ℝ}^{27}}f(\mathit{\xi }){\psi }_i(\mathit{\xi })\rho (\mathit{\xi })\mathrm{d}\mathit{\xi }\approx \sum _{q=1}^{N_1^{\mathrm{s}\mathrm{p}}}f({\mathit{\xi }}^q)(1+\sigma \zeta ){\psi }_i({\mathit{\xi }}^q)w^q,$$ (4.12)

where ζ is a standard Gaussian random variable and σ is the magnitude of the noise. We repeat each test with 1000 independent sets of noise and present the mean and standard deviation of the L₂ error in Table 4.1. As σ increases from 10⁻⁴ to 10⁻³, the relative L₂ error further increases. Moreover, the 2nd-order expansion yields much larger error than 1st-order expansion due to the more inhomogeneous weight distribution.

Table 4.1.

L₂ error of of the 1st-order and 2nd-order expansion of f constructed by level-1 (ε₁) and level-2 (ε₂) sparse grid method with sampling data superimposed with different magnitude of numerical error.

σ	ε1	ε2
1 × 10−3	0.04 ± 0.02	1.1 ± 0.8
5 × 10−4	0.03 ± 0.01	0.5 ± 0.4
1 × 10−4	0.029 ± 0.002	0.14 ± 0.09

Similar to the simple numerical example presented above, the surrogate model error of the biomolecular system constructed by the sparse grid method is determined by both the aliasing error and the numerical error on the sampling point. Here we systematically investigate the L₂ error of the surrogate model of the target property X (e.g., the SASA value) on residue P14. The target quantity X on sampling point is computed under various accuracy levels with relative error from approximately 10⁻⁵ to 10⁻³. The different accuracy levels are achieved by choosing different number of probe points on the solvent particle when computing the SASA value of the target residue. For each accuracy level, we conduct a random 3D rotation of the molecule and conduct 32 computation of the SASA value on residue P14. We approximate the relative error by σ_X/𝔼(X) with σ_X and 𝔼(X) defined by

$${\sigma }_X=\frac{1}{S}\sum _{i=1}^S{\sigma }_{X^i},𝔼(X)=\frac{1}{S}\sum _{i=1}^S{X}^i,$$ (4.13)

where σ_Xⁱ is the standard deviation of 32 independent computation values of X on sample point ξⁱ and S is the total number of sample points. We emphasize that σ_X defined by Eq. (4.13) is not equal to ϕ (e.g., the difference between the observation and true solution). However, σ_X provides a useful guide to understand the magnitude of the disturbance on the sample data. We also note that all the numerical results presented in Sec. 4.1 were computed using sampling data with relative error σ_X/𝔼(X) ≈ 5 × 10⁻⁵.

Fig. 4.10 shows the relative L₂ error of the gPC expansion using the compressive sensing, level-1 sparse grid, and level-2 sparse grid methods. The results from the sparse grid methods are very sensitive to accuracy level of the sample point. For high accuracy levels, the L₂ error is mainly due to the aliasing error. Increasing σ_X/𝔼 (X) from 2 × 10⁻⁵ to 1.2 × 10⁻³, the mean value of the relative L₂ error increases from 2.20% to 3.36% for the level-1 sparse grid method and from 9.66% to 135.13% for the level-2 sparse grid method. In contrast, the compressive sensing method is insensitive to the imposed error on X for the present system; the resulting error is nearly constant for δ ∈ [10⁻⁵, 10⁻³]. This result suggests another advantage of the present method: the present method is more stable in the presence of limited accuracy in the computed target quantity. For high dimensional systems, the performance of sparse grid method strongly depends on the accuracy of the evaluation of X at collocation points. Similar phenomena have been reported previously [59]. In practice, it may be computationally infeasible to evaluate X at the accuracy level required for stable sparse grid results. However, our new method based on compressive sensing shows a much weaker dependence on accuracy at individual sample points.

Fig. 4.10.

Relative L₂ error of the SASA value on residue 14 predicted by 1st-order gPC expansion constructed by the level-1 sparse grid method (“▼” symbol), 2nd-order gPC expansion constructed by the level-2 sparse grid method (“■” symbol), and 2nd-order gPC expansion constructed by compressive sensing (“●” symbol using 200 sample points) under different accuracy levels σ_X/𝔼(X). For each accuracy level, 32 sets of independent computations are conducted to compute the L₂ error of the constructed response surface.

In order to explore the sensitivity of the accuracy level on the constructed gPC expansion, we conducted 32 independent computations on the target quantity X by randomly rotating the biomolecule 32 times on each sampling point. However, we are cautious to claim that numerical error ϕⁱ superimposed on the Xⁱ is i.i.d. among the sampling points. The i.i.d. assumptions adopted in Eqs. (4.7) and (4.12) are used to demonstrate that the numerical error ϕ may further induce error to the constructed gPC expansion. The study presented in this section demonstrates that the sparse grid method may induce relatively large errors to the constructed surrogate model. Rigorous error analysis of the sparse grid method in high-dimensional/complex systems is beyond the scope of this work. However, there appear to be at least two important error sources (aliasing and numerical error on X) that could lead to erroneous results when applying the sparse grid method to high-dimensional systems such as biomolecules. We note that other specific structured or adaptive sparse grid methods [20, 30, 33, 34] may alleviate the instability issue in high-dimensional systems. However, these methods either have less exibility (the required number of sampling points is fixed for each accuracy level) or require a specialized design for adaptivity criteria.

4.3. Surrogate model for total molecular SASA

Finally, we apply our method to quantify the uncertainty of the total SASA for the entire molecule. Unlike the previous local per-residue SASA, this target quantity depends on the conformation states of all residues. We construct the gPC expansion within the full random space ℝ¹⁶⁸. Due to the high dimensionality, we use a second-order gPC expansion with 14365 basis functions. Fig. 4.11 shows the relative L₂ error of the surrogate model and the K-L divergence of the PDFs.

Fig. 4.11.

(a) Relative L₂ error of the total molecular SASA by gPC expansion $\tilde{X}(\mathit{\xi })$ (“●”) and $\tilde{X}(\mathit{\chi })\tilde{X}(\mathit{\chi })$. The dash-dot line represents the relative L₂ error obtained from sampling on level-1 sparse grid points. Sampling over the level-2 sparse grid points generates erroneous results, as discussed in the text. (b) Kullback-Leibler divergence between the PDF of the reference solution and the PDFs obtained from surrogate models $\tilde{X}(\mathit{\chi })$ (“■”), level-1 sparse grid (dash-dot line) and direct MD sampling (dash line, 2400 sample points).

We note that the Hermite basis functions associated with the normal distribution are unbounded which leads to inhomogeneous error distributions in the random space. Fig. 4.12 shows the average error distribution of the surrogate model within different regimes of the SASA value. The average error of the surrogate model of X within [x₁, x₂] is defined by

$$𝔼(\epsilon (x_1,x_2))={\left(\frac{\sum _i{(X^{\mathrm{g}\mathrm{P}\mathrm{C}}({\mathit{\xi }}^i)-X({\mathit{\xi }}^i))}^2{I}_{(x_1,x_2)}(X({\mathit{\xi }}^i))}{\sum _i{I}_{(x_1,x_2)}(X({\mathit{\xi }}^i))}\right)}^{\frac{1}{2}},$$ (4.14)

where I_{(x1, x2)}(X(ξⁱ)) is an indicator function which is 1 if X(ξⁱ) ∈ [x₁, x₂] and 0 otherwise. As shown in Fig. 4.12, the error exhibits a minimum value near the equilibrium state and increases as the X approaches the tails of the SASA PDF. This demonstrates that the constructed surrogate model is not a global approximation of the target quantity X over the entire random space. Instead, it provides an approximation of X with respect to the local points near equilibrium within the random space. Nevertheless, in practice, we are generally interested in the variation of X with response to conformation fluctuation near the equilibrium state; e.g., the relatively small thermally induced molecular fluctuations considered in the present work.

Fig. 4.12.

The average error distribution of the surrogate models of the total-SASA within different regimes [X − ΔX,X + ΔX]. ΔX is chosen as 5 Ų. The surrogate models are constructed using 800 (blank) and 1600 (filled) sample points, respectively. The mean value of total SASA is approximately 3351 Ų (denoted by “⋆” symbol), corresponding to conformations near the equilibrium state with respect to the thermal fluctuation.

Similar to the “local” properties discussed above, the gPC expansion recovered by our compressive sensing method yields the smallest error. However, the advantage of our new method over direct Monte Carlo sampling for the global SASA is not as large as in the case of local properties. By constructing multi-D basis function through tensor product of one-dimensional basis functions, the upper bound (Here, the upper bound exists because the sampling of the Gaussian random variables is truncated in practice.) of the basis function becomes larger, which decreases the efficiency of the compressive sensing method. This is similar to the phenomenon observed by others [39, 54]. If only statistical information such as expectation values or PDFs are needed, other methods such as quasi-Monte Carlo [35, 46] may be suitable for high-dimensional systems.