Physics-Based Machine Learning Trains Hamiltonians and Decodes the Sequence–Conformation Relation in the Disordered Proteome

Abstract

Intrinsically disordered proteins and regions (IDPs) are involved in vital biological processes. To understand the IDP function, often controlled by conformation, we need to find the link between sequence and conformation. We decode this link by integrating theory, simulation, and machine learning (ML) where sequence-dependent electrostatics is modeled analytically while nonelectrostatic interaction is extracted from simulations for many sequences and subsequently trained using ML. The resulting Hamiltonian, combining physics-based electrostatics and machine-learned nonelectrostatics, accurately predicts sequence-specific global and local measures of conformations beyond the original observable used from the simulation. This is in contrast to traditional ML approaches that train and predict a specific observable, not a Hamiltonian. Our formalism reproduces experimental measurements, predicts multiple conformational features directly from sequence with high throughput that will give insights into IDP design and evolution, and illustrates the broad utility of using physics-based ML to train unknown parts of a Hamiltonian, rather than a specific observable, in combination with known physics.

Graphical Abstract

1. INTRODUCTION

Machine learning (ML) can decipher complex patterns in data that are not typically amenable to closed-form mathematical equations. Problems in physical sciences are rife with such data sets and are routinely subjected to ML to ultimately make predictions. However, numerous problems are governed in part by known quantitative laws of physics, which are analytically tractable. In such a case, do we ignore that knowledge or leverage it to our advantage? This intriguing possibility presents itself in an important biophysical problem when modeling conformations of intrinsically disordered proteins and regions (collectively termed IDPs). Conformations of IDPs depend on both electrostatics and nonelectrostatic interactions. While physics of electrostatics is analytically tractable, nonelectrostatics is not amenable to closed-form mathematical relation. Hence, the approach we take builds and applies a ML model only to address the intractable nonelectrostatic contribution while preserving and leveraging analytically tractable electrostatic contribution. We showcase this hybrid usage of physics-based analytical theory and ML to model the IDP conformation that will be relevant not only in IDP biophysics but also in general problems of physical sciences.

IDPs participate in vital biological functions including transcriptional regulation, cellular differentiation, and the formation of membraneless organelles, to name a few. A growing body of data suggests that features of IDP sequence and conformation, which in turn depend on sequence,^1–6 provide insights into function.^7–17 Consistent with this, IDPs can modulate their conformation and function by biological regulators such as mutations and post-translational modifications (e.g., phosphorylation) that alter their sequence. Furthermore, IDPs can sensitively regulate their conformation by responding to environmental conditions such as changes in ionic strength, pH, or crowders (chemical regulators).^6,18–21 To advance IDP biophysics further, we need to decode the sequence–conformation relation and its response to chemical and biological regulators to ultimately decipher the sequence–conformation–function link.

IDPs are naturally abundant in the human proteome, and there is a need to understand their evolution across multiple proteomes. Advances in synthetic biology also require the design of IDPs with specific conformational features. Recognizing the demand from both IDP evolution and design that requires high-throughput modeling, it is timely to establish a mathematical link between IDP sequence and conformation. Different experimental tools have been developed to probe IDP configuration, from single-molecule FRET,^19,22,23 NMR,^24,25 to SAXS.^26,27 However, these specialized experiments on IDPs have yet to reach the necessary throughput. All-atom molecular dynamic simulations, despite their tremendous success in modeling folded proteins, are limited in IDPs due to their computational cost, sampling challenges,^28,29 and inaccuracies in force fields. More recently, coarse-grain simulations emerged as an alternate approach due to their computational efficiency and ability to accurately reproduce experimentally measured chain dimensions.^17,30–32 Coarse-grain simulations, despite their impressive throughput, are computationally costly for long chains. Simulation throughput is also limiting when designing an IDP with targeted conformational properties. Mutagenesis-based design often requires enumerating all possible mutations⁶ and their effect on the conformation, causing a combinatorial explosion. ML or mathematical models based on theoretical physics can mitigate these challenges. Furthermore, mathematical models can give valuable insights and generate new hypotheses to be subsequently tested by detailed experiment and/or simulation.

ML models such as AlphaFold,³³ used in modeling folded proteins with unique structure, are inapplicable to IDPs³⁴ due to their ensemble nature and paucity of data. To overcome the lack of direct experimental data, coarse-grain simulations are used to generate ensemble average properties for a large set of sequences to train ML models.^17,32,35 These approaches train a large data set of a specific observable generated from the simulation of diverse sequences to predict the same observable for a new sequence. A separate approach has used machine learning to directly train the ensemble and its change upon mutation.³⁶ Naturally, these training schemes are limited to the prediction of the specific observable and/or an ensemble under a given environmental condition.

We provide a novel approach integrating physics-based theory, simulation, and ML that is not limited to one specific observable used to train the model and is versatile in its prediction. Our proposed formalism begins with a sequence-dependent analytically tractable Hamiltonian from which several ensemble average conformational properties can be derived with an approximate closed-form relation. These analytical models are scalable and capable of handling the aforementioned combinatorial explosion faced in IDP design and generating new hypotheses. A challenge, however, is to model sequence-dependent nonelectrostatic interaction parameters of the Hamiltonian. This parameter for a specific sequence is not known a priori. Sequence-dependent electrostatics, on the other hand, can be directly calculated using Coulomb’s law and its screening based on laws of ionic equilibrium, amenable to analytical treatment using tools from theoretical polymer physics.^4,37 An appealing feature of our theory is its ability to infer interaction parameters of the Hamiltonian for a sequence if values of a specific observable is known. For example, the ensemble average end-to-end distance may be known from a simulation or experiment. Given this knowledge, the inference of the unknown parameter is straightforward with simple algebraic manipulation at no computational cost or optimization. We leverage this special feature of our formalism to infer sequence-dependent non-electrostatic parameters from simulated values of a specific observable for a large set of sequences (see Figure 1 left panel). For this purpose, we utilize two recent large-scale simulation studies.^17,30 Since the underlying mapping between these nonelectrostatic parameters and their parent sequences is difficult to decipher analytically, we use ML to train these inferred parameters for a large set of sequences. For a new sequence, we then predict the nonelectrostatic patterning from the machine-learned model. The predicted nonelectrostatic interaction is further combined with our charge-patterning theory, not machine learned, derived from physicochemical laws (see Figure 1 right panel). This is the essence of physics-based ML (PML) in the present context. The resulting Hamiltonian can accurately reproduce multiple simulated conformational properties, e.g., ensemble average distance between any two residues and not just end-to-end distance for a given sequence.

Figure 1.

PML of IDP trains Hamiltonian and not a specific observable. (Left) Simulated values of end-to-end distance $\left(⟨R_{\mathrm{e}\mathrm{e}}⟩\right)$ for different sequences (S1, S2, S3…) are used to infer sequence-dependent nonelectrostatic interaction parameters (${\omega }_2$) from a physics-based theory in step 1. (Right) These sequence-dependent interaction parameters are trained with CNN in step 2. In step 3, for a new sequence, nonelectrostatic interaction Hamiltonian ($H_{\text{non-elec}}$) parametrized by ${\omega }_2$ is predicted from the trained CNN model and combined with physics-based analytically tractable electrostatics ($H_{\text{elec}}$) to construct the overall Hamiltonian (H) from which several observables (such as distance $⟨R_{ij}⟩$ between two residues i and j beyond $⟨R_{\mathrm{e}\mathrm{e}}⟩$) are predicted. For the purpose of this figure, $\sqrt{⟨R_{i,j}^2⟩}$ is denoted by $⟨R_{i,j}⟩$ to avoid cumbersome notation.

Thus, PML integrates ML with a physics-based mathematical formalism and provides a powerful platform to predict both trained and untrained observables for a new sequence going beyond traditional approaches that are limited to a specific observable. Furthermore, this proof of concept principle shows the importance of training a Hamiltonian directly from experimental data, when such data of the IDP conformation becomes available at a large scale, even if measurements are limited to a specific observable. This will provide a high-throughput modeling of IDP conformation that will be useful in IDP design and applicable to predict conformations of disordered proteomes. Beyond IDP biophysics, our approach will also contribute to the emerging frontier at the interface of ML and physics.^38,39

2. METHODS

2.1. Inferring Sequence-Dependent Nonelectrostatic Parameters from Simulations.

The global dimension of an IDP can be characterized by ensemble average end-to-end distance squared ($⟨R_{\mathrm{e}\mathrm{e}}^2⟩$) or a normalized swelling factor $x=⟨R_{\mathrm{e}+}^2⟩/\left(N{l}^2\right)$, where N is the number of amino acids and l = 5.5 Å.²¹ We have recently derived a sequence-dependent free energy (F(x)) as a function of the swelling factor (x) to determine the most likely x (by minimizing the free energy) for a given sequence using a model that accounts for (i) chain connectivity, (ii) electrostatic interaction and its screening by salt described by Debye–Huckel theory, (iii) residue-pair-specific nonelectrostatic interaction modeled by two-body delta function potential with unknown parameters, and (iv) a three-body mean-field repulsive interaction (with strength ${\omega }_3$) as a product of two delta functions²¹ needed to avoid collapse. Specifically, F(x) is given by

$$\beta F(x)=\frac{3}{2}(x-\text{ln}x)+{\left(\frac{3}{2\pi }\right)}^{3/2}\frac{\text{Ω}}{x^{3/2}}+{\omega }_3{\left(\frac{3}{2\pi }\right)}^3\frac{B}{2x^3}+\frac{l_{\text{b}}}{b}\frac{Q}{x^{1/2}}\sqrt{\frac{6}{\pi }}$$ (1)

where $\beta =1/\left(k_{\mathrm{b}}T\right)$, $k_{\mathrm{b}}$ is Boltzmann constant, T is the absolute temperature, l_b is the Bjerrum length, assumed to be l_b = 7.12 Å (293/T), and b = 3.8 Å is the bond length. Sequence and chain length dependence is embedded in three terms: Ω accounting for nonelectrostatic two-body interaction, B similarly capturing three-body interaction, and $\mathrm{\Omega }$ for two-body electrostatic interactions (at zero salt, see refs 21 and 40 for arbitrary salt conditions).

$$\begin{gathered} \Omega =\frac{1}{N}\sum _{m=2}^N\sum _{n=1}^{m-1}{\omega }_{m,n}{\left(m-n\right)}^{-1/2}; \\ B\text{=}\frac{1}{N}\sum _{p=3}^N\sum _{m=2}^{p-1}\sum _{n=1}^{m-1}\frac{\left(p-n\right)}{{\left[\left(p-m\right)\left(m-n\right)\right]}^{3/2}} \\ Q=\frac{1}{N}\sum _{m=2}^N\sum _{n=1}^{m-1}{q}_m{q}_n{\left(m-n\right)}^{1/2} \end{gathered}$$ (2)

Q is also known as sequence charge decoration (SCD)⁴ and is calculated from sequence charge information by assigning q = −1 to glutamic, aspartic acids, and the C-terminal end, q = +1 to lysines, arginines, and the N-terminal end, and possibly q = +0.5 to histidines to be consistent with coarse grain simulation (see Supporting Information for more). The residue-pair-specific nonelectrostatic interaction between the two residues at m and n is ${\omega }_{m,n}$, related to the residue-pair-specific Lennard-Jones potential (or its variants) used in the simulation.³⁰ However, the mapping between the parameters used in simulation and ${\omega }_{m,n}$ used as a prefactor of the delta function potential in theory are unknown. We circumvent this by defining a two-body sequence-dependent effective parameter ${\omega }_2$

$${\omega }_2=[\sum _{m=2}^N\sum _{n=1}^{m-1}{\omega }_{m,n}{(m-n)}^{-1/2}]/[\sum _{m=2}^N\sum _{n=1}^{m-1}{(m-n)}^{-1/2}]$$ (3)

Minimizing free energy, the equation for the most likely x becomes

$$x^{3/2}\left(1-\frac{1}{x}\right)={\left(\frac{3}{2\pi }\right)}^{3/2}{\omega }_2\left(\sum _{m=2}^N\sum _{n=1}^{m-1}{(m-n)}^{-1/2}\right)\frac{1}{Nx}+\frac{l_{\text{b}}}{l}\sqrt{\frac{2}{3\pi }}Q+{\omega }_3{\left(\frac{3}{2\pi }\right)}^3\frac{B}{x^{5/2}}$$ (4)

which can be used to predict x for a sequence if ${\omega }_2$ and ${\omega }_3$ are known with Q computed from the sequence. Conversely, for a given value of ${\omega }_3$, the corresponding value of ${\omega }_2$ for a sequence can be inferred if x is known. Ensemble average end-to-end distance obtained from coarse-grained simulation or experiment can be used to extract x for a given sequence. We use the latter approach to infer ${\omega }_2$ values for different sequences using two separate large-scale coarse grain simulations performed by Zheng et al.³⁰ and Tesei et al.¹⁷

2.2. Training and Predicting ${\omega }_2$ with Neural Networks.

The inferred ${\omega }_2$ values were trained against their sequence using a convolutional neural network (CNN_Z for the Zheng set and CNN_T for the Tesei set) model (see Supporting Information and Figure S1 for details of the CNN model). Tesei data set was further modified by removing highly homologous proteins to ensure that proteins in the test set are significantly different from the training set (see Supporting Information for details of detecting homologous sequences). Excluding homologous proteins from the original Tesei set resulted in a modified Tesei set with 27,060 proteins. We used k-folds to predict ${\omega }_2$ values for the resulting Tesei data set and the Zheng set. For each set, we randomly shuffled and split the set into ten equally sized folds. We then trained ten different models, each using a different fold for testing and validation, and the remaining eight folds for training. Compiling the results of all test sets provides predicted ${\omega }_2$ values for the full data set.

2.3. Predicting Conformational Properties from the Hamiltonian.

For an unknown sequence, ${\omega }_2$ is first predicted from the trained CNN model and used (in eqs 1 and 3) to predict most likely x and the ensemble average end-to-end distance. However, predictions are not limited to $⟨R_{\mathrm{e}\mathrm{e}}^2⟩$ only. For example, ensemble average distances $\left(⟨R_{ij}^2⟩\right)$ between any two amino acid residues i and j, beyond just end-to-end distance, can be determined by minimizing the corresponding free energy $F\left(x_{ij}\right)$

$$\beta F\left(x_{i,j}\right)=\frac{3}{2}\left(x_{i,j}-\text{ln}{x}_{i,j}\right)+{\left(\frac{3}{2\pi }\right)}^{3/2}\frac{{\text{SHDM}}_{i,j}}{x_{i,j}^{3/2}}+{\left(\frac{3}{2\pi }\right)}^3\frac{{\omega }_3{T}_{i,j}}{2(i-j)x_{i,j}^3}+\frac{l_{\text{b}}}{b}\sqrt{\frac{6}{\pi }}\frac{{\text{SCDM}}_{i,j}}{x_{i,j}^{1/2}}$$ (5)

with $x_{ij}|i-j|l^2=\langle R_{ij}^2\rangle$. The electrostatic contribution (in the zero salt limit) is calculated from sequence charge decoration matrix (SCDM); the details of which, along with its salt dependence, can be found in ref 21 and Supporting Information. The noncharge patterning contribution given by sequence hydropathy decoration matrix (SHDM) is defined as

$${\text{SHDM}}_{i,j}=\frac{1}{\left(i-j\right)}[\sum _{m=j}^i\sum _{n=1}^{j-1}{\omega }_{m,n}\frac{{\left(m-j\right)}^2}{{\left(m-n\right)}^{5/2}}+\sum _{m=j+1}^i\sum _{n=j}^{m-1}{\omega }_{m,n}{\left(m-n\right)}^{-1/2}+\sum _{m=i+1}^N\sum _{n=j}^i{\omega }_{m,n}\frac{{\left(i-n\right)}^2}{{\left(m-n\right)}^{5/2}}+\sum _{m=i+1}^N\sum _{n=1}^{j-1}{\omega }_{m,n}\frac{{\left(i-j\right)}^2}{{\left(m-n\right)}^{5/2}}]$$ (6)

For a specific sequence, ${\mathrm{S}\mathrm{H}\mathrm{D}\mathrm{M}}_{ij}$ is estimated by replacing ${\omega }_{m,n}$ (in eq 6) by sequence-specific ${\omega }_2$ from the CNN model. With this mean-field approximation, ${\omega }_3$, and the definition of $T_{ij}$ (see ref 21), the most likely $x_{ij}$ and hence $⟨R_{ij}^2⟩$ can be predicted for a given sequence. However, an improved (still approximate) model (${\mathrm{S}\mathrm{H}\mathrm{D}\mathrm{M}}_{\mathrm{I}}$) can be envisioned beyond the mean-field model. We note that there are four interaction terms (I1, I2, I3, and I4) in eq 6 with different summation ranges, corresponding to interactions between different segments of the parent sequence. These are I1, interaction between the segment bounding (j, i) and the N-terminal dangling end between (1, j – 1); I2, interaction between amino acids in the segment bounded by (j, i); I3, interaction between segment (j, i) and the C-terminal dangling end (i + 1, N); and I4, interaction between (1, j – 1) and (i + 1, N), the two dangling ends. These different sets of interactions can be modeled using ${\omega }_2$ for specific sequence segments. For I1, ${\omega }_{m,n}$ can be replaced by ${\omega }_2$ for the sequence bounded by (1, i), similarly for I3, ${\omega }_2$ could be estimated for the sequence bounded by (j, N). For I2 and I4, ${\omega }_2$ can be estimated for the sequence bounded by (j, i) and the parent sequence (1, N), respectively.

3. RESULTS

3.1. Simulated Hamiltonian Can Be Mapped to an Analytically Tractable Hamiltonian.

We built two separate neural network models, CNN_Z and CNN_T, to train sequence-dependent nonelectrostatic interaction parameters (${\omega }_2$) inferred from coarse-grain simulations of Zheng et al. and Tesei et al.,^17,30 respectively. Zheng and colleagues simulated 15,390 designed protein sequences and provided a linear regression model using two patterning metrics to describe ensemble average radius of gyration and scaling exponent.³⁰ More recently, Tesei et al. curated and simulated 28,058 disordered regions of the human proteome using a similar but different coarse-grain force field. Both CNN_Z and CNN_T models accurately reproduce sequence-specific ${\omega }_2$ values in their respective test sets (Figure 2). We conclude that nonelectrostatic potentials used in the simulation can be effectively mapped to an analytically tractable delta function potential with a sequence-specific strength (${\omega }_2$). Furthermore, these effective parameters can be predicted by CNN_Z and CNN_T for any arbitrary sequence. The optimal value of the three-body interaction (${\omega }_3=0.2$) was determined such that the predicted and true values (inferred from the simulation) of ${\omega }_2$ correlate best for both Zheng and Tesei data sets simultaneously (see Supporting Information Figure S2 for details). While the CNN model accurately predicts ${\omega }_2$ for most sequences, there are also a few outlier sequences with notable differences between the predicted and true ${\omega }_2$ values. We compared the average fraction of 20 amino acids between the outlier and nonoutlier sets and found that outliers tend to be enriched in charges. Furthermore, outlier sequences tend to have a higher net charge compared to the nonoutlier sequences (see Supporting Information Figures S3–S5).

Figure 2.

Nonelectrostatic effective potential can be mapped to a sequence-dependent parameter using a neural network. Sequence-dependent nonelectrostatic interaction strength ${\omega }_2$ predicted from the neural network model well reproduces true ${\omega }_2$ values extracted from two sets of simulations: (left) simulation set performed by Tesei et al. with 27,060 sequences (correlation R = 0.94 between the predicted and true values) and (right) simulation set performed by Zheng et al. with 15,390 sequences (correlation R = 0.97).

3.2. Nonelectrostatic Patterning Can Be Predicted Directly from the Sequence.

Sequence-specific ${\omega }_2$ predicted from the trained model yields a measure of nonelectrostatic patterning, analogous to charge patterning given by SCD metric.⁴ We thus define a machine-learned sequence hydropathy decoration metric (SHD_ML) as

$${\text{SHD}}_{\text{ML}}={\omega }_2[\sum _{m=2}^N\sum _{n=1}^{m-1}{(m-n)}^{-1/2}]/N\approx \frac{4}{3}{\omega }_2{N}^{1/2}$$ (7)

where in the second equality, we approximated the summation by an integral when N is sufficiently large. The primary nomenclature SHD was borrowed from the work of Zheng et al.³⁰ In the present work, we use ML using CNN_T or CNN_Z to compute SHD_ML so it can be used directly to predict size. Furthermore, SHD_ML (or ${\omega }_2$ removing the chain length dependence) can be used in addition to the traditional patterning metric (hydropathy patterning³⁰ or charge patterning^3,4,41) to build ML models for functions that are beginning to emerge.^17,42

3.3. PML-Derived Hamiltonian Can Predict Size across the Disordered Proteome.

SCD⁴ and similar metrics³ so far have been used to qualitatively determine the relative trend of expansion/compaction between sequences of different charge content and patterning.^6,12,43 However, our theoretical formalism (eq 1) with machine-learned ${\omega }_2$ values can predict IDP size (not just trend) for a sequence, accounting for both electrostatic and nonelectrostatic patterning. We reiterate that this is different from traditional ML approaches that train protein size and not the parameter of the Hamiltonian. We leverage this unique feature of our formalism and predict ensemble average end-to-end distance ($⟨R_{\mathrm{e}\mathrm{e}}^2⟩$) for a given sequence. Predicted chain dimensions compare well with the simulated values for the Zheng and Tesei data sets (see Figure 3). CNN_Z was used to predict ${\omega }_2$ values when comparing chain dimensions from Zheng simulations for 15,390 designed sequences. CNN_T was used for the Tesei set to predict chain dimensions of 27,060 disordered proteins of the entire human proteome. PML-predicted R_ee values agree well with the true values for most sequences, with some outliers. We notice that some of these outliers were outliers in the predicted ${\omega }_2$ values as well (see Supporting Information Figure S6). However, PML also predicts the dimension well for a significant fraction of the sequences identified as outliers when predicting ${\omega }_2$. This finding suggests that modest inaccuracy in predicting ${\omega }_2$ is masked in the prediction of the chain dimension, possibly due to the dominant role of electrostatics on the chain dimension for these sequences. To further investigate the nature of the outlier sequences, we divided the original set in seven bins based on the net charge of the proteins. We computed the correlation coefficient between the predicted and true values of R_ee within each bin (see Figure S7 in Supporting Information). While each bin has high correlation coefficient, there is a slight drop in correlation with increased net charge. While dimensions of these proteins were already predicted from these large-scale simulations, our goal here is not to derive further biological insights by reproducing these results. Instead, we want to highlight that our formalism provides a fast and accurate model to predict ensemble average end-to-end distances directly from the sequence. The end-to-end distance for the entire proteome could be predicted in about 16 h. Thus, if new sets of sequences—either for design or for another proteome—are constructed, our methodology will provide a quick prediction of conformations for further analysis without having to run lengthy simulations at first. From these preliminary predictions, new hypotheses can be generated and further tested with detailed but limited simulations afterward.

Figure 3.

PML can predict chain dimensions for large sets of proteins consistent with simulation. Predicted end-to-end distance from PML reproduces simulation results for two sets of simulations: (left) Tesei set of 27,060 (correlation R = 0.99) and (right) Zheng set of 15,390 (correlation R = 0.97). Theoretical predictions were made with the conditions used in the simulation, i.e., charge assignments, ionic strength, temperature, etc., and ${\omega }_3=0.2$.

3.4. Ensemble Average Distance Maps Can Be Predicted Directly from the Sequence.

Mathematical models grounded on Hamiltonian models can formally predict several ensemble average properties. Consistent with this expectation, our theoretical formalism can predict multiple ensemble average properties beyond just end-to-end distance, for example, ensemble average distance $⟨R_{ij}^2⟩$ between any two amino acid residues i and j. To showcase this broad predictive power of the PML approach, we have chosen 25 representative sequences from the Tesei list of the human proteome (see Supporting Information for details of the selection criteria). Predicted distance maps for each of the sequences compare well with the maps generated from the simulation. Following Tesei, the CALVADOS simulation package was used to generate simulated conformations to benchmark theoretical predictions (see Supporting Information for details of CALVADOS). The results for the three sequences with the most compact chain dimensions (quantified by x) are shown here for illustration (Figure 4); the remaining comparisons can be found in the Supporting Information (see Figures S8–S12 in the Supporting Information).

Figure 4.

PML can predict local and global measures of chain conformation beyond end-to-end distance. Ensemble average inter-residue distances from Tesei set show close agreement between PML predicted (bottom triangle) and simulated maps (top triangle). Shown are the three most compact sequences out of the representative subset of 25 randomly drawn; see Supporting Information for 22 others and for correlations across all residue pairings for each sequence. Different colors show distances (in nanometers) between the two residues i and j shown in the two axes of the triangles. From left to right, sequences are A0A1B0GVY4_31–71, A0A1W2PP97_1–45, and A6NHP3_1–187.

Simulated maps highlight regions of relative expansion and compaction even when sequence separation is fixed (|i – j| is a constant), reflecting deviation from the homopolymer model. These detailed features are well reproduced in the predicted maps. Quantitative comparisons between predicted distances are also provided as correlations in the Supporting Information (see Figures S8–S12 in Supporting Information). Similar comparisons were performed for the Zheng set using 29 sequences, with reasonable agreement between simulation and prediction (see Figures S13–S17 in Supporting Information for details). For these predictions, we used CNN_Z. As an additional test of our PML prediction, we predicted distance maps of these 29 sequences from the Zheng list but using CNN_T and compared against simulations carried out using CALVADOS (different from Zheng simulation force field). Again, PML predictions agree well with these new simulations (Figures S18–S22 in Supporting Information). This additional analysis shows that simulated distance maps for the same sequence can be different depending on the force field. Not surprisingly, PML predictions made using different models of ${\omega }_2$ (i.e., CNN_T and CNN_Z) are also different. Finally, we notice that specific sequences (13,550, 14,175, 14,782, and 15,339 in the Zheng list) can have noticeable differences (Pearson correlation below 0.95) between the predicted and simulated (using Zheng force field) distance maps. We hypothesized that the discrepancy is possibly due to the mean-field assumption ${\omega }_{m,n}\approx {\omega }_2$ used to estimate SHDM. We used the improved model SHDM_I to predict these distance maps for these specific sequences from the Zheng list. SHDM_I yields improved prediction of the distance maps (see Supporting Information Figure S23). However, it is important to note that SHDM_I is still an approximate model and further improvement would require building models that retain amino acid pair-specific interactions ${\omega }_{m,n}$. This would be needed when modeling an IDP without any charge residues where variations in the distance map will purely arise from nonelectrostatics, not accounted for in our mean-field model (${\omega }_{m,n}\approx {\omega }_2$) at present.

3.5. PML Can Reproduce Experimental Data.

How well do PML predictions compare against the experimental measurement of protein size? To answer this, we curated 64 protein sequences (including wild type and mutants) for which radius of gyration values were reported using SAXS and/or FRET.^30,31 PML predictions were made using the CNN_T model (Figure 5). Furthermore, we accounted for experimental salt, pH, and temperature conditions varying across experiments (see Table S1 in Supporting Information for different sequences and conditions). These conditions directly impact the electrostatic contributions in our model but ignore their impact on ${\omega }_2$. CNN_T-based prediction is chosen here for comparison for two reasons. First, Tesei simulation was carried out with the force field that was originally adjusted to reproduce the majority of the proteins, although not all, in the benchmark set. No such adjustments were made for the force field used by Zheng. Furthermore, CNN_T was trained against sequences from human proteome that are expected to well represent the benchmark set consisting of natural sequences or their mutations. However, we ensured that none of these proteins in the benchmark set were used in the original Tesei set. In contrast, CNN_Z was trained against protein sequences that were designed and not naturally occurring. These could be the reasons why CNN_T performs better than CNN_Z (see Figure S24 in Supporting Information). We also notice minor deviations between PML prediction and experimental measurement, even when using CNN_T. The discrepancy could arise from two additional sources beyond training inaccuracies. First, the solution dependence of ${\omega }_2$ is not included in the current model. Furthermore, we have estimated the radius of gyration from the ensemble average end-to-end distance by using a simple conversion based on homopolymer theory. Nevertheless, the close agreement between prediction and experiment (Figure 5) shows that PML with CNN_T offers a reliable and fast tool to predict protein size directly from the sequence.

Figure 5.

PML well reproduces experimental measurement of radius of gyration $\left(R_{\mathrm{g}}\right)$ across 64 different protein sequences. PML prediction was made assuming ${\omega }_3=0.2$ and using CNN_T to estimate ${\omega }_2$. $R_{\mathrm{g}}^{\text{pML}}$ was estimated from $\sqrt{\left(⟨R_{\mathrm{e}\mathrm{e}}^2⟩/6\right)}$. We trained ten models, each with different training, validation, and test set. ${\omega }_2$ and subsequently $R_{\mathrm{g}}$ values were calculated from these models and averaged to yield $R_{\mathrm{g}}^{\text{pML}}$. Error bars are standard deviation of the ten $R_{\mathrm{g}}$ values. Different colors denote data reported by different groups; details of which can be found in Supporting Information.

4. DISCUSSION

We present a Hamiltonian-based mathematical formalism to determine conformational properties of IDPs accounting for sequence-specific charge and noncharge interactions simultaneously. While the effect of charge decoration can be analytically computed, noncharge patterning is difficult to describe mathematically. We overcome this difficulty by using the formalism in reverse: with the knowledge of the protein size such as ensemble average end-to-end distance, we infer the sequence-specific noncharge interaction parameter. In the absence of large-scale experimental data, we determine the protein size from recently performed high-throughput coarse-grain simulations originally benchmarked against experimental data. These simulated values serve as a proxy for experimental data and are used to extract noncharge parameters for a large set of sequences. Next, we trained these inferred sequence-dependent interaction parameters using a neural network to predict their values for a new sequence outside of the training set. The predicted values were combined with electrostatic contributions to reproduce a simulated observable (ensemble average end-to-end distance), demonstrating our ability to predict the ensemble average end-to-end distance directly from a sequence in an accurate and efficient manner. We further benchmarked our approach against pre-existing experimental data on multiple (64) protein sequences including wild type and mutants. Critically, our formalism, due to its reliance on Hamiltonian, can predict observables beyond end-to-end distance such as the ensemble average distance between any two arbitrary residues. These predictions provide a detailed knowledge of the sequence-dependent distance map, capturing both local and global measures of chain dimension. As a result, it can now predict conformational differences due to differences in both charge and noncharge patterning.^44,45 These broad-ranging predictions reiterate the power of an integrated approach that (i) deploys ML to selectively train part of the Hamiltonian that is not analytically tractable and (ii) combines with interactions that are known from the laws of physics and are analytically tractable. This integrated approach has been termed PML.

Although our theory has been applied to describe the equilibrium properties of a single IDP, it can extend far beyond. For example, it can serve to develop a theory of IDP dynamics under dilute conditions or even IDP solution when a single-chain Hamiltonian is extended to a multichain Hamiltonian. Modeling IDP solution will have implications in building a sequence-specific quantitative theory of liquid–liquid phase separation (LLPS). Analytical theories of LLPS are currently limited to predicting effects of charge patterning only due to their inability to quantify and model noncharge patterning. Consequently, qualitative trends are predicted when comparing two sequences with different charge patterning neglecting variations in noncharge interactions or, at best, phase diagrams are fitted.^46,47 Several models of noncharge interactions are being investigated to account for their effect on the phase diagrams.⁴⁸ A multichain Hamiltonian accounting for noncharge patterning, building on the PML formalism presented here, will be able to make quantitative predictions avoiding these approximations. It can also model differences in phase separation propensity between two sequences that only differ in noncharge patterning.⁴⁴ However, there are some limitations to our approach. The overall success of the proposed formalism to model IDP conformation, dynamics, and phase behavior will only be as accurate as the force fields from which interaction parameters were derived and trained. If better force fields for IDPs are generated, new simulations should be performed, and the neural network model should be retrained. The role of force field is apparent when comparing distance maps of 29 sequences (compare Figures S13–S17 and S18–S22 in the Supporting Information) generated from two different PMLs trained with different force fields used in Zheng and Tesei simulation. This is also evident in our comparison of the two models against known experimental data (see Figures 5 and S24 in Supporting Information). We have also ignored the presence of folded domains that may surround intrinsically disordered regions (IDRs) and alter the conformation due to additional interactions arising between IDR and the folded domain. However, inferred ${\omega }_2$ would still serve as a measure of intrachain nonelectrostatic interaction. Similarly, these parameters can be further augmented to model salting out effect,⁴⁹ not included in the present model.

In summary, our approach makes four important points. First, it forms a bridge between simulation and theory. Specifically, it maps potentials used in coarse-grain simulations to effective interactions used in analytical theory. The ability to map these potentials allows us to predict numerous observables of multiple sequences without having to run new simulations. This would be critical when high-throughput calculation is needed in the design of IDPs and understanding conformations of multiple proteomes that would be computationally prohibitive, even for coarse-grain simulations. The gain in computational efficiency would be even more drastic if the formalism is extended to multichain problem or dynamics of IDPs that are typically plagued by associated computational costs of modeling multiple chains or hydrodynamic interactions. This is not to say that PML should completely replace simulation; rather, PML formalism should be used for preliminary analysis to generate insights and hypotheses that should be further tested with detailed but limited simulation and experiment. Second, the machine-learned noncharge patterning parameter will add a new metric to understand and train other models of protein function such as LLPS. Third, our approach shows a proof of concept principle to train interaction parameters and not observables when experimental data of IDP dimensions become available at a large scale. This is in contrast with traditional ML approaches. Finally, our PML approach shows how to integrate quantitative laws of physics and ML to problems not only in IDP biophysics but also in other areas in physical sciences where part of the problem is analytically solvable.

Supplementary Material

The Supporting Information is available free of charge at https://pubs.acs.org/doi/10.1021/acs.jctc.4c01114.

Details of the theory, CNN model, selection of training and test sets, comparison between outlier and nonoutlier sets, comparison graphs between PML prediction and simulation, and details of the proteins for which experimentally measured R_g values were compared against PML prediction (PDF)