IDPForge: Deep Learning of Proteins with Global and Local Regions of Disorder

Abstract

Although machine learning has transformed protein structure prediction of folded protein ground states with remarkable accuracy, intrinsically disordered proteins and regions (IDPs/IDRs) are defined by diverse and dynamical structural ensembles that are predicted with low confidence by algorithms such as AlphaFold and RoseTTAFold. We present a new machine learning method, IDPForge (Intrinsically Disordered Protein, FOlded and disordered Region GEnerator), that exploits a transformer protein language diffusion model to create all-atom IDP ensembles and IDR disordered ensembles that maintains the folded domains. IDPForge does not require sequence-specific training, back transformations from coarse-grained representations, nor ensemble reweighting, as in general the created IDP/IDR conformational ensembles show good agreement with solution experimental data, and options for biasing with experimental restraints are provided if desired. We envision that IDPForge with these diverse capabilities will facilitate integrative and structural studies for proteins that contain intrinsic disorder, and is available as an open source resource for general use.

1. Introduction

Structures of biomolecules have driven functional insight ever since Watson and Crick solved the structure of DNA, establishing the structure-function paradigm that has been critical to progress in molecular biology and biochemistry. X-ray crystallography and cryo-EM along with nuclear magnetic resonance (NMR) have informed the traditional structure-function paradigm for folded proteins for many decades. The recent advent of machine learning (ML) models for protein structure prediction, most notably AlphaFold (AF)^1,2 and RoseTTAFold³, have made stunning breakthroughs in producing accurate ground state structures of monomeric folded proteins of high quality, similar to that of experimental structures. Although static structures of folded proteins can yield significant insights, protein function is inherently controlled by dynamics.^4–8 But the surprising depth of deep learning models suggest that they too can create not just a single structure, but a diverse and representative protein structure-dynamical space for functional insight.

In fact the ML/AI field has seen an emergence of a number of approaches to learn structural variants of folded states.⁹ One expected class of ML algorithms are generative models that learn conformational distributions from molecular dynamics (MD) data via autoencoders, Boltzmann generators¹⁰ and flow-matching¹¹, or to drive enhanced conformational sampling^12,13 in learned lower manifolds of collective variables¹⁴. Notably, denoising diffusion probabilistic models (DDPMs) have emerged as a powerful framework for structural variant sampling and generative protein design^2,15–20. Compared to traditional generative approaches, DDPMs provide a more flexible mechanism to incorporate diverse conditioning signals, such as functional constraints, secondary structure preferences, or binding site requirements, enabling the generation of proteins tailored to specific tasks^15–17,21. Other approaches involve inference time manipulations of structure prediction models. The Meiler lab cleverly demonstrated that reducing the depth of the input multiple sequence alignments (MSAs) in AF2 led to predictions of protein structures in multiple conformational states²², an idea that was later elaborated upon by other groups using sequence clustering or sub-sampling the MSAs to isolate the origin of conformational states of certain classes of folded proteins.^23,24

But approximately two-thirds of the canonical human proteome are predicted to be within intrinsically disordered proteins (IDPs) or intrinsically disordered regions (IDRs) that do not adopt a dominant folded structure, but instead utilize their conformational range to execute diverse cellular functions.^25,26 Because X-ray crystallography and cryo-EM methods are non-viable for this class of protein, computational models must bridge the gap by creating IDP ensembles that agree with the highly averaged experimental solution data from NMR, small angle X-ray scattering (SAXS), single molecule Förster Resonance Energy Transfer microscopy (smFRET) and pulsed EPR double electron-electron resonance (DEER-EPR).^27–29 The generation of putative disordered ensembles can be based on knowledge based methods such as IDPConformer Generator³⁰, atomistic molecular dynamics³¹ or a $C_{\alpha }$ based coarse-grained representation^32,33. These structural “pools” of disordered protein conformers are often reweighted to give an ensemble average that agrees with experimental observables^34–38, although there is a risk that the underlying pool is incomplete and missing important populations. Recently generative algorithms have become popular for generating all-atom or coarse-grained IDP ensembles^39–43. For example idpGAN uses a generative adversarial network that learns a $C_{\alpha }$ based coarse-grained representation of disordered sequences, then superseded by a latent diffusion model with enhanced transferability to their test sequences^41,42. IDPFold developed by Zhu et al. employed a two-stage training of a $C_{\alpha }$ backbone denoising model with PDB and all-atom molecular dynamics (MD) simulation structures, which showed promising performances on sampling structured and disordered proteins⁴³. Still they often require postprocessing by some type of reweighting step to gain better experimental relevance.

Our group introduced a generative recurrent neural network (GRNN), DynamICE (Dynamic IDP Creator with Experimental Restraints)⁴⁴, that uses experimental data to bias the torsion angle distributions of an underlying structural pool to generate new conformational states that conform to experimental averages. The significance of the DynamICE approach is that we remove the iterative guesswork about relevant sub-populations and subsequent reweighting of arbitrarily drawn conformations. But the underlying long short term memory (LSTM) architecture uses an internal coordinate representation of protein conformers, which is sub-optimal for meeting distance restraints such as nuclear overhauser effect (NOEs) and paramagnetic relaxation enhancement (PREs). Hence a method that can exploit Cartesian coordinates would be a more natural framework. Furthermore DynamICE like other Bayesian and generative models mentioned above also requires training each new IDP sequence and/or with its own experimental structural restraints, which reduces its generalizability.

Here we shed the limitations of generic generative models and sequence specific training to create experimentally valid conformational ensembles for IDPs as well as IDRs from proteins containing folded domains, with all-atom resolution. We adapt the attention and structure modules from the ESMFold network⁴⁵ to a generative model for disordered protein structures in a Denoising diffusion probabilistic models (DDPM) framework, by supplying an additional noisy structure input as exemplified in RFdiffusion¹⁶ and AlphaFlow (ESMFlow)¹¹. Re-purposing established deep learning models has several benefits: it saves the considerable effort in developing a model from scratch that combines sequence and inter-residue feature extraction for polymer-like biomolecules that is already handled well by other methods. Instead we enable structure predictions of disordered regions in conjunction with folded domains with simple adjustments of diffusion steps at inference time. The resulting IDPForge (Intrinsically Disordered Protein, FOlded and disordered Region GEnerator) unites ensemble generation of IDPs and IDRs with folded domains within one model, demonstrating excellent performance in sampling conformational ensembles in agreement with solution experimental data. We also introduce a sampling procedure for guidance based on experimental data to further align ensemble generation to the experimental averages that are effective for distance restraints, without additional training cost. Finally, unlike almost all predictors that emphasize the minimization of $R_g$ error, a strength of IDPForge is an all-atom representation that allows for evaluation all local and global data types such as chemical shifts, J-couplings, NOEs/PREs, in the evaluation. With these diverse capabilities we expect that IDPForge can be applied across structural and integrative biology tasks for proteins that contain intrinsic disorder, and is available as an open source resource.

2. Results

The general framework of IDPForge is shown in Figure 1, and details of the architecture and training protocols can be found in Methods. We first demonstrate the ability of IDPForge to generate single chain IDPs on a collection of 32 test sequences^32,41,43 with NMR or SAXS data (Supplementary Table S1). We assess the generated ensembles in terms of their secondary structure propensities and agreements with experimental data including chemical shifts (CS), J-Couplings (JCs), NOEs and PREs, smFRET, and radius of gyration $\left(R_g\right)$. We also compare IDPForge performance with other published methods, including an all-atom force field a99SB-disp³¹ and the coarse-grained force field CALVADOS³² and ML models idpGAN⁴¹, STARLING⁴⁶ and IDPFold⁴³. In addition to the generation of single chain IDPs, we showcase two inference time generation examples with shifted distributions: 1) sampling with experimental data guidance and 2) sampling local IDRs within the context of folded domains. In the latter case, some proteins in the test set were decided upon based on availability of CS data in order to validate and compare methods. Basic structural validation for bond, angle and torsion statistics of IDPForge over sampled conformations is provided in Supplementary Fig. 1.

Figure 1: The schematic of IDPForge framework for IDP/IDR prediction.

(A) DDPM network that takes a protein sequence and diffused conformer coordinates $x_t$ that are featured into a sequence and a pair representation to predict protein structure $x_0$ by iterative denoising for time steps $t=T$. Secondary structure propensities are inferred from the protein sequence, and candidate secondary structure encodings are sampled based on these propensities using DSSP⁴⁷ for helices (H) and sheets (E) assignment and further classifies DSSP coils into poly-proline II (P) and other Ramachandran regions as defined in Supplementary Fig. S8. Details of network architecture and input features are discussed in Section 4.5 and Supplementary Section 1.1. Generated structures are relaxed using an empirical force field. Notation: $N$ residue number, $C_s$ sequence representation dimension and $C_z$ pair representation dimension. (B) Breakdown of a single denoising step $x_{t-1}\leftarrow x_t$ into a network forward pass ${\widehat{x}}_0\leftarrow ϵ\left(x_t\right)$ and a reverse step mapping $x_{t-1}\leftarrow \mathrm{Interp}\left(x_t,{\widehat{x}}_0\right)+\sigma$ based on Algorithm S2, with optional experimental data guidance via gradients formulated based on back-calculators $\nabla P\left({\widehat{x}}_0\right)$. During sampling, $x_T$ is initiated as random noise and the denoising step is repeated while decrementing the time step $t$. (C) Sampling of IDRs with folded domains via conditioning on a given folded template and partial denoising of the disordered sequence at inference time.

2.1. Generating single chain disordered ensembles

Table 1 reports the test set evaluation of the generated single chain IDP ensembles across the various ensemble generation methods. These evaluations include chemical shift (CS) ${\chi }^2$ values for 15 sequences with CS values taken from the BMRB⁴⁸, normalized radius of gyration error with respect to reported experiments for 30 IDPs, and a Bayesian score using X-EISD³⁸ (see Eqn. S11) for additional experimental data types for sequences reported in Supplementary Table S1. The X-EISD score is a Bayesian model that takes into account the various sources of experimental and back-calculation uncertainties when measuring the fit between the structural ensemble and the experimental observable^34,38; a higher X-EISD score corresponds to a higher likelihood that the structural ensemble is in agreement with the given data type or all data types together (Total).

Table 1: Experimental benchmark of the ensembles generated by IDPForge and other methods.

The experimental data for X-EISD calculations include J-Couplings (JC), nuclear overhauser effect (NOE), paramagnetic relaxation enhancement (PRE) and chemical shifts (CS). The X-EISD scores (Eq. S10) are normalized for each protein between 0–1 using Eqn. S11, and consider proteins with available data types reported in Table S1. X-EISD accounts for error and uncertainty for both experiments and back-calculations. The uncertainties in values are reported in the last significant figure. The radius of gyration error with respect to the experimental values $\text{Δ}{R}_g$ are calculated using Eqn. S5. The relative performance among the methods is broken down by test evaluation category in Supplementary Fig. S3.

	Normalized X-EISD score ↑				${\chi }_{CS}^2$ ↓	$\text{\%}\left|\text{Δ}{R}_g\right|/R_g^{exp}$ ↓
	Total	CS	JC	NOE/PRE
IDPForge	0.890	0.917	0.956	0.728	0.156	6.92
IDPConformerGenerator	0.576	0.471	0.829	0.477	0.272	7.43
IDPFold	0.582	0.627	0.393	0.456	0.271	11.75
idpGAN	0.209	0.167	0.347	0.727	0.367	10.32
STARLING46	0.395	0.475	0.380	0.489	0.207	4.74
CALVADOS	0.584	0.677	0.694	0.491	0.272	5.03

When averaged across a large variety of IDPs, IDPForge demonstrates excellent agreement with the solution experimental data, outperforming all other methods when accounting for chemical shifts, J-couplings, and NOEs/PREs as evaluated from the total X-EISD score (Table 1 and Supplementary Fig. S3(A)). We find that the generated ensembles by all methods yield ${\chi }^2$ values within the uncertainties of the CS back calculator (UCBShift^49,50) for all IDP proteins as seen in Table 1 and Supplementary Fig. S3(B), with the exception of PaaA2 and the unfolded state of drkN-SH3 which degrades the performance of methods such as idpGAN, IDPFold, and CALVADOS.

Supplementary Fig. S4 sheds light on the $C_{\alpha }$ CS inconsistencies of these methods from the perspective of secondary structure features derived from NMR versus their heavier weighting of Rg from SAXS data. As already reported for the conformational ensemble derived from SAXS and NMR chemical shift, PaaA2 exists in solution with two preformed helices, connected by a flexible linker, believed to be two types of molecular recognition elements for toxin inhibition.⁵¹ Sterckx and co-workers showed that a random coil pool containing no α-helices and a pool with α-helical populations are both in agreement with SAXS (i.e. Rg), but only ensembles selected from the pool containing the preformed α-helices can adequately reproduce the NMR data in addition. Hence the two coarse-grained methods CALVADOS and idpGAN report a higher ${\chi }^2$ due to their lack of secondary structure features. The observed connection between chemical shifts and secondary structure profile highlights that IDPs are not featureless coil-like structures, and thus may miss an important factor in biological relevance. The drkN-SH3 ensemble generated by IDPFold features a considerable β-sheet population between residue 35–50, likely reminiscent of its folded states rather than the unfolded states⁵², thus leading to its poor fit to the experimental chemical shifts compared to the other methods. IDPForge improves upon chemical shift predictions due to the presence of localized regions of transient helical/turn secondary structure across the sequences of the unfolded state of drkN-SH3.

In regards the $R_g$ error for the 30 test IDPs in Table 1 and Supplementary Fig. 3C, it is found that STARLING performs the best, likely benefiting from its distrogram prediction representation⁴⁶, followed by CALVADOS and IDPForge, while IDPFold and idpGAN perform the most poorly. Even so the best performing STARLING and CALVADOS methods for $R_g$ perform much worse than other methods on NOEs/PREs. To gain more insight into this imbalance across what should be related data types, we compare the subset of 9 disordered proteins for which long MD simulation trajectories using the a99SB-disp forcefield are available³¹ in Table 2. On this test set we see that the a99SB-disp ensembles have the smallest $R_g$ errors while also showing excellent agreement with NOEs/PREs, indicating that these data types are mutually reinforcing such that the underlying a99SB-disp ensembles are more self-consistent. Thus it appears that STARLING and CALVADOS are mostly designed to create ensembles that match experimental $R_g$’s, but are less consistent with the NOE/PRE restraints and other NMR data types as already seen for PaaA2. The strength of IDPForge is its overall agreement and uniformity in performance across local (CS and JC) and global (NOE/PRE and $R_g$) data types.

Table 2: Experimental benchmark of the ensembles generated for test proteins compared to molecular dynamics.

The subset of proteins for which MD using the a99SB-disp forcefield are available³¹: Aβ40, ACTR, Ash1, α-Syn, drkN-SH3, $N_{TAIL}$, $p{15}^{PAF}$, PaaA2 and Sic1. The experimental data for X-EISD calculations include J-Couplings (JC), nuclear overhauser effect (NOE), paramagnetic relaxation enhancement (PRE) and chemical shifts (CS). The X-EISD scores (Eq. S10) are normalized for each protein between 0–1 using Eqn. S11, and consider proteins with available data types reported in Table S1. X-EISD accounts for error and uncertainty for both experiments and back-calculations. The uncertainties in values are reported in the last significant figure. The $R_g$ error with respect to the experimental values $\text{Δ}{R}_g$ are calculated using Eqn. S5.

	Normalized X-EISD score ↑				${\chi }_{CS}^2$ ↓	$\text{\%}\left|\text{Δ}{R}_g\right|/R_g^{exp}$ ↓
	Total	CS	JC	NOE/PRE
IDPForge	0.782	0.892	0.904	0.699	0.135	5.97
IDPConformerGenerator	0.627	0.508	0.910	0.566	0.262	5.35
IDPFold	0.482	0.505	0.282	0.435	0.285	7.53
idpGAN	0.260	0.139	0.554	0.730	0.376	4.78
STARLING	0.210	0.288	0.199	0.381	0.229	4.77
CALVADOS	0.519	0.584	0.724	0.429	0.300	4.83
a99SB-disp	0.772	0.843	0.529	0.871	0.215	2.91

2.2. Generation of IDP/IDR ensembles guided by experimental data

For cases in which experimental information is available across data types, using it to bias ensemble generation has proved fruitful for IDP structural ensemble analysis.²⁸ For example, some of the IDP test proteins such as drkN-SH3 and RNaseA are folded proteins, but the experimental data that is available relates to the unfolded state. Experimental data also contains environmental signatures, for example that α-Syn is disordered in aqueous solution but is folded into helices when membrane-bound. We also showed in our DynamICE work that Hst5 structural ensembles depend on solvent conditions, which is also reflected in J-couplings.⁴⁴ So while IDPForge predicts disordered states first, it can be sculpted by experimental data that changes secondary structure (influenced by J-couplings), or long-range interactions (like NOEs/PREs) or collapsed and expanded states (such as Rg) dependent on environment or post-translational modifications (PTMs).

Hence we augment IDPForge with sampling biases to further align the generated ensembles towards better agreement with the experimental data as shown in Table 3. We highlight that this sampling guidance strategy requires no training but is done at the inference stage. When comparing the X-EISD score for each IDP (Supplementary Fig. S3(A)), IDPForge ranks at or near the top by total X-EISD score across all proteins with the exception of α-Syn, whose disordered state is a precursor to toxic fibers found in Parkinson’s disease^53,54, and the intrinsically disordered cyclin-dependent kinase (CDK) inhibitor Sic1⁵⁵. In both cases this arises due to a high PRE error, a problem shared by the other methods including for CALVADOS, STARLING, and a99SB-disp simulations. Table 3 shows that IDPForge when guided with long-range PRE experimental restraints (only using PREs for residue number differences greater than 10) significantly reduces the mean absolute error (MAE) error for this property for both Sic1 and α-Syn, while maintaining good agreement for other data types. This IDPForge capability allows ensemble generation that enforces distance-based requirements, which for Sic1 creates a heterogeneous compaction and expansion pattern as seen in Supplementary Fig. S5. This property of Sic1, such as overall compactness and large end-to-end distance fluctuations, are consistent with Sic1’s ultrasensitive binding to its partner Cdc4.²⁸

Table 3: Evaluation of $\alpha$−Syn and Sic1 ensembles from unbiased and experimentally biased IDPForge compared to other methods.

Mean absolute errors (MAEs) for J-Couplings (JCs), paramagnetic relaxation enhancement (PREs), chemical shifts $(\delta )$ and $R_g$ error $\left(\text{Δ}{R}_g\right)$ noting experimental uncertainties $\left({\sigma }_{\alpha Syn}=3.5,{\sigma }_{Sic1}=4\right)$. Standard deviation (in parenthesis) over 30 ensembles of 100 structures each.

	Experimental data type MAE ↓
	JC (Hz)	PRE (Å)	$\text{Δ}{R}_g$ (Å)	${\delta }_{C\alpha }$ (ppm)	${\delta }_{C\beta }$ (ppm)	${\delta }_H$ (ppm)
α-Syn ensembles
IDPForge Unbiased	0.343 (0.009)	5.554 (0.215)	−0.63 (0.76)	0.241 (0.012)	0.251 (0.007)	0.086 (0.002)
IDPForge PRE biased	0.414 (0.012)	2.953 (0.247)	−1.04 (0.67)	0.315 (0.012)	0.338 (0.006)	0.102 (0.002)
IDPConformerGenerator	0.224 (0.013)	4.028 (0.291)	0.085 (0.004)	0.511 (0.058)	0.293 (0.015)	0.088 (0.005)
IDPFold	0.459 (0.013)	4.218 (0.236)	−4.16 (0.46)	0.465 (0.011)	0.400 (0.008)	0.101 (0.002)
idpGAN	0.438 (0.007)	8.511 (0.388)	−9.62 (0.34)	0.326 (0.019)	0.471 (0.009)	0.091 (0.002)
STARLING	1.021 (0.016)	4.154 (0.272)	1.45 (0.70)	0.209 (0.004)	0.260 (0.004)	0.091 (0.002)
CALVADOS	0.310 (0.009)	3.689 (0.147)	1.97 (0.81)	0.200 (0.005)	0.283 (0.008)	0.092 (0.003)
a99SB-disp	0.413 (0.013)	6.043 (0.236)	0.09 (1.09)	0.220 (0.011)	0.287 (0.008)	0.083 (0.002)
Sic1 ensembles
IDPForge Unbiased		4.644 (1.091)	0.95 (0.45)	0.567 (0.017)	0.463 (0.015)	0.087 (0.002)
IDPForge PRE biased		2.047 (0.462)	0.81 (1.45)	0.552 (0.013)	0.438 (0.018)	0.089 (0.003)
IDPConformerGenerator		5.133 (0.615)		0.764 (0.017)	0.449 (0.012)	0.105 (0.003)
IDPFold		9.687 (1.359)	5.48 (0.67)	0.393 (0.007)	0.595 (0.008)	0.101 (0.002)
idpGAN		1.820 (0.230)	−3.68 (0.62)	0.346 (0.005)	0.739 (0.009)	0.116 (0.003)
STARLING		7.245 (0.923)	1.23 (0.66)	0.326 (0.004)	0.813 (0.006)	0.120 (0.002)
CALVADOS		5.816 (0.574)	−1.52 (0.38)	0.519 (0.012)	0.507 (0.008)	0.105 (0.002)
a99SB-disp		2.253 (0.239)	−6.14 (0.66)	0.631 (0.030)	0.501 (0.013)	0.109 (0.002)

2.3. Sampling of IDRs along with their folded domains

Although most disorder in the proteome takes the form of IDRs, at present few approaches are available for modeling IDRs within the context of folded domains at all-atom resolution. For IDPForge, we focus specifically on modeling IDRs by conditioning on available folded structures or structure predictions, rather than attempting to predict folded domains de novo. This decision is motivated by several factors: 1) Numerous successful structure prediction programs exist for folded proteins, and retraining a model to replicate these results incurs a high computational cost. 2) Using existing folded templates allows us to exploit the model architecture to predict disordered regions in a context-aware manner, without requiring additional training. 3) it aligns with the broader objective of unified sequence-based structure prediction for folded and disordered regions.

While IDPForge did not see training examples of IDRs within fixed folded domains, we show the model’s ability to be conditioned on folded template structures by substituting noisy conformations with folded features and applying minimal denoising in those folded domains. We demonstrate IDPForge’s zero-shot generation of terminal and linker IDRs given the structures of folded domains for two cases: modeling the missing electron density from experimentally determined IDR structures (Figure 2), and replacing the low confidence regions longer than 15 amino acids for IDR proteins predicted with AF2 (Figure 3).^1,56

Figure 2: All-atom IDPForge ensembles generated for IDRs with folded domains from experimental structures.

Ai) NMR ensemble from PDB ID 2KK1⁵⁷. Bi) Intervening 85 residues of the IDR region within the sulfate transporter anti-sigma domain of solute carrier family 26 member 9 (SLC26A9 STAS domain); PDB ID 7CH1⁵⁸. (ii) The generated ensembles of ABL2 and SLC26A9 using IDPForge with folded domains shown in grey and IDRs in orange. (Aiii) C_α chemical shifts for IDPForge ABL2 ensemble and solution NMR⁵⁷. (Biii) Secondary structure propensities for SLC26A9 of the generated ensembles are shown as dark dashed lines and folded templates in light solid colors. Error bars estimate the mean and standard deviation of 100 conformer ensembles from 10 trials. Minor β-sheets variations of the modeled folded domains of SLC26A9 STAS domain.

Figure 3: All-atom IDPForge ensembles compared with AlphaFold2 predictions with IDRs representing low confidence regions.

A) PRAME family member 20 entry Q5VT98 and B) Cullin-1 entry Q13616 with AF2 predictions (i) colored by pLDDT. Yellow and orange indicate low confidence regions (pLDDT < 70). The generated ensembles of (Aii) PRAME20 and (Bii) Cullin-1 using IDPForge with folded domains shown in grey and IDRs in orange. Secondary structure propensities of (Aiii) PRAME20 and (Biii) Cullin-1 of the generated ensembles (dashed) and the folded templates (solid and in lighter colors).

In the first category, Figure 2Ai considers residues 1–36 in the C-terminal Domain of Tyrosine-protein kinase ABL2 from Homo⁵⁷ solved with solution NMR, as well as the more difficult case involving 86 unresolved residues in the center of the sulfate transporter anti-sigma (STAS) domain of solute carrier family 26 member 9 (SLC26A9) solved by X-ray crystallography⁵⁸ shown in Figure 2Bi. We also examine residues 90–123 from human prion protein (HuPrp) with E219K⁵⁹ and Interleukin-6 (IL-6) with 41 missing residues at the N-terminus⁶⁰ in Supplementary Fig. S6.

Figures 2Aii and 2Bii and corresponding Supplementary Figs. S5ii show the generated ensembles using IDPForge for all four IDR systems, and are seen to be well aligned to the backbone torsion angles from the folded templates while sampling evenly across the alpha, beta and poly-proline II in the disordered regions (shown for IL-6 and STAS in Supplementary Fig. S7). We see that ensemble averages over the IDPForge conformers create good agreement in chemical shifts for the disordered regions while maintaining agreement with experimental CS values in the folded domains (Figures 2Aiii and Supplementary Figs. S5Aiii). The structural metrics show that, like CALVADOS, IDPForge preserves the secondary and tertiary structure in the folded domains, whereas in the disordered regions it is seen that IDPForge predicts more transient secondary structure, unlike CALVADOS which is largely featureless throughout the disordered regions (Figures 2Biii and Supplementary Figs. S5Biii).

In Figure 3Ai we consider the low confidence regions of AF2 structures of the preferentially expressed antigen in melanoma (PRAME) family member 20, replacing the low confidence residues 116–145. We also consider Cullin-1 with two IDR regions: an N-terminal IDR 1–15 and a linker IDR between 56–82⁶¹ in Figure 3Bi. In the latter case we generated the 1–196 residue region for Cullin-1 by first predicting the linker IDR and holding that prediction fixed to then complete the N-terminal IDR generation. Because Cullin-1 beyond residue 196 is distant enough from the IDRs to ignore their influence on sampling, we simply aligned the generated 1–196 regions to the rest of the protein. As expected, IDPForge conformers sample a much greater range of conformations and secondary structure types compared to the primarily featureless coil structures from AF2 for both IDRs (Figures 3ii and 3iii).

We also compare the IDPForge generated ensembles with the multi-domain ensembles modeled by CALVADOS and AFflecto⁶² across secondary structure categories in Supplementary Fig. S8. AFflecto is a web server that also identifies IDRs from AF predictions and generates ensembles with stochastic sampling and robotic-inspired loop closure. As already seen for the ABL2, HuPrP, IL-6 and STAS IDRs, IDPForge creates more diverse secondary structure types for the disordered regions than those from CALVADOS, while AFflecto exhibits no secondary structure at all.

In and around the sequence junction between the ordered and disordered regions, both IDPForge and CALVADOS exhibit more conformational variations compared to AFflecto as seen in Figure 4. When the junction is helical IDPForge shows more variations across 3₁₀helix, ${\alpha }_R$-helix, and $\pi$-helix states, with some transient helical structure that continues into the beginning of the disordered domain. If the junction involves a folded domain “loop”, as near residue 655 in STAS, more variation is sampled. Evidence of increased conformational sampling near the sequence junctions between folded and disorder regions have been observed previously⁶³, and IDPForge and CALVADOS preserves this feature unlike AFflecto (Figure 4). Both IDPForge and CALVADOS sample the folded domains with average RMSDs to template structures of ≤ 2 Å across all IDRs, whereas AFflecto is seen to introduce very little structural variations in the folded domains with respect to the AF2 predictions (Figure 4). Given the known limitations of the lack of dynamics for AF2 folded structure predictions, we believe that IDPForge and CALVADOS better reflect the true fluctuating ensembles of the folded domains and is a desirable feature of these methods.

Figure 4: Per residue RMSDs of folded domains for ABL2, HuPrp, IL-6, SLC26A9 STAS, PRAME20 and Cullin-1 ensembles generated by IDPForge, CALVADOS and AFflecto.

RMSDs are calculated across ensembles of 100 conformers, based on the folded domain heavy atoms of reference structures from PDB ID 2KK1, 2LFT, 4CNI and 7CH1, and AF2 entry Q5VT98 and Q13616 respectively.

3. Discussion and Conclusion

IDPForge is a transformer-based diffusion model that generates both all-atom IDPs and IDRs with folded domains. IDPForge uses attention modules for structure denoising, enabling communication between sequence and pairwise residue information while also providing a suitable channel for conditioning on folded templates. On a benchmark of 32 test IDPs/IDRs, IDPForge achieves comparable or superior agreement with NMR and SAXS experimental data compared to existing methods. It captures both local structural features as quantified by J-Couplings and global shape characteristics as measured by NOEs, PREs, and the radius of gyration, and benefits from all-atom data training to conform to chemical shift data.

Additionally, its diffusion model framework offers a natural mechanism for incorporating experimental data guidance at inference time. We demonstrate this capability for α-Syn and Sic1 in which sampling is guided with PRE data that shifts the conformational distribution towards enhanced agreement for multiple data types with the underlying structural pool better delineating long-ranged residue interactions. We attribute the improvement to the use of the residue frame representation, which enables the model to more effectively integrate distance- and contact-biased experimental restraints—a limitation observed with the torsional representation in DynamICE⁴⁴. We again highlight that this sampling guidance strategy requires no training but is done at inference, and is generalizable to any sequence with experimental data that can provide direct gradient information.

IDPForge also models regions of disorder within the context of folded domains at inference with a targeted diffusion step strategy by initiating from a structured template instead of a fully noisy state, even though IDPForge is not explicitly trained to perform a partial denoising task. Alternative to stitching based methods that screen a pre-generated pool of single chain conformations⁶⁴, the attention modules in IDPForge extract and exchange information from both the folded and noisy residue frames through pair and sequence representation for each denoising step. Hence IDPForge more intuitively enforces the spatial relation between IDRs and folded domains as well as conditioning on a folded template during generation, an advantage for interpreting linker IDRs. When conditioned on a folded template, IDPForge minimizes perturbations to the structured regions but still introduces local changes in the folded domains and at their junction with IDRs which better reflect an extension of dynamic movements that are present in solution.

There are many exciting directions to use and expand IDPForge over a range of IDP/IDR applications. First we believe that we have already demonstrated that, because IDPForge does not train a new model for each IDP/IDR sequence but instead is trained on large variations in sequence data, we anticipate that sequence mutations can be modeled in the disordered domain, just as AF2 is capable of modeling mutations in the folded regions of IDRs. Furthermore, the introduction of PTMs appears to be a general mechanism for regulation of IDPs/IDRs⁶⁵. We can train IDPForge quite easily through data generated not only on canonical amino acids, but phosphorylated sequences given our recent creation of a rotamer library for PTMs⁶⁶. Finally, given that a significant number of proteins contain IDRs that engage in dynamic interactions with other domains in either discrete dynamic complexes or within condensed states^26,67,68, future work will pursue simultaneous modeling of multiple IDR domains and IDP/IDR complexes to further enhance its ability to capture the complex interplay between domains for disordered proteins such as observed for hnRNPA1⁶⁹.

4. Methods

4.1. Denoising diffusion probabilistic models.

Denoising diffusion probabilistic models (DDPMs)⁷⁰ approximate a distribution by parameterizing the reversal of a discrete-time diffusion process. The forward diffusion process increasingly corrupts a sample $x_0$ from an unknown data distribution $q\left(x_0\right)$ for a sequence of $T$ steps, such that the final step $x_T$ is indistinguishable from a reference distribution that has no dependence on the data. DDPMs approximate the data distribution with a second distribution $p\left(x_0\right)$ through learning a backward transition kernel $p\left(x_{t-1}\mid x_t\right)$ at each $t$. In the reverse process, one first samples from the reference distribution $x_T\sim p\left(x_T\right)\approx q\left(x_T\right)$, and then repeatedly denoises by sampling $x_{t-1}\sim p\left(x_{t-1}\mid x_t\right)$ until $x_0\sim p\left(x_0\right)$ is obtained.

Under Gaussian noise $ϵ$ and $\alpha$ that parametrizes a noise schedule (with ${\overline{\alpha }}_t={\prod }_{s=1}^t{\alpha }_t$), a forward step is defined as,

$$x_t=\sqrt{{\overline{\alpha }}_t}{x}_0+\sqrt{1-{\overline{\alpha }}_t}ϵ,ϵ~𝒩(0,I)$$ (1)

and the conditioned probability in the reverse process as,

$$p_{\theta }\left(x_{t-1}\mid x_t\right)=𝒩\left(x_{t-1};{\mu }_{\theta }\left(x_t,t\right),{\text{Σ}}_{\theta }\left(x_t,t\right)\right)$$ (2)

Thus the network can learn the mean of the Gaussian posterior distribution ${\mu }_{\theta }$ from predicting the noise ${ϵ}_{\theta }$ for the reverse process by

$${\mu }_{\theta }\left(x_t,t\right)=\frac{1}{\sqrt{{\alpha }_t}}(x_t-\frac{1-{\alpha }_t}{\sqrt{1-{\overline{\alpha }}_t}}{ϵ}_{\theta }\left(x_t,t\right))$$ (3)

which then allows us to derive a time series of $x$ by

$$x_{t-1}=\frac{1}{\sqrt{{\alpha }_t}}(x_t-\frac{1-{\alpha }_t}{\sqrt{1-{\overline{\alpha }}_t}}{ϵ}_{\theta }\left(x_t,t\right))+\sqrt{1-{\alpha }_t}z,z~𝒩(0,I)\text{for}t>1$$ (4)

4.2. Diffusion over residue rigid frames and torsions

We use DDPMs in the rigid-frame representation of residues, where the forward noising process is defined independently over the rotational and translational components of the backbone rigid bodies, as well as the torsional components of the sidechains. This treatment has been a popular choice for diffusing protein backbones in previous studies^17,71,72. The diffusion over residue rigid frames is modeled as a discretization of a continuous-time diffusion process with Brownian motion defined on a product manifold $\mathbb{T}(3)\times SO(3)$ that separates the rotations and translations for each residue^16,21,43. The translational case is simply defined by a standard Gaussian, whereas the 3D rigid body rotation $SO(3)$ can be represented by the isotropic Gaussian distribution $~ℐ{𝒢}_{SO(3)}$^73,74, which is sampled in the axis-angle parametrization with an angle of rotation $\omega$ according to

$$p(\omega )=\frac{1-\cos \omega }{\pi }\sum _{l=0}^{\infty }(2l+1)\exp \left(-l(l+1){\sigma }^2/2\right)\frac{\sin ((l+1/2)\omega )}{\sin \omega /2}$$ (5)

When forward noising the sidechain torsion angles with $SO(2)$, we use a wrapped normal distribution where $wrap(\chi )=((\chi +\pi )mod2\pi )-\pi$, to handle the periodic nature of torsion angles, a strategy used in FoldingDiff¹⁹ and DiffDock⁷¹.

While DDPM commonly models the added noise, here we trained the folding block and structure module to predict the final denoised protein structure ${\widehat{x}}_0$. We provide the pseudocode for a training step in Algorithm S1. In this case, a typical denoising step in Eqn. 4 would include a forward network pass and a reverse step that maps the prediction ${\widehat{x}}_0$ to $x_{t-1}$ given the noised input $x_t$ with independent backward transition kernels on the $\mathbb{T}(3)\times SO(3)\times SO(2)$ space. The translations $T_t$ and torsions ${\chi }_t$ are updated by interpolating the Gaussian distribution parametrized by the noise schedule ${\beta }_t=1-{\alpha }_t$; this reverse distribution for torsion angles also needs to observe the periodic boundary. On the $SO(3)$ space, the backward kernel is approximated with the score matching identity on the corresponding rotations $R_t$ and the predicted noiseless ${\widehat{R}}_t$ such that ${\nabla }_{R_t}\log q_t\approx {\nabla }_{R_t}\log ℐ{𝒢}_{SO3}(R_t;{\widehat{R}}_0,{\sigma }_t^2)$. Thus to start a conformer generation, we first randomly sample $x=\left(T,R,{\chi }^{1,\dots ,4}\right)$, $T~𝒩\left(0,I_3\right)$, $R~SO(3)$,${\chi }^{n=4}~wrap(𝒩(0,I))$ for all disordered residues, and repeatedly apply the denoising step until $T=0$ (see Algorithm S2). More details of the training scheme and sampling procedures are provided in the Supplementary Text Section 1.1.

4.3. Conditioned generation towards experimental data

Conditioned generative models extend the capabilities of generative algorithms by incorporating auxiliary information to meet specific data generation criteria. DDPMs provide a convenient framework for conditioned generation, as the iterative nature of the reverse process offers natural integration points to influence the trajectory of sample synthesis with auxiliary information at each step. In classifier-guided diffusion models, this guidance is achieved by directly incorporating the gradients of a classifier trained to predict the conditional label of interest $\nabla \log f_{\varphi }\left(y\mid x_t\right)$, to steer the reverse diffusion process toward generating samples that align with condition $y$⁷⁵. With Bayes’ rule, we can write the log density of the joint distribution $q\left(x_t,y\right)$ as

$${\nabla }_{x_t}\log q\left(x_t,y\right)={\nabla }_{x_t}\log q\left(x_t\right)+{\nabla }_{x_t}\log q\left(y\mid x_t\right)$$ (6)

$$\approx -\frac{1}{\sqrt{1-{\overline{\alpha }}_t}}{ϵ}_{\theta }\left(x_t,t\right)+{\nabla }_{x_t}\log f_{\varphi }\left(y\mid x_t\right)$$ (7)

Introducing a guidance scale $\omega$, the classifier-guided noise prediction takes the form as,

$${\overline{ϵ}}_{\theta }\left(x_t,t\right)={ϵ}_{\theta }\left(x_t,t\right)-{\omega }_t\sqrt{1-{\overline{\alpha }}_t}{\nabla }_{x_t}\log f_{\varphi }\left(y\mid x_t\right)$$ (8)

Within the context of generating protein conformers towards desired properties, this is analogous to defining a biasing potential as a function of the structural coordinates $x_{t-1}\leftarrow x_{t-1}+{\overline{\omega }}_t{\nabla }_{x_t}P\left(x_t\right)$. Similar strategies in protein design guided by pseudo-potentials have been adopted by RFdiffusion¹⁶ and Chroma¹⁷ for tasks such as functional-scaffolding, generating symmetry, or shape constrained protein monomers and oligomers. Here, to bias the generation towards experimental data, we take advantage of the potentials readily defined by the back-calculators.³⁸ Experimental observables such as J-Couplings, NOEs, PREs and $R_g$ (often estimated from a SAXS or smFRET measurement) have direct analytical back-calculations and we limit our studies with experimental guided generation for these data types as no training of an additional classifier is needed. We discuss details of the back-calculators in the Supplementary Text Section 1.1 and 1.3.

4.4. Generation of IDRs within the context of folded domains.

While IDPForge was originally trained for denoising on completely disordered single chains, we extend IDPForge to infer ensembles of local IDRs with folded domains. This involved conditioning the generation with structural featurizations of the folded domains and their secondary structure assignment in which the model effectively denoises the IDR sequences only by zero-ing the diffusion time steps of the folded domains. This partial denoising is controlled by a user defined denoising mask. Inference time modeling of IDRs has the most transferability for generating proteins with a single IDR. Thus, for the case of generating multiple IDRs separated by folded domains, we can model segments of protein containing one IDR at a time and use the denoising mask to treat previous predictions as fixed templates. Sampling of IDRs conditioned on folded domains also requires an estimation of the orientation of the folded domains with respect to the noise initiation. We prepare the folded template input by extracting its coordinates from a reference structure centered by its IDR. We randomly rotate the folded template coordinates and add a noise that scales with the residue number of the IDR to its center of mass. The detailed algorithm is provided in Algorithm S3.

4.5. Network architecture

The network architecture illustrated in Figure 1 takes two main module components, the “folding block” and the structure module from ESMFold⁴⁵. The structure module consists of Invariant Point Attention (IPA) followed by backbone update network as introduced in AF2¹. The structure module is implicitly $SE(3)$-equivariant by frame-localized geometric reasoning. The folding block contains stacks of triangular attention and communications between the sequence and pair representations that closely mimic the Evoformer described in AF2, but with the exception that it does not have a multiple sequence alignment (MSA) processing transformer module. We use two multilayer perceptrons (MLPs): one for the sequence representation to concatenate the diffused sidechain torsion angles and time step encoding, and the other for the pair representation that features a 2D transformation of the diffused coordinates before adding a pairwise relative positional encoding for the residue indices. We then define two learnable embeddings for the protein sequence and for secondary structure encoding (Supplementary Fig. S8), which are added to the sequence MLP outputs before being fed into the attention block. In an ablation study, we also investigated the effect of adding a learnable weighted sum of the ESM2 embedding to the sequence representation or a reshaped ESM2 attention map to the pair representation; we didn’t find significant improvement to IDP ensemble generation performances. Details of these ESM2 appending networks are discussed in the Supplementary Text Section 1.2 and their configurations are listed in the Supplementary Table S2.

4.6. Data preparation

We collected the non-overlapping disordered sequences (30–200 residues in length) from the idpGAN training set⁴¹ consisting of sequences from DisProt(version 2021–06)⁷⁶ and the human intrinsically disorder proteome (IDRome) database³³. We excluded the native folded proteins characterized in the presence of denaturant in the original idpGAN train set that are also in the CASP12 dataset. 32 disordered sequences with experimental data (in Supplementary Table S1) and 6 sequences for systems of folded domains and IDRs (human prion protein (E219K), C-terminal domain of kinase ABL2, interleukin-6, SLC26A9 STAS domain, cullin-1 and PRAME20), were set aside as our test set. In addition to the disordered states, we expanded our data with CASP12 folded states from Sidechainnet⁷⁷ with 30% thinning, to enrich the training data with secondary structural features. Only CASP12 sequences or segments with completely defined coordinate information and secondary structure annotations were included. We performed sequence similarity clustering with MMseqs2⁷⁸ and removed any sequence with ≥ 50% identity to the test sequences from our train-val set, which in total generated 1,892 disordered sequences and 10110 CASP12 sequences. We used IDPConformerGenerator³⁰ with sequence-based sampling to generate 100 conformers for each included idpGAN sequences with FASPR⁷⁹ sidechain building. For the IDRome sequences simulated with CALVADOS, we randomly extracted 100 conformers from the coarse-grained trajectories for each sequence, converted to all-atom conformers using cg2all⁸⁰. All conformers were relaxed for a maximum of 5000 steps with the Amberff14SB forcefield using OpenMM⁸¹. We used MDTraj⁸² to assign the secondary structure for each of the generated conformers. While this yielded an extensive structural dataset of 199,440 conformers, for training efficiency a 15–3% split was used for training-validation.

4.7. Timing and memory usage

IDPForge network inference using basic attention calculations on a 132 residue sequence for 40 time steps takes 11.9 seconds per conformer on average on a A40 GPU, in which relaxation takes 8.1 seconds per conformer. Detailed runtime scaling with protein length on a A40 GPU and 32 core CPUs is shown in Figure S9. There can be room for computational efficiency improvements with attention optimization, such as flash attention⁸³ and axial attention chunking, which we did not pursue here.

6. Data Availability

A list of training sequences, conformer data and training weights can be found in a Figshare repository https://doi.org/10.6084/m9.figshare.28414937.