Rapid prediction and analysis of protein intrinsic disorder

Abstract

Protein intrinsic disorder is found in all kingdoms of life and is known to underpin numerous physiological and pathological processes. Computational methods play an important role in characterizing and identifying intrinsically disordered proteins and protein regions. Herein, we present a new high‐efficiency web‐based disorder predictor named Rapid Intrinsic Disorder Analysis Online (RIDAO) that is designed to facilitate the application of protein intrinsic disorder analysis in genome‐scale structural bioinformatics and comparative genomics/proteomics. RIDAO integrates six established disorder predictors into a single, unified platform that reproduces the results of individual predictors with near‐perfect fidelity. To demonstrate the potential applications, we construct a test set containing more than one million sequences from one hundred organisms comprising over 420 million residues. Using this test set, we compare the efficiency and accessibility (i.e., ease of use) of RIDAO to five well‐known and popular disorder predictors, namely: AUCpreD, IUPred3, metapredict V2, flDPnn, and SPOT‐Disorder2. We show that RIDAO yields per‐residue predictions at a rate two to six orders of magnitude greater than the other predictors and completely processes the test set in under an hour. RIDAO can be accessed free of charge at https://ridao.app.

1. INTRODUCTION

Anfinsen's classic work on bovine pancreatic ribonuclease ¹ helped establish the protein folding problem. ² Since then, extensive work has been done to decipher the protein folding code and build computer models capable of predicting a protein's native structure based solely on its amino acid sequence. While some proteins require a unique and nearly time‐independent structure before becoming functional, most proteins exhibit some degree of time‐dependent structural heterogeneity in their native, functional state due to intrinsic disorder. ³ Protein intrinsic disorder can manifest globally or locally giving rise to intrinsically disordered proteins (IDPs) and intrinsically disordered protein regions (IDPRs), respectively. ⁴ These proteins and protein regions have been found in all kingdoms of life, ⁵ , ⁶ , ⁷ , ⁸ , ⁹ , ¹⁰ , ¹¹ often serve as actors in important physiological processes, ³ , ¹² , ¹³ , ¹⁴ and are implicated in numerous human diseases. ¹⁵ , ¹⁶ , ¹⁷ , ¹⁸ , ¹⁹ , ²⁰ IDPs and IDPRs have challenged the structure–function paradigm, ²¹ , ²² , ²³ complicated the protein folding problem, ²⁴ , ²⁵ , ²⁶ and hindered structural proteomics studies. ²⁷

There are significant differences between the amino acid sequences of proteins with stable, well‐defined structures and IDPs. ²⁸ These differences, which include attributes, such as hydrophobicity, content of charged residues, ²⁹ and sequence complexity, ³⁰ serve as the basis for predicting if a protein is expected to be wholly or partially disordered under physiological conditions. To date, more than a hundred disorder predictors have been developed. ³¹ Some predictors, for example, the Predictors of Natural Disordered Regions (PONDRs), ³⁰ produce disorder profiles by providing continuous per‐residue disorder propensity predictions from N‐termini to C‐termini (Figure 1). Per‐residue predictors, like the PONDRs, are useful for identifying IDPRs. Other predictors, including charge‐hydropathy (CH) plots, ²⁹ and those based on cumulative distribution functions (CDFs), ¹¹ yield binary predictions that indicate whether a given sequence is expected to be wholly disordered or ordered under physiological conditions. These predictors, and binary disorder predictors in general, are used to identify IDPs.

FIGURE 1.

Human p53 Rapid Intrinsic Disorder Analysis Online (RIDAO) disorder profile. Plotting the residue number against disorder score, also known as disorder propensity, for all residues in a sequence yields a disorder profile. Residues with a disorder score exceeding 0.5 are predicted to be disordered. Residues with a disorder score below the 0.5 are predicted to be ordered. The 0.5 threshold is represented with a dashed line. Here we show a complete RIDAO disorder profile, including the mean disorder profile (MDP) error, PONDRs VLXT, VL3, and VSL2, as well as IUPred‐Long, IUPred‐Short, and PONDR‐FIT

Disorder predictors have a history of being implemented into web‐servers and made available to the public free of charge. However, with a few exceptions, ³² , ³³ these servers are not designed with genome‐scale structural bioinformatics or comparative genomics/proteomics in mind. We previously developed a web‐crawler style disorder predictor for internal usage that was implemented into a tool we refer to as Disorder Spider (DiSpi) in our previous studies, see for example, refs. ³⁴ , ³⁵ , ³⁶ , ³⁷ , ³⁸ , ³⁹ DiSpi works by first aggregating disorder profiles from six well‐known disorder predictors: PONDR VLXT, ⁴⁰ PONDR VL3, ⁴¹ PONDR VLS2B, ⁴² PONDR‐FIT, ⁴³ IUPred‐Short, and IUPred‐Long. ⁴⁴ Then, a mean disorder profile (MDP) is computed along with the standard error. The final output consists of all seven sets of predictions along with the error computed from the MDP as well as a disorder profile plot (Figure 1). Additionally, DiSpi performs CH‐CDF analysis ⁴⁵ , ⁴⁶ , ⁴⁷ and yields CH‐CDF plots (Figure 2), which enables rapid discrimination between flavors of disorder. ⁴⁸ Despite the fact that DiSpi was designed to facilitate the application of disorder prediction in comparative genomics, it suffers from the same computational bottlenecks as the web‐servers it crawls.

FIGURE 2.

Charge‐hydropathy (CH)‐CDF plots. Plots of CH versus cumulative distribution function (CDF) predictions yield multinomial classification of protein disorder. Classification is based on which quadrant a protein is placed in. When both methods predict disorder, the protein is found in quadrant 4 and is predicted to be disordered. Alternatively, when both methods predict a protein to be well‐structured it is found in quadrant 2 and is predicted to be structured. If the two methods disagree, then the protein is predicted to be “mixed” or “rare” and is found in quadrant 3 or quadrant 1, respectively. Here, we present 100 mapped proteins pulled from the human proteome (left) and slime mold proteome (right)

In this work, we construct a high‐efficiency unified disorder predictor based on DiSpi that does not rely on web‐crawling. This command‐line predictor, henceforth referred to as Rida, subsequently serves as the basis for the development of a second, web‐based, disorder predictor. Apart from rounding errors, the disorder scores produced by these two unified predictors exactly match those produced by each of their original, stand‐alone counterparts—PONDR VLXT, PONDR VSL2B, PONDR VL3, PONDR‐FIT, IUPred‐Short, and IUPred‐Long. Herein, we present RIDAO, an easy to use, publicly accessible, high‐efficiency, web‐based disorder predictor. We compare the accessibility and computational efficiency of RIDAO with five powerful and popular disorder predictors ⁴⁹ —AUCpreD, ⁵⁰ flDPnn, ⁵¹ IUPred3, ⁵² metapredict V2, ⁵³ and SPOT‐Disorder2. ⁵⁴ Using a test set containing more than one million sequences comprising over 420 million residues, we find that RIDAO is the most efficient predictor, analyzing the complete dataset in under an hour with a per‐residue prediction rate (Equation (1)) at least two orders of magnitude greater than the other predictors.

2. RESULTS AND DISCUSSION

2.1. Implementation

Rida is a single‐threaded C program that is designed to run on 1 CPU core. RIDAO, on the other hand, is a Ruby on Rails web application with a C backend that implements the logic of Rida. Unlike Rida, the RIDAO backend takes advantage of multi‐threading; as a result, RIDAO can run on as many CPU cores as are available to it. This in turn allows RIDAO to treat multiple sequences simultaneously without a loss in efficiency. In principle, this design paradigm could be applied to any predictor to yield improved efficiency.

Unlike existing web‐servers, RIDAO requires users to login using an email address and password. This design choice may be viewed unfavorably due to concerns regarding privacy and FAIR data principles ⁵⁵ —findability, accessibility, interoperability, and reusability. However, in our view, the benefits of this choice outweigh these drawbacks. One benefit of user logins is the ability of RIDAO to create dedicated user workspaces, wherein results can be aggregated, stored, and shared. Within the user workspace, three types of job submissions can be made: (a) users can enter sequences manually, (b) users can upload multi‐FASTA files, and (c) users can upload a compressed file (.zip) containing any number of multi‐FASTA files. In all cases, RIDAO requires FASTA formatted sequences and allows, but does not require, FASTA headers. Once submitted, a job is added to the user's workspace and placed into the run queue to wait for system resources to become available. The run queue is implemented in a first in, first out fashion and is intended to prevent over‐subscription of system resources. Once the necessary system resources are available, the next scheduled job in the queue moves into an active state and processing begins. After processing, the results are compressed (.zip) before the job moves into the final, completed state. Once a job has completed its run, users can view RIDAO generated figures by accessing the job within their workspace. Users can also download or share their results, including raw data, generated figures, and the scripts used to generate said figures. Job sharing among RIDAO users is a straight forward, in‐app process, designed to facilitate collaboration. Results can also be shared outside of RIDAO via a downloadable link that does not require credentials to access.

2.2. Efficiency

Each predictor's efficiency was evaluated on the test set over a 24 hr period (Figure 3). We included Rida in this evaluation for two reasons. First, unlike the other predictors, RIDAO is web‐based and can only be accessed via a web browser. Second, RIDAO leverages parallel processing to analyze multiple sequences simultaneously while the other predictors do not. Rida, on the other hand, is restricted to serial processing and is therefore more directly comparable to the others. Rida and RIDAO completed processing the entire dataset within the allotted 24 hr window. Specifically, RIDAO completed the job in ≈42 min and Rida finished in ≈613 min. The other predictors were unable to process the entire dataset during the 24 hr period. On average, we found that metapredict V2 processed 64.0% of the dataset (644,823 sequences); IUPred3 processed 25.0% of the dataset (252,267 sequences); AUCpreD processed 23.8% of the dataset (240,016 sequences); flDPnn processed 0.7% of the dataset (6,557 sequences); and SPOT‐Disorder2 processed less than 0.1% of the dataset (34 sequences).

FIGURE 3.

Efficiency Evaluation. Each predictor was evaluated on the same hardware over a 24 hr period using a set of 1,007,540 sequences. Rapid Intrinsic Disorder Analysis Online (RIDAO) and Rida completed the entire set within the allotted time; metapredict V2 completed 644,823; IUPred3 completed 252,267; AUCpreD completed 240,016; flDPnn completed 6,557; and SPOT‐Disorder2 completed 34. (Top left) test set performance over 24 hr, (top right) test set performance over 24 hr in log ₁₀ scale, (bottom left) per‐residue performance (Equation (1)). RIDAO outperforms other predictors by two orders of magnitude. (Bottom right) per‐residue prediction rate (Equation (1)) in log ₁₀ scale

Based on the 24 hr performance, we evaluate the per‐residue prediction rate, R _p , of each predictor, p, as,

$$R_p=t^{-1}\sum _{s=0}^S{l}_s,$$ (1)

where t is the observed runtime in seconds, S is the number of processed sequences, s is the sequence index, and l _s is the length of sequence s. Using Equation (1), we find that RIDAO performed predictions at a rate of 170,153.28 residues per second (RPS) and Rida performed at a rate of 11,658.13 RPS. For metapredict V2, IUPred3, AUCpreD, flDPnn, and SPOT‐Disorder2 the observed rates were: 3,155.12 RPS; 1,378.18 RPS; 1,265.44 RPS; 21.62 RPS; and 0.13 RPS, respectively.

Considering these rates and the total number of residues in the test set (428,786,255), the estimated total runtime, $T_{\text{total}}^{*}$, can be computed as,

$$T_{\text{total}}^{*}=\frac{N_r}{R_p},$$ (2)

where N _r is the total number of residues in the test set and R _p is the per‐residue prediction rate defined in Equation (1). For metapredict V2, IUPred3, AUCpreD, flDPnn, and SPOT‐Disorder2, the $T_{\text{total}}^{*}$ is computed to be 37.8 hr; 86.4 hr; 94.1 hr; 5,509.1 hr (≈229 days); and 916,209.9 hr (≈104 years), respectively (Figure 4).

FIGURE 4.

Runtime evaluation. Actual and estimated runtimes reported in log scale. Rapid Intrinsic Disorder Analysis Online (RIDAO) required 42 min of runtime while Rida required 613 min. Runtimes for the other predictors, which did not completely process the test set during the 24 hr test period, are estimated (Equation (2)) based on observed per‐residue prediction rates shown in Figure 3. Then, 38 hr are estimated for metapredict V2, 86 hr for IUPred3, and 94 hr for AUCpreD; flDPnn is estimated to require 229 days and SPOT‐Disorder2 is estimated to require 104 years of runtime

We show in Figure 3 that RIDAO yields disorder predictions at a per‐residue rate that is two to six orders of magnitude greater than metapredict V2, IUPred3, AUCpreD, flDPnn, and SPOT‐Disorder2. Rida is also shown to out‐perform each predictor by at least an order of magnitude.

The difference in efficiency between these predictors may be explained in terms of design principles, prediction methods, and implementation choices. AUCpreD, metapredict V2, Rida, and RIDAO have all been developed with computational efficiency in mind and employ some form of machine‐learning to make predictions. While neural networks serve as the underpinnings of flDPnn and SPOT‐Disorder2, these predictors were not designed for high‐throughput applications. Consequently, both flDPnn and SPOT‐Disorder2 rely on inefficient system calls to various supplemental tools prior to performing disorder predictions. These system calls bog down the prediction process dramatically. This is especially the case for SPOT‐Disorder2, which must invoke HHBlits, PSI‐BLAST, CCMpred, SPIDER3, SPOT‐1D, SPOT‐Contact, and DCA prior to making its disorder prediction. The inclusion of PSI‐BLAST in particular exacts a heavy toll, despite running on 32 CPU cores.

Each predictor we examined produces per‐residue disorder propensities that can be viewed as a disorder profile or converted to binary predictions. Rida, RIDAO, flDPnn, and SPOT‐Disorder2 yield additional outputs as well. For Rida and RIDAO, these outputs include disorder propensities from seven different predictors. Additionally, they compute mean disorder propensities and their standard errors. Furthermore, RIDAO tabulates results for whole sequences based on each of its constituent predictors. These tabulated results come with an increase in computational cost and include the rank—or average disorder score over the entire sequence, the percentage of disordered residues based on a disorder propensity threshold of 0.5, the average distance from the CDF boundary line (dCDF) with respect to PONDR VLXT, and the distance from the boundary line in CH phase space (dCH). When taken together the dCDF and dCH allow for the construction of CH‐CDF plots shown in Figure 3. The production of these data points is a unique feature of RIDAO. For flDPnn, the additional outputs include eight data points in addition to the disorder propensities and their respective binary predictions. These data points describe propensities and binary predictions for linker‐specific disorder, protein‐binding, DNA‐binding, and RNA‐binding. While SPOT‐Disorder2 does not directly produce additional outputs itself, the supplemental tools it requires do. These supplemental outputs include a contact map, secondary structure predictions, a position‐specific scoring matrix and more. With the exception of RIDAO's tabulated results, the additional outputs of the aforementioned predictors also serve as inputs for their final predictions. SPOT‐Disorder2 and flDPnn would see improved computational efficiency by combining their sublayers, as RIDAO does, instead of daisy chaining them together.

The energy estimation method was elaborated and implemented in the IUPred family of predictors. ⁴⁴ The IUPred predictors are unique in that they employ a physical model describing the interaction energy between residues to make predictions instead of relying on machine‐learning. RIDAO outperformed IUPred3 by two orders of magnitude with respect to its per‐residue prediction rate. This difference in performance may be ascribed to the parallel processing strategy employed by RIDAO. However, Rida also outperforms IUPred3 by an order of magnitude despite being restricted to serial processing on a single CPU core. This is surprising given that the energy estimation method is incorporated into Rida and RIDAO and suggests that a more efficient implementation of IUPred3 could be constructed.

Rida and RIDAO are most comparable to AUCpreD and metapredict V2, as each of these predictors are self‐contained and designed with proteome‐scale analysis in mind. These predictors all employ neural networks to make disorder predictions. AUCpreD leverages a deep convolutional neural network and is implemented in C++; metapredict V2 uses a bidirectional recurrent neural network with long short‐term memory and is implemented in Python. Rida and RIDAO employ feed‐forward neural networks, as well as support vector machines (SVMs). Rida is implemented in C and RIDAO is implemented in Ruby with a C backend. Although implementations in C and C++ can be expected to run faster than those in Python, ⁵⁶ , ⁵⁷ the choice of language in this case does not explain the dramatic difference in per‐residue prediction rates compared to RIDAO. Especially given that metapredict V2, which is implemented in Python, outperforms AUCpreD. Therefore, these differences must be a consequence of the neural network architecture ⁵⁸ and program design.

2.3. Accessibility

RIDAO was designed to be accessible to a large number of end‐users, without requiring command‐line experience. Nevertheless, RIDAO is a web‐based predictor and is therefore inaccessible to individuals without an internet connection. For these individuals, Rida is available for download as a precompiled command‐line program. RIDAO does not restrict user input with respect to file size or the number of submitted sequences nor does it restrict the frequency of job submissions. To the best of the authors' knowledge existing web‐servers providing disorder prediction services are all limited in some capacity. A consequence of these restrictions is the need for end‐users to setup local installations in order to process large datasets.

With the exception of AUCpreD, each predictor described herein has an associated web‐server. Although AUCpreD is not available as a web‐server, the software acquisition and installation is easy and straight forward for users with command‐line experience. The acquisition and installation of metapredict V2 is streamlined as well. The authors have prepared a package that is available through the de facto python package management system, which can be used as a stand‐alone program or a Python package. The simplicity of this process is appreciated given that the metapredict V2 web‐server only accepts a single sequence at a time, requiring multi‐FASTA datasets to be processed locally.

SPOT‐Disorder2, flDPnn, and IUPred3 also offer web‐servers for end‐users but restrict user submissions. SPOT‐Disorder2 limits submissions to 10 sequences, each containing less than 750 residues. Similarly, flDPnn limits submissions to 20 sequences and IUPred3 limits submission to 1 MB of data. For larger jobs, flDPnn provides a stand‐alone version in an easy to use docker container, as well as source code for local installations. IUPred3 and SPOT‐Disorder2 also offer stand‐alone versions for download. In the case of IUPred3, the setup is easy and only requires one additional Python package, that is, SciPy. In contrast, setting up SPOT‐Disorder2 requires a number of additional tools to also be acquired and installed, each with their own dependencies, and an appreciable amount of storage to house the required NCIB database. The accessibility of SPOT‐Disorder2 could be improved through the addition of an installation script to automate the acquisition and installation of all dependencies. Alternatively, SPOT‐Disorder2 could provide a ready‐to‐run container, similar to flDNpp.

3. CONCLUSIONS

This article introduces a new high‐efficiency web‐based disorder prediction tool, RIDAO. The purpose of this tool is to facilitate the application of disorder analysis in comparative genomics/proteomics and genome‐scale structural bioinformatics. To this end, we have focused on investigating computational efficiency and accessibility (i.e., ease of use) instead of accuracy in this article. Readers interested in the accuracies of the predictors integrated into RIDAO as well as those examined in this work are referred to the original publications ⁴² , ⁴³ , ⁴⁴ , ⁵⁰ , ⁵¹ , ⁵² , ⁵³ , ⁵⁴ , ⁵⁹ , ⁶⁰ and the recent critical assessment of protein intrinsic disorder experiment. ⁴⁹ Importantly, we note that the accuracies of the disorder predictors integrated into RIDAO remain unchanged from their original publications. Indeed, with the exception of rounding errors, RIDAO reproduces disorder scores from its constituent predictors with perfect fidelity. Moving forward we plan to expand the analytical functionality of RIDAO. This includes but is not limited to the development and integration of additional disorder predictors, as well as tools for annotation of functional disorder sites, such as molecular recognition features and posttranslational modification sites. RIDAO can be accessed for free at: https://ridao.app.

4. MATERIALS AND METHODS

In this section, we outline the concepts and methods of disorder prediction which are leveraged by Rida and RIDAO. For any given sequence, six per‐residue disorder predictors are used to construct an MDP. These predictors, that is, PONDRs VLXT, VL3, and VSL2B along with IUPred‐Long, IUPred‐Short, and PONDR‐FIT, are introduced below. In addition to creating disorder profiles, RIDAO also performs multinomial classification by creating CH‐CDF plots. These plots along with the underlying CH and CDF analysis is also discussed. Finally, we detail how predictor efficiency is evaluated.

4.1. CH phase space

Unfavorable thermodynamic interactions between nonpolar solutes and their aqueous environments give rise to the hydrophobic effect, ⁶¹ , ⁶² which is one of the major driving forces behind protein folding. ⁶³ Early work on the protein folding problem led to the elaboration of different amino acid scales describing the affinity and antipathy between each amino acid and water. ⁶³ , ⁶⁴ , ⁶⁵ George Rose, for example, used empirical water–ethanol transfer free energies ⁶⁶ to derive his scale. ⁶⁷ Kyte and Doolittle, on the other hand, based their scale ⁶⁸ on interior–exterior amino acid distribution data collected from experiment ⁶⁹ as well as empirical water‐vapor transfer free energies. ⁷⁰ , ⁷¹ Today, these scales and others like them serve as the basis for CH plots and as features for machine‐learning‐based approaches.

4.1.1. CH plots and FoldIndex

Uversky et al. brought to bear Kyte and Doolittle's hydropathy scale to examine the relationship between sequence composition and disorder in the native state. ²⁹ They showed, with a set of 336 proteins, that a border between natively folded and IDPs could be drawn in CH phase space. The so‐called “Uversky plot” takes the mean net charge at pH 7.0, $⟨R⟩$, and graphs it against the mean hydropathy, $⟨{\widehat{H}}^{\mathit{KD}}⟩$, derived from Kyte and Doolittle's scale. Their borderline can be expressed by the equation

$$⟨R⟩=2.785⟨{\widehat{H}}^{\mathit{KD}}⟩-1.151$$ (3)

which can be rearranged to yield the FoldIndex ⁷² :

$$I_F^{\mathit{KD}}=2.785⟨{\widehat{H}}^{\mathit{KD}}⟩-⟨R⟩-1.151$$ (4)

CH plots enable rapid binary classification of IDPs without the need for experimental isolation and characterization.

Moreover, when Equation (4) is applied to a sequence using a sliding window, disorder profiles can be constructed to detect IDPRs.

4.2. Predictors of natural disordered regions

Work by Romero et al. in 1997 resulted in the first formal predictors of disorder, ⁷³ which were eventually called PONDRs. ³⁰ To distinguish various PONDRs from one another, a descriptive nomenclature is used. ⁶⁰ For example, XL1 and NL1 are both first generation predictors trained on long disordered regions (i.e., more than 30 residues); however, the XL1 dataset contains X‐ray‐characterized sequences while the NL1 dataset contains sequences characterized by nuclear magnetic resonance. ⁷⁴ Regardless of how the training data were obtained, all PONDRs employ machine‐learning techniques and operate on a sliding window of residues. Typically features are selected based on conditional probabilities ⁷⁵ and include but are not limited to: amino‐acid compositions, attributes derived directly from compositions, and attributes derived from some function of compositions. ⁷⁴ The chosen features are weighted and combined in a nonlinear fashion byway of SVMs, artificial neural networks (ANNs), or logistic regression to yield a final prediction.

4.2.1. PONDR VLXT

The early PONDRs were all specialized—with respect to length or position—and therefore were unable to accurately predict disorder along the entire length of a protein sequence. ⁴⁰ , ⁷³ , ⁷⁶ , ⁷⁷ , ⁷⁸ Despite this limitation, these predictors paved the way for future development and, for the time, were accurate within their own domains. ⁷⁹ PONDR VLXT ³⁰ , ⁴⁰ was the first predictor designed to work over the entire length of a sequence. VLXT integrates three feed‐forward neural networks predictors, two for the terminal regions (i.e., XT) and one for the internal region (i.e., VL1). The final prediction is the result of merging the individual results of each underlying PONDR byway of two averaging passes over the entire sequence.

4.2.2. PONDR VL3

The first generation VL dataset consisted of 15 variously characterized long regions of disorder. ³⁰ This number was increased to 145 for the second generation dataset ⁸⁰ and was further refined to 152 samples to produce a third generation dataset, VL3. ⁴¹ PONDR VL3 feeds 20 features through an ensemble of 10 feed‐forward ANNs before selecting the final result by a simple majority vote. In the fifth Critical Assessment of Structured Prediction (CASP5) experiment the VL3 predictor, along with two variants (i.e., VL3‐H, and VL3‐P) were shown to outperform VLXT on long regions of disorder by margins of 7–16%. However, VLXT—with an accuracy of 58%—outperformed the same VL3 predictors on short regions of disorder by margins of 23–33%.

4.2.3. PONDR VSL2B

In CASP6, a significant improvement in predicting short regions of disorder was achieved with PONDR VSL1. ⁸¹ Later, the VSL1 dataset was refined and a new two‐level predictor, PONDR VSL2, along with two variants, VSL2P and VSL2B, were developed using three linear SVMs. ⁴² VSL2 predictors work in two stages. First, two length‐specific predictors, VSL2‐L and VSL2‐S, make their own predictions for every residue in the sequence. Then, a meta‐predictor, VSL2‐M1, weights and combines their outputs to provide a final per‐residue result. VSL2, VSL2P, and VSL2B differ only with respect to their input features. VSL2 uses a total of 54 features, including 22 derived from multiple sequence alignments and 6 from secondary structure predictions. In contrast, VSL2P and VSL2B use reduced feature sets of 48 and 26 features, respectively. VSL2B strictly employs features derived from the amino acid composition of the sequence; and, is therefore less computationally expensive than VSL2 and VSL2P. While the overall accuracy of VSL2B is 2–3% less than VSL2P and VSL2, its lower computational cost makes it well‐suited for genome‐scale studies.

4.3. Pairwise energy estimations

Machine learning‐based disorder predictors, like the PONDRs, are inherently biased due to the difficulty of constructing large unbiased training datasets. Ideally, each disordered segment in a training set would have been characterized by all available experimental techniques. Additionally, the distributions of segment size and location would be expected to be uniform and free of false positives. Despite a continuous influx of empirical data on IDPs and IDPRs, much more work is needed before an ideal training dataset can be constructed. The IUPred predictor circumvents this issue by relying on an energy estimation method, instead of machine learning, to generate disorder profiles.

4.3.1. IUPred

The energy estimation method incorporated into IUPred is based on statistical inter‐residue inter‐action potentials, which can be approximated from solved protein structures. ⁸² Starting with an initial 20 by 20 matrix, M ⁽⁰⁾, where the superscript (0) denotes the zeroth approximation, and each element, m _ij , represents the statistical potential for the pairing of amino acid types i and j, a final matrix M ^*, may be derived iteratively. At each iteration, the error is computed between the predicted energies and observed energies as:

$$∆M_{\mathit{ij}}^{\left(n\right)}=-\mathit{ln}\left(\frac{p_{\mathit{ij}}\left(\mathit{\text{observed}}\right)}{p_{\mathit{ij}}^{\left(n\right)}\left(\mathit{\text{predicted}}\right)}\right),$$ (5)

where p _ij is the pairing frequency of amino acid type i with amino acid type j. Then, the computed error is applied as a correction to yield the next approximation:

$$M_{\mathit{ij}}^{\left(n\right)}=M_{\mathit{ij}}^{\left(n-1\right)}+∆M_{\mathit{ij}}^{\left(n-1\right)},$$ (6)

This process is repeated until the magnitude of the error falls below a predetermined threshold. Provided this process converges, the final matrix M ^* can be used to predict form from sequence. For any given protein in a known conformation, the calculated energy, $E_j^p$, of a residue of type i at position p can be then be expressed as:

$$E_i^p=c_j^p,$$ (7)

where $c_j^p$ is the conformation‐specific interaction frequency between the residue of type i at position p and residues of type j. Alternatively, the energy of a given residue may be estimated purely from the primary structure without prior knowledge of the actual conformation.

$$e_i^p=\sum _{j=1}^{20}{P}_{\mathit{ij}}{n}_j^p,$$ (8)

where $e_i^p$ is the estimated energy for the residue at position p with type i, P _ij is a pairwise energy prediction matrix entry for atom types i and j, and $n_j^p$ is the jth element of the amino acid composition vector, which is specific for position p. The elements of matrix P are determined beforehand by least‐squares fitting of a reference set of proteins with known conformations. To develop IUPred, a reference set comprised of 785 nonredundant globular proteins was used.

4.4. Combinatory techniques (metapredictors)

The initial results on disorder prediction implied that there are at least two distinct types or “flavors” of protein intrinsic disorder. ⁷³ Models trained on long segments saw sharp decreases in accuracy when applied to short segments and vice versa. The ability of VLXT ³⁰ , ⁴⁰ to accurately predict disorder along an entire sequence stems from the fact that it combines three other domain specific predictors. Indeed, numerous examples of combinatory predictors outperforming their individual constituents exist in the literature.

4.4.1. PONDR‐FIT

PONDR‐FIT ⁴³ is an ANN meta‐predictor that integrates three PONDRs (i.e., VLXT, VSL2B, and VL3) with FoldIndex, IUPred, and TOP‐IDP. The underlying predictors all generate disorder profiles given an amino acid sequence and, with the exception of TOP‐IDP, are described elsewhere. TOP‐IDP is a numerical amino acid disorder propensity scale elaborated from an initial pool of 517 sequence attributes using genetic algorithms. Similar to the FoldIndex, it can be applied in a sliding window to yield a disorder profile. Each prediction made by PONDR‐FIT depends on 126 input features which are the outputs of its 6 constituent predictors operating on sliding windows of 21 residues. Overall, PONDR‐FIT yields predictions that are, on average, 11% more accurate when compared to VLXT, VSL2B, VL3, FoldIndex, IUPred, or TOP‐IDP alone.

4.4.2. CH‐CDF

Disorder profiles are useful for identifying IDPRs as they provide a continuous measure of disorder throughout a sequence. Alternatively, the data in a disorder profile can be viewed as a histogram of disorder propensities. This histogram then serves as input for the CDF. ⁸³ Afterward, for a given predictor, the optimal boundary line between ordered and disordered proteins can be computed and CDF plots can be used for the binary classification of IDPs, ¹¹ , ⁸⁴ similar to CH plots. PONDR VLXT was the first per‐residue predictor to be given this treatment. ¹¹

Oldfield et al. compared the CDF and CH methods and found disagreements between their predictions when applied to the same data sets. ⁸⁵ Their results may be interpreted through the lens of the protein quartet model, ⁸⁶ which classifies soluble proteins as being natively extended, relaxed (pre‐molten), collapsed (molten), or well‐structured. When the two methods are in agreement, the protein can be expected to belong to either the extended or the well‐structured class. When the two methods do not agree, the protein likely belongs to an intermediate class.

Combined CH‐CDF plots (Figure 2) can be constructed to perform multinomial classification. ⁴⁵ , ⁴⁶ , ⁴⁷ , ⁴⁸ Classification is based on which quadrant a protein is placed in. When both methods predict disorder, the protein is found in quadrant 4 and is predicted to be disordered. Alternatively, when both methods predict a protein to be well‐structured it is found in quadrant 2 and is predicted to be structured. If the two methods disagree, then the protein is predicted to be “mixed” or “rare.” These proteins are found in quadrant 3 or quadrant 1, respectively. Together, CH and CDF analysis provides sought after insight into the sequence‐based partitioning of functional disorder flavors. ⁷³ , ⁷⁸ , ⁸⁰ , ⁸⁷ , ⁸⁸

4.5. Efficiency evaluation

To evaluate the efficiency of each predictor, a test set comprised of 1,007,540 sequences taken from a 100 UniProt ⁸⁹ reference proteomes is constructed. Complete reference proteomes are selected from all kingdoms of life (Table 1). Specifically, the test set contains: 76,772 sequences from 9 fungi; 210,052 sequences from 8 protists; 329,717 sequences from 8 animals; 66,211 sequences from 14 bacteria; 59,809 sequences from 19 archaea; 259,300 sequences from 6 plants; and 5,679 sequences from 36 viruses. Sequences in the dataset containing less than 31 residues are extended by appending a poly‐alanine tail at the C‐termini such that the sequence meets the minimum length requirement for all predictors—31 residues. Additionally, we convert U to C, B to N, O to K, Z to N, and X to G so that no sequences are rejected from any of the predictors. After modification, a grand total of 428,786,255 residues are contained in the dataset, which is available for downloaded at https://ridao.app/publications. Each predictor is given 24 hr of runtime to process as much as this dataset as possible.

TABLE 1.

Test set reference proteomes. Reference proteomes from all kingdoms of life used to construct the test set for predictor efficiency evaluation. The final test set is available for download at https://ridao.app/publications. It contains 1,007,540 sequences from 100 organisms comprising 428,786,255 residues

Kingdom	UniProtIDs	Total sequences
Animalia	UP000001940, UP000000803, UP000005640, UP000002358,	329,717
	UP000410492, UP000000437, UP000186698, UP000000589
Archaea	UP000000536, UP000000590, UP000000792,	59,809
	UP000001018, UP000001974, UP000002613,
	UP000006794, UP000008243, UP000011555,
	UP000305339, UP000000554, UP000000758,
	UP000001013, UP000001903, UP000002487,
	UP000006663, UP000007722, UP000011511, UP000297295
Bacteria	UP000000428, UP000000537, UP000000625, UP000000807,	66,211
	UP000001364, UP000001414, UP000001425, UP000001570,
	UP000001584, UP000002332, UP000028875, UP000095713,
	UP000184536, UP000192674
Fungi	UP000000560, UP000000561, UP000000591, UP000001805,	76,772
	UP000001861, UP000002149, UP000002311, UP000002485,
	UP000007431
Plantae	UP000001514, UP000006548, UP000006727, UP000006729,	259,300
	UP000011750, UP000595140
Protist	UP000000542, UP000000600, UP000001449, UP000001542,	210,052
	UP000002195, UP000009168, UP000011087, UP000013827
Virus	UP000000354, UP000000730, UP000000854, UP000000938,	5,679
	UP000001134, UP000001240, UP000001451, UP000002241,
	UP000002242, UP000002500, UP000006720, UP000007639,
	UP000008158, UP000008286, UP000008288, UP000008597,
	UP000009253, UP000009255, UP000009294, UP000012678,
	UP000018636, UP000101154, UP000105007, UP000114976,
	UP000145840, UP000202558, UP000203665, UP000204142,
	UP000217350, UP000241349, UP000247059, UP000249273,
	UP000281246, UP000383418, UP000464024, UP000514836

The total time required to read inputs, make predictions, and write outputs is taken into consideration when evaluating efficiency. The input order of sequences is kept constant throughout this work and any predictors capable of reading multi‐FASTA files are given the entire set as a single input. If a multi‐FASTA file cannot be used, the test set is broken up into individual FASTA files. Importantly, we note that flDPnn could not load the test set using a single multi‐FASTA file without experiencing catastrophic failure; for that reason, flDPnn was evaluated using individual FASTA files. Furthermore, because it has been reported that a majority of the metapredict V2 runtime is spent loading its neural network, ⁵³ we evaluated metapredict V2 using individual FASTA files and a single multi‐FASTA file. Herein, we report the results of the latter, for which metapredict V2's code was modified to write outputs as they are generated instead of at the end of processing. This modification was necessary as metapredict V2 does not finish processing the multi‐FASTA file input within the allotted time and therefore does not produce any output otherwise. Each predictor is evaluated on the same server housing RIDAO. This server employs a 2.60 GHz Intel Xeon E5‐2640 v3 CPU, 64 GB 2,133 MHz DDR4 SDRAM, and two NVIDIA GeForce GTX 980 s.

AUTHOR CONTRIBUTIONS

Vladimir N. Uversky motivated and guided the work. Guy W. Dayhoff conceived the project, designed and developed the web application, performed analysis, made figures, and wrote the manuscript.

CONFLICT OF INTEREST

Both the authors declare no conflict of interest.

Dayhoff GW II, Uversky VN. Rapid prediction and analysis of protein intrinsic disorder. Protein Science. 2022;31(12):e4496. 10.1002/pro.4496

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available from the corresponding author upon reasonable request.