ColGen: An end-to-end deep learning model to predict thermal stability of de novo collagen sequences

Abstract

Collagen is the most abundant structural protein in humans, with dozens of sequence variants accounting for over 30% of the protein in an animal body. The fibrillar and hierarchical arrangements of collagen are critical in providing mechanical properties with high strength and toughness. Due to this ubiquitous role in human tissues, collagen-based biomaterials are commonly used for tissue repairs and regeneration, requiring chemical and thermal stability over a range of temperatures during materials preparation ex vivo and subsequent utility in vivo. Collagen unfolds from a triple helix to a random coil structure during a temperature interval in which the midpoint or T_m is used as a measure to evaluate the thermal stability of the molecules. However, finding a robust framework to facilitate the design of a specific collagen sequence to yield a specific T_m remains a challenge, including using conventional molecular dynamics modeling. Here we propose a de novo framework to provide a model that outputs the T_m values of input collagen sequences by incorporating deep learning trained on a large data set of collagen sequences and corresponding T_m values. By using this framework, we are able to quickly evaluate how mutations and order in the primary sequence affect the stability of collagen triple helices. Namely, we confirm that mutations to glycines, mutations in the middle of a sequence, and short sequence lengths cause the greatest drop in T_m values

1. Introduction

Type I collagen is the most abundant protein in animals, forming the matrix of the skin, tendons, bone, and vasculature. Collagen consists of amino acid sequences, generally in the form of repeat units of glycine-X-Y where amino acids proline and hydroxyproline are especially abundant¹. The fixed angles of proline or hydroxyproline enable collagen sequences to fold into a triple helical structure, or tropocollagen, which forms the basic structural unit of collagen. In this triple-helical arrangement, glycine is the only amino acid that can be incorporated in the helix without distortion and is, therefore, a strict requirement for fibril-forming collagens. This fundamental structural unit is typically 300 nm long, and 1.5 nm in diameter and can pack with other three-stranded collagen molecules to form hierarchical structures, including fibrils and fibers ^2-8. This hierarchical structure enables significant mechanical tensile loading of collagen in physiological conditions ^9-13; collagen fibrils can reach a tensile modulus of 0.2 to 0.86 GPa while maintaining elasticity in the human body ^11,14-17.

With such remarkable structure and mechanics, as well as biocompatibility, collagen-based biomaterials are routinely sought for use in vivo tissue repairs. Collagen-based biomaterials have been successfully used in drug delivery systems, skin repair, and other biomedical applications ^18,19. However, the use of collagen as biomaterial constructs remains limited in large part due to the inability to design and assemble collagen structures in vitro that emulate the structural hierarchy seen in vivo, reflective of triple-helical stability as well as higher order assembly. Such limits in the ability to recapitulate collagen triple helical structures in vitro, are reflected in reduced temperature stabilization and, in practice, in thus reduced mechanics and more rapid degradation in vivo. Such limitations remain as significant hurdles to the more widespread use of collagen in a broader range of tissue repairs. Thus, one metric to evaluate collagen’s mechanical integrity is from its melting point or T_m, defined as the midpoint during the temperature window in which the triple helix unfolds.

Several researchers have examined various biological species to discover or design collagen peptides with greater thermal stability ^20-25. Others have tried to add various additives to increase T_m ²¹. A significant effort by Persikov et al. developed equations to predict T_m values based on local interactions between different amino acid chemistries in collagen tripeptides ^26-29. While significant work has sought to understand the effects of variables including primary sequence and additives on the thermal stability of collagen, having a predictive framework that facilitates a priori design of collagen sequences with specific T_m values without prior knowledge of chemical interactions would enable researchers to more efficiently design and synthesize thermally stable collagen for desired applications. Such a framework for discovery, when combined with advancing concepts in controlled self-assembly, would propel the field of collagen-based biomaterials forward.

However, a major challenge to this goal has been to uncover sequence-structure-function relationships in collagen across its length scales ^6,11,30,31. Most of the earlier studies have reported progress using atomistic-based methods, such as molecular dynamics or coarse-graining, however, these approaches tend to be computationally expensive and cannot easily explore vast variations of sequences and mutations. In this paper we propose to develop a method that overcomes some of these limitations to set the foundation for an effective machine-learning based approach towards identify sequence-structure-function relationships in collagen molecules. We hypothesize that machine learning can effectively relate complex sequences to effective physical, chemical and biological functions without knowledge of underlying biological interactions. Machine learning has emerged as a useful tool in the analysis of large datasets to help develop design principles of biological materials ^32-36. Through careful design and application of neural networks and use of appropriate datasets, machine learning can provide useful information for predicting various behaviors of biological materials without prior knowledge of chemical interactions ^33,34

This paper reports the development of a deep neural network machine learning model to predict T_m values of de novo collagen sequences. Through the application of a self-evolutionary algorithm, 1D convolution, bidirectional long short-term memory (LSTM), and dropout features, the model trains on a dataset of existing collagen sequences and T_m values derived from the literature. Using this framework, the effect of mutations in specific residues, sequence length, and order on T_m values is quickly analyzed to derive design principles about the thermal stability of collagen. The goal of this work is to twofold. First, this machine learning method applied to generate thermally stable de novo collagen sequences creates a general framework for the use of machine learning models in biomaterial applications. Second, the insights gained from rapid testing of collagen sequences contribute general design principles about the thermal stability of collagen, with implications not only in biomaterial designs, but also in understanding collagen-related mutations and associated disease states.

2. Materials and Methods

2.1. Collagen Dataset

We collected 566 collagen sequences with reported T_m values from a survey of literature (see Table S1). These available melting temperatures were collected from the PubMed, Web of Science, Scopus, Directory of Open Access Journals (DOAJ) Google Scholar databases ^{26,27,45-54,37,55}-^57,38-44. Experimental thermal stability data sets for the observed T_m values (in °C) for the Gly-X-Y tripeptide units in triple helical collagens-like peptides are based on host-guest peptides and are integrated to produce an algorithm for predicting global melting temperatures. These experimental results are expanded further using predictions of Tm values of host-guest sequences from Persikov et. al.²⁶ The distribution of the dataset is presented in Figure 1 where sequences have experimentally measured melting temperatures ranging from a few degrees C to 70°C with the mean at ~30°C. The data shows a normal distribution which is used in the machine learning model (Figure 1c). Outliers in the data, specifically the negative T_m value sequences, are generally from extrapolated experimental data.⁵⁸

Fig. 1. Distribution of data from literature, based on experimental results.

a) Overview of the problem studied here, to predict the melting point Tm from the sequence of collagen molecules. b) Experimental melting temperatures collected. c) Normalized T_m value distribution. Thermal stability data sets for observed Tm values for Gly-X-Y tripeptide units in triple helical collagen-like peptides are integrated here to produce an algorithm for predicting global melting temperatures. Data from ^{26,27,45-54,37,55-57,38-44}.

2.2. Development of the End-to-end Deep Learning Model

We provide a summary of the deep learning model reported here, as shown in Figure 2. This model is named ColGen to represent a collagen sequences generator which is capable of generating new type I collagen sequences with known T_m values. A natural language processing (NLP) model was adopted ^59,60.

Fig. 2. Overview of machine learning model.

a) We design a deep learning network to discover hidden features of collagen sequences by introducing embedding layer. b) The structure of our deep learning model starts at an embedding layer, followed by two 1D convolution layers, then we flatten all the features and send them into a fully connected layer for regression to determine T_m value. Figures S1 and S2 provide details of the neural network model.

Tokenization is used, reflecting a common approach when processing the raw texts or sequences of symbols. Tokens can be considered as the fundamental building blocks of our data which was represented in single letter code for each amino acid along the main chin^61,62. Tokenization is first applied to the collagen sequences before passing them through the deep learning model. Every collagen sequence was first encoded into a series of digital tokens, that is, every amino acid is treated as a unique number from 1 to 21 for all the 20 essential amino acids and one nonessential amino acid, hydroxyproline, found in collagen.

2.3. Neural net structure

The tokenized sequences were passed through an embedding layer which is able to recognize the relationship between tokens during the training process. After the embedding layer, data flows through a 1D convolutional layer to harness the internal features from the input. The data is then routed into bidirectional LSTM layers to learn all hidden features from each collagen sequences. Finally, a fully connected neural network - composed by two dense layers – is used to ultimately output the predicted T_m value as a scalar value. This flow of information provides an end-to-end model that relates a sequence of amino acids of varying lengths to its melting temperature.

Figures S1 and S2 provide detailed information into the architecture of the neural network used. The neural network features a total of 49,041 trainable parameters.

2.4. Model training

The dataset was randomly split into training dataset testing dataset, where 80% of the data is used for training and the other 20% is used for testing to examine the ability of prediction of our model.

The model is trained on a Xeon workstation with a GTX-3090 GPU, for 200 epochs. It is worth noting here, the well-trained model can be deployed in a laptop or desktop computer without further requirement of GPUs.

3. Results

We begin the analysis by training the machine learning model. We find that ColGen demonstrates good predictive accuracy of T_m values in the testing set (Figure 3a). The data shows that testing data is generally well predicted, and a large range of temperatures can be predicted by the model. In addition, the training and validation error remain consistent at a mean squared error of ~0.2 after 120 epochs (Figure 3b).

Fig. 3. Predictive accuracy of ColGen, and training performance.

a) Data comparing training with test set demonstrates a 95% confidence interval. Plotting R² of training / testing / generation. b) Training and validation error over epochs demonstrate a well fit model. The validation and training errors reach a plateau around 150 epochs.

Because the ColGen model enables fast characterization of the T_m values of different collagen sequences, the model was used to understand how mutations and chain length affect T_m value. ColGen enables a rapid search of these effects. (GPO)₁₀ was used as a standard comparison, referred to here as the “pristine sequence,” as it has the highest known thermal stability value in literature ^20-56-63-64 due to stereoelectronic effects from hydroxyproline ^65-67. Mutations were made in either the G, P, or O position, where either the G, P, or O positions were replaced with another amino acid, and this process is repeated over all amino acid substitutions. The resulting T_m values from each of these calculations are then averaged.

Based on the GolGen model, mutations in the middle of the pristine collagen sequence have the greatest destabilizing effect on collagen T_m values (Figure 4a). This suggests the presence of a critical transition location along the length of the sequence, potentially near the midpoint of the sequence or at least away from the chain ends depending on overall length, that is critical in holding the full chain length together. Further, mutations made along the first several residues at the N terminus of the collagen sequence are more destabilized and have a lower T_m value than mutations made along the last several residues at the C terminus. This indicates a directionality along the sequence and is validated by experimental data which shows that the N-terminal regime is required for the trimerization of other triple helical collagens ^68,69. This is in contrast to other work on fibril-forming procollagens which suggests that type 1 collagen molecules in vivo have a C-terminal that is responsible for chain selection and trimerization ^70-74. This difference between our model and procollagen fibrillar formation results is likely because several peptides included in this training data may have been folded in N-to-C direction with a nucleation domain at the N terminus ^75,76.

Fig. 4. Characterization of the effects of various types of mutations, predicted by ColGen.

a) T_m values of mutations in G, P, or O position demonstrates that mutations in the middle of the sequence are the most destabilizing for Tm values. Mutations in the G position are the most destabilizing to the peptide. Error bars indicate standard deviation of all amino acids that were mutated. b) Thermal stability as a function of collagen sequence length, where length is number of repeat units (GOP) demonstrates that there is a critical length at which the Tm can no longer be increased significantly. This critical length is consistent with other studies.⁸⁵

The effect of chain length on T_m was also measured. As shown in Figure 4b, the model faithfully captures that T_m values increase upon increasing number of amino acids (triplets) due to increasing hydrogen bonding between triplets. However, there is a limit to the increase in thermal stability that is also captured by the model. This leveling off is achieved at about 14 triplet repeats at around 80C and is consistent with experiments and a thermal stability prediction algorithm developed by Persikov et. al ²⁶. Similar intrinsic strength limits have been found in hydrogen bonded alpha helix and beta sheet structures ^77-79.

To quantify how the increasing number of mutations affects the thermal stability of collagen, we define the term “disorder parameter” which means the number of repeating triplets with mutations in the G, P, or O position compared to the pristine sequence. Thus, increasing disorder parameter increases deviation from the pristine sample. As expected, increasing disorder decreases the thermal stability of the pristine sequence consistent with experimental data (Figure 5a) ²⁶. Further, disruptions in glycines are the most destabilizing. This is consistent with experimental findings that disruptions in glycine severely impact stability and often constitute disease states, though we cannot directly correlate the position of our glycine mutations to the position of naturally observed mutations due to the shorter sequence length we employ in this study ^58,80-82.

Figure 5. Characterization of effect of disorder on T_m, as predicted by the model.

A) T_m values of disorder arranged by G, P, or O position confirm that increasing mutations along the chain decreases thermal stability of the triple helix. Error bars indicate standard deviation of all amino acids that were mutated. b) T_m values of disorder in the O position demonstrates that initial mutations to polar, positive charged, and negative charged amino acids confer the same degree of stability in the molecule. However, upon increasing mutations, polar amino acids are the least destabilizing to the triple helix, suggesting that they should be used for bacterial expression of collagen where expression of O is not possible.

This model could be useful in informing sequence design of bacterially-produced collagen, which enable larger production of tailored collagen sequences ^23,50,83. Given that bacteria are unable to express hydroxyproline for bacterially-produced collagen, the machine learning model developed here could also serve as a tool to predict which amino acids may help as a replacement to hydroxyproline. Figure 5b demonstrates these results, showing that positively charged, negatively charged and polar amino acids are only slightly destabilizing compared to (GPO)₁₀ up to a certain extent, but positively charged amino acids (R, H, K) are the least destabilizing compared to other types of amino acids if significant mutations are introduced in the O position.

4. Discussion and conclusions

We developed a platform that adopts a deep learning model trained with input sequences collected from the literature to predict T_m values and a self-evolutionary module to optimize T_m values. To evaluate the long-term interference within a cut-off, the size of kernel function is prescribed and the data flows through several fully connected neuron layers with non-linear activation functions. The deep learning model predicts T_m values of de novo sequences as the output.

Our model shows good prediction power in extrapolating T_m values of the testing set for collagen sequences not included in our training dataset. The trained deep learning model is able to predict T_m values within an acceptable error range considering the amount of experimental data. The machine learning algorithm also allows us to quickly determine how specific amino acids mutations, the amount of disorder in the sequence, and the sequence length affect the thermal stability of collagen. We determine that mutations in the middle of the sequence greatly affect stability and that there the maximum achievable temperature is already reached at a sequence length of 14 repeat units.

While the model enables a quick prediction of T_m values, there are some limitations that should be expanded upon in future work. These limitations primarily arise from the data set used to train the model, which could be further expanded to include a wider range of sequences and sequence lengths. For example, the current dataset has a maximum T_m range of 70C. While the collagen model is able to predict sequences up to 80C, the model’s predictive capacity beyond this temperature range is unclear. New sequences with higher T_m ranges would provide one way to validate the model. Further, the model is only trained on the standard amino acids and hydroxyproline. As such, it would be unable to predict the T_m values of other non-native amino acids and this problem too should be resolved by an expanded data set. Further, experimental melting temperature tests of the predictions from the collagen sequence mutation would help validate the model.

Despite these limitations, the application of this algorithm to collagen sequences more broadly would enable researchers and engineers to design specific collagen sequences with desired T_m values to match specific processing steps related to collagen materials formation and subsequent biomedical applications. More importantly, this approach would lead to more stable and thus mechanically more robust collagen biomaterials to meet new medical applications. Such design control would also provide a foundation for understanding collagen degradation rates in vivo, thus correlating T_m to rates of collagen-based biomaterial remodeling/regeneration in vivo. Tuning such rates would be a significant advance the field of regenerative medicine.

Many collagen-based diseases are based on mutations in the primary sequence, which relate to the models and predictive tools offered here. Thus, this new method would offer insight and new perspectives on disease states in the context of collagen stability, along with implications for possible interventions and regeneration/repair routes in the future. Further, this approach could be propagated up length scales to enable predictions of macroscale biomaterial features and assembly.

In addition to expanding the utility of thermally stable collagen sequences, this work represents a starting point future work on a priori design of protein sequences with specific properties without prior chemical knowledge, for instance through the use of genetic algorithms in conjunction with the machine learning model reported here – to solve the inverse problem ⁸⁴ A more immediate application may be other fibrous proteins, such as silks, keratins, resilins, reflectins and elastins, which have repeating protein sequences and could be optimized with such an algorithm to maximize mechanics, stability and functions. The formulation reported here could find many other generalizable applications.

Data availability:

The processed data required to reproduce these findings cannot be shared at this time as the data also forms part of an ongoing study. The raw data set is provided as Table S1