Scaling SMILES-based chemical language models for therapeutic peptide engineering

Abstract

Therapeutic peptides occupy a unique middle ground in drug discovery, offering the high specificity of protein interactions with the chemical diversity of small molecules, yet they currently fall in a computational blind spot. Existing foundation models cannot handle them effectively: protein models are restricted to natural amino acids, while chemical models struggle to process large, polymer-like sequences. This disconnect has forced the field to rely on static chemical descriptors that fail to capture subtle chemical details or on complex multi-embedding pipelines that are custom tailored to specific datasets. To bridge this gap, we present PeptideCLM-2, a suite of chemical language models trained on over 100 million molecules to natively represent complex peptide chemistry. This modeling approach both simplifies the application of machine learning to therapeutic peptides and results in improved performance over alternative approaches for predicting development endpoints including membrane diffusion, tumor homing, and half life.

Graphical Abstract

Introduction

Peptides are a rapidly evolving modality in the therapeutic space.^1–6 They occupy a unique chemical niche situated between small molecules and proteins. Like small molecules, they possess immense chemical diversity.^7–11 Yet, they retain the modularity of biological polymers, allowing for accessible and efficient synthesis.^12–14 While modern synthetic methods enable the creation of a vast diversity of modified peptides, the computational tools required to rationally select among these options have not kept pace. Attempts to adapt existing frameworks have faced significant limitations. Protein language models (pLMs) are restricted to fixed amino-acid alphabets, rendering them unable to encode noncanonical or chemically modified residues.¹⁵ Conversely, chemical language models (CLMs) are typically trained on small molecules, lacking the contextual range to interpret peptide-specific motifs.^16,17

Large-scale protein corpora such as UniProt¹⁸ have fueled the rise of self-supervised pLMs. Architectures like the ESM family^19–21 and ProtTrans²² have proven that transformers can learn structural and functional constraints directly from sequence data. These models capture deep dependencies predictive of structure,^23–25 variant effects,^26,27 and molecular function.^28,29 In parallel, CLMs utilizing symbolic representations like SMILES or SELFIES^30,31 have successfully encoded chemical syntax via masked or autoregressive objectives,^32–34 enabling prediction of various chemical properties.^35,36 Extensions such as ChemBERTa-2³⁷ and ProLLaMA³⁸ have further suggested that augmenting self-supervision with physicochemical regression introduces a valuable inductive bias, improving generalization. Despite these advances, however, deep learning has shown minimal, if any, improvement over molecular fingerprints for peptide property prediction.^39,40

In this work, we introduce PeptideCLM-2, a suite of nine SMILES-based transformer encoders⁴¹ designed to unify therapeutic peptide modeling. Building upon our previous architecture, PeptideCLM,⁴² we train models across a parameter range of 32 to 337 million using three distinct objectives: masked language modeling (MLM), multi-task regression (MTR)⁴³ to RDKit-derived descriptors,⁴⁴ and a dual objective combining both. This systematic design provides a framework to rigorously decouple the effects of model scale and inductive bias on representation learning. Because therapeutic peptides often adopt transient rather than static conformations, we employed a string-based architecture to capture topological connectivity without the bias of a single, rigid 3D structure.

We demonstrate that these models outperform molecular fingerprints and specialized architectures, while relying only on simple string-based inputs. Our evaluation also reveals a critical scaling transition: while descriptor-guided pretraining provides a necessary scaffold for smaller models, larger architectures trained solely on MLM spontaneously recover these physicochemical relationships, a capability evident in both the embedding manifold and downstream evaluations. Together, these results establish PeptideCLM-2 as both an open, scalable resource for peptide engineering and a framework for understanding the interaction between model capacity and pretraining for the field of computational peptide chemistry.

Results

Transformer architecture and multi-objective pretraining for diverse peptide chemistries

The PeptideCLM-2 suite represents a unified framework for embedding therapeutic peptides. Unlike standard protein language models, which are restricted to a fixed alphabet of 20 canonical amino acids, PeptideCLM-2 is designed to explore the full spectrum of chemical space. By combining scalable transformer architectures with interpretable tokenization strategies, the framework provides a bridge between symbolic chemical representation and chemical function.

To rigorously evaluate chemical deep learning for peptides and the emergence of chemical intuition, we designed a controlled grid of nine models. We varied model capacity across three orders of magnitude—32M, 114M, and 337M parameters—and trained each scale using three distinct pretraining objectives: masked-language modeling (MLM), multi-task regression (MTR) to physicochemical descriptors, and a combined dual objective. This systematic design allows us to disentangle the effects of parameter scale from the learning paradigm used.

Our architecture processes inputs as raw SMILES strings. This allows for the native encoding of canonical residues, noncanonical modifications, cyclic scaffolds, and complex conjugations like lipidation or PEGylation (Fig. 1A). The backbone of the framework is a BERT-style transformer encoder (Fig. 1B). To ensure training stability and effective handling of long-range chemical dependencies within large macrocycles, we incorporated modern architectural features including rotary positional embeddings (RoPE),⁴⁵ SwiGLU activation functions,⁴⁶ and pre-layer normalization.⁴⁷ Model depth and hidden dimension scale with parameter count, maintaining a consistent per-head dimensionality of 64.

Figure 1: PeptideCLM-2 workflow: enabling encoding of chemical diversity for predictive modeling.

(A) The model can input SMILES strings from peptides with various modifications, allowing it to encode cyclic peptides, noncanonical amino acids, and synthetic modifications. (B) Models were trained with both span masking (predicting grey [MASK] tokens) and regression to physicochemical descriptors (e.g., LogP, TPSA) from a pooled sequence embedding. (C) Examples of prediction targets for downstream tasks, including membrane permeability, cellular interactions, and aggregation propensity.

We investigated three distinct learning paradigms to train the architecture. The masked language modeling (MLM) objective utilized span masking⁴⁸ to force the model to reconstruct missing chemical fragments, effectively learning the syntax of molecular graphs. The multi-task regression (MTR) objective explicitly grounded the embeddings with regression to 99 RDKit-derived physicochemical descriptors (Tab. S1) from a mean-pooled embedding. Finally, a hybrid objective optimized both losses simultaneously, testing whether syntax and semantics could be learned in tandem. For downstream evaluation, the pretrained encoder is coupled with a lightweight, 2-hidden layer feed-forward regression head employing GeLU activation, allowing the model to adapt its learned representations to specific biological tasks such as membrane permeability, cellular interactions, or aggregation (Fig. 1C).

Composite pretraining corpus and sequence compression via k-mer tokenization

To construct a representation space that spans the full continuum from small molecules to biological polymers, we curated a composite pretraining corpus combining three distinct datasets. We aggregated lipids from the LIPID MAPS Structure Database (n = 50,450),⁴⁹ small drug-like molecules from PubChem (n = 108,583,157),⁵⁰ and diverse peptide sequences from ESMAtlas (n = 9,634,945).²⁰ As visualized in Figure 2A, these sources inhabit unique, complementary regions of the chemical manifold, ensuring the model encounters a heterogeneous distribution of molecular syntax.⁵¹ Due to the flexibility of SMILES encoding, we are able to train on small molecules to learn chemical features, which can directly translate to therapeutic peptides that contain chemistries not seen in the natural amino acid space.

Figure 2: Data Diversity and Representational Efficiency.

(A) A 2D t-SNE projection of Morgan Fingerprints from the pretraining corpus, demonstrating that the composite dataset (LMSD, PubChem, and ESMAtlas) bridges distinct chemical subspaces. (B) Comparative length distributions of tokenized molecules. The k-mer tokenization strategy yields a substantial compression factor relative to atom-level encoding, reducing sequence length by 64% for peptides, thereby enabling the efficient processing of long biological chains.

A central bottleneck in applying chemical language models to biopolymers is the computational cost of self-attention, which scales quadratically with sequence length $\left(𝒪\left(n^2\right)\right)$. Because direct character encoding of peptides generates exceptionally long SMILES strings, standard models become prohibitively expensive to train. To resolve this, we developed a specialized k-mer tokenizer (Tab. S2 & S3) that compresses these symbolic representations by mapping recurring sub-structural motifs to single tokens. This strategy significantly reduces the effective sequence length, and thus the quadratic compute burden, while maintaining full compatibility with standard SMILES syntax.

Quantitatively, this strategy reduces sequence lengths by 38% for small molecules and 64% for natural peptides relative to standard atom-level frameworks like DeepChem⁵² (Fig. 2B). In order to conduct a fair evaluation, we held the data composition and tokenization schema constant across all experiments (Tab. S4). We similarly standardized finetuning workflows using nested cross-validation (Tab. S5), ensuring that observed differences in downstream tasks, such as permeability or aggregation, are attributable solely to the emergence of learned chemical intuition rather than artifacts from pretraining.

We found that this compression comes at no cost to accuracy (Tab. S6); benchmarking confirms that models trained with either tokenizer achieve equivalent performance on membrane permeability prediction. Thus, the k-mer approach successfully resolves the trade-off between computational tractability and semantic fidelity, allowing the model to attend to long-range dependencies in complex backbones without the prohibitive cost of character-level encoding.

Parameter scaling drives the spontaneous emergence of physicochemical organization

We evaluated PeptideCLM-2 using the CycPeptMPDB dataset,⁵³ a standard benchmark for cyclic peptide permeability with robust evaluation splits.⁴² To ensure a rigorous comparison, we focused our analysis on nine distinct model configurations selected via ablation studies, covering three distinct parameter sizes.

We first investigated whether the models had learned a chemically meaningful latent space. By projecting the embeddings of the 337M MLM model into two dimensions, we observed that the model organizes the chemical manifold according to fundamental physical properties. Without explicit instruction, MLM training organized embeddings by molecular weight and aromaticity (Fig. 3A, B). We also notice this across a larger list of physicochemical descriptors including charge, logP, and available hydrogen bonding sites (Fig. S1). This unsupervised organization shows that the embedded structure also has some organization with measured PAMPA permeability (Fig. 3C), confirming that the model has encoded many of the structural determinants of membrane diffusion.

Figure 3: Emergence of chemical structure and predictive capability.

(A-C) A t-SNE projection of embeddings from the 337M model. The model spontaneously organizes the chemical manifold by (A) molecular weight and (B) aromaticity, which correlates with (C) measured permeability. (D) Transfer learning performance (linear probing) remains low across all scales, indicating permeability requires non-linear features. (E) Full finetuning reveals a scaling transition: while small models rely on explicit supervision (MTR) to perform well, large self-supervised models (MLM) spontaneously recover this performance, matching the supervised baseline.

To disentangle the quality of these raw representations from the model’s capacity to adapt, we compared layer-wise transfer learning against full-model finetuning. We found that linear probes trained on frozen features performed poorly (R² < 0.3) regardless of model scale (Fig. 3D), suggesting that the model embeddings are complex, non-linear properties that cannot be extracted via simple linear combinations of features.

We further analyzed where this information is stored by probing each layer of the network. This revealed a significant difference in stability between model sizes. In smaller models, the final hidden layer—typically used for downstream tasks—often underperformed compared to intermediate layers (Fig. S2), suggesting a forgetting of physicochemical details at the output bottleneck. In contrast, larger models exhibited high stability, maintaining rich representations from the middle layers all the way to the final output.

The most significant finding, however, emerged during full-model finetuning (Fig. 3E). At the smallest scale (32M), inductive bias is dominant: models pretrained with explicit physicochemical regression (MTR) significantly outperformed those trained only on language modeling (R² ≈ 0.38 vs. 0.13). This indicates that small models improve when pretraining is grounded in physical reality.

As capacity increased to 337M parameters, this dependency vanished. The purely self-supervised MLM models show equivalent predictions to the supervised MTR models, with both objectives converging at an R² ≈ 0.58. This is nearly double the performance of traditional molecular fingerprints (R² ≈ 0.3). Here we demonstrate a fundamental scaling law: while small models improve prediction when explicitly taught physical properties of molecules, sufficiently large transformers are able to derive these priors from the syntax of chemical language alone.

Predictive generalization to complex biological phenotypes and non-canonical chemistries

Having established that PeptideCLM-2 captures physicochemical constraints, we next asked whether this intuition translates to complex biological function. We evaluated the model on three diverse classification benchmarks—tumor homing, cell penetration, and antimicrobial activity (Tab. S7)—using Matthews Correlation Coefficient (MCC) to rigorously account for class imbalance. The datasets used for these tasks all require modeling of noncanonical chemistry.

Visual inspection of the pretrained embedding space reveals that the model has already learned to distinguish bioactive species. Embedding projections from PeptideCLM-2 Hybrid 337M parameter model show distinct separation between positive and negative classes across all three benchmarks (Fig. 4A, C, E). This latent structure provides a robust foundation for finetuning, where PeptideCLM-2 models were able to outperform specialized architectures, often relying on generation of chemical features or complicated embedding methods.

Figure 4: Predictions on three benchmark datasets for biological interaction.

Left panels: t-SNE projections (A, C, E) of embeddings from the 337M Hybrid model. Even before finetuning, the model’s latent space exhibits strong linear separability between active (red) and inactive (blue) peptides across all three tasks. Right panels: Performance comparison (MCC) between Transfer Learning (linear probe) and End-to-End Finetuning (B, D, F). Grey dashed lines indicate the performance of prior state-of-the-art methods (THPep, Random Forest+PaDEL, AmpHGT). Red dashed lines indicate a standard Random Forest baseline using RDKit topological fingerprints.

Tumor Homing:

Tumor-Homing Peptides (THPs) are notoriously difficult to classify because they rely on subtle recognition motifs (e.g., RGD, NGR) to target tumor vasculature.⁵⁴ The method published with this dataset, THPep, achieved an MCC of ≈ 0.71 by relying on engineered features like Pseudo Amino Acid Composition (PseAAC).⁵⁵ PeptideCLM-2 surpassed this baseline (MCC 0.732) using only raw SMILES strings (Fig. 4B), demonstrating that the model effectively learns motif-driven features without explicit feature engineering.

Cell Penetration:

We next evaluated the model on CellPPD-Mod,⁵⁶ a dataset explicitly containing chemically modified peptides. Standard sequence models fail here, forcing previous baselines to rely on extracted 2D/3D chemical descriptors (e.g., PaDEL) for the side-chain modifications. The use of PeptideCLM-2 allows for encoding of the entire molecule in one pass, and achieved an MCC of 0.875, outperforming the descriptor-based baselines (≈ 0.85) while operating directly on the chemical syntax (Fig. 4D).

Antimicrobial Activity:

Finally, we tested the model on a rigorous benchmark by He et al.,⁵⁷ designed to challenge models by training on natural peptide sequences and making predictions on peptides containing noncanonical amino acids. The specialized baseline, AmpHGT, employs a complex Heterogeneous Graph Transformer to explicitly model atoms and fragments, achieving an MCC of 0.797. PeptideCLM-2 surpassed this graph-based architecture with an MCC of 0.813 (Fig. 4F). This result shows that a SMILES-based transformer can capture the intrinsic chemistry of noncanonical residues as effectively as explicit molecular graphs.

Capacity-dependent prediction of non-linear biophysical properties evaluated via peptide half-life and aggregation propensity

Peptide stability dictates therapeutic viability through two primary mechanisms: resistance to enzymatic degradation in circulation and physical resistance to aggregation during storage. We evaluated PeptideCLM-2 on both fronts to determine if learned chemical syntax captures these complex biophysical properties.

To assess enzymatic stability, we utilized the recently curated PepMSND dataset,⁵⁸ a benchmark of peptide half-life across 635 diverse samples in varying blood environments. The published modeling approach for this dataset was a multimodal ensemble combining 0D physicochemical descriptors, 1D sequences, 2D molecular graphs, and 3D structural conformations. They include a Kalmogorov-Arnold Network (KAN)⁵⁹ to process molecular descriptors, which, in ablation studies, had the greatest impact to predictive capability.

Visualizing the pretrained PeptideCLM-2 embeddings for the PepMSND dataset reveals that the model innately organizes the latent space by stability class without requiring explicit spatial or physical inputs (Fig. 5A). There are several clusters containing a single class (stable vs unstable), and upon finetuning, our single string-based architecture accurately captures the determinants of degradation. PeptideCLM-2 outcompetes the multimodal baseline without the KAN network (Fig. 5B), demonstrating that large-scale representation learning can successfully replace complex, multi-level feature engineering for blood stability prediction. Additionally, combining PeptideCLM-2 with the same KAN network used with PepMSND, applying their same method of combining output features into a joint regression model, improves predictive capability above the PepMSND+KAN baseline.

Figure 5: Scaling model size in predicting peptide stability.

(A) t-SNE projection of PeptideCLM-2 MLM model embeddings for all 10 clusters from PepMSND, colored by either passing or failing stability checkpoint. (B) Boxplot of PeptideCLM-2 predictions on binary stability for 10 folds from PepMSND splits, colored by model type. KAN represents a Kolmogorov-Arnold Network trained on molecular descriptors, as used in the PepMSND model. Grey dashed line is the baseline model score from the original publication. t-SNE visualizations of the peptide fibrillation data show significant overlap between non-aggregating (C) and aggregating (D) peptides, illustrating the difficulty of linearly separating these classes based on structure alone. (E) Density plots of model predictions. Traditional molecular fingerprints (Morgan FP) fail to distinguish the two populations, performing near random chance (AUROC 0.579). In contrast, PeptideCLM-2 exhibits increasing accuracy with model size, reaching an AUROC of 0.823 at 337M parameters. Curves are scaled independently for visibility.

Beyond enzymatic degradation, physical aggregation represents a critical failure mode driven by condition-dependent interactions between lipids, excipients, and secondary structures. To evaluate this, we utilized a large proprietary dataset of Thioflavin T (ThT) fluorescence assays. This benchmark encompasses engineered macrocycles and endogenous peptides containing diverse protraction moieties measured across varying pH conditions. We appended the pH value to the embedding prior to the regression head, allowing the network to learn environment-dependent stability.

The inherent difficulty of predicting fibrillation is visible in the latent space. Pretrained embeddings show extensive overlap between aggregating and non-aggregating peptides (Fig. 5C, D), indicating that simple structural clustering cannot separate the classes. Consequently, a Random Forest trained on Morgan Fingerprints fails completely, yielding an area under the receiver operating characteristic curve (AUROC) of 0.579.

In contrast, PeptideCLM-2 demonstrates a dramatic recovery of predictive power that scales directly with model capacity (Fig. 5E). While the 32M parameter model provided a baseline discrimination (AUROC 0.694), scaling to the 114M and 337M architectures improved performance to 0.751 and 0.823, respectively. This distinct scaling trajectory confirms that large chemical language models can capture the subtle, non-linear biophysical drivers of aggregation that remain invisible to static chemical fingerprints.

Summary of predictive performance

Our comprehensive evaluation of PeptideCLM-2 across six distinct datasets demonstrated its superiority when compared to existing computational baselines. Notably, all of these datasets feature peptides incorporating noncanonical amino acids and complex chemical modifications that traditional protein language models cannot process. As summarized in Table 1, the 337M parameter model achieves state-of-the-art performance on each task, consistently outperforming specialized deep learning architectures and standard chemical descriptors.

Table 1:

Performance comparison of PeptideCLM-2 (337M) against benchmark baselines. AUROC: area under the receiver operating characteristic curve; MCC: Matthews correlation coefficient. RF denotes a Random Forest classifier trained on the specified descriptors.

Task (Dataset)	Metric	Alternative Model	Score	PeptideCLM-2
Membrane Permeability	AUROC	PeptideCLM42	0.781	0.830
Tumor Homing	MCC	THPep55	0.710	0.732
Cell Penetration	MCC	RF + PaDEL56	0.850	0.875
Antimicrobial Activity	MCC	AmpHGT57	0.797	0.813
Blood Stability	MCC	PepMSND Multimodal58	0.537	0.609
Fibrillation Propensity	AUROC	RF + Morgan FP	0.579	0.823

Discussion

The development of therapeutic peptides has long been hindered by a representational dilemma. While machine learning has revolutionized protein engineering, peptides remain stranded between two paradigms: protein language models cannot handle noncanonical chemistry, while standard molecular fingerprints lack the contextual depth to model long biopolymers. Our initial foray into this space, PeptideCLM,⁴² demonstrated the feasibility of SMILES-based modeling for cyclic peptides. In this work, we expand this concept into a comprehensive framework, PeptideCLM-2, incorporating significant advances in data scale, architectural depth, and rigorous benchmarking that demonstrates that these transformer-based encoders consistently outperform classical fingerprint embeddings.

Our analysis identifies two critical drivers for effective peptide modeling. First, we observe a parameter scaling effect where larger models move beyond simple pattern recognition to spontaneously learn fundamental chemical rules from SMILES notation alone. Second, we introduce a novel k-mer tokenizer to process these complex molecules efficiently. This innovation allows PeptideCLM-2 to analyze intricate structures that were previously too computationally demanding, resolving the conflict between deep chemical accuracy and processing speed.

Large-scale transformers spontaneously derive physical rules from chemical syntax alone

Our results demonstrate that the optimal pretraining strategy is strictly a function of model capacity. At the 32M parameter scale, the model lacks the representational capacity to infer thermodynamic laws from raw syntax alone. In this regime, explicit physicochemical supervision provides a decisive advantage, acting as a scaffold that guides the model toward a structured embedding space where permeability can be accurately predicted.

However, this advantage vanishes at scale. The convergence of self-supervised and descriptor-guided models at 337M parameters confirms that sufficiently large transformers outperform chemical features when trained only on token co-occurrence. This finding offers a mechanistic explanation for prior observations in the field, such as the reported superiority of descriptor-guided training in ChemBERTa-2.³⁷ We posit that ChemBERTa-2, being a relatively lightweight architecture, operated on the lower end of this scaling curve, where inductive bias is still beneficial, while MoLFormer,³⁴ trained purely on SMILES, operated on the upper end of this curve. PeptideCLM-2 demonstrates that with sufficient depth, the model transcends a dependence on supervised learning, developing learned representations from sequence alone that are functionally equivalent.

String-based architectures resolve geometric bias and enable functional prediction for diverse chemistries

Our reliance on SMILES strings is a deliberate design choice to address the limitations of geometric deep learning in this domain. Unlike globular proteins, which fold into stable tertiary structures, therapeutic peptides are often intrinsically disordered or exist as dynamic ensembles. Current 3D methods force these labile molecules into a single static conformation, introducing a geometric bias that misrepresents their behavior in solution. By operating on SMILES, PeptideCLM-2 captures precise chemical connectivity while implicitly allowing the model to learn representations that account for conformational flexibility. This is supported by our results in membrane permeability, where the model successfully inferred 3D-dependent constraints directly from the 1D syntax without requiring explicit structural input.

Notably, PeptideCLM-2 generalizes beyond simple physicochemical descriptors to predict complex biological phenotypes. This capability stems from the fundamental advantage of SMILES-based architectures: the ability to represent a residue not as a fixed label, but as a composite of its constituent chemical parts. While protein language models are structurally limited to the 20 canonical amino acids, our k-mer tokenization strategy allows PeptideCLM-2 to efficiently process the chemical diversity that defines modern peptide therapeutics, natively encoding cyclizations, stereochemical inversions, and diverse chemical modifications. Additionally, this tokenization method allows the model to learn from small molecule chemistry during pretraining and apply that knowledge on downstream tasks.

This structural flexibility not only enabled simple encoding, but also resulted in improved performance over custom architectures and molecular fingerprints on benchmarks such as AMP and CellPPD-Mod. The model also has demonstrated chemical generalization when successfully predicting the antimicrobial activity of peptides containing noncanonical residues that were held out during training. The finding that embeddings pretrained on chemical syntax (SMILES reconstruction) transfer effectively to functional tasks implies that the chemical features necessary to generalize across molecules are present in the grammar of chemical language.

Limitations and future directions

While PeptideCLM-2 establishes a new standard for discriminative modeling, several frontiers remain. First, peptide aggregation is inherently dynamic. While our static embeddings predict propensity effectively, capturing true fibrillation kinetics may require training on molecular dynamics trajectories to explicitly model conformational flexibility. Second, our reliance on SMILES strings prioritizes topological connectivity over explicit 3D geometry. While the model successfully infers 3D constraints—as evidenced by its permeability predictions—integrating geometric deep learning could further refine performance on highly structure-dependent targets.

As high-throughput screening⁶⁰ and advanced synthesis⁶¹ continue to generate large-scale datasets, extending this architecture to the billion-parameter scale promises even greater predictive fidelity. Coupling these predictive oracles with generative frameworks like diffusion models⁶² brings us closer to de novo design of noncanonical peptides with precise, multi-parametric profiles.

Conclusion

We present PeptideCLM-2 as an open, scalable resource designed to accelerate the transition from empirical screening to rational engineering in peptide drug discovery. By successfully resolving the representational trade-off between semantic depth and computational tractability, our framework empowers the community to design the next generation of stable, potent, and chemically diverse therapeutics. To foster reproducibility and catalyze further innovation, we release all model weights, tokenizers, and training datasets to the public.

Methods

Dataset curation and preprocessing

To construct a chemically diverse pretraining corpus bridging the gap between small molecules and proteins, we curated data from three primary sources: PubChem,⁵⁰ ESMAtlas,²⁰ and the LIPID MAPS Structure Database (LMSD).⁴⁹

Small molecules:

We retrieved the complete compound set from PubChem and applied a rigorous filtering cascade to remove non-drug-like entities. Entries were excluded if they were shorter than 20 characters or contained silicon chains. We further removed salts (leading/trailing Br and Cl) and disconnected components (e.g., solvent molecules indicated by ‘.’ within brackets). Polymeric artifacts, specifically repeating silicon oxide motifs (e.g., [Si](=O)[Si](=O)), were identified and discarded. After splitting remaining disconnected components into independent entries and deduplicating the dataset, the final small-molecule corpus contained 108,583,157 unique SMILES strings.

Peptides:

Peptide sequences were sourced from ESMAtlas. To ensure high-quality structural priors, we filtered for sequences with high predicted confidence (pTM > 0.7, pLDDT > 0.7) and lengths ≤ 100 amino acids. To reduce redundancy, sequences were clustered at 30% identity using MMseqs2 (--min-seq-id 0.3 -c 0.8 --cov-mode 1), retaining the centroid with the highest product of pTM and pLDDT as the cluster representative. Amino acid sequences were converted to SMILES strings using p2smi.⁶³

Lipids:

All available lipid structures were obtained directly from LMSD.

Data Balancing:

To prevent model collapse into the dominant small-molecule modality, we employed a balanced sampling strategy during training. Each epoch consisted of an upsampled lipid set (250K), the full peptide set (~10M), and a random downsampled subset of small molecules (10M), ensuring the model encountered a heterogeneous distribution of chemical syntax. All molecules were canonicalized to standard SMILES format using RDKit; entries failing conversion were discarded.

K-mer SMILES tokenization

Standard atom-level tokenization often results in excessively long sequences for peptide biopolymers, increasing the computational cost of self-attention $\left(𝒪\left(n^2\right)\right)$. To mitigate this, we developed a custom k-mer tokenizer based on the concepts from SMILES Pair Encoding⁶⁴ that compresses peptide SMILES while preserving chemical semantics.

We first constructed a pre-tokenizer that segments SMILES into atom-level primitives, preserving multi-character elements (e.g., ”Br”, ”Cl”) and stereochemical/charge brackets (e.g., [C@@H], [N+]). We then mined the most frequent contiguous k-mers (up to length 6) from a reference corpus of 200,000 PubChem small molecules and 200,000 SmProt peptides.⁶⁵ This candidate list was filtered based on chemical validity and frequency (Tab. S2), yielding a compact vocabulary of 405 tokens (160 single-atom, 245 k-mer). This vocabulary reduces sequence length by approximately 60% compared to character-level encoding, enabling efficient training on longer peptide chains.

Model architecture and pretraining

Architecture

PeptideCLM-2 models are trained as BERT-style transformer encoders.⁴¹ We instantiated three model scales:

Small (32M):

6 layers, 384 hidden dimension, 6 heads.

Base (114M):

12 layers, 768 hidden dimension, 12 heads.

Large (337M):

24 layers, 1024 hidden dimension, 16 heads.

All models utilize Rotary Positional Embeddings (RoPE) to better capture relative distances in chemical space, along with SwiGLU activation functions and pre-layer normalization for training stability. Full hyperparameters can be found in Table S1.

Masking distributions

To implement span masking, we determined span lengths using a Gaussian distribution ($\mu =3.5,\sigma =1.0$) with a minimum length of one token. The total number of tokens to be masked was calculated per sequence based on a specified masking percentage. For each span, a random starting position was selected, with adjustments made to prevent boundary overruns or overlaps with previously masked regions; selected positions were then replaced with the mask token ID until the target masking budget was satisfied.

Training objectives

We employed a hybrid objective function combining self-supervision with property regression:

$$ℒ={\lambda }_{\mathrm{M}\mathrm{L}\mathrm{M}}\cdot {ℒ}_{\mathrm{M}\mathrm{L}\mathrm{M}}+{\lambda }_{\mathrm{M}\mathrm{T}\mathrm{R}}\cdot {ℒ}_{\mathrm{M}\mathrm{T}\mathrm{R}}$$

Masked language modeling (MLM):

We applied a 25% masking rate using a span-masking strategy to encourage the reconstruction of chemical fragments rather than trivial atoms. Span lengths were sampled from a Gaussian distribution ($\mu =3.5,\sigma =1$), with each masked position replaced by the [MASK] token.

Multi-task regression (MTR):

In parallel, a regression head predicted 99 physicochemical descriptors (computed via RDKit) from the mean-pooled sequence embedding. The MTR head consists of two fully connected layers with SiLU activation.⁴⁶ Descriptors were normalized to zero mean and unit variance prior to training.

For the combined objective, we set ${\lambda }_{\mathrm{M}\mathrm{L}\mathrm{M}}=0.6$ and ${\lambda }_{\mathrm{M}\mathrm{T}\mathrm{R}}=0.4$. To enforce structural invariance, we applied dynamic randomization of SMILES strings during training,⁶⁶ ensuring the model sees different valid string representations of the same molecule.

Optimization

Models were trained using the AdamW optimizer (β₁ = 0.9,β₂ = 0.98, weight decay 0.01). Pretraining was performed on 8× NVIDIA H100 GPUs with a global batch size of 512 for 3 pseudo-epochs. Each pseudo-epoch consisted of a sample of ~20.25M molecules, comprising 10M small molecules, the full ESMAtlas, and a 5× upsampling of the LMSD. The learning rate followed a linear warmup for the first 5,000 steps to a peak of 3 × 10⁻⁴, followed by cosine annealing to 10% of the peak rate.

Downstream evaluation protocols

Finetuning strategy

For all downstream tasks, we replaced the pretraining heads with a task-specific feed-forward network consisting of two hidden layers with GeLU activation and dropout (p = 0.1). Optimization was performed using AdamW with a learning rate of 1 × 10⁻⁵ and a batch size of 16 unless otherwise noted.

Train/test splitting was strictly replicated from the benchmark literature for each dataset to ensure fair comparison. For aggregation prediction, we performed stratified 5-fold cross-validation with random splits.

Models were trained until validation loss plateaued (patience of 5 epochs, checking validation every 0.5 epoch). To improve robustness, final predictions are reported as the ensemble mean of models trained on the inner cross-validation folds.

Classification Baselines

As a primary baseline in classification tasks, we computed RDKit topological fingerprints (2048-bit, path length 1–7, 2 bits/hash)⁴⁴ and trained Random Forest classifiers using scikit-learn.⁶⁷ These models utilized 100 estimators (n_estimators=100) with the standard Gini impurity criterion. Tree depth was unconstrained, allowing nodes to expand until all leaves were pure. For specific tasks, we compared against published specialized architectures including AmpHGT⁵⁷ and THPep.⁵⁵

Layer-wise transfer learning, CycPeptMPDB

To assess the intrinsic quality of learned representations without weight updates, we froze the pretrained encoder and extracted embeddings from each layer. These embeddings were mean-pooled over the sequence length and passed to a LassoCV regressor (5-fold cross-validation). We allowed the model to automatically select the regularization parameter ($\alpha$) over a path of 100 values with a maximum of 10,000 iterations.

Supplementary Material

Supporting Information Available

Supplementary tables and figures are included in this publication.

Data and Software Availability

All code for pretraining, finetuning, and data processing for this project has been made available at https://github.com/AaronFeller/PeptideCLM-2. Model code and weights for all models are released on huggingface at https://huggingface.co/collections/aaronfeller/peptideclm-2. Pretraining data has been released on Zenodo (doi:10.5281/zenodo.17993164) and on Huggingface datasets in the PeptideCLM-2 collection for ease of use. For advancement of modeling therapeutic peptides with noncanonical chemistry, the datasets used for downstream tasks including for membrane permeability, tumor homing, cell penetration, antimicrobial activity, and blood stability have been organized and can be found on the project github.