Identifying developability risks for clinical progression of antibodies using high-throughput in vitro and in silico approaches

ABSTRACT

With the growing significance of antibodies as a therapeutic class, identifying developability risks early during development is of paramount importance. Several high-throughput in vitro assays and in silico approaches have been proposed to de-risk antibodies during early stages of the discovery process. In this review, we have compiled and collectively analyzed published experimental assessments and computational metrics for clinical antibodies. We show that flags assigned based on in vitro measurements of polyspecificity and hydrophobicity are more predictive of clinical progression than their in silico counterparts. Additionally, we assessed the performance of published models for developability predictions on molecules not used during model training. We find that generalization to data outside of those used for training remains a challenge for models. Finally, we highlight the challenges of reproducibility in computed metrics arising from differences in homology modeling, in vitro assessments relying on complex reagents, as well as curation of experimental data often used to assess the utility of high-throughput approaches. We end with a recommendation to enable assay reproducibility by inclusion of controls with disclosed sequences, as well as sharing of structural models to enable the critical assessment and improvement of in silico predictions.

Introduction

The study of the developability of antibodies has been an active area of research in recent years. For example, focusing exclusively on the top journals specialized in antibody research (mAbs, Antibodies, and Antibody Therapeutics), we observe that 6% of all published articles since the start of 2018 to November of 2022 include the term “developability” in the title or abstract, with percentages from individual journals ranging from 4% to close to 8%. It follows that the field has also been reviewed extensively, and we refer the reader to three recent comprehensive examples.^1–4 As has been often stated, while a successful antibody drug must first satisfy biological requirements of target choice and potency, it also needs to have favorable biophysical properties and chemical stability, among other attributes. The set of such properties is commonly referred to as “developability.”

Here, we focus on the results of studies involving a substantial number of antibodies samples, and, preferably, those including antibody sequence information or other data (e.g., antibody international nonproprietary name designations) that link to sequences. The studies we examined in detail include reports of experimental measurements, as well as accounts of computational methods aimed at predicting specific biophysical attributes or aimed at discriminating drug-like antibodies from others that may not possess such qualities. As indicated above, the focus is mainly on the results, with less attention being paid to the details of the methodology, experimental or computational, used in the different studies.

The Results section is organized into five parts. First, we present a comparative review of in vitro metrics reported, primarily, for sets of antibody samples generated from variable region sequences corresponding to clinical-stage molecules.⁵ Second, we review a selection of studies using in silico assessments for similar as well as other antibody molecules. Third, we evaluate the ability of in vitro assay-based metrics to correlate with progression in the clinic of the associated antibody molecules. Next, we look critically at the ability of in silico metrics to predict extreme, usually undesirable, behaviors of the in vitro measurements. Lastly, we examine issues of reproducibility from study to study for both computational and experimental assessments with nominally the same antibodies.

Results

In vitro measurements

In early 2017, we published a study⁵ in which we generated 137 IgG1 samples based on sequences of antibodies that had gained approval or had reached Phase 2 or Phase 3 clinical trials at any time during their clinical development. For each sample, 12 assays assessing biophysical properties were carried out. Part of the analysis in this work included estimation of 90% thresholds for 10 of the 12 assays using the readouts for samples corresponding to approved antibodies at that time. Tables 1 and 2 summarize the updated clinical status, as of late 2022, for the 137 antibodies. Sixteen additional antibodies in this set have since been approved, bringing the total number of approvals to 64. We additionally annotated 55 mAbs as having their development terminated at any point, including prior to 2017, or moved down a phase since 2017.

Table 1.

Clinical progression counts for the 137 mAbs with in vitro experimental data.

Active Status in 2022	Status in 2017
	Approved	Phase 3	Phase 2
Approved	48	14	2
Phase 3		4	3
Phase 2		6	9
Phase 1		1	2
Terminated		18	30

Status in 2017 annotation from prior publication⁵ considered the furthest progression in the clinic as of 2017. Status in 2022 annotation is current clinical status of molecules as of October 2022. Antibodies whose development was discontinued at any time, including prior to 2017, are considered as Terminated. the sets of mAbs above and below the main diagonal are collectively denoted as Progressed and Regressed, respectively.

Table 2.

Summary of in vitro assay clusters determined using hierarchical clustering following Spearman rank correlation analysis.

Assay	Cluster	Number	Type
Fe2+ FVIII	1	115	Induced polyreactivity6
Fe2+ C3	1	115
Fe2+ LysM	1	115
Heme FVIII	2	113	Induced polyreactivity7
Heme C3	2	113
Heme LysM	2	113
SGAC-SINS	3	137	Hydrophobicity5
HIC	3	137
SMAC	3	137
Tm	4	137	Thermal stability5
AS	5	137	Aggregation5
ELISA	6	137	Polyreactivity5
BVP	6	137
DNP	6	112	Proxy for polyreactivity
PSR	7	137	Polyreactivity5
CIC	7	137
ACSINS	7	137	Self-interaction5
CSI	7	137
FcRn Rel RT	7	130	FcRn interaction8
Hep RT	8	130	Proxy for pinocytosis8
Heme	8	113	Proxy for polyreactivity7
FA	8	113

The number of mAbs with experimental values and nature of biophysical interaction probed by the experiment are indicated.

Following the original publication, several additional in vitro studies have been performed on a subset of the 137 mAbs. These investigations have provided valuable data on neonatal Fc receptor (FcRn) column or heparin column retention times,⁸ polyreactivity to cofactors heme or folate,⁷ nitroarenes,⁹ induced polyreactivity on exposure to oxidative agents,⁶ polyreactivity to chaperone proteins¹⁰ or protein mixtures,¹¹ and induced aggregation on flow stress.¹²

We repeated the clustering analysis from our prior work using these additional in vitro measurements. To reduce bias arising from the choice of subsets of mAbs used in subsequent studies, we only used in vitro measurements where values were available for at least 100 mAbs. The resulting assays and number of measurements are summarized in Table 2. A compilation of all data and assay descriptions is provided in the supplementary information (Definitions and In vitro measurements). Hierarchical clustering following Spearman rank correlation calculation identified eight clusters. The original cluster containing hydrophobic interaction chromatography (HIC), standup monolayer adsorption chromatography (SMAC) and salt-gradient affinity-capture self-interaction nanoparticle spectroscopy (SGAC-SINS) assays remained unchanged, as did the accelerated stability (AS) assay, which shows low correlation to others. The retention time on the FcRn column was highly correlated with the affinity-capture self-interaction nanoparticle spectroscopy (AC-SINS), polyspecificity reagent (PSR), clone self-interaction using bio-layer interferometry (CSI) and cross-interaction chromatography (CIC) assays, which constituted one of the polyspecificity clusters in our prior work. Binding to 2,4-dinitrophenol (DNP) correlated with the prior identified polyspecificity cluster of binding to baculovirus particles (BVP) and the average binding in enzyme-linked immunosorbent assay (ELISA) to insulin, ssDNA, dsDNA, Keyhole limpet hemocyanin (KLH), lipopolysaccharide (LPS) and cardiolipin.¹³ Heparin chromatography retention time and heme or folate binding measurements were correlated and constituted a new cluster. Finally, separate clusters were also identified for induced polyspecificity to FVIII, C3 and LysM proteins on exposure to heme and Fe²⁺. Table 2 summarizes the cluster assignments. Figure 1 shows the hierarchical clustering dendrogram, and the correlation matrix arising from this analysis. While considered as separate clusters, we note that the polyspecificity clusters 6 and 7, and cluster 8 show higher mutual correlation than with other clusters.

Figure 1.

Spearman rank correlation analysis for the set of in vitro assays with measurements for 100 or more mAbs. A. Hierarchical clustering with average linkage used to merge clusters. B. Matrix of calculated correlations between the in vitro measurements. The red boxes and black rectangles outline the individual assay clusters. The eccentricity of the ellipses is proportional to the magnitude of the correlation coefficient. The slope of the major axis has the same sign as the correlation coefficient. The numeric suffixes in the names map to literature sources for the data (‘Definitions’ in the supplementary information).

The updated 90% thresholds for the in vitro measurements are listed in Table 3 alongside the original thresholds estimated from the 48 approved mAbs as of 2017; this was carried out for the original 10 assays examined in this way in 2017, as well as for additional assay results published since using similar sets of antibody samples. The updated thresholds are close and within error of the prior recommendations except for binding to DNP and heme. Based on an investigation of pharmacokinetics assessed in Tg32 h-FcRn mice,¹⁴ cutoffs of 11 for ACSINS wavelength shift, and 1.6 min for FcRn retention time (RT) (which corresponds to 1.1 FcRn relative RT⁸ from Figure 10 C) to identify mAbs with fast clearance were recommended. Additionally, using human clearance data for a set of 64 mAbs, a threshold of 0.35 for PSR score was proposed to identify mAbs with fast clearance.¹⁶ These independent threshold or cutoff recommendations, based on pharmacokinetics, are close to the ones estimated in this study solely from the measurements on the Approved set, highlighting the promise of using early in vitro polyspecificity screening to de-risk for poor pharmacokinetics.

Table 3.

Determination of 90% thresholds for the approved mAbs within the set of 137 mAbs with experimental data.

Assay	Approved – 2017	Number – 2017	Approved – 2022	Number – 2022
Fe2+ FVIII	6.68 ± 0.67	34	6.67 ± 0.55	50
Fe2+ C3	4.67 ± 1.03	34	4.69 ± 0.62	50
Fe2+ LysM	8.28 ± 0.72	34	8.27 ± 0.49	50
Heme FVIII	6.06 ± 0.79	32	5.87 ± 0.72	48
Heme C3	7.40 ± 0.88	32	7.36 ± 0.78	48
Heme LysM	10.84 ± 2.14	32	10.10 ± 1.36	48
SGAC-SINS	370 ± 136	48	330 ± 128	64
HIC	11.63 ± 0.41	48	11.53 ± 0.27	64
SMAC	12.82 ± 1.20	48	12.62 ± 1.34	64
Tm	64.9 ± 0.93	48	64.2 ± 0.81	64
AS	0.08 ± 0.03	48	0.09 ± 0.04	64
ELISA	1.88 ± 1.03	48	2.06 ± 1.00	64
BVP	4.28 ± 2.22	48	5.30 ± 1.92	64
DNP	0.61 ± 0.25	32	1.24 ± 0.57	48
PSR	0.26 ± 0.06	48	0.31 ± 0.05	64
CIC	10.09 ± 0.46	48	10.09 ± 0.39	64
ACSINS	11.78 ± 6.23	48	13.84 ± 4.8	64
CSI	0.01 ± 0.02	48	0.02 ± 0.02	64
FcRn Rel RT	1.04 ± 0.14	44	1.18 ± 0.11	62
Hep RT	0.81 ± 0.06	44	0.84 ± 0.05	60
Heme	6.66 ± 1.89	32	10.73 ± 3.50	48
FA	2.90 ± 1.20	32	3.52 ± 1.46	48

Figure 10.

Comparison of in vitro measurements between independent assessments. A. Binding to CHO-derived polyspecificity reagent on yeast-presented mAbs⁵ vs. mAbs captured on ProA beads.¹¹ B. Aggregation rate in HBS pH 7.3 40 ºC mAbs⁵ vs. His-HCl pH 6.0 40 ºC¹⁵ C. Self-interaction measurements using the ACSINS assay in Jain et al. vs. Avery et al. D. FcRn column retention time in Kraft et al. vs. Avery et al. E. HIC column retention time in Jain et al. vs Fekete et al. The numeric suffixes in the names map to literature sources for the data (‘Definitions’ in the supplementary information).

In silico assessments

In addition to high-throughput experimental assessment, efforts have been made to develop predictive tools for providing quantitative estimates of developability and in silico boundaries to identify drug-like mAbs. While a detailed review of these methods is beyond the scope of this work, we collected published values for in silico descriptors for clinical and approved mAbs from several studies.^16–19 Table 4 lists the descriptors used for further analysis in this study. Using the methodology described earlier, we identified 15 clusters reflecting size, patches and distributions of hydrophobicity, overall charge, charge asymmetry, patches of positive and negative charges, and buried surface area at the variable heavy chain (VH): variable light chain (VL) interface. Figure 4 shows the resulting hierarchical clustering dendrogram, and the correlation matrix for the in silico descriptors. While most descriptors require 3D structure, the calculation of isoelectric point and net charge requires only the sequence. We note that the biophysical rationale of the in silico metrics is consistent with the obtained clusters. For example, a high correlation is seen for the estimated negative charge patch metrics calculated using different algorithms. The overall charge metrics and isoelectric point calculations have high mutual correlation as expected, but are also additionally correlated to the SFvCSP metric that is the product of exposed non-salt bridged VH and VL charges at pH 7.4. Since VH and VL are typically positively charged at pH 7.4, this relationship is to be expected even though the metric only considers the subset of exposed amino acids that are not participating in a salt bridge. The charge asymmetry metrics such as the dipole moment and Fv_chml (VH minus VL charge at pH 7.4) show higher mutual correlation than with other charge and positive or negative patch descriptors. The hydrophobicity descriptors show greater disagreement with Avg_HI and HI_sum being singleton clusters that are distinct from other measures. The Therapeutic Antibody Profiler¹⁸ (TAP) recommended patches of surface hydrophobicity (PSH) metric shows higher correlation to the solvent-accessibility weighted hydrophobicity score (asa_hyd) than the patch-based calculations.

Table 4.

Summary of in silico descriptor groups determined using hierarchical clustering following Spearman rank correlation analysis.

In silico metric	Cluster	Number	Type	Reference
HI_sum.12	1	64	Hydrophobicity	16
CP:HP.14 (pH 7.0)	2	137	Ratio of charge to hydrophobicity	19
CDR.PNC.13 (pH 7.4)	3	137	Negative charge patches	18
CDR PNC.12 (pH 7.4)	3	64		16
ion.12 (pH 7.4)	3	64	Negative and positive charge patches
cdr_ion.12 (pH 7.4)	3	64
neg.12 (pH 7.4)	3	64	Negative charge patches
cdr_neg.12 (pH 7.4)	3	64
cdr_neg.11 (pH 7.4)	3	137		17
Avg_HI.14	4	137	Hydrophobicity	19
r_gyr.12	5	64	Size	16
asa_hph.12	6	64	Hydrophilicity
asa_vdw.12	6	64	VdW interactions
CDR.Length.13	6	137	Size	18
r_solv.12	6	64		16
mass.12	6	64
volume.12	6	64
cdr_hyd.11	7	137	Hydrophobicity	17
cdr_hyd.12	7	64		16
hyd.12	7	64		16
CDR.PSH.13 (pH 7.4)	8	137	Hydrophobicity
CDR.PSH.12 (pH 7.4)	8	64		16
asa_hyd.12	8	64
hyd_moment.12	9	64	Hydrophobicity	16
CDR.PPC.12 (pH 7.4)	10	64	Positive charge patches	16
CDR.PPC.13 (pH 7.4)	10	137		18
SFvCSP.13 (pH 7.4)	11	137	VH:VL charge asymmetry
SFvCSP.12 (pH 7.4)	11	64		16
fvcharge5.5.12 (pH 5.5)	11	64	Charge
ens_charge_Fv.11 (pH 7.0)	11	137		17
pI_seq.12	11	64		16
pro_pI_3D.14	11	137
pI_3D.12	11	64		16
mobility.12	11	64
net_charge.12 (pH 7.0)	11	64
pos.12 (pH 7.4)	12	64	Positive charge patches	16
cdr_pos.11 (pH 7.4)	12	137		17
cdr_pos.12 (pH 7.4)	12	64		16
DM.HM.14 (pH 7.0)	13	137	Ratio of dipole to hydrophobicity	19
BSA_HCLC.14	14	137	VH:VL buried surface area	19
Fv_chml.11 (pH 7.0)	15	137	VH:VL charge asymmetry	17
dipole_moment.12	15	64	Overall charge asymmetry	16

The number of mAbs with calculated values and molecular property assessed by the descriptor are indicated. The numeric suffixes in the names map uniquely to literature sources for the values (‘Definitions’ in the supplementary information).

Figure 4.

Spearman rank correlation analysis for the set of in silico descriptors with values for 60 or more mAbs. A. Hierarchical clustering with average linkage used to merge clusters. B. Matrix of calculated correlations between the in silico measurements. The red boxes and black rectangles outline the individual descriptor clusters. The numeric suffixes in the names map to literature sources for the data (‘Definitions’ in the supplementary information)..

Beyond the calculation of individual values, strategies have been proposed to flag mAbs based on the number of violations of these descriptors.^17–19 The general principle is to determine the distribution of these descriptors for a reference set of clinical, approved, and representative human repertoire mAbs, and flag those that lie at the extremes, or tails, of these distributions. Surface-area weighted compositional rules²⁰ to classify specific and nonspecific mAbs were proposed from an analysis of antibodies with experimental measurements of polyspecificity.^5,20,21 A violation of eight out of 12 proposed rules was recommended as the threshold to flag nonspecific mAbs. Predictions from sequence for the 12 biophysical assays in our original work were made using Abpred²² for the expanded set of clinical mAbs and sequences isolated from B cells with 349 experimental HIC²¹ and 967 polyspecificity measurements.^20,21 We collected or recalculated the flag assignments based on the published guidelines, supplementary information from individual studies, or from a local installation of software, as detailed in Materials and Methods. Finally, we calculated the predicted HIC RT for 64 mAbs using the model²³ developed using patch descriptors.¹⁶ The supplementary information provided with this work includes the flag assignments (In silico flags), descriptors (In silico descriptors), and predictions (In silico predictions, In silico predictions B-cell) from all methods described earlier.

Ability of flags to identify progression in the clinic

Considering the primary drivers of clinical success include biological function and unmet medical need, and the likely bias for members of this small set of advanced mAbs toward better developability, it is unlikely to expect a strong statistical significance based solely on developability characteristics. Nevertheless, in our original study, we found that the number of violations for assay clusters decreased with progression in the clinic leading to approval. Considering additional in vitro assay data, we extended our prior analysis using a modified flag assignment scheme as described in Materials and Methods. An assay violation is assigned to a mAb if its assay readout exceeds the corresponding 90% threshold. Instead of assigning a cluster violation if any single assay was flagged, we adopt an alternate approach where a mAb is assigned a cluster violation if it violates one or more individual assay flags in a cluster with three or fewer assays, and two or more assay flags in clusters with three or more assays. Since the DNP and heme polyreactivity assays did not have measurements for 25 of the 137 mAbs and showed a large change in the 90% threshold between the 2017 and 2022 approved mAbs, we did not include them for assigning a cluster violation. We sought to establish if the number of violations assigned based on the 2017 set of 48 approved mAbs was predictive of changes to the clinical progression of these antibodies. For this analysis, we focused on 73 mAbs annotated as having either Progressed or Regressed in the clinic. Figure 2 shows the fraction of mAbs with violations for the set of Progressed and Regressed mAbs. We notice the intriguing trend, albeit slight, for a higher proportion of mAbs with favorable properties in the Progressed set, especially for cluster 3 (HIC, SGAC-SINS, SMAC), cluster 6 (BVP, ELISA), and cluster 7 (PSR, CSI, ACSINS, FcRn, CIC). Figure 3 shows the distributions of the biophysical assessments for the Approved, Progressed and Regressed sets of mAbs. The mAbs in the Regressed set generally show larger tails in the unfavorable developability regime for the hydrophobicity and polyspecificity assay clusters, compared to the Approved and Progressed sets. However, we caution against strong interpretations given the small dataset size. Instead, combining these results with the current state of knowledge in the field, the overall thesis of using a set of high-throughput assays capturing distinct biophysical attributes as tools for early screening remains intact.

Figure 2.

Proportion of mAbs with in vitro cluster violations as a function of clinical progress. The order of the clusters follows from top left to bottom right in Figure 1. The number of individual assay violations determining the cluster violation is indicated in the text above each panel.

Figure 3.

Distribution of in vitro measurements as a function of clinical progress. The values for the 2017 Approved set (48 mAbs) are shown for reference. The blue and red crossbars indicate the median and 90% values for each distribution. The red crossbars for the Approved set are the thresholds used to assign assay violations.

In Table 5, we summarize the ability of the in silico and in vitro flagging rules to identify progression in the clinic. Since the studies were performed on differing subsets of mAbs and at distinct timepoints, which affects the clinical phase for a mAb, we adopted the following approach: annotations of Approved mAbs were taken from the supplementary information of the respective studies. The non-approved mAbs are denoted as the Clinical set. The assumption is that, even if not comprehensive, the collection of sequences in these studies is representative of the clinical landscape at the time of those investigations. Based on a list of approved mAbs as of October 2022, we then denote the subset of the Clinical set that has since been approved, as the Clinical to Approved set. For each of three sets, we calculate the number and proportion of mAbs with different thresholds for flags or violations. We argue that, since the utility for early assessments is to identify, or enrich for, mAbs becoming approved therapeutics, we expect that the Clinical to Approved set should have a higher proportion of non-flagged mAbs than the Clinical set. The TAP metrics show an increase from 72% to 78% for no violations between the Clinical to Clinical to Approved sets.

Table 5.

Summary of counts and proportions for mAbs without violations in Approved, Clinical, and Clinical to Approved sets for different studies.

Guidelines	Definition for no violation	Approved at time of publication		Clinical at time of publication		Clinical to Approved since publication
		% Without violations	Counts	% Without violations	Counts	% Without violations	Counts
Thorsteinson et al.−2021 4 rules	No flags	78	88/113	69	356/515	71	18/25
	≤1 flag	97	110/113	94	486/515	92	23/25
Raybould et al.−2019 5 rules	No flags	72	49/68	72	126/174	78	21/27
	≤1 flag	91	62/68	95	165/174	100	27/27
Ahmed et al.−2021 5 rules	No flags	75	57/76	65	174/269	56	10/32
	≤1 flag	95	72/76	95	255/269	94	30/32
Ahmed et al.−2021 Z-distance	Z-score >2.67	75	57/76	68	181/269	69	22/32
Zhang et al. – 2020 12 rules	≤7 flags	90	43/48	62	55/89	56	9/16
Jain et al.−2017	Cluster 7 PSR, CIC, ACSINS, CSI, FcRn ≤1 flag	83	40/48	71	63/89	81	13/16
	Cluster 6 BVP, ELISA No flags	88	42/48	64	57/89	81	13/16
	Cluster 3 HIC, SMAC, SGAC-SINS No flags	79	38/48	69	61/89	88	14/16
	Clusters 7 + 6 + 3 ≤1 flag	85	41/48	71	63/89	88	14/16

Values for assigned flags are from the supplementary information of the respective publications or were recalculated using published criteria.

For in silico flags proposed in the other studies, we observe little to no difference. However, we note that the TAP metrics were proposed based on Phase 2 and higher mAbs, while the other studies contain Phase 1 antibodies as well. Based on fewer rule violations⁸ observed in Phase 1 (28%) vs. Phase 2 + 3 (40%), it was argued by Ahmed et al.¹⁹ that newer antibodies entering the clinic have cleaner biophysical profiles due to greater attention being paid to developability. However, a similar analysis using the rules proposed by Thorsteinson et al.¹⁷ finds that 31% of antibodies have no flags in either Phase 1 or Phase 2 + 3. Additionally, since the in silico rules^2,3,20 have been determined on the entirety of the Approved and Clinical sets, an argument could be made that the in silico measures are more suited to filtering at the pre-clinical and early discovery stages of drug development, rather than clinical candidates. However, from an assessment of flags for 3,120 internal hit antibodies from Boehringer Ingelheim and 14,037 human sequences,¹⁹ the percent of flagged mAbs is very similar, though the distributions show some differences compared to the Approved mAbs. A similar conclusion was also reached by Raybould et al.,¹⁸ where all descriptors, except for the hydrophobic metric, were similar between the clinical and human representative set. The average of the in silico hydrophobicity descriptor for the human sequences was higher than the clinical set, which was attributed to Vλ germlines. While higher HIC RTs were also observed for mAbs from naïve B-cells compared to approved mAbs,²¹ the differences were attributed to the complementary determining region (CDR) H2 of heavy chain germline IGHV1–69, whereas no significant differences were seen between $Vk$ and Vλ subsets.

For the flags assigned using in vitro measurements, we find that, consistent with Figures 2 and 3, outliers in polyreactivity and hydrophobicity are depleted in the Clinical to Approved set. However, we note that experimental data is limited to 137 mAbs, while in silico assessments were done over a significantly larger and more current set of sequences. Additional experiments on a larger set of samples with known sequences would be needed to confirm these observations and will be pursued in a future study. In the absence of additional experimental measurements over a larger set of mAbs, we used predictions from Abpred²² for the set of 12 in vitro assays in our original work and assigned cluster violations as described in Materials and Methods. The results in Table 6 show that mAbs with predicted violations for cluster 6 (BVP + ELISA) are depleted in the Clinical to Approved compared to the Clinical set for all three studies. For the other assay clusters, the observed percentages show minor differences. We speculate that this result could be driven by the higher model quality (Table 1 in Ref. 21) for BVP and ELISA with R² of 0.355 and 0.383, respectively, compared to worse predictions for other polyspecificity assessments. Further experiments will be needed to confirm whether these predictions agree with in vitro assessments. Negron et al.²⁴ assessed the ability of 910 descriptors to discriminate between 4929 repertoire and 339 clinical antibodies. The final Therapeutic Antibody Developability Analysis (TA-DA) score contained contributions from framework aggregation scores that are driven by hydrophobic clusters of atoms, light chain CDR positive patch energy, overall atomic contact energy, and amino-acid penalties based on their relative enrichment in ordered vs. intrinsically disordered proteins. Since the TA-DA score and values of descriptors were not part of the publication, we could not include them in our analysis. However, even though the TA-DA score was determined based on its ability to discriminate clinical from repertoire antibodies, it was found to correlate best with PSR, BVP and ELISA scores, further highlighting the importance of polyspecificity as a key predictive developability metric for clinical progression.

Table 6.

Summary of counts and proportions for mAbs without violations in Approved, Clinical, and Clinical to Approved sets for different studies using predictions from Abpred.

Dataset	Definition for no violation	Approved at time of publication		Clinical at time of publication		Clinical to Approved since publication
		% Without violations	Counts	% Without violations	Counts	% Without violations	Counts
Thorsteinson et al.−2021	Cluster 7 PSR, CIC, ACSINS, CSI ≤1 flag	82	93/113	85	437/515	84	21/25
	Cluster 6 BVP, ELISA No flags	74	84/113	72	372/515	80	20/25
	Cluster 3 HIC, SMAC, SGAC-SINS No flags	81	92/113	82	423/515	56	14/25
	Cluster 7 + 6 + 3 ≤1 flag	80	90/113	77	398/515	72	18/25
Raybould et al.−2019	Cluster 7 PSR, CIC, ACSINS, CSI ≤1 flag	81	55/68	83	145/174	85%	23/27
	Cluster 6 BVP, ELISA No flags	78	54/68	70	121/174	74	20/27
	Cluster 3 HIC, SMAC, SGAC-SINS No flags	81	55/68	78	136/174	81	22/27
	Cluster 7 + 6 + 3 ≤1 flag	88	57/68	84	127/174	8	19/27
Ahmed et al.−2021	Cluster 7 PSR, CIC, ACSINS, CSI ≤1 flag	77	61/76	85	227/269	88	28/32
	Cluster 6 BVP, ELISA No flags	75	58/76	69	186/269	82	26/32
	Cluster 3 HIC, SMAC, SGAC-SINS No flags	77	60/76	79	212/269	82	26/32
	Clusters 7 + 6 + 3 ≤1 flag	84	62/76	85	200/269	91	25/32

Ability of in silico metrics to predict in vitro measurement flags

To identify which in silico metrics are predictive of outlying in vitro measurements, we performed a receiver operating characteristic (ROC) analysis and penalized logistic regression on the dataset. Only in silico metrics computed for 100 or more mAbs were used for further analysis. Figure 5 summarizes the area under the ROC (AUROC) curve obtained for the individual in silico descriptors for the in vitro measurements binarized on their exceeding the 90% threshold on the 48 Approved mAbs in 2017. The final two columns show the estimated AUROC for logistic regression using lasso and ridge penalties on coefficients for in silico descriptors. Figure 6 and Supplementary Figure S1 show the estimated coefficients for the lasso and ridge regressions, respectively. Supplementary Figure S2 shows the Spearman rank correlations between the in vitro measurements and the in silico descriptors.

Figure 5.

AUROC for discriminating mAbs with in vitro measurements exceeding the 90% threshold determined on 48 approved mAbs using in silico descriptors. Clustering for in silico descriptors was repeated for the subset with values for 100 or more mAbs. Columns Fit1 and Fit0 contain values obtained from cross-validation predictions using lasso and ridge regression, respectively. The numeric suffixes in the names map to literature sources for the data (‘Definitions’ in the supplementary information).

Figure 6.

Penalized logistic regression using glmnet for discriminating mAbs with in vitro measurements exceeding the 90% threshold (based on the 48 Approved 2017 mAbs) using in silico descriptors. Average coefficients from 10 repeats of 10-fold cross validation using the lasso regularization penalty are shown. The numeric suffixes in the names map to literature sources for the data (‘Definitions’ in the supplementary information).

We observe from the AUROCs and coefficient magnitudes that the in silico descriptors have greater predictive ability for HIC, SMAC, heparin RT, binding to folate, and Fe²⁺ induced polyreactivity, than for the other in vitro assays. Consistent with the underlying physics driving hydrophobicity, the cdr_hyd calculated in Thorsteinson et al.¹⁷ is identified as an important predictor for HIC and SMAC. Additionally, higher ratios for charge to hydrophobic patch, and clusters representing higher negative and positive charges, favor lower hydrophobicity as well. Several investigators have published models for predicting HIC RT directly from sequence^22,25–28 or from molecular mechanics calculations on crystal structures or homology models^23,25,26 with good predictive performance. The negative charge patch and charge asymmetry descriptors from the TAP metrics also show an impact on SGAC-SINS. We note that this assay is related to the ACSINS assay where these descriptors are deemed important as well.

Heparin RTs are positively correlated with the cluster of in silico metrics capturing overall higher charge, which is consistent with their use to mimic the negatively charged glycocalyx on endothelial cells.⁸ Similarly, binding to either folate or heme also shows a tendency to weaken with high overall charge. However, compared to binding to heme, increasing values of several hydrophobic descriptors correlate with lower binding to folate and heparin. This is also consistent with a negative correlation seen between the folate and heparin binding to HIC and SMAC measurements, and a compositional analysis⁷ that found increasing aromatic amino acids correlated with decreased binding to folate.

The prediction of heme-induced polyreactivity is weaker, though some hydrophobic descriptors are identified as important. We note that higher coefficients in both higher hydrophobic imbalance and higher ratio of hydrophobic to charge patches carries increased risk for this assay. The original study⁷ identified a positive correlation between Tyr in CDR H2 and H3 and heme-induced polyreactivity. However, the presence of other aliphatic amino acids, especially Ala and Leu, was found to be negatively correlated, possibly indicating a specific mechanism beyond overall hydrophobicity.

While the overall predictive performance for polyreactivity is weak, the AUROC values and estimated coefficients for the cluster consisting of PSR, CIC, ACSINS, CSI, and FcRn RT assays indicates that higher overall positive charges have a detrimental effect, while the presence of larger negative patches is beneficial. While the estimated coefficients are smaller than for the charge dominant metrics, higher hydrophobicity may also contribute to higher polyreactivity. Similarly, increased binding to BVP, ELISA and DNP also correlate to the presence of greater positive charge and decreased negative charge. These observations are consistent with several studies investigating polyreactivity as a function of the presence of motifs containing Gly, Arg, Val and Trp²⁹ and the enrichment of exposed positively charged, aliphatic and aromatic amino acids. Similarly, depletion of negatively charged and polar amino acids in 12 compositional rules,²⁰ high net CDR charge,³⁰ and highly basic mAbs³¹ have been associated with increased non-specificity as well.

For polyreactivity induced on exposure to Fe^2+,6 we observe that higher overall positive charge, lower net VH minus VL charge and higher product of VH:VL charges can drive higher in vitro readouts. This is consistent with the observation from amino acid composition analysis⁶ that higher counts of Arg and Lys, and lower counts of Glu in the light-chain framework correlated with higher induced polyreactivity.

In Figure 7, we assess the ability of HIC and PSR predictions from Abpred²² to identify mAbs exceeding the proposed 90% thresholds for a set of mAbs isolated from B cells. While the quantitative predictions show low correlations to experimental measurements (Supplementary Figure S4B and S5), we find that the AUROC for discriminating HIC and PSR outliers are 0.78 and 0.66–0.70, respectively. These results are consistent with the observations that in silico descriptors for hydrophobicity have stronger predictive ability for HIC RT than PSR, as shown in Figure 5 and Supplementary Figure S2.

Figure 7.

Identification of outliers for experimental PSR scores^20,21 and HIC RT²⁰ from Abpred²² predictions on sequences from B cells. ROC curves and areas are shown for two PSR thresholds, corresponding to the 90% thresholds for the 48 and 64 approved mAbs in 2017 and 2022, respectively. Only one curve is plotted for HIC, since they are identical for the two thresholds.

Reproducibility of assessments

While in silico assessments are valuable tools for early screening, challenges still lie in their transferability across organizations. Since a majority of in silico descriptors require 3D-structure, a meaningful difference can arise depending on the software used to generate homology models. As an example, we examined the correlations between the four TAP metrics from ABodyBuilder^18,32 and Antibody Modeler in MOE 2019.01¹⁶ homology models (Figure 8). The R² values range from 0.31 to 0.69 for 64 mAbs common to both studies. Differences can also arise in cases where homology models were generated by different versions and protocols of the same software. Figure 9 shows the correlations between the hydrophobic and charge patch descriptors calculated using Antibody Modeler in MOE 2020.0901¹⁷ and MOE 2019.01¹⁶ with R² values of 0.76 to 0.93. Differences in the methodology for charge assignment, and use of static structures versus ensemble averages can influence the values of patch descriptors. For example, typical calculations using MOE^16,17,19 include averaging over conformations generated using molecular dynamics with a sampling of alternate protonation states over a range of pH values using Protonate3D.³³ Differences in determination of overall protein charge can arise from use of PROPKA,^16,17,19,34 which considers the 3D structure of the protein compared to assignments based solely on the pKa of isolated residues as done for calculation of the TAP¹⁸ metrics.

Figure 8.

Comparison of four TAP descriptors reported in Grinshpun et al. (calculated from MOE homology models) on the x-axis vs Raybould et al. (calculated from ABodyBuilder homology models) on the y-axis. The numeric suffixes in the names map to literature sources for the data (‘Definitions’ in the supplementary information).

Figure 9.

Comparison of CDR hydrophobic, positively charged and negatively charged patch descriptors calculated using MOE 2020.09.01 in Thorsteinson et al. (x-axis) vs. MOE 2019.01 in Grinshpun et al. (y-axis). The numeric suffixes in the names map to literature sources for the data (‘Definitions’ in the supplementary information).

Similarly, in vitro measurements that rely on complex reagents and experimental setups can also be difficult to reproduce quantitatively, lessening the confidence in resulting guidelines and thresholds. Figure 10A shows a comparison of binding to Chinese hamster ovary cell membrane-derived polyspecificity reagent by the same mAb captured on beads using Protein A¹¹ versus presented on the surface of yeast.^5,35 A comparison of ACSINS wavelength shifts is shown in Figure 10B (right) from two studies.^5,14 In both cases, while there is high overlap between the outlier mAbs, the quantitative correlations are low. By contrast, Figure 10D shows that measurements of FcRn RT show a high correlation between two independent studies.^8,14 Similarly, Figure 10E demonstrates good agreement between two sets of HIC measurements.^5,36 In these cases of reproducible assays, the approach outlined in this study focusing on identifying outliers should be applicable after calibration of assays with respect to the original ones used to establish the thresholds.

Beyond the challenges in reproducing in vitro and in silico assessments, it is also challenging to curate high quality in vivo readouts to establish datasets which serve to evaluate the utility of high-throughput upstream metrics. Figure 11 shows a comparison of human clearance data from several studies.^14,16,37,38 Clear discrepancies are observed in most comparisons, even for mAbs showing fast clearance, which are precisely the ones that need to be identified during early screening using the in vitro assessments and in silico descriptors discussed above.

Figure 11.

Comparison of human clearance values for mAbs from different studies. A. Grinshpun et al. vs. Chung et al. B. Grinshpun et al. vs. Hu, Datta-mannan, et al. C. Grinshpun et al. vs. Avery et al. D. Avery et al. vs. Chung et al. The numeric suffixes in the names map to literature sources for the data (‘Definitions’ in the supplementary information).

Discussion

The goal of high-throughput in vitro and in silico assessments is to identify potential downstream risks that occur during manufacturing, undesired modifications and aggregation during long-term storage, poor solubility and high viscosity precluding formulation for subcutaneous administration, and poor pharmacokinetics and off-target interactions in vivo affecting the therapeutic objective. Early identification of mAbs that are developable using platform approaches³⁹ reduces risk and accelerates the time from discovery to clinic. It should be mentioned that developability assessments may be useful to prioritize candidates with a lower likelihood of presenting problems, which may lead to increased costs and or prolonged timelines. This factor, which may be termed “ease of development”, is different from a molecule having a higher chance of reaching approval. It is still a valuable and important motivation, especially in the context of a portfolio of clinical candidates, where it will result in more efficient allocation of resources. Challenges arising from poor solution behavior of clinical candidates can often be addressed during formulation without necessitating changes to the sequence. Omalizumab (Xolair) is an example of a mAb with high viscosity⁴⁰ that has been formulated for subcutaneous administration at 150 mg/ml. By contrast, polyreactivity and interactions with unintended targets that occur after administration are a function of the mAb sequence itself, which drives its therapeutic function.

High-throughput in vitro assays designed to be surrogates for known physiological mechanisms offer huge promise for de-risking at the screening stage^8,14,41,42 and as depletion reagents³⁵ during discovery and optimization as well. While quantitative agreement between different polyspecificity assays cannot be expected due to reagent and assay complexity, we note that multiple assays can identify the outliers in other assays, especially within the same cluster (Supplementary Figure S3). Similarly, several experimental strategies have been proposed for flagging mAbs with poor solution properties,^43–46 such as opalescence, high viscosity, and self-association in typical formulation buffers. Depending on material requirement, throughput and organizational expertise, a judicious combination of representative assays could be used to filter mAbs during early discovery, followed by a more comprehensive assessment at a later stage. Establishing boundaries based on distributions of biophysically motivated in silico descriptors can offer early warning signs for developability. Beyond filtering for extreme outliers, it remains to be seen whether they can meaningfully enrich discovery pipelines with mAbs that succeed in the clinic. Given the recency of such approaches, we believe the field should continue to collect data and collaborate further to refine the descriptors and thresholds that constitute these guidelines. Several rational approaches targeting disruption of hydrophobic^47–50 and charge patches^49,51–53 using 3D structure have yielded significant reduction in hydrophobicity, non-specificity, and viscosity. A recent review¹ summarizes the current understanding of the physicochemical nature of patches and the resulting developability outcomes. The utility of such rational approaches to improve developability, after issues have been identified, shows great promise. However, quantitative predictions over a diverse set of sequences remains an outstanding challenge even as the community continues to develop models for predictions of solubility,^54–56 viscosity,^57–61 aggregation,^15,62–64 and chemical stability;^65–68 a recent review⁶⁹ includes discussion of these methods. As an example, we highlight a recent study⁵⁷ where predictions of viscosity using prior published models showed low correlations to experiments for a set of 27 approved mAbs. Similarly, while a HIC prediction model²³ showed a promising R² of 0.6 on a set of 152 mAbs, the predictions on a distinct subset of 64 clinical mAbs in this study using published descriptors¹⁶ resulted in R² of 0.21 (Supplementary Figure S4A).

The potential concerns on assay reproducibility should be addressed by inclusion of multiple controls with known sequences that span a range of measurement values. We highlight a recent study⁴³ that does an exemplary job on protocol capture and recommendations for assay calibration. However, the lack of sequence disclosure, even for control mAbs, in many investigations continues to hinder the growth of datasets that can be analyzed in aggregate to identify future research directions. Collaborative in silico efforts can be enabled by sharing not only the structures used in a study, but also the algorithmic implementations for the calculated metrics to aid reproducibility and data accumulation for larger analyses. Descriptors based on patches, atomic and residue neighborhoods, and atomistic energy calculations, can be sensitive to local conformation, choice of forcefields and methods for charge assignment, and structure modeling protocols. We cite a recent systematic exploration of HIC RT prediction²⁶ using different hydrophobicity scales which showed worse performance using MOE homology models compared to those from MoFvAb⁷⁰ and DeepAb.⁷¹ The use and sharing of structures from accurate open-source protein modeling methods^71–73 may help eliminate an important source of variability between investigations across industry and academic groups.

Material and Methods

Antibody Expression and Production

Our prior publication⁵ includes details for the production of the 137 clinical antibodies. Briefly, VH and VL encoding gene fragments (Integrated DNA Technologies) were subcloned into heavy- and light-chain pcDNA 3.4+vectors (ThermoFisher) and expressed in HEK293 cells. Regardless of the clinical isotype, all mAbs were expressed as IgG1.

Clinical Status Update

We annotated the current clinical status for the 137 mAbs in our original study. Antibodies moving up a phase in clinical trial or obtaining approval were annotated as Progressed. Conversely, antibodies terminated for development at any time or having moved down a phase in clinical trials since 2017 were marked as Regressed. This information was compiled from a combination of AdisInsight (https://adisinsight.springer.com), Tabs-Therapeutic Antibody Database (https://tabs.craic.com), Thera-SAbDab⁷⁴ (https://opig.stats.ox.ac.uk/webapps/newsabdab/therasabdab) and the in silico studies referenced herein.^17–19 Table 1 summarizes the changes up to October 2022. It is inevitable that the status for many antibodies has changed since the compilation of information for this study. The annotations used for the analysis in this study are provided in the supplementary information.

Corrections due to incorrect published sequences

The in silico descriptors and flag assignments for abciximab, tositumomab, motavizumab, ibritumomab,⁸ and zolimomab¹⁷ were removed due to incorrect sequences in the referenced studies.

Rank correlation analysis and clustering

Since most metrics investigated in this study do not show normal distributions, we calculated Spearman’s rank correlations for all pairwise combinations of metrics. Missing data was ignored via the use of use.pairwise.obs=complete option in the cor function of the R stats package. Since hierarchical clustering can be dependent on the input order of values, we reordered the correlation matrix using optimal-leaf-ordering implemented in the R seriation package. The reordered matrix was subsequently used for hierarchical clustering using the R hclust package with method=’average’ option for progressively merging clusters. The hierarchical clustering dendrogram and correlation coefficient matrices were inspected visually to determine the number of clusters. We find that the biophysical underpinning of the experimental assays and in silico metrics is consistent with the obtained clusters.

In silico calculations

The docker image (ID 800e30189fa1) containing the Abpred program²² was downloaded and run according to the provided instructions. A fasta file with concatenated VH and VL sequences was input to the program based on a provided example on https://protein-sol.manchester.ac.uk/abpred.

Predictions for HIC RT for 64 mAbs were made using patch descriptor values¹⁶ as inputs to the following model^23:

HIC.RT.15 = 42.23687–0.02859*cdr_ion.12 + 0.12656*cdr_hyd.12–0.02909*hyd.12–0.00949*ion.12

Assignment of flag violations

Following our earlier protocol, we calculated the 90% threshold for in vitro measurements for the variable regions of Approved antibodies on a common IgG1 backbone. The error bars on the thresholds were estimated using bootstrapping in the boot R package. An assay violation is assigned to a mAb if its assay value exceeds the corresponding 90% threshold. Instead of assigning a cluster violation if any single assay was flagged, we adopt an alternate approach where a mAb is assigned a cluster violation if it violates one or more individual assay flag in a cluster with three or fewer assays, and two or more flags in clusters with greater than three assays.

The assignment of violations for the five TAP descriptors¹⁸ was determined based on published amber and red thresholds. Similarly, based on the recommended guidelines, we calculated the violations for the four metrics recommended by Thorsteinson et al.¹⁷ The flags based on z-scores for the metrics in Ahmed et al.¹⁹ were recalculated using the values in the published supplementary information.⁷ Since Abpred²² returns scaled predictions for several in vitro measurements, we calculated the 90% thresholds for the predictions on the 48 approved mAbs. These thresholds were then used akin to the in vitro thresholds to assign flags. Finally, using the published data in Zhang et al.,²⁰ we calculated the number of rule violations on the set of 137 mAbs. As per the recommendation, violation of 8 or more of 12 rules was assigned as a flag.

For each of the above studies, we relied on the provided annotation of clinical status in the respective publications. We assume that the mAbs annotated as Approved or Clinical are a representative snapshot at the time of the published study, i.e., while every individual annotation might not be perfect, the distribution of calculated flags or biophysical properties is representative of the clinical landscape for the mAb subsets and can be used to compare mAb subsets within a study. Finally, since the determination of approved mAbs in unambiguous, we can further annotate the set of mAbs that have progressed from Clinical to Approved since the time of publication. For the Thorsteinson et al.¹⁷ study, the specific clinical annotation was obtained from the authors since it is not available with the publication.

Receiver operating characteristic curve and penalized logistic regression

To assess the ability of various in silico metrics to discriminate violations in the in vitro assays, we calculated the area under receiver operating characteristic curve using the R package pROC. The violations were defined as the measurements exceeding the 90% threshold determined on the subset of 48 mAbs that were approved in 2017.

Since several in silico metrics are mutually correlated, we also performed a penalized logistic regression using glmnet R package. The in silico descriptors were scaled to zero mean and unit variance in order to compare the estimated coefficients across the different magnitudes for the independent variables. The function cv.glmnet was used with 10-fold cross validation done via specification of the foldid parameter. To ensure the presence of mAbs with an assay threshold violation in every fold, a stratified sampling on the dependent variable was carried out. Both ridge and lasso penalties were investigated by setting alpha to 0 or 1, respectively, in the function call. The parameter keep=TRUE was passed to the function call to retain cross-validation predictions for the observations. These predictions were used to compute the AUROC values in the last two columns of Figure 5. The coefficients corresponding to s=lambda.min, and cross-validation metrics were obtained and averaged over 10 repeats.

Funding Statement

The author(s) reported there is no funding associated with the work featured in this article.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplemental material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/19420862.2023.2200540.

Abbreviations

ACSINS

Affinity-capture self-interaction nanoparticle spectroscopy

AS

Accelerated stability

asa

Accessible surface area

AUROC

Area under receiver operating characteristic

Avg_HI

Average Hydrophobic Imbalance

BSA

Buried surface area

BVP

Baculovirus particles

CDR

Complementarity determining region

CHO

Chinese hamster ovary cells

CIC

Cross-interaction chromatography

CP

Charge patches

CSI

Clone self-interaction using bio-layer interferometry

DM

Dipole moment

DNP

2,4-Dinitrophenol

ELISA

Enzyme-linked immunosorbent assay

ens_charge_Fv

Forcefield charge of the Fv averaged on a structural ensemble

FcRn

Neonatal Fc receptor

Fv

Variable domain of an antibody

Fv_chml

VH minus VL charge at pH 7.4

fvcharge5.5

Net antibody Fv charge at pH 5.5

HI_sum

Hydrophobic index along three CDRs

HIC

Hydrophobic interaction chromatography

HM

Hydrophobic moment

HP

Hydrophobic patches

hph

Hydrophilic

hyd

Hydrophobic

IgG

Immunoglobulin G

ion

Ionic charge patches

KLH

Keyhole limpet hemocyanin

LPS

Lipopolysaccharide

mAb

Monoclonal antibody

neg

Patches of negative charge – MOE

pI

Isoelectric point

pI_3D

pI estimation using structure

pI_seq

pI estimation using sequence

PNC

Patches of negative charge – TAP

pos

Patches of positive charge – MOE

PPC

Patches of positive charge – TAP

PSH

Patches of surface hydrophobicity – TAP

PSR

Polyspecificity reagent

r_gyr

Radius of gyration

ROC

Receiver operating characteristic

RT

Retention time

SFvCSP

Product of exposed non-salt bridged VH and VL charges at pH 7.4

SGAC-SINS

Salt-gradient affinity-capture self-interaction nanoparticle spectroscopy

SMAC

Standup monolayer adsorption chromatography

TAP

Therapeutic Antibody Profiler

VH

Variable heavy chain

VL

Variable light chain