Assessment of the Transmembrane Domain Structures in GPCR Dock 2013 Models

Abstract

The community-wide blind prediction of G-protein coupled receptor (GPCR) structures and ligand docking has been conducted three times and the quality of the models was primarily assessed by the accuracy of ligand binding modes. The seven transmembrane (TM) helices of the receptors were taken as a whole; thus the model quality within the 7TM domains has not been evaluated. Here we evaluate the 7TM domain structures in the models submitted for the last round of prediction - GPCR Dock 2013. Applying the 7 x 7 RMSD matrix analysis described in our prior work, we show that the models vary widely in prediction accuracy of the 7TM structures, exhibiting diverse structural differences from the targets. For the prediction of the 5-hydroxytryptamine receptors, the top 7TM models are rather close to the targets, which however are not ranked top by ligand-docking. On the other hand, notable deviations of the TMs are found in in the previously identified top docking models that closely resemble other receptors. We further reveal reasons of success and failure in ligand docking for the models. This current assessment not only complements the previous assessment, but also provides important insights into the current status of GPCR modelling and ligand docking.

Introduction

The GPCR Dock is a community-wide assessment organized by Abagyan and Stevens with the purpose of evaluating the status of the G-protein coupled receptor (GPCR) structural modeling and ligand docking (Kufareva et al., 2014; Kufareva et al., 2011; Michino et al., 2009). Three rounds of assessment have been conducted since 2008, shortly after the technique breakthroughs in membrane protein crystallography (Cherezov et al., 2007; Jaakola et al., 2008; Rasmussen et al., 2007; Rosenbaum et al., 2007). During each assessment, participants made blind predictions for receptor-ligand complex structures given the information of amino acid sequences of the receptors and ligand chemical structures. The models were then evaluated by comparing with the experimentally solved structures in several aspects, including the seven transmembrane (7TM) domains, the extracellular loops, the ligand-binding pocket definition, the ligand positions and the atomic contacts between receptor residues and ligands. For evaluation of the 7TM domain structures, the seven TMs were taken as a whole and the models were compared by the overall 7TM root-mean-square-deviations (RMSDs) against the target structures. The results showed that the median 7TM RMSDs of the models were around 2 Å for class A receptor targets and around 6 Å for the non-class A receptor target. It appears that it is no challenge to build a GPCR model with reasonable accuracy for the overall structure given the conserved 7TM topology and the availability of template structures. However, similar overall structures may represent rather dissimilar receptors as we have seen that the GPCR structures solved to date typically show overall 7TM RMSDs in the range of 2–3 Å, but the receptors span several phylogenetically distant families, such as from amine to lipid receptors. Looking into the 7TM bundles, one can find that the relative orientations of the seven helices vary significantly with families, functions and activation states of the receptors (Wang et al., 2017). Therefore, it is necessary to evaluate the structural differences within the 7TM bundles of the models.

Another major result from the three rounds of assessment was that 7TM domain prediction accuracy did not correlate with ligand docking prediction accuracy (Kufareva et al., 2014; Kufareva et al., 2011; Michino et al., 2009). Receptor modeling and ligand docking appear to be two distinct steps in the generation of the final receptor-ligand complex models. In addition, factors such as placement of the extracellular loop 2 between TM4 and TM5 (ECL2) heavily affect ligand-docking results. In the assessment, the models were primarily evaluated by ligand binding modes and ranked by how much they reproduced the receptor-ligand contacts as observed in the target structures. It is possible that the top docking models deviate from the targets in the exact arrangements of the seven TMs and the “bad” models may have 7TM structures close to the targets. It is thus interesting to find out how the models distribute in the structural space of the exact arrangements of the 7TM helices and furthermore compare top docking models with top 7TM models.

We have recently developed a novel method, called 7 x 7 RMSD matrix to specifically compare the 7TM bundle structures of GPCRs (Wang et al., 2017). Briefly, a 7 x 7 RMSD matrix records the RMSDs of the backbone atoms between reference and target structures for each TM pair when superposing only one TM in turn. As there are seven TMs, the matrix is composed of seven rows and seven columns, thus 7 x 7 = 49 RMSD values. Specifically, the seven elements in the i-th row are the backbone RMSDs calculated for TM1 through TM7 when superposing only the i-th TM. If the i-th TM transforms significantly relative to the reference structure, large RMSDs may appear in the i-th row. This correspondence is very pronounced if the TM undergoes rotational transformation. The 49 parameters in the matrix thus contain information about changes in conformations and relative orientations of the seven TMs: the seven diagonal elements serve as an indicator of helical conformational changes or conservations within the TMs themselves and the off-diagonal elements at each row reveal whether the corresponding TM moves from the references structure relative to the other TMs. Rotational angles of each TM relative to the reference structure can also be computed. The 7 x 7 RMSD matrix has been applied to identify and quantify helix movements in active GPCR structures, compare the X-ray structures of 33 unique GPCR receptors, and derive the structural relationships of different GPCRs by their 7TM arrangements (Wang et al., 2017).

In this work, we apply the 7 x 7 RMSD matrix analysis to evaluate the 7TM domain structures of the receptor models that were submitted for the last round of GPCR Dock, which was performed in 2013. The targets in GPCR Dock 2013 include three receptors, the human 5-hydroxytryptamine (5HT, or serotonin) receptors 1B and 2B (5HT1B and 5HT2B, respectively) and the human smoothened receptor (SMO). Compared to previous assessment rounds, these targets are more challenging in both receptor modeling and ligand docking due to difficulties in modeling of agonist-bound activation states, ligand-interacting ECL2 loops, and a non-class A receptor with distant homology to the structure-known GPCRs. The models have previously been extensively evaluated by using criteria including overall accuracy of the 7TM prediction and ligand-docking (Kufareva et al., 2014), but the structural differences within the 7TM domains have not been evaluated nor the relationships with the experimental structures. Our current assessment complements the previous assessment by focusing on the exact 7TM arrangements of the receptor models. Combined together, the results provide a more detailed and comprehensive picture about the status of GPCR modeling and docking.

Methods and Materials

Source of GPCR Dock 2013 models

In GPCR Dock 2013, the participants submitted 181, 171, 88, and 88 models predicting four receptor-ligand complex structures: 5HT1B and 5HT2B, both with an agonist ergotamine(Wacker et al., 2013; Wang et al., 2013b) and SMO with two distinct antagonists, LY-2940680 (Wang et al., 2013a) and SANT-1 (Wang et al., 2014), respectively. We obtained the PDB files of all models from the GPCR Dock 2013 web site at http://ablab.ucsd.edu/GPCRDock2013.

Structures of targets and templates

The PDB IDs of the four target structures are 4IAR for the 5HT1B-ergotamine complex ¹⁰, 4IB4 for the 5HT2B-ergotamine complex ¹¹, 4JKV for the SMO-LY-2940680 complex (Wang et al., 2013a), and 4N4W for the SMO-SANT-1 complex (Wang et al., 2014), respectively.

According to the methods description provided by the participants (Kufareva et al., 2014), a number of GPCR structures available at the time of prediction were used as templates for model generation. These include rhodopsin (RHO)(PIDB ID: 1U19 (Okada et al., 2004)), adrenoceptors β1AR (PDB ID : 2VT4 (Warne et al., 2008)) and β2AR (PDB ID: 2RH1 (Cherezov et al., 2007)), muscarinic receptors M2 (PDB ID: 3UON (Haga et al., 2012)) and M3 (PDB ID: 4U15 (Thorsen et al., 2014)), dopamine receptor D3(PDB ID: 3PBL(Chien et al., 2010)), histamine receptor H1(PDB ID: 3RZE (Shimamura et al., 2011)), adenosine receptor A2A (PDB ID : 3EML (Jaakola et al., 2008)), opioid receptors NOP(PDB ID: 4EA3 (Thompson et al., 2012)), δOR(PDB ID : 4N6H (Fenalti et al., 2014)), κOR (PDB ID: 4DJH (Wu et al., 2012)) and μOR (PDB ID: 4DKL (Manglik et al., 2012)), chemokine receptor CXCR4(PDB ID : 3ODU (Wu et al., 2010)), sphingosine 1-phosphate receptor S1P1 (PDB ID: 3V2Y (Hanson et al., 2012)). Some of the structures were solved with low resolution at that time (> 3.0 Å); we replaced them with the recent structures of higher resolution. In addition, several agonist-bound or active structures were also used as templates, including agonist-bound β1AR (PDB ID: 2Y02), agonist-bound A2AR (PDB ID: 3QAK), agonist-G-protein-bound β2AR (PDB ID: 3SN6) and ligand-free rhodopsin (PDB ID: 3CAP).

TM residue ranges

In our assessment of 5HT1B and 5HT2B models, the TM residues are defined as 1.36 – 1.60, 2.38–2.65, 3.23–3.55, 4.39–4.62, 5.36–5.63, 6.32–6.58 and 7.33–7.53 in the Ballesteros-Weinstein notion(Ballesteros and Weinstein, 1995). These residues are the maximal TM residues that are solved and present in all target, template and model structures.

In the assessment of SMO models, because of the shorter TM4 in target structures 4JKV and 4N4W, the TM4 range was changed to 4.45 – 4.62.

Ligand binding-pocket residues

We defined the ligand-binding pocket residues to be the protein residues within 4.0 Å distance of any of the ligand atoms. In 5HT1B-ergotamine complex structure (PDB id: 4IAR), the binding pocket residues are W125, L126, D129, I130, C133, T134, C199, V200, V201, S212, A216, W327, F330, F331, S334, L348, F351, D352, T355, and Y359. In 5HT2B-ergotamine complex structure (PDB id: 4IB4), the binding pocket residues are D135, V136, S139, T140, V208, L209, T210, K211, M218, A225, W337, F340, N344, L347, and Q359

7 x 7 RMSD matrix calculations

Structural superposition and RMSD calculations are performed with the VMD software (Humphrey et al., 1996), including calculations of the overall 7TM RMSDs and the 7 x 7 RMSD matrices. The RMSD values are calculated for the backbone atoms of each TM, i.e. the CA, C, O, N atoms. A 7 x 7 RMSD matrix is obtained by successively superposing one of the seven TMs and recording RMSDs for all seven TMs. For comparison between models and targets, the matrix of each model is computed against the corresponding target structure. For all-against-all structural comparison, each of the structures in the data set serves as a reference structure to compute the 7x7 RMSD matrices of all other structures.

Rotational angle calculations

Rotations of the TMs are defined as around the principal axes of the target structure. It is worth noting that the principal axes of the target structure are not the helical axis of any TM. To compute the rotational angles, we first need to align the target and model structures into a homologous coordinate system. The alignment is done in two steps: 1) the coordinates of the target structure are reset by aligning the principal axes of the molecule in the X, Y, and Z-directions. As a result, the Z-axis is approximately perpendicular to the membrane, pointing from the intracellular to extracellular side; 2) the coordinates of the model structure are transformed by being superposed onto the re-oriented target structure using all TMs. Figure S4 shows the principal axes of 5HT1B target (4IAR) and 5HT2B target (4IB4).

After the target and models are aligned into a homologous coordinate system, structural superposition is performed for each of the target-model structural pair using one of the seven TMs at a time. The VMD software (Humphrey et al., 1996) automatically generated a 4 x 4 transformation matrix for each structural superposition. A python script is written to convert the transformation matrix into rotational angles of the corresponding TM. More examples of rotational angle computations can be found in reference (Wang et al., 2017).

Clustering analysis

We perform all-against-all structural comparison for each of the three data set: 5HT1B, 5HT2B and SMO, using the 7 x 7 RMSD matrix as structural similarity index. First, each of the N structures in the data set serves as a reference structure to compute the 7x7 RMSD matrices of all other structures, which generate a similarity matrix that has a dimension of N rows and N x 49 columns. Then, the N x (N x 49) matrix of pairwise similarities is input into the R program (R-core-Team, 2014) to compute the Euclidean distances between the rows of the matrix and perform an average linkage clustering by calling the hclust function. Finally, a fan-shaped dendrogram plot is generated to present the clustering tree in the R program (R-core-Team, 2014).

Results

In this assessment, we focus our analysis on the 7TM domains of the receptor structures in the models. For each set of the 5HT1B, 5HT2B and SMO models, we first compute the 7 x 7 RMSD matrices of the models against the corresponding target structure. These matrices reveal similarity and differences in 7TM arrangements between models and target. The models with lowest maximal 7 x 7 RMSDs in the matrices are identified as top 7TM models. As all models have been previously evaluated by criteria that primarily focused on ligand docking and the models have been ranked by their ligand-docking quality (Kufareva et al., 2014), it is thus interesting to compare the model quality between ligand-docking and the exact arrangement of the 7TM helices. We then highlight the top 7TM models identified in this current assessment as well as the top docking models that were identified in the previous assessment (Kufareva et al., 2014). Finally, we perform all-against-all structural comparison for all models and related template and target structures by using the 7 x 7 RMSD matrix as a 49-parameter similarity index, which generates a classification tree highlighting the models close to each template and target in terms of their 7TM structures.

5HT1B models

181 5HT1B-ergotamine complex models were submitted by the participants. In Figure S1, we present the heat map plot of the 7 x 7 RMSD matrices of all 181 models against the target structure (PDB ID : 4IAR). The models are ordered by the maximal RMSDs in the matrices, with the lowest listed first. The maximal RMSDs in the matrices range from 4.1 Å to 23.7 Å with a median of 8.5 Å. We can see that large RMSDs appear in the blocks labeled by “Superpose-TM4”, “Superpose-TM5”, “Superpose-TM6” for most models, indicating large deviations from the target in the placements of TM4, TM5 and TM6. In addition, TM1 and TM7 deviate largely in the models listed on the bottom of the plot. The varied patterns of colors in almost every row exhibit diverse structural differences between the models and the target.

To further demonstrate how the individual TMs deviate from the target within the 7TM bundle, we present the scatter plot (Figure 1) of the overall 7TM bundle RMSD versus the average of the RMSDs when superposing only one TM, that is the average in each block of Superpose-TM1, Superpose-TM2, Superpose-TM3, Superpose-TM4, Superpose-TM5, Superpose-TM6, and Superpose-TM7. Whereas the overall 7TM bundle RMSD are small between 1.1 Å and 3.1 Å, the average RMSD of superposing individual TMs spread from 1 Å to 23 Å. Of the seven TMs, TM2 and TM3 are the best predicted, indicated by the relatively small RMSDs in the blocks Superpose-TM2 and Superpose-TM3.

Figure 1.

Scatter plot of overall 7TM bundle RMSD (horizontal) versus average of RMSDs when individually superposing each of the seven TMs (vertical), labelled by S-TM1, S-TM2, S-TM3, S-TM4, S-TM5, S-TM6, and STM7, in which “S” stands for “superpose”. The top 7TM models and top docking models are highlighted by green color and red color, respectively.

We further analyzed the top five models ranked by the lowest maximal RMSDs in the matrices, which are the models with labels 5447-1, 3157-1, 4203-4, 3157-3, and 1458-5 (Figure 2A). The first part of a model label is the group ID of the participant and the second part is the model number given by the participant (For the participant names, please refer to reference (Kufareva et al., 2014)). The maximal RMSDs in the matrices of the top 7TM models are rather small and close, between 4.1 Å and 4.8 Å. Model 5447-1 with the lowest maximal RMSD of 4.1 Å also shows the lowest overall 7TM RMSD of 1.15 Å. However, the ranking correspondence is not as straightforward for other models, as some models with low overall 7TM RMSDs show large maximal RMSDs due to displacement of a particular TM. For instance, the model with the second lowest overall 7TM RMSD of 1.19 Å (model 6100-1) is ranked # 37 with a maximal RMSD of 6.9 Å in block “Superpose-TM4”. Model ranking by overall 7TM RMSDs is presented in Table S2 and the scatter plot of the maximal RMSDs versus the overall 7TM RMSDs is presented In Figure S5. The top five 7TM models are ranked # 1, #59, #78, #54 and #8, respectively by the overall 7TM RMSDs ranging from 1.15 Å to 1.46 Å. On the other hand, the five models with the lowest overall 7TM RMSDs (models 5447-1, 6100-1, 5641-4, 5641-2 and 5641-3) are ranked # 1, #37, #65, #67 and #64 by the maximal RMSDs ranging from 4.1 Å to 7.7 Å. We then computed the Kendall’s tau coefficients for the rank correlations, which are 0.26 between the maximal 7 x 7 RMSD and the overall 7TM RMSD. Such weak correlations indicate that 7 x 7 RMSD matrix contains information that cannot be represented by the single-parameter overall 7TM RMSD.

Figure 2.

Analysis of top 5HT1B models. a) Heat map plot of 7 x 7 RMSD matrices against the target (4IAR). The maximal RMSDs are labeled. The top 7TM models are with green labels and the top docking models with red labels; b) Superposition of each TM after superposing only TM4. Grey: target, green: model 4203-4 and red: model 6882-2; c) Ligand-docking in models 5447-1 (ligand in purple sticks) and 4203-4 (ligand in orange sticks) in comparison with the target structure (ligand in silver sticks), viewed from the extracellular side. For each model, the left-side image highlights the incorrect placement of the cyclic tripeptide moiety of ergotamine and ECL2 is not shown; the right-side image highlights the placement of ECL2 (in red color); d) Rotational angles of each TM in the top 7TM models. Rotations are computed around the X, Y and Z-directions, which are aligned along the principal axes of the target. The Z-axis is approximately perpendicular to the membrane, pointing from the intracellular to extracellular side (Figure S4).

As in the previous assessment (Kufareva et al., 2014), model evaluation has been primarily focused on ligand-docking and all models have been ranked by their ligand-docking quality, it is thus interesting to highlight the top docking models for the quality evaluated by the exact arrangement of the 7TM helices. For the top five docking models identified in the previous assessment (models 6882-2, 6882-3, 4628-3, 4628-1 and 2128-2) (Kufareva et al., 2014), the maximal 7 x 7 RMSDs range from 7.6 Å to 9.9 Å and the models are ranked # 132, # 93, # 120, # 119, and #56, respectively (Figure 2A). These models are ranked # 84, # 13, # 91, # 90 and # 50, respectively by the overall 7TM RMSDs ranging from 1.38 Å to 1.48 Å. The maximal 7 x 7 RMSDs of the top docking models are all present in block S-TM4, indicating that TM4 significantly deviates from the target and is the worst predicted among the seven TMs (Figure 2B). On the other hand, although the top 7TM models show rather small RMSDs for almost every TM, their ligand-docking results are not accurate and the five models are ranked # 92, # 85, # 175, # 61 and # 126, respectively, based on ligand-docking criteria (Kufareva et al., 2014). In the best 7TM model 5447-1 (Figure 2C), while the ergoline core of ergotamine is correctly placed in the primary ligand-binding pocket, the cyclic tripeptide moiety points toward TM2 in error. This is probably due to inaccuracy in the ligand-docking program as the binding pocket residues including side chains show a RMSD of 1.77 Å and the ECL2 segment is also predicted relatively well, with a RMSD of 4.95 Å including side chains of residues from SER197 to HIS205, for comparison, which is 5.11 Å in the best docking model 6882-2. Another major reason preventing correct ligand docking is the errors in placement of ECL2. For instance, in model 4203-4, ECL2 is too close to TM7, blocking the correct placement of the cyclic tripeptide moiety, resulting it pointing towards TM4 (Figure 2C).

To investigate the structural relationships between the computational models and experimental structures, we perform all-against-all structural comparison using the 7 x 7 RMSD matrix as a structural similarity index. Among the maximal five receptor-ligand models submitted by each participant, some models have identical receptor structures. We define identical receptor models by an overall 7TM RMSD less than 0.001 Å and for each set of identical models, we keep the models with lower index numbers. As a result, 145 unique receptor models are kept for further analysis (The duplicated models are listed in Table S1). For the experimental structures, besides the two 5HT1B and 5HT2B target structures, we include another 18 GPCR structures that had been used as templates by the participants to build their models (See Methods and Materials section for the detail of the 18 GPCR structures). In total, we have 165 structures in our all-against-all structural comparison analysis.

In Figure 3, we show the 7 x 7 RMSD matrix-based clustering dendrogram of the 5HT1B data set. To facilitate identification of the models, we colored the labels of the models by arbitrarily cutting the tree into 50 clusters so that most models can be separated by different colors because of being in different clusters. We can see that the target, 5HT1B, lacks a close group member either from the models or the experimental structures. Instead, receptors that served as templates, such D3, H1, M2, and the active structure of receptor β2AR have close group members from the models. The top five docking models, except for model 2128-2, all are grouped closely with receptors β1AR and β2AR. For instance, the maximal 7 x 7 RMSD calculated between β2AR and its closest member model 3276-5 is only 1.5 Å. But, the top 7TM models have not reached such a level of similarity to the 5HT1B target. However, their RMSDs are encouragingly small, revealing minor rotations of the TMs from the target. In Figure 2D, we can also see that except for TM6, the minor rotations (< 10 degree) of the TMs are mainly around the Z-axis (red bars). As the Z-axis is approximately perpendicular to the membrane (Figure S4), this indicates that these TMs show minor spiral shifts along the direction perpendicular to the membrane. Differently, rotations of TM6 are mainly around the Y-axis, reflecting the facts that the outwards shift of TM6 in the target structure is not predicted in the models.

Figure 3.

Clustering of the 5HT1B data set. Target 5HT1B is marked with a red star. The top 7TM models (5447-1, 3157-1, 4203-4, 3157-3 and1458-5) and top docking models (6882-2, 6882-3, 4628-3, 4628-1 and 2128-2) are marked with solid green dots and red dots, respectively. Note, model 4628-3 and model 2128-2 are represented by model 4628-2 and model 2128-1 with lower index numbers because of identical receptor structures.

For the purpose of comparison, we also perform all-against-all structural comparison using the overall 7TM RMSD as a single-parameter structural similarity index. In Figure S6, we show the overall 7TM RMSD-based clustering dendrogram of the 5HT1B data set. The difference at a first glance is that the hierarchical structure in Figure S6 has fewer levels than in Figure 3. Some structures that are distantly placed in Figure 3 are now placed closely, such as the two serotonin receptors (5HT1B and 5HT2B), and the fully activated β2AR receptor (β2AR:3SN6) with the fully activated rhodopsin (RHO:3CAP). These differences are consistent with those revealed in the our prior work, in which we compared the clustering results of using 7 x 7 RMSD matrix and three other metrics including the overall 7TM RMSD for 33 unique GPCR receptors (Wang et al., 2017). Because the 7 x 7 RMSD matrix explicitly measures the relative orientation of each TM pair, it is more sensitive to variations in the 7TM arrangements than other methods. Due to this property, structures that are determined by 7 x 7 RMSD matrix as similar highly likely remain similar in the clustering by other metrics. For instance, the top docking models that are placed close to receptors β1AR and β2AR in Figure 3 are still placed close to those receptors in Figure S6. However, it is not the other way. The top 7TM models 5447-1 and 1458-5 placed close to receptor D3 in Figure S6 are far from receptor D3 in Figure 3. Model 5203-2, the closest to target 5HT1B in Figure S5 is close to opioid receptors in Figure 3.

5HT2B models

There are 171 5HT2B-ergotamine complex models. Except for five models, all receptor models show reasonable overall accuracy, with overall 7TM RMSDs between 1.3 Å and 2.9 Å from the target (PDB id: 4IB4) (Figure 1). The heat map plot of the 7 x 7 RMSD matrices of the 171 models are shown in Figure S2. The models are ordered by the maximal RMSDs in the matrices, which range from 5.7 Å to 42.1 Å with a median of 10.5 Å. We can see that the majority of the models show moderate to large RMSDs (5 Å to 19 Å) in blocks “Superpose-TM1” and “Superpose-TM7”. This indicates that TM1 and TM7 in the majority of the models have notable deviations from the target. In addition, many models also show large deviations for TM4, TM5 and TM6. Relatively, TM2 and TM3 are the best predicted, similar to the 5HT1B prediction (Figure 1). Overall, the RMSDs of 5HT2B models spread in a wider range than those of 5HT1B models, except for TM2 and TM3 (Figure 1).

The top 7TM models ranked by the maximal RMSDs in the matrices are models 4452-2, 5447-5, 4113-3, 7527-2 and 4204-3 (Figure 4A). Ranking by overall 7TM RMSDs is presented in Table S3. The top 7TM models all show moderate overall 7TM RMSDs ranging from 1.68 Å to 1.98 Å (ranked # 47, # 59, # 94, # 127 and # 62, respectively). On the other hand, the five models with the lowest overall 7TM RMSDs (models 6407-3, 6055-3, 6055-1, 6055-4, 6055-5) show maximal RMSDs from 8.67 to 10.63 (ranked # 27, # 87, # 85, # 86 and # 78, respectively). The Kendall rank correlation coefficients are 0.19 between the maximal 7 x 7 RMSD and the overall 7TM RMSD in the 5HT2B models.

Figure 4.

Analysis of top 5HT2B models. a) Heat map plot of 7 x 7 RMSD matrices against the target (4IB4). The maximal RMSDs are labeled. The top 7TM models are with green labels and the top docking models with red labels; b) Ligand-docking in top 7TM model 4452-2 (ligand in orange sticks) in comparison with the target structure (ligand in silver sticks), viewed from the extracellular side. The left-side image highlights the correct conformation of the ligand and the correct placement of the binding pocket; the right-side image highlights the incorrect placement of ECL2 (in red color); C) Rotational angles of each TM in the top 7TM models. Rotations are computed around the X, Y and Z-directions, which are aligned along the principal axes of the 5HT2B target (4IB4). The Z-axis is approximately perpendicular to the membrane, pointing from the intracellular to extracellular side (Figure S4).

Similar to the 5HT1B assessment, none of the best docking models identified in the previous assessment (models 4628-3, 7554-3, 7554-2, 7554-1, and 6882-2) is among the top 7TM models, which instead are ranked #105, #158, #161, #160 and #63, respectively. These models also show relatively high overall 7TM RMSDs (1.70 Å, 2.30 Å, 2.31 Å, 2.32 Å, and 1.73 Å, respectively). Also, similar to the 5HT1B assessment, the top docking models all show large RMSDs when superposing TM4, indicating large errors in TM4 orientation. However, worse than in 5HT1B prediction, both the top 7TM and top docking models of 5HT2B prediction show moderate to large RMSDs when superposing TM1 and TM7, indicating inaccurate prediction for the orientations of TM1 and TM7, as well. Better than in the 5HT1B prediction in which the highest ligand-docking rank of the top 7TM models is #61, the top 7TM model 4452-2 is ranked # 8 by the ligand-docking criteria in reference(Kufareva et al., 2014). The ligand is in the correct conformation and the TM residues of the binding pocket show a backbone RMSD of 1.6 Å from the target (Figure 4B). But the ECL2 residues (THR206 to LYS211) of the binding pocket show a large backbone RMSD of 7.3 Å, pushing the ligand shifting towards TM7. In the other top 7TM models, errors in prediction of ECL2 are also the major source of inaccurate ligand-docking.

For all-against-all structural comparison, we include the same experimental GPCR structures as in the 5HT1B assessment, here the target being the x-ray structure of ergotamine-bound 5HT2B structure (PDBid: 4IB4). We filtered out 28 duplicated receptor models and use the 143 unique receptor models in analysis. The 7 x 7 RMSD matrix-based clustering dendrogram of the 5HT2B data set is shown in Figure 5. The tree is arbitrarily cutting into 50 clusters so that most models can be separated by different colors because of being in different clusters. We can see that two models, 4204-3 and 4204-5 are close to the target 5HT2B. These two models are ranked #5 and #6 by the maximal 7 x 7 RMSD (6.7 Å and 6.9 Å, respectively) in the matrices. These results indicate that the relative orientations of the seven TMs in models 4204-3 and 4204-5 resemble the targets in the current structural space defined by the computational models and experimental structures. The top four 7TM models, although showing smaller maximal 7 x 7 RMSDs, do not resemble the target or any of the templates in terms of the relative orientations of the seven TMs. In the plot showing rotational angles of the TMs in the top 7TM models (Figure 4C), we can see that the signs and components of the rotational angles are diverse among the models, indicating diverse structural differences from the target, although the models show similar maximal RMSDs.

Figure 5.

Clustering of the 5HT2B data set. Target is marked with a red star. The top 7TM models (4452-2, 5447-5, 4113-3, 7527-2 and 4204-3) and top docking models (4628-3, 7554-3, 7554-2, 7554-1, and 6882-2) are marked with solid green dots and red dots, respectively. Note, model 4628-3 is represented by model 4628-1 with a lower index number that has an identical receptor structure.

Of the top five docking models, two models (4628-1 and 6882-2) closely resemble receptors β1AR and β2AR, similar to four of the 5HT1B top five docking models. Also, similar to the 5HT1B prediction, of the models that resemble experimental structures, most resemble receptors β1AR and β2AR, and a small number resemble receptors H1, D3, M2, M3, and the active structures of rhodopsin, A2A and β2AR.

We also perform all-against-all structural comparison using the overall 7TM RMSD as a single-parameter structural similarity index (Figure S7). Similar to the 5HT1B data set, the hierarchical structure by overall 7TM RMSD has fewer levels than that by 7 x 7 RMSD matrix, indicating that more models are classified as similar by overall 7TM RMSD. Interestingly, the best 7TM model 4452-2 is placed closest to target 5HT2B in Figure S7, suggesting that this model resembles most the target in the overall shape of the 7TM domain while the individual arrangements of the 7TMs are also close to the target.

SMO models

There are 88 models submitted targeting the complex structures of receptor SMO bound with ligand LY-2940680 (PDB id: 4JKV) and another 88 models for SMO bound with SANT-1 (PDB id: 4N4W). In both predictions, 14 models submitted by three groups have a number of missing residues and these models are excluded from our analysis, leaving 74 models in each case. Although every model shows a heptahelical structure as a core structure, the transmembrane regions are incorrectly predicted in many models. This leads to large overall 7TM RMSDs, spreading to 17.8 Å, with a median of 5.2 Å (Figure 1). The heat map plot of the 7 x 7 RMSD matrices of the models against the corresponding target are shown in Figure S3. We can see that almost every model shows large RMSDs of exceeding 15 Å (colored in green and purple) in one or more blocks. In the top models 6055-3 and 5447-4, the maximal RMSDs are already large, being 20.5 Å and 18.5 Å, respectively (Figure 6). Therefore, predictions of receptor SMO are far less accurate than those of receptors 5HT1B and 5HT2B. Large errors in receptor structures directly resulted in incorrect ligand-binding pocket prediction. Very few to no receptor-ligand contacts are achieved in the models. According to the previous assessment (Kufareva et al., 2014), only one model (7527-3) achieved docking correctness of above 0.1 % targeting 4JKV and three models (1458-5, 6055-3, 5641-5) targeting 4N4W. The extremely large RMSDs in the SMO models are due to alignment errors. For instance, in the top 7TM model 6055-3, the sequence alignment of TM6 with the target was shifted by three residues, which resulted in almost one-turn shift of the helical structure.

Figure 6.

Heat map plot of 7 x 7 RMSD matrices of the top SMO models. The maximal RMSDs are labeled. The top 7TM models are with green labels and the top docking models with red labels. The labels of the models targeting 4JKV are ended with “_I” and models targeting 4N4W with “_II”. Model 6055-3_II is ranked # 2 by both 7TM and docking criteria and colored in orange.

In Figure 7, we show the 7 x 7 RMSD matrix-based clustering dendrogram of the SMO data set, including 148 SMO receptor models, two target structures (4JKV and 4N4W) and the X-ray structures of 16 receptors used in the 5HT1B and 5HT2B assessments. We can see that the two target structures are grouped into a same clade and separated from all other receptors, indicating an unique 7TM arrangement in receptor SMO, which is consistent with our previous clustering result for 33 unique GPCR receptors (Wang et al., 2017). Also, none of the models is grouped together with any of the non-SMO receptors, indicating that none of the models shows similarity in the 7TM arrangement with those receptors. The ten models by group GaTech (model ID 1458) that have the fewest alignment error (one residue shift in TM5) with the lowest overall 7TM RMSDs (Table S4) are placed closest to the targets, among which model 1548-5 is also the top docking model targeting 4N4W. In addition, the five top 7TM models are also placed relatively close to the targets in Figure 7. However, in the overall 7TM RMSD-based clustering dendrogram (Figure S8), targets 4N4W and 4JKV are placed close to class A GPCR experimental structures and far from the models.

Figure 7.

Clustering of the SMO data set. Two targets are marked with red stars. The top 7TM models (6055-3_I, 5447-4_I, 6135-4_I, 5447-4_II, and 5447-5_II) are marked with solid green dots. The top docking models (7527-3_I, 1458-5_II, 6055-3_II and 5641-5_II) are marked with red dots. Model 6055-3_II that is ranked on top by both 7TM and docking criteria is marked with an orange dot.

Discussion

With the 7 x 7 RMSD matrix analysis, our assessment reveals that the models vary widely in prediction accuracy of the 7TM structures despite with similar 7TM folds. This indicates that the current GPCR modeling methods show a varied level of performance in predicting 7TM arrangements. Encouragingly, several high quality 7TM models for receptors 5HT1B and 5HT2B are identified, highlighting the corresponding modeling methods. According to the method description provided in reference(Kufareva et al., 2014), the top five 7TM models of receptor 5HT1B are generated by RosettaCM(Song et al., 2013) (model 5447-1), Modeller (Eswar et al., 2007) (models 3157-1 and 3157-3), I-TASSER(Roy et al., 2010) (model 4203-4) and the methods used by group GaTech (model 1458-5). The methods generating the top five 7TM models of receptor 5HT2B are PRIME(Jacobson et al., 2004) (model 4452-2), RosettaCM(Song et al., 2013) (model 5447-5), Modeller (Eswar et al., 2007) (4113-3), Rosetta(Nguyen et al., 2013) (model 7527-2) and the methods used by group Caltech-Poland-CSMC (model 4204-3). In comparison, the top five models of 5HT1B ranked by overall 7TM RMSD are generated by RosettaCM (Song et al., 2013) (model 5447-1, also the best 7TM model), Modeller (Eswar et al., 2007) (model 6100-1), and Galaxy (Ko et al., 2012) (models 5641-1, 5641-2, and 5641-3). The top five models of 5HT2B ranked by overall 7TM RMSD are generated by Modeller (Eswar et al., 2007) (model 6407-3) and GPCR-I-TASSER (Zhang et al., 2015) (models 6055-3, 6055-1, 6055-4 and 6055-5). Modeller is also the major method used to build the receptor structures in the top docking models. As methods such as Rosetta, RosettaCM, I-TASSER, and PRIME explicitly explore backbone motions, their successes of generating top 7TM models imply the importance of conformational sampling of protein backbone in accurate prediction of the 7TM arrangements.

It appears that 5HT2B represents a more challenging target than 5HT1B in terms of 7TM prediction. The maximal 7 x 7 RMSDs of the 5HT2B top 7TM models are 1–2 Å larger than those of the 5HT1B top 7TM models, and the 5HT2B top docking models show large deviations not only in TM4 but also in TM1 and TM7. This correlates with the structural differences between the target structures and the adrenergic receptors (β1AR and β2AR) - the template structures the models resemble most (Figure 8). Relative to β2AR (PDB id: 2RH1), 5HT2B shows large helix movements in TM1 and TM7, which are absent in 5HT1B. Also, helix movement in TM6 is slightly larger in 5HT2B than in 5HT1B. Moderate helix movements in TM6 and TM7 have been considered as the activation features in the intermediate-active structures of 5HT2B bound with agonist ergotamine, but has no analog in the available crystal structures. These results indicate that it remains an unsolved problem to predict receptor activation state when it is beyond the template structures.

Figure 8.

Pair-wise 7 x 7 RMSD matrices of receptors β2AR (PDB ID: 2RH1), 5HT1B (PDB ID: 4IAR), and 5HT2B (PDB ID: 4IB4).

For the purpose of docking ergotamine into the serotonin receptors, a high quality of 7TM domain structure is not enough. This is because binding of ergotamine to 5HT1B and 5HT2B involves not only the 7TM domains but also the extracellular loop region. Ergotamine makes extensive interactions with the ECL2 loop residues including CYS199 to VAL201 in 5HT1B and VAL208 to LYS211 in 5HT2B, which requires correct predictions of the ECL2 loop as well as the extracellular tip residues of the TMs. Unfortunately, most top 7TM models in the current data sets failed in ligand-docking either because of errors in ECL2 prediction or inaccuracy in the ligand-docking program used. This emphasizes the importance of improving ECL2 loop modeling and ligand-docking algorithms. Nevertheless, correct prediction of TMs forming the primary ligand-binding site is the very first prerequisite for accurate ligand-docking. For ergotamine binding to 5HT1B and 5HT2B, the primary binding site is formed by TM3, TM5, TM6 and TM7. Despite large deviation in TM4, the top docking models still achieved high accuracy in ligand-docking because TM4 is not involved ligand binding at all. In fact, TM4 has not been found to participate in ligand binding in any of the known GPCR structures because of its location in the 7TM bundle. TM4 is located behind TM3, which shields TM4 from access to the center of the 7TM bundle. Although not participating in ligand binding, the extracellular tip of TM4 is the starting position of ECL2 and thus incorrect placement of TM4 may affect the placement of the ECL2 loop. Also, TM4 forms an interface between the 7TM bundle and cell membrane. The highly conserved residue W4.50 in TM4 is a conserved lipid binding site in class A GPCRs(Hanson et al., 2008; Jaakola et al., 2008; Manglik et al., 2012). TM4 has been found to form dimer interfaces in GPCR crystal structures including receptors CXCR4 (Wu et al., 2010) and SMO (Wang et al., 2013a). Therefore, Incorrect TM4 placement, though not affecting ligand-docking, may lead to incorrect modeling of lipid binding or dimerization and thus affect the prediction of GPCR functions (Bouvier, 2001; Ferre et al., 2014).

As shown in the Results section, model ranking by the maximal 7 x 7 RMSD and the overall 7TM RMSD shows rather weak correlation. It requires a model to be close to the target in every TM to have a low maximal 7 x 7 RMSD, and on the other hand, a model may have a low overall 7TM RMSD even if some TMs have large deviations from the target because superposition of multiple TMs can average out the deviations. In addition, although we have demonstrated model ranking by the maximal RMSD in this work, we do not think such a single parameter is sufficient to conclusively determine whether a model is better than another. Depending on the purpose of the models, the whole matrix and in which positions the large RMSDs occur may need to be considered as well. For instance, if the TMs of interest all show small deviations, one may consider the model as good even if the other TMs show large deviations, thus a large maximal 7 x 7 RMSD. In addition, performing a hierarchical clustering analysis for the models and experimental structures can help identify whether the models selected by the lowest maximal 7 x 7 RMSDs are close enough to the targets. It is worth noting that in the 7 x 7 RMSD matrix–based clustering analysis, the 49 positions may have different contributions to pair-wise similarity or distance. If the top models do not have sufficiently low RMSDs in the positions that significantly contribute to similarity, they may not be grouped closely to the target. Unfortunately, like in other multivariate analysis (most machine-learning methods), it is difficult to find out which variables contribute to the model significantly or which not. The maximal 7 x 7 RMSD and clustering analysis are two different methods for evaluating the models: the former being a convenient single-parameter ranking metric and the latter taking all 49 parameters into account to investigate structural relationships. We recommend using them complementarily.

Finally, as we have demonstrated in this work and our prior work (Wang et al., 2017), the 7 x 7 RMSD matrix provides a simple and efficient measure of structural differences within the 7TM domains, which can not only reveal deviations of each individual TM from the target, but also the structural relationships of the models in terms of the exact 7TM arrangements. A comprehensive evaluation of the status of GPCR modeling and docking should include evaluation of the exact 7TM arrangements, the ECL2 loops and ligand-receptor binding interactions.