Towards the development of universal, fast and highly accurate docking/scoring methods: a long way to go

Abstract

Accelerating the drug discovery process requires predictive computational protocols capable of reducing or simplifying the synthetic and/or combinatorial challenge. Docking-based virtual screening methods have been developed and successfully applied to a number of pharmaceutical targets. In this review, we first present the current status of docking and scoring methods, with exhaustive lists of these. We next discuss reported comparative studies, outlining criteria for their interpretation. In the final section, we describe some of the remaining developments that would potentially lead to a universally applicable docking/scoring method.

Introduction

Molecular discovery: high-throughput screening vs virtual screening

Since the advent of the human genome project and functional genomics, there has been a continual increase in the number of new therapeutic targets available for drug design. During this time, advances in crystallography and nuclear magnetic resonance spectroscopy have contributed many structural details of protein and protein–ligand complexes. In response to these newly discovered targets and their structural elucidation, cost- and time-efficient design and syntheses of bioactive compounds targeting these became a priority for the field of drug design. In recent years, this field has partially turned away from the traditional approaches, such as rational and semi-rational design, to the high-throughput screening of large combinatorial libraries. However, the identification of novel lead compounds via these traditional approaches has been more fruitful compared with the low hit rates observed with combinatorial methods (Shoichet et al., 2002). In practice, identifying leads or hits in silico rather than via library generation and screening is both faster and more economical, while being easier to setup. Indeed, screening of large libraries has been used in combination with (or in parallel to) or sometimes substituted by computational approaches. Among the most commonly used virtual screening (VS) tools are docking methods, which have been successfully used to predict the binding modes and affinities of many potent enzyme inhibitors as well as receptor antagonists. As a result, many drugs developed in part by computer-aided structure-based drug design methods are in late-stage clinical trials or have now reached the market (Borman, 2005). Following these success stories, the pharmaceutical companies are increasingly relying on computational methods as one of the primary platforms for designing new potential leads.

Current and future challenges

With the ever-growing interest in using computational structure-based drug design tools, there is a need to scrutinize the field and examine its current state and its future. An overview of the latest progress in the structure-based drug design field shows that significant improvements must still be achieved in order to develop highly accurate molecular docking and VS methods (Taylor et al., 2002; Krumrine et al., 2003; Brooijmans and Kuntz, 2003; Mohan et al., 2005; Ferrara et al., 2006; Klebe, 2006; Rester, 2006; Sousa et al., 2006). As most of the discussed programmes have been reported, we will refer the reader to the many reviews cited throughout this review and original articles for more information about each piece of software.

Instead, we review herein the state-of-the-art of docking/scoring methods and discuss the achievements covering the literature until June 2007, the challenges tackled over the recent years and the remaining bottlenecks. This review does not cover low-throughput approaches including molecular dynamics simulation-based approaches (Aqvist et al., 1994; Jones-Hertzog and Jorgensen, 1997; Gervasio et al., 2005) and de novo design programmes such as LEGEND (Nishibata and Itai, 1993), DREAM++ (Makino et al., 1999) and LUDI (Bohm, 1992).

The available docking and scoring methods

Docking and scoring methods

To date, over 60 docking programmes (Table 1) and more than 30 scoring functions (SFs) (Table 2) have been disclosed (see Supplementary Information for more detailed tables). However, only some of these programmes were made available and a limited number of them are widely used (AutoDock, DOCK, FlexX, FRED, Glide, GOLD, ICM, QXP/Flo+, Surflex). In the following sections, the references will appear only in tables and will not be repeated when a programme or a SF is given as an example.

Table 1. Reported docking programmes and, when available, the latest version number, the key references and web sites.

Program	Reference(s)	Website	Search algorithma
Affinity	N/A	www.accelrys.com/products/datasheets/i2_affinity_data.pdf	MC
ADAM	(Yamada and Itai, 1993; Mizutani et al., 1994, 2006)	www.immd.co.jp/en/product_2.html	RBD
Autodock 4.0	(Morris et al., 1998; Osterberg et al., 2002)	autodock.scripps.edu/	GA
CDOCKER	(Wu et al., 2003a)	N/A	MD-SA
CHARMM	(Vieth et al., 1998b, 1998a)	www.charmm.org/	GA and MC
CLIX	(Lawrence and Davis, 1992)	N/A	RBD
DARWIN	(Taylor and Burnett, 2000)	N/A	GA
DIVALI	(Clark and Ajay, 1995) ,	N/A	GA
DOCK 6	(Oshiro et al., 1995; Knegtel et al., 1997; Kang et al., 2004; Moustakas et al., 2006)	dock.compbio.ucsf.edu/	IC
DockIt	N/A	www.metaphorics.com/products/dockit.html	N/A
DockVision 1.0.3	(Hart and Read, 1992) ,	www.dockvision.com	MC
DoMCoSAR	(Vieth and Cummins, 2000)	N/A	SA
DragHOME	(Schafferhans and Klebe, 2001)	N/A	RBD
EADock	(Grosdidier et al., 2007)	N/A	EA
eHiTs	(Zsoldos et al., 2006)	www.simbiosys.ca/ehits/index.html	RBD of fragments followed by reconstruction
EUDOC	(Pang et al., 2001) ,	N/A	RBD
FDS	(Taylor et al., 2003)	N/A	MC
DAIM-SEED-FFLD	(Majeux et al., 1999; Budin et al., 2001; Kolb and Caflisch, 2006)	biocroma.uzh.ch/SDL/DAIM-SEED-FFLD_agreement.pdf	RBD
FITTED	(Corbeil et al., 2007)	www.fitted.ca	GA
FlexX 2.2	(Rarey et al., 1996)	www.biosolveit.de/	IC
FlexE	(Rarey et al., 1999; Claußen et al., 2001)
FlipDock (formerly pyDock)	(Zhao and Sanner, 2007)	www.scripps.edu/~yongzhao/FLIPDock/	GA
FLOG	(Miller et al., 1994)	N/A	IC
FRED	(McGann et al., 2003)	www.eyesopen.com/products/applications/fred.html	RBD
FTDock	(Gabb et al., 1997)	www.bmm.icnet.uk/docking/ftdock.html	RBD
GAMBLER	(Charifson et al., 1999)	N/A	GA
GasDock	(Li et al., 2004)	N/A	GA
GEMDOCK	(Yang and Chen, 2004)	N/A	EA
GlamDock	(Tietze and Apostolakis, 2007)	N/A	MC/SA
Glide 4.0	(Friesner et al., 2004; Sherman et al., 2006)	www.schrodinger.com/ProductDescription.php?mID=6&sID=6&cID=0	Hierarchical filters and MC
GOLD 3.1	(Verdonk et al., 2003, 2005)	www.ccdc.cam.ac.uk/products/life_sciences/gold/	GA
Hammerhead	(Welch et al., 1996)	N/A	IC
HADDOCK	(Dominguez et al., 2003)	www.nmr.chem.uu.nl/haddock/	SA
HierDOCK/HierVLS	(Floriano et al., 2004; Trabanino et al., 2004)	N/A	IC
ICM	(Abagyan et al., 1994b; Totrov and Abagyan, 1997)	www.molsoft.com/docking.html	MC
LibDock	(Diller and Merz, 2001)	N/A	MA
LIDAEUS	(Wu et al., 2003b)	N/A	MA
LIGIN	(Sobolev et al., 1996)	N/A	RBD
MacDOCK	(Fradera et al., 2004)	N/A	IC
MCDOCK	(Liu and Wang, 1999)	N/A	MC
MolDock	(Thomsen and Christensen, 2006)	N/A	DE
MVP	N/A	N/A	IC
PatchDOCK	(Schneidman-Duhovny et al., 2005)	N/A	Shape complementarity
PAS-Dock	(Tøndel et al., 2006)	N/A	TS
PhDOCK	(Joseph-McCarthy et al., 2003)	N/A	MA
Ph4DOCK	(Goto et al., 2004)	N/A	MA
PIPER	(Kozakov et al., 2006)	N/A	Fast Fourier transform
Plants	(Korb et al., 2006)	N/A	Ant colony optimisation
Prodock	(Trosset and Scheraga, 1999)	N/A	MC
PRO_LEADS	(Murray et al., 1999)	N/A	TS
ProPose	(Seifert, 2005)	N/A	IC
PSI_DOCK	(Pei et al., 2006)	N/A	GA with TS
Q-fit	(Jackson, 2002)	N/A	MA
Quantum	N/A	www.q-lead.com/cnt/quantum	N/A
QXP/Flo+	(McMartin and Bohacek, 1997)	N/A	MC
RiboDock	(Morley and Afshar, 2004)	N/A	MC
ROSETTALIGAND	(Meiler and Baker, 2006)	N/A	MC
SANDOCK	(Burkhard et al., 1998)	N/A	MA
SDOCKER	(Wu and Vieth, 2004)	N/A	Random walk
SLIDE	(Schnecke and Kuhn, 2000; Zavodszky and Kuhn, 2005)	N/A	IC
SKELGEN	(Alberts et al., 2005)	N/A	IC
SODOCK	(Chen et al., 2007)	N/A	Swarm Optimisation
SG-DOCK/SP-DOCK	(Fradera et al., 2000)	N/A	IC
Surflex 2.1	(Jain, 2003, 2007)	N/A	IC with MA
Yucca	(Choi, 2005)	N/A	MC

Abbreviation: N/A, not available.

^aDE, differential evolution; EA, evolutionary algorithm; GA, genetic algorithm; IC, incremental construction; MA, matching algorithm; MD, molecular dynamics; RBD, rigid-body docking; SA, simulated annealing; TS, Tabu search.

Table 2. Reported SFs.

Scoring function	Software implementations	Class	Reference(s)
ChemScore	GOLD, FRED, CScore, PRO_LEADS	Empirical	(Eldridge et al., 1997)
eHiTS SF	eHiTS	Empirical	(Zsoldos et al., 2003)
FlexX SF	FlexX	Empirical	(Rarey et al., 1996)
Fresno	Standalone	Empirical	(Rognan et al., 1999)
GlideScore	Glide	Empirical	(Friesner et al., 2004, 2006)
Hammerhead SF	Hammerhead, Surflex, LigandFit	Empirical	(Jain, 1996)
HINT	Standalone	Empirical	(Cozzini et al., 2002)
LigScore	LigandFit	Empirical	(Krammer et al., 2005)
PLP	LigandFit, FRED, DockIt	Empirical	(Gehlhaar et al., 1995)
RankScore	FITTED	Empirical/FF	(Moitessier et al., 2006a)
SAFE_p	None	Empirical/FF	(Sussman et al., 2002)
SCORE	Standalone	Empirical	(Wang et al., 1998)
SCORE 3.0	PSI-DOCK	Empirical	(Pei et al., 2006)
SCORE1	LUDI	Empirical	(Böhm, 1994)
SCORE2	LUDI	Empirical	(Böhm, 1998)
ScreenScore	FRED	Empirical/consensus	(Stahl and Rarey, 2001)
SIE	Standalone	Empirical/FF	(Naim et al., 2007)
SLIDE SCORE	SLIDE	Empirical	(Schnecke and Kuhn, 2000)
VALIDATE	Standalone	Empirical/FF	(Head et al., 1996)
X-Score	Standalone	Empirical/consensus	(Wang et al., 2002)
AutoDock SF	AutoDock, SODOCK	FF/empirical	(Morris et al., 1998)
DockScore	DOCK, CScore	FF	(Meng et al., 1992)
GoldScore	GOLD, CScore	FF	(Jones et al., 1997)
HADDOCK Score	HADDOCK	FF	(van Dijk et al., 2006b)
ICM SF	ICM	FF	(Abagyan et al., 1994b)
QXP SF	QXP/MCDOCK	FF	(McMartin and Bohacek, 1997)
BLEEP	Standalone	Knowledge based	(Mitchell et al., 1999)
DrugScoreCSD	Standalone	Knowledge based	(Velec et al., 2005)
DrugScorePDB	Standalone	Knowledge based	(Gohlke et al., 2000)
M-Score	Standalone	Knowledge based	(Yang et al., 2006)
PMF	CScore, LigandFit, BioMedCAChe, DockIt	Knowledge based	(Muegge and Martin, 1999; Muegge, 2006)
SMoG	SMoG	Knowledge based	(DeWitte and Shakhnovich, 1996; Ishchenko and Shakhnovich, 2002)

Abbreviations: FF, force field; SF, scoring function.

Each docking programme relies on two complementary components: (1) a method to explore the conformational space of the ligand and/or the protein target and (2) a SF to evaluate the proposed binding modes referred to as poses. An SF should first assign the best score to the ‘correct pose' (that is, the native pose observed in crystal structures), thus ‘guiding' the conformational sampling algorithm. This first aspect is critical to accurately predict the binding mode. Second, the docked poses of highly active compounds should be attributed better scores than those of non-binders or poor binders. This second aspect is critical in lead optimization and in VS, where potential hits are to be extracted from large libraries. In fact, some programmes use multiple SFs, such as a crude SF to direct the docking and a more refined SF for scoring the final poses.

Whenever a docking programme is selected, two inversely correlated factors are to be considered. While speed is essential for effective virtual high-throughput screening of large libraries, accuracy is critical for lead optimization.

Conformational sampling methods

Multiconformer docking algorithms

Rigid-body docking methods use either a single conformation or multiconformer libraries to consider ligand flexibility. These approaches often dock the small molecules using shape complementarity or interaction site matching algorithms (for example, ADAM, DOCK, FRED, FTDock, LIGIN, SANDOCK and YUCCA). In matching algorithms (Figure 1a), a pharmacophore representing the protein is initially developed and used to guide the docking. An initial ligand conformation is generated and a ligand pharmacophore is derived from this conformation. The distance matrices (listings of all the distances between each of the pharmacophoric points) of the ligand and protein pharmacophore are examined for a match. If there is a match, rotational and translational vectors are calculated and applied to the ligand. These vectors position the ligand in the same frame of reference as the protein. While speed is the major strength of these approaches, their predictive power is reduced as ligand poses are not fully refined, making it difficult for the correct poses to be found or for an accurate score to be assigned. Post-processing of selected poses using an additional local search algorithm addresses the former issue. Owing to their great speed, this first class of programmes is very useful in selecting medium sized libraries from extremely large libraries with acceptable enrichment in active compounds.

Figure 1.

Schematic representation of a matching algorithm (a) and an incremental construction algorithm (b) in the context of docking of a flexible ligand.

Incremental construction

Docking programmes implementing incremental construction (Figure 1b) methods build up the ligand on-the-fly in the active site, often relying on libraries of preferred conformations (for example, MIMUMBA) to connect the fragments and to consider the ligand flexibility. To do so, the ligand is split into a set of fragments, one of which is selected as an anchor and rigidly docked to the binding site using methods such as matching algorithms (for example, DOCK, FlexX, FLOG, Hammerhead, Surflex). Another variation is the docking of fragments followed by reconnection, exemplified by the algorithm implemented in eHiTS.

Stochastic methods

The second class of conformational sampling methods explores the ligand conformational space on the fly. There are many different methods to search stochastically with the most common methods being genetic algorithms (GAs) and Monte Carlo (MC) search (Figure 2). Many programmes make use of GAs (for example, AutoDock, DARWIN, DIVALI, GOLD, EADock, FITTED and PSI-DOCK.), first implemented in docking programmes in the mid-90s to dock flexible ligands (Clark and Ajay, 1995; Jones et al., 1995; Oshiro et al., 1995). GAs (Figure 2a) are based on Darwin's theory of evolution. The pose of the ligand is represented by a chromosome, which is made up of genes. These genes code for each torsional angle, as well as the ligand's rotation and translation in space. These poses then evolve through transmission of genetic information (reproduction), altered over time by genetic operators such as crossover and mutation. There are many different methods for the selection of the next generation, but the most popular is the survival of the fittest, where the two lowest scoring conformations are passed to the next generation. Modifications of GAs (Lamarckian GA in AutoDock or tabu search-enhanced GA in PSI-DOCK) have been found to efficiently explore the conformational space of the ligands.

Figure 2.

Schematic representation of a genetic algorithm (a) and a Monte Carlo search (b) in the context of docking a flexible ligand.

With Monte Carlo (for example, DockVision, ICM, MCDOCK, ProDOCK, SLIDE) (Figure 2b), the pose of the ligand is sequentially modified through bond rotation, translation and/or rigid body rotation, one or more parameters at a time, and the new conformation is then evaluated. If the new conformation has a lower score, it is kept. If a conformation is higher in energy, one can reject it or use a selection criterion such as the Metropolis criterion. Metropolis allows for higher energy conformation to exist by allowing a temperature dependence (the higher the temperature, the more likely that a higher energy conformation is kept), resulting in the crossing of energy barriers on the potential energy surface. If the new pose does not pass the Metropolis criterion, then it is discarded and another set of modifications is tried. Other stochastic optimization algorithms, inspired by analogy to biology, are the ant colony optimization (PLANTS) and the particle swarm optimization (SODOCK/AutoDock) algorithms, both of which are combined with a local search algorithm. Tabu searches (Pro_Leads) have also been employed to consider ligand flexibility.

Multistep procedures

Using one or more of the algorithms mentioned above in conjunction with refinement of the poses has also been performed. For instance, EUDOC and Glide combine the random generation of poses with their refinement to obtain increased accuracy.

Consensus docking

All the methods described so far have shown great success in predicting the binding modes of co-crystallized ligands. However, none of them has been identified as a universally applicable method. Furthermore, CONS_DOCK, a consensus docking method using FlexX, GOLD and DOCK (Paul and Rognan, 2002); AutoxX, combining AutoDock and FlexX interaction models (Wolf et al., 2007) and FITTED, combining a GA and interaction site matching, have been found to be more accurate than the individual search methods. Another way to improve a programme's performance is to provide additional information such as pharmacophore(s) to orient the docking (Hindle et al., 2002; Moitessier et al., 2004; Verdonk et al., 2004), as well as any other knowledge about the target that may reduce the number of false positives and false negatives in VS studies (Jansen and Martin, 2004).

Scoring functions

As previously discussed, SFs have a twofold function: to direct the docking and to predict the binding affinity of the final pose. In this section, only functions that are developed to score the final poses will be discussed. Customized SFs (developed for a specific target only) and functions used to improve the docking accuracy will not be considered. In the past 15 years, a number of SFs have been reported and implemented in docking programmes. Table 2 summarizes the literature in the field (see also Supplementary Information for more detailed data). This table contains SFs used to score the final pose while scoring functions used to orient the docking algorithm towards the correct pose (e.g., knowledge-based interaction fingerprint scoring – (Mpamhanga et al., 2006)) are not included. In general, SFs attempt to predict the binding free energy or to rank-order compounds by their bioactivity. These SFs are classified as force field (FF)-based, empirical and knowledge-based (Tame, 1999; Schulz-Gasch and Stahl, 2004; Mohan et al., 2005; Tame, 2005; Jain, 2006). As SF accuracy is critical for a successful VS campaign (Klebe, 2006), considerable progress over the years has been made. However, commonly used SFs are known to have limitations as the protein–ligand complex formation often includes subtleties not captured by SFs (Gohlke and Klebe, 2002). This in turn causes poor ranking of compounds by predicted activities. Many oversimplifications, discussed in the following sections (for example, solvation and entropy contributions to the binding free energy), are believed to be the main cause of this pitfall. In addition, the SFs tend to be more accurate with the proteins and the docking programmes used to calibrate them. Indeed, customized SFs calibrated for specific targets have also been devised (Catana and Stouten, 2007). To overcome this hurdle, more than one SF can be alternatively employed with docking programmes. For instance, GoldScore and ChemScore have been implemented in GOLD, while ChemScore, ScreenScore and PLP can be used with FRED. As for consensus docking, consensus scoring, which combines several scoring methods, has been found to be superior to the use of a single function in some cases (X-Score, formerly known as X-Cscore; Wang et al., 2002).

Empirical SFs

Since the pioneering work of Böhm (1994) in the development of LUDI, a significant amount of work has been devoted to the development and improvement of empirical SFs. With empirical SFs, the evaluation of the energetics of the ligand binding (essentially from protein/ligand crystal structures) is decomposed into simpler, scalable contributions arising from, for example, hydrogen bonds, metal ligation, hydrophobic effects and freezing of rotatable bonds (Equation (1)). The various scaling factors (ΔG_i in Equation (1)) are then defined by regression to fit experimentally determined protein–ligand affinities.

Among the most commonly used SFs is ChemScore, which has been implemented in various docking programmes (for example, GOLD, FRED). Standalone SFs have also been devised and include X-Score, DrugScore, VALIDATE and HINT. Each empirical SF differs by the number and nature of the terms used to make up its equation. For instance, several include an explicit directional hydrogen bond energy term (for example, ChemScore, X-Score and the SFs implemented in eHiTS, FlexX, Surflex), while only a few include an explicit directional metal–ligand interaction term (for example, eHiTS, Surflex and X-Score). Functions such as the eHiTS and PLP SFs evaluate the internal energy of the ligand in its bound conformation, while solvation and/or predicted captured water molecules (within GlideScore) are computed in a different manner. Many empirical SFs take into account the hydrophobic effect in the binding, mostly either by computing the hydrophobic surface buried in the complex (for example, SCORE1/2, LigScore), or by evaluating the match of the hydrophobicity of an atom with its environment (for example, FlexX, SCORE, SLIDE), while several combine both approaches (for example, eHiTS, GlideScore, HammerHead, X-Score). On the other hand, HINT computes the logP of the ligand as a measure of its water solvation. The entropic contribution to the binding energy due to the freezing of torsional degrees of freedom upon binding is often estimated by a term proportional to the number of sp³–sp³ and sp²–sp³ rotatable bonds. In some cases (for example, ChemScore, GlideScore, VALIDATE, X-Score), the environment of a bond is taken into consideration to assess the extent of its effect, while RankScore attempts to include the freezing of protein side chains by scaling the interaction with flexible side chains.

FF-based SFs

Force fields were originally developed to reproduce conformational behaviour and thermodynamic and kinetic properties of small molecules and macromolecules. When applied to protein–ligand complexes, FFs are often found to significantly overestimate the binding affinity (Equation (2)) even when applied in conjunction with highly accurate, time-consuming techniques (for example, Linear Interaction Energy method), which consider the bulk water either explicitly or implicitly (for example, PB/SA, GB/SA; Michel et al., 2006). Scaling factors applied to the non-bonded terms (van der Waals and electrostatics) were found to restore the predictiveness of FFs in this area.

When compared to empirical SFs, a smaller number of SFs were developed exclusively from FFs. More commonly, FF terms (illustrated in Equation (2)) are combined with terms from empirical SFs, such as the solvation and ligand entropy terms in the AutoDock SF. The choice of FF parameters is varied; AutoDock, DOCK and RankScore SFs combine the van der Waals, electrostatic and hydrogen bond interaction energy computed using the AMBER FF, while GoldScore makes use of the Tripos parameters and ICM implements a hybrid AMBER-ECEPP/2 approach.

Knowledge-based SFs

Other popular SFs, such as DrugScore and PMF, have been developed from statistical analysis of crystal structures of ligand–protein complexes. These analyses report the distribution of ligand–protein atom-type pairs (histogram in Figure 3) and convert these data into pairwise potentials (blue and green curves in Figure 3). In the interaction between charged species (blue line), there is a sharp minimum at a relatively close distance and a secondary minimum at a larger separation, accounting for the interaction via a bridging water molecule. In contrast, the potential for a pair of aliphatic carbons (green line) shows little preference over a wide range of interatomic distances. The score is calculated by the sum of all interaction pairs between each ligand and protein atom lying within a sphere of a given cutoff (usually 6–12 Å). Although these functions are expected to capture all the data needed for predicting the free energy of binding, some of the interactions are underrepresented in the available crystal structures (for example interactions with metals and/or halogens) and are not well parameterized. As for FF-based SFs, correcting/additional terms were implemented as exemplified by the solvation term included in DrugScore.

Figure 3.

Potentials of mean force for a pair of aliphatic carbons (green) and a pair composed of a positively charged nitrogen and a negatively charged oxygen (blue).

Comparative studies

Comparative studies

Due to the rapid evolution of docking and scoring algorithms, comparing their efficiency was needed. In the past decade, many comparative studies have been reported that evaluate the relative performance of the most popular programmes (Westhead et al., 1997; Bissantz et al., 2000; Perez and Ortiz, 2001; Stahl and Rarey, 2001; Terp et al., 2001; Doman et al., 2002; Diller and Li, 2003; Jenkins et al., 2003; Schulz-Gasch and Stahl, 2003; Wang et al., 2003, 2004; Ferrara et al., 2004; Kellenberger et al., 2004; Kontoyianni et al., 2004; Kroemer et al., 2004; Perola et al., 2004, 2007; Chen et al., 2006; Cummings et al., 2005; Kontoyianni et al., 2005; Warren et al., 2006). These studies can be classified into target-oriented studies and broad comparative studies. This first class is of great interest for the medicinal chemistry community, and aims to help identify the best programme for a specific target. For instance, programmes were evaluated for their ability to dock to matrix metalloproteases (Ha et al., 2000; Hanessian et al., 2001; Hu et al., 2004) and to score BACE-1 inhibitors (Schafferhans and Klebe, 2001; Moitessier et al., 2006a; Holloway et al., 2007) or mannosidase inhibitors (Englebienne et al., 2007). The goal of the broader studies is to evaluate the programmes' accuracy on a set of proteins and is threefold. They aim to evaluate and compare the ability of different docking programs and/or SFs (either separately or in combination) to (1) properly dock compounds to proteins, (2) predict the ligand binding affinities and (3) extract active compounds from libraries of decoys. In Figures 4, 5 and 6, selected comparative studies are shown. Table 3 lists reported comparative studies. While this manuscript was under revision, other comparative studies were published (McGaughey et al., 2007; Onodera et al., 2007; Zhou et al., 2007) and are included in Table 3 but are not discussed.

Figure 4.

Comparative studies of docking programmes. A criterion of r.m.s.d. <2 Å has been used in all the studies except in Kontoyianni and co-workers, where a more suggestive criterion has been used (see text). R.m.s.d., root mean-square deviation.

Figure 5.

Comparative studies of docking-based VS programmes. Top panel: average enrichment factors computed for a variety of enzymes and programmes; bottom left panel: application of MVP to various enzymes; bottom right panel: evaluation of SFs for identifying actives in large libraries. SF, scoring function; VS, virtual screening.

Figure 6.

Comparative studies of docking/scoring programmes. Top panel: average Spearman coefficient factors computed for a variety of enzymes and programmes; bottom panel: Spearman coefficient factors computed for a set of scoring functions and consensus SFs. SF, scoring function.

Table 3. Reported comparative studies.

Programs	Scoring function (SF)	Reference
Glide, GOLD, DOCK	Native scoring functions	(Zhou et al., 2007)
FLOG, FRED, Glide	Native scoring functions	(McGaughey et al., 2007)
DOCK, AutoDock, GOLD	Native scoring functions	(Onodera et al., 2007)
N/A	MMFFs, LigScore1, LigScore2, PLP1, PLP2, PMF, LUDI, X-Score	(Holloway et al., 2007)
GOLD, Glide, FlexX, AutoDock, eHiTS, LigandFit, FITTED	Native scoring functions	(Englebienne et al., 2007)
FlexX, GOLD, Glide, ICM	Native scoring functions	(Chen et al., 2006)
DOCK, DockIt, FlexX, Flo, Fred, Glide, Gold, LigFit, MOE, MVP	Native scoring functions, FlexX, DrugScore, Mcdock, Mcdock+, Fulldock, Sdock, Zdock, ChemScore, ScreenScore, CVFF,DOCKPLP, PMF. Dockit-Score, Cscore, DockScore, GoldScore, Sdock+, Fulldock+, LigScore1, LigScore2, PLP1, Dreiding, DOCK-Energy, DOCK-Chemical, DOCK-Contact	(Warren et al., 2006)
N/A	PLP1, PLP2, LUDI, PMF, DockScore, GoldScore, ChemScore, LigScore1, LigScore2, CFF91, AMBER94, RankScore	(Moitessier et al., 2006a)
Glide, LigandFit, FlexX, DOCK	LigScore1, LigScore2, PMF, PMF, ChemScore, GoldScore, GoldScore, FlexX, PLP1, PLP2	(Kontoyianni et al., 2005)
DOCK, DOCKVISION, Glide, GOLD	Native scoring functions, consensus scoring	(Cummings et al., 2005)
FlexX, DOCK, GOLD, LigandFit, Glide	Native scoring functions	(Kontoyianni et al., 2004)
DOCK, FlexX, FRED, Glide, GOLD, SLIDE, SurFlex, QXP	Native scoring functions	(Kellenberger et al., 2004)
Glide, GOLD, ICM	ChemScore, GlideScore, OPLS-AA	(Perola et al., 2004)
N/A	BLEEP, PMF, GoldScore, DockScore, ChemScore	(Marsden et al., 2004)
DOCK, FlexX, GOLD, AutoDock	Native scoring functions, DrugScore, X-Cscore	(Hu et al., 2004)
N/A	X-Score, DrugScore, D-Score, PMF-Score, GoldScore, ChemScore, F-Score, LigScore, PLP, PMF, LUDI, HINT	(Wang et al., 2004)
N/A	CHARMm, DOCK-Energy, DOCK-Chemical, DOCK-contact, DrugScore, ChemScore, AutoDock, PMF, GoldScore	(Ferrara et al., 2004)
FlexX, GOLD, ICM, LigandFit, the Northwestern University version of DOCK, QXP	Native scoring functions	(Kroemer et al., 2004)
Glide, FRED	FlexX, ScreenScore, GlideScore, GlideComp, ChemScore, PLP	(Schulz-Gasch and Stahl, 2003)
DockVision, GOLD	LUDI, GoldScore	(Jenkins et al., 2003)
AutoDock, DOCK, FlexX, GOLD, ICM	Native scoring functions	(Bursulaya et al., 2003)
FlexX	FlexX, PLP, DrugScore, PMF, ScreenScore	(Stahl and Rarey, 2001)
GOLD	SCORE, LUDI, GRID, PMF, DockScore, GoldScore, ChemScore, F-Score	(Terp et al., 2001)
DOCK, AutoDock	Native scoring functions	(Hanessian et al., 2001)
DOCK, FlexX	PMF, FF	(Ha et al., 2000)
DOCK, FlexX, GOLD	Chemscore, DockScore, FlexX, Fresno, GoldScore, Pmf, Score	(Bissantz et al., 2000)

Abbreviation: N/A, not available.

Limitations of comparative studies

While comparative studies provide insight into the various programmes' accuracy, speed, applicability to a range of targets and other factors, they should be considered with great care for many reasons.

First, the versions of the programmes in question are not always specified and may vary from one study to another.

Second, the accuracy is greatly dependent on the settings employed and fine-tuning parameters may lead to significant changes in accuracy. As a result, greater expertise with a specific programme may bias the study. Comparative studies published as part of the development of new programmes will not be discussed herein, as the authors are more proficient with their own programme, thus lowering the objectivity of the study. We believe that the most useful programmes for medicinal chemists are those that can provide accurate and reliable results without extensive training and/or labour-intensive optimizations.

Third, the preparation of receptors and ligands for docking requires knowledge of the active site and careful consideration of all possible isomers (for example, tautomers) and protonation states. To address this issue, Warren and co-workers have divided the work between chemists with expertise on each of the proteins studied, and others with expertise on each of the programmes considered. This strategy significantly reduces bias, making this comparative study very fair and objective.

Fourth, as pointed out by Cole et al. (2005), using the root mean-square deviation (r.m.s.d.) as a criterion of docking accuracy is questionable. For instance, an r.m.s.d. of less than 2.0 Å used as a criterion of success yields misleading results when specific interactions such as directional hydrogen bonds or floppy solvent exposed groups are to be considered. In fact, Kontoyianni used a criterion that was more subjective yet more representative of the true accuracy of the programmes. In this study, in addition to examining the r.m.s.ds, all of the poses were also visually inspected and success of the docking was then described as ‘close' (correct pose), ‘active site' (fair pose) or ‘inaccurate' (wrong pose). Unfortunately, this criterion would be difficult to implement in programmes and therefore cannot be applied to large testing sets. During the same year, interaction-based accuracy classification was also proposed and found to be a more meaningful criterion of success than the r.m.s.d. (Kroemer et al., 2004). Recently, Marcou and Rognan (2007) came to the same conclusion when they used interaction fingerprints combined with a Tanimoto coefficient. Additionally, the metrics for success in virtual screening (usually enrichment factors and ROC curves) are sometimes difficult to interpret (Truchon and Bayly, 2007).

Fifth, self-docking is a good indication of a programme's ability to identify native poses amongst several others, but provides little information about the accuracy in a real drug discovery scenario. In practice, cross-docking is a more valuable experiment. In fact, although Warren and co-workers pointed out that a careful selection of the protein structure can reduce the impact of the induced fit upon binding, many other studies have shown the significant impact of protein flexibility on the docking accuracy (Claußen et al., 2001; Carlson, 2002; Osterberg et al., 2002; Zentgraf et al., 2006; Corbeil et al., 2007).

Sixth, in certain cases, no mention is made (or insufficient details are provided) regarding the CPU time required to run the study. As many programmes (for example, GOLD and Glide) provide various levels of speed and accuracy, a comparative study should reveal the accuracy to expect within a specified period of time. For instance, the early version of our own programme, FITTED, performs as well as other programmes, but was found to be much slower and therefore not competitive.

For medicinal chemists, the best indicator of a programme's accuracy is its ability to identify novel compounds in VS studies that are then experimentally confirmed. However, it would be difficult to compare studies as they often deal with different proteins and compound libraries. A number of successful applications can be found in (Kitchen et al., 2004; Ghosh et al., 2006; Muegge and Oloff, 2006). To date, the available programmes (that is, AutoDock, DOCK, GOLD, FlexX, Glide) have been very successful.

Docking accuracy

Several conclusions can be drawn from the fair and exhaustive comparative study by Warren and co-workers. First, as can be seen in Figure 4 (top panel), the accuracy is highly dependent on the protein considered. Second, none of the 10 programmes assessed clearly outperforms all the others. However, in this study, MVP, Glide, GOLD and FlexX may be considered the best four, while DOCK, DockIt and MOE reproduced the native pose in only a few cases. Other comparative studies showed that Glide, GOLD and FlexX are among the best programmes. Chen and co-workers reported that ICM outperformed the other three programmes investigated. Interestingly, Perola et al. (2007) commented on this same article that they were not able to reproduce the high accuracy of ICM, although they found that ICM is still at least as accurate as the other programmes assessed. A series of comparative studies looked at metalloenzymes and revealed that the metal coordination is still not well reproduced by many docking programmes. Englebienne et al. recently looked at the α-mannosidase and reported Glide as the best of the seven assessed programmes (Englebienne et al., 2007).

Virtual screening accuracy

Comparing programmes for their ability to identify actives within libraries is an even more difficult task. In fact, each of these types of comparative studies must be considered separately. Figure 5 summarizes some of the data from selected comparative studies. From a more global point of view, one can draw three major conclusions. First, none of the programmes is universally accurate as the data indicate that the accuracy is highly protein dependent. Second, until recently, none of the assessed programmes were consistently better than others, as most of them exhibit similar overall accuracies. However, the recent study by Warren and co-workers, which appeared to be the largest study of this kind, clearly showed that MVP stood out. Further investigation of this result on other proteins is needed to confirm this important discovery. Third, all the programmes, except for MOEDock, provide enrichments of the top of the hit list in active compounds. This shows that in virtually every case, it is worth running a VS to guide the development of a focused library as enrichment is likely to be obtained. Similarly, studies were reported that identified the best SFs in the context of VS as illustrated in Figure 5 (bottom right panel).

SF accuracy

Other comparative studies aim to identify the most accurate SF in rank-ordering compounds by affinity. The Spearman coefficient describing the ability to reproduce a rank-ordered list has often been used. Figure 6 summarizes the accuracy of various SFs as determined by a number of different comparative studies. Comparing SFs for their ability to identify actives within libraries is an even more difficult task. In contrast to the docking programmes, which often demonstrate similar accuracies, these published studies revealed a wide range of accuracies. For instance, Bissantz and co-workers determined that the poses produced by GOLD were the most suitable for their VS targets (thymidine kinase and oestrogen receptor), while identifying which SF to use proved to be a very difficult task. Two of the SFs (PMF and FlexX SF) that performed well for the thymidine kinase screening were among the worst for the oestrogen receptor screening, highlighting the target specificity of SFs. Bissantz et al. (2000) have also shown that regardless of the docking programme, GoldScore was often the most accurate SF of their set. It has also been shown that consensus scoring is often better (not always significantly) than the individual SFs (Wang et al., 2004; Figure 6).

The challenges of docking/scoring programmes

Although considerable efforts have been devoted to the development of accurate and fast docking/scoring methods, a more universal method is yet to be developed. Although developing a universal method might not be the major goal, increasing the transferability from one protein to another of existing methods is certainly to be improved. Among the issues that remain to be addressed and those that are currently being addressed are the treatment of protein flexibility, and in particular the scoring of modelled protein conformations, and the presence of water molecules including the evaluation of their binding free energy. Ligands forming covalent complexes and macromolecules such as nucleic acids and metal-containing enzymes are subjects in need of further study. Each challenge requires examination of both docking and scoring. Considering that the free energy of binding is a combination of subtle enthalpic and entropic effects but that high-throughput scoring is desired, careful approximations have to be made when developing SFs. Unfortunately these approximations inevitably add noise to the SFs, resulting in reduced accuracy. In the following section we will describe what we believe are the remaining issues facing the developers. To illustrate the state-of-the-art, examples are given in each of these sections. However, the following descriptions are limited to the reported features of each programme, although we are aware that some features such as the docking of covalent inhibitors with AutoDock are described only in user manuals and/or on web pages.

Exhaustive sampling vs speed

Despite the considerable efforts to address the problem of conformational sampling using a large panel of algorithms, one is always left with the task of balancing speed and accuracy. On one end of the spectrum, one can apply crude but fast docking (for example, rigid-body docking) and scoring (for example, shape complementarity) approaches. In practice, many fast programmes that only consider discrete values for the many degrees of freedom were enhanced by local search algorithms (simplex, Solis and Wets in AutoDock, conjugate gradients), although significantly extending the time required for the computations. On the other end of the spectrum, one can apply molecular dynamics-based approaches that would consider the Boltzmann's distribution of conformations (for example, Linear Energy Interaction). While being accurate in predicting relative binding affinity of congeneric compounds, these latter methods cannot be applied to large libraries of ligands (for example 100 000 compounds) due to their speed.

Protein flexibility

Another major factor that significantly limits the accuracy of today's docking methods is protein flexibility upon ligand binding (Bursavich and Rich, 2002). Najmanovich et al. (2000) pointed out in a statistical analysis of Brookhaven protein databank (PDB) structures that 85% of the proteins contain one to three flexible residues. Although this number of flexible side chains can be considered small, a slight adjustment of the protein can have a significant effect on the molecular recognition process. As a result, the success rate drops significantly when going from self- to cross-docking (Osterberg et al., 2002). Due to the large fraction of flexible proteins reported, flexibility cannot be ignored. Many approaches were proposed and recently reviewed (Cavasotto et al., 2005), although very few of these have been made available to the greater medicinal chemistry community.

Among these approaches, the docking of ligands to soft structures (reduced van der Waals penalties at short distances; for example, LJ 8-4 in GOLD) simulates slight adjustments of the protein. A similar practice reduces the van der Waals optimal distances thereby uniformly enlarging the binding site (smooth potential in AutoDock 3.0 and vdW offset in ADAM). Local optimization of hydrogens is also possible (polar hydrogens in GOLD, all hydrogens in PLANTS). Overall, these approaches were found to improve the docking accuracy but not when considerable side chain rotations are observed. The first attempt to account for larger moves has been reported by Leach (1994) and relied on libraries of side chain rotamers. Similar approaches have since been devised (Frimurer et al., 2003) and even implemented in available programmes. ICM offers the use of biased probability moves of the side chains coupled with Monte Carlo search of the ligand pose (Totrov and Abagyan, 1997). Docking to multiple structures can also account for side chain movement (EUDOC) while significantly increasing the required CPU time.

Ideally, one would like to dock ligands to virtual proteins where the flexibility, including backbone adjustments and side chain rotamers, is considered on-the-fly. To this effect, two other major approaches have been proposed. For programmes relying on grids for scoring, combination of grids derived from more than one protein conformations can be either combined leading to one (for example, DOCK; Knegtel et al., 1997) or to a set of individual grids (for example, AutoDock; Osterberg et al., 2002) or aligned into a larger grid modelling the conformational ensemble (Sotriffer and Dramburg, 2005). The great advantage of this approach is that the time required for a single run is roughly similar to the time necessary to dock to a rigid protein. For programmes using atomic representations, docking to composite structure ensembles has been shown to significantly enhance the docking accuracy (for example, FlexE, FITTED). These composite structure ensembles can be implemented in different ways. Within FlexE, multiple crystal structures are merged where similar, while the dissimilar areas are marked as different alternatives. Upon docking, the ligand is compared with each alternative and the best scoring protein structure is selected. Within FITTED, the evolution of the individual allows for the crossover of the conformation of the backbone and active site residues independently of each other. An induced-fit docking method has also been developed (SLIDE), removing clashes between the ligand and the protein by directed, single bond rotation of either the ligand or the side chains of the protein.

Although conformational flexibility of proteins is now implemented in some programmes, one major challenge remains to be addressed. As soon as the correct protein structure is found, it must be attributed a score. Considering the subtle protein entropy and enthalpy changes upon binding is very challenging. To our knowledge, ICM is the only programme that approximates these two contributions (Abagyan and Totrov, 1994a), while FITTED implicitly considers the entropy cost by reducing the interaction energy between the docked ligand and flexible residue side chains but disregards the protein enthalpy change. ROSETTALIGAND computes all the intramolecular interactions of the proteins when reconstructing and evaluating side-chain conformations.

Water molecules

The presence of water molecules critical to the binding of ligands adds another dimension to the docking problem (Ladbury, 1996; van Dijk and Bonvin, 2006a; Barillari et al., 2007; Li and Lazaridis, 2007). Although these water molecules are often removed, other options exist. First, one can keep them explicitly. This approach has the drawback of preventing the docking of ligands requiring displacement of a water molecule. Another approach is to implement a protocol that will select the best option (keep or displace each of the water molecules). Such methods have only been implemented in GOLD, FlexX and SLIDE in which waters are toggled on or off, and in FITTED, which makes use of a specific potential energy term. The FlexX method is unique in the way it ‘predicts' potential locations of water molecules, thereby not relying on their crystallographic positions.

Unfortunately, these approaches have not been extensively applied and validated by independent studies. In addition, water locations should be optimized upon docking of ligands, a property that has only partially been addressed by SLIDE. The latter translates the water molecule to remove collisions with protein or ligand atoms but, to our knowledge, does not consider the energetics associated with these moves. Once more, scoring the presence/absence of water molecules is a challenge that remains to be dealt with. The key issue in this context is the consideration of the binding free energy of these specific water molecules. For instance, FITTED, GOLD and SLIDE add a penalty for each displaced water molecule to the final score. However, each water molecule has a different free energy of binding that cannot be accurately represented by a single value (Gohlke and Klebe, 2002).

Covalently bound inhibitors

Although many enzymes are targeted by covalent inhibitors (reversible or irreversible), little effort has been dedicated to this specific area. For instance, GOLD (Jones et al., 1997), FlexX (Kramer et al., 1999) and AutoDock allow the user to manually select which atoms of the ligand and protein are to be joined by a covalent bond and add the protein atom to the ligand input file. However, these methods are hardly amenable to VS of large libraries of compounds given that the user must manually locate the covalent bond and insert it in each individual input file. MacDOCK combines ligand–protein docking (DOCK 4.0) with ligand–ligand superposition to mimic the pose of a covalently bound inhibitor (using the module MIMIC). Within MacDOCK, anchor-guided AG-DOCK includes the location and directionality of the atoms involved in the covalent bond in the docking (Fradera et al., 2004). The functionalities within the ligand that can form covalent bonds and the site of covalent binding of the protein are automatically identified. The geometrical arrangement is then modified to match that of the product structure. If several functional groups can be used, poses are generated for each one and docked to generate a set of product structures, which are then compared. The SF is modified to ignore the van der Waals clashes at the site of formation of the covalent bond. To our knowledge, no other programmes consider the formation of a covalent bond upon ligand binding, and scoring the formation of this bond remains to be explored.

Nucleic acids

Proteins have been the major targets of docking methods. However, nucleic acids are also targets for medicinal chemistry and should be further investigated. One of the major challenges lies in the fact that binding pockets on nucleic acids are highly charged and more solvent exposed than the binding sites of most proteins, which can be deeply buried. Docking DNA intercalators is even more challenging, as a major opening of the bases must occur to bind the ligand. The first attempts for docking to nucleic acids relied on the use of existing programmes originally developed for docking small molecules to proteins. For instance, Chen et al. (1997) used DOCK 3.5 to identify possible binders to the RNA major groove, while Leclerc and Karplus (1999) used the MCSS method. The authors noted that the effect of solvent on the ligand–RNA interaction was not evaluated and represented a necessary improvement. In just a few cases, existing programmes were modified to account for the high polarity of nucleic acids (Pan et al., 2003; Kang et al., 2004).

In addition to the high charge density, flexibility of the nucleic acid strands and presence of key water molecules have been some of the hurdles to the successful applications of docking programmes in this area. In fact, flexibility and water molecules were found to be necessary to improve docking accuracy (Moitessier et al., 2006b). DNA flexibility has been addressed using the HADDOCK programme and semi-flexible refinement using simulated annealing methods following rigid docking (van Dijk et al., 2006b). To our knowledge, few SFs have been specifically designed to score RNA binders. HADDOCK has its own SF, HADDOCK Score, which includes FF terms, solvation energies and a buried surface area term. The DNA conformations are scored using deformation energies calculated from statistical preferences. RiboDOCK evaluates specific nucleic acid–ligand interactions such as aromatic stacking onto the nucleic acid bases and interaction between guanidinium and carbonyl functions. In these two programmes, the water molecules are not directly addressed (Morley and Afshar, 2004). Chen et al. (2004) developed an SF that can identify native protein–RNA structures from the incorrectly docked decoys based on hydrogen-bonding geometries and scores, which they believe to be nucleic acid specific. Finally, Ge and co-workers developed a knowledge-based SF that considers the binding patterns of ligands around individual bases. This approach does not directly address the presence of water, instead considers the frequency of the observed water at a hydration site (Ge et al., 2005).

Entropy

Computational evaluation of the entropy is often based on normal mode analysis or other CPU-expensive approaches. Using such approaches would significantly reduce the throughput of docking methods. To date, the translational and rotational entropy loss estimated in the range of 15 to 20 kJ mol⁻¹ is ignored (Murray and Verdonk, 2002) and most SFs consider the number of rotatable bonds as an estimate of the entropic contribution to the binding free energy. However, not all the bonds are equal and penalties should be assigned accordingly. Indeed, ChemScore considers rotatable bonds in the context of their hydrophobicity; apolar portions tend be more flexible in a binding site than polar fragments and are less penalized. A more advanced approach has recently been proposed that evaluates the restriction of the ligand conformational space by considering a large number of poses (Ruvinsky, 2007) at the expense of CPU time. As mentioned above, the changes in protein entropy are seldom implemented in available programmes (for example, ICM and FITTED).

Solvation

Apart from the bridging water molecules, the effect of solvation by bulk water is a key player in ligand protein binding. Although fairly accurate methods have been reported such as GB/SA and PB/SA (Taylor et al., 2003; Morreale et al., 2007), their implementation in a given docking programme affects the time required to perform a single run unless pre-computed on a grid (DOCK 5) or on the unbound protein (FITTED 2.0). In order to reduce the CPU cost, one can look at ligand lipophilicity as a measure of its desolvation without considering the protein desolvation, which can be assumed to be constant regardless of the pose of the docked ligand. However, larger ligands imply larger desolvation of the binding site and various poses for a single ligand can require the desolvation of different residues. Approaches with higher throughput have been devised, such as the use of atom-based solvation parameters evaluating the buried surface, as exemplified by the AutoDock and Hammerhead SFs, which also penalize the hydrogen bond-capable sites that are not fulfilled. Solvation is known to be partly proportional to the change in solvent-exposed surface area. Based on this concept, DrugScore and ICM evaluate the solvation contribution to the free energy of binding using a term proportional to this area.

Metals

Many groups have looked closely at the scoring of metalloprotease inhibitors (Hu et al., 2004; Irwin et al., 2005; Khandelwal et al., 2005; Englebienne et al., 2007; Jain and Jayaram, 2007) and identified the scoring of metal coordination as a major issue in SFs. In fact, metals can adopt various coordination geometries and docking to metals remains a challenge. Coordination templates have been exploited by GOLD to orient the docking towards potential coordination geometries. However, this has been found to increase only marginally the docking accuracy (Englebienne et al., 2007). As soon as the pose is generated, scoring the binding to the metal is the next issue, and very few SFs such as the eHiTS SF include a specific metal binding term. Only eHiTS, SLIDE and Surflex SF take the geometry restraints into account. Again, evaluation of the impact of these additional terms has not been often investigated, and one cannot state that they significantly improve the docking/scoring accuracy.

Specific interactions

Most of the developed programmes account for the most common interactions such as hydrogen bonds, hydrophobic and ionic interactions. However, the seldom studied and observed interactions, such as cation–π interactions (Scharer et al., 2005) or CH–π interactions (Gil et al., 2007), are not captured by the available SFs and specific functions remain to be developed. This lack is due in part to the low number of complexes featuring these interactions in the PDB. Other interactions such as weak hydrogen bonds required considerable efforts to be considered by SFs or docking programmes (Verdonk et al., 2004).

Training set quality

An accurate SF has to be trained and tested against a high quality set of protein ligand complexes and care must be taken to select the protein–ligand complexes (Hartshorn et al., 2007). Efforts are being made to prepare large databases of high-quality protein–ligand complexes (for example, PDBbind, BindingDB, Relibase) to calibrate SFs. However, automated preparation of some of these publicly available large data sets does not allow for their refined preparation. For instance, a careful identification and orientation of bridging water molecules or the optimal protonation state of each residue and ligand has rarely been carried out, even though it has a great impact on the accuracy of the derived SF. In addition, some of the biological data collected is difficult to interpret and use, such as the very different affinity values reported for an adenosine deaminase inhibitor (PDB codes 1fkx and 2ada; K_i=6 mM and 0.1 pM, respectively).

The PDBbind-refined set of 800 complexes has been used to assess the accuracy of available SFs (Wang et al., 2004). In contrast to our unpublished study, Wang and co-workers have found that all the SFs were poorly predictive for HIV-1 protease inhibitors. In fact, this particular enzyme binds ligands with the need of a water molecule and ligand-dependent protonation state of the two catalytic aspartic acids (Yamazaki et al., 1994; Kulkarni and Kulkarni, 1999; Czodrowski et al., 2007). However, in their protocol, water molecules were all removed and protonation was automatically assigned to ionized aspartic acids.

A disadvantage of using crystal structures of protein–ligand complexes to derive predictive methods is that for the former to exist, the ligand needs to bind to the protein to a certain extent. As a result, SFs are not trained with non-binders. As obvious as it sounds, SF should be trained against completely inactive compounds as well in order to be able to discriminate the latter. In a recent development, Pham and co-workers recently showed that docked compounds can help train an SF against inactive compounds (Pham and Jain, 2006).

Although various sets have been reported and made available, the ultimate benchmark set should be a set for cross-docking studies. As mentioned above, self-docking is not representative of a true medicinal chemistry scenario, while cross-docking is a more reliable evaluation of a programme's performance.

Docking to homology models

It has been observed that moving from crystal structures to homology models often leads to a decrease in accuracy. Solutions to reduce this discrepancy by inclusion of additional information (QSAR data in DRAGHOME; Schafferhans and Klebe, 2001, and SAR information in DoMCoSAR; Vieth and Cummins, 2000) have been proposed although not extensively validated. Rognan and co-workers successfully used homology models of G-protein-coupled receptors to dock antagonists to three human G-protein-coupled receptors; the docking of agonists remains elusive (Bissantz et al., 2003). More recently, Ferrara and Jacoby (2007) evaluated the docking accuracy on homology models and showed that the accuracy can in some cases be as high as with crystal structures.

Conclusion

We have reviewed herein the state-of-the-art in the field of small-molecule-protein docking and scoring methods. Comparative studies, the only available tools to evaluate relative performance of docking programmes, are often carried out by chemists with varying expertise and using different metrics to define accuracy/success. Conclusions and results are difficult to extrapolate to other proteins. An interesting proposal would be to set up a large blind competition analogous to CAPRI in the field of protein–protein docking (http://capri.ebi.ac.uk/). Overall, these comparative studies indicate that none of the docking programmes truly outperforms the others, and the high accuracy observed for ICM and MVP in the two largest and most recent comparative studies remains to be cross-validated.

Ultimately, docking/scoring programmes should be able to identify novel potential ‘binders' very accurately. However, to reach this goal, many issues, including those discussed throughout this review, have yet to be addressed. Currently other strategies, such as post-docking strategies or smart selection of docked compounds, are used to reduce the number of false positive and negatives.

Glossary

FF

force field

PDB

Brookhaven protein databank

r.m.s.d.

root mean-square deviation

SF

scoring function

VS

virtual screening