Skip to main content
PMC home page
Environmental Health Perspectives logoLink to Environmental Health Perspectives
. 2003 Aug;111(10):1361–1375. doi: 10.1289/ehp.5758

Methods for reliability and uncertainty assessment and for applicability evaluations of classification- and regression-based QSARs.

Lennart Eriksson 1, Joanna Jaworska 1, Andrew P Worth 1, Mark T D Cronin 1, Robert M McDowell 1, Paola Gramatica 1
PMCID: PMC1241620  PMID: 12896860

Abstract

This article provides an overview of methods for reliability assessment of quantitative structure-activity relationship (QSAR) models in the context of regulatory acceptance of human health and environmental QSARs. Useful diagnostic tools and data analytical approaches are highlighted and exemplified. Particular emphasis is given to the question of how to define the applicability borders of a QSAR and how to estimate parameter and prediction uncertainty. The article ends with a discussion regarding QSAR acceptability criteria. This discussion contains a list of recommended acceptability criteria, and we give reference values for important QSAR performance statistics. Finally, we emphasize that rigorous and independent validation of QSARs is an essential step toward their regulatory acceptance and implementation.

Full Text

The Full Text of this article is available as a PDF (328.6 KB).

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Berglund A., De Rosa M. C., Wold S. Alignment of flexible molecules at their receptor site using 3D descriptors and Hi-PCA. J Comput Aided Mol Des. 1997 Nov;11(6):601–612. doi: 10.1023/a:1007983320854. [DOI] [PubMed] [Google Scholar]
  2. Cooper J. A., 2nd, Saracci R., Cole P. Describing the validity of carcinogen screening tests. Br J Cancer. 1979 Jan;39(1):87–89. doi: 10.1038/bjc.1979.10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Cronin Mark T. D., Jaworska Joanna S., Walker John D., Comber Michael H. I., Watts Christopher D., Worth Andrew P. Use of QSARs in international decision-making frameworks to predict health effects of chemical substances. Environ Health Perspect. 2003 Aug;111(10):1391–1401. doi: 10.1289/ehp.5760. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Cronin Mark T. D., Walker John D., Jaworska Joanna S., Comber Michael H. I., Watts Christopher D., Worth Andrew P. Use of QSARs in international decision-making frameworks to predict ecologic effects and environmental fate of chemical substances. Environ Health Perspect. 2003 Aug;111(10):1376–1390. doi: 10.1289/ehp.5759. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Cruciani G., Watson K. A. Comparative molecular field analysis using GRID force-field and GOLPE variable selection methods in a study of inhibitors of glycogen phosphorylase b. J Med Chem. 1994 Aug 5;37(16):2589–2601. doi: 10.1021/jm00042a012. [DOI] [PubMed] [Google Scholar]
  6. Eriksson Lennart, Johansson Erik, Lindgren Fredrik, Sjöström Michael, Wold Svante. Megavariate analysis of hierarchical QSAR data. J Comput Aided Mol Des. 2002 Oct;16(10):711–726. doi: 10.1023/a:1022450725545. [DOI] [PubMed] [Google Scholar]
  7. Feinstein A. R. Clinical biostatistics XXXI. On the sensitivity, specificity, and discrimination of diagnostic tests. Clin Pharmacol Ther. 1975 Jan;17(1):104–116. doi: 10.1002/cpt1975171104. [DOI] [PubMed] [Google Scholar]
  8. Giraud E., Luttmann C., Lavelle F., Riou J. F., Mailliet P., Laoui A. Multivariate data analysis using D-optimal designs, partial least squares, and response surface modeling: A directional approach for the analysis of farnesyltransferase inhibitors. J Med Chem. 2000 May 4;43(9):1807–1816. doi: 10.1021/jm991166h. [DOI] [PubMed] [Google Scholar]
  9. Golbraikh Alexander, Tropsha Alexander. Beware of q2! J Mol Graph Model. 2002 Jan;20(4):269–276. doi: 10.1016/s1093-3263(01)00123-1. [DOI] [PubMed] [Google Scholar]
  10. Gramatica P., Corradi M., Consonni V. Modelling and prediction of soil sorption coefficients of non-ionic organic pesticides by molecular descriptors. Chemosphere. 2000 Sep;41(5):763–777. doi: 10.1016/s0045-6535(99)00463-4. [DOI] [PubMed] [Google Scholar]
  11. Hong Huixiao, Tong Weida, Fang Hong, Shi Leming, Xie Qian, Wu Jie, Perkins Roger, Walker John D., Branham William, Sheehan Daniel M. Prediction of estrogen receptor binding for 58,000 chemicals using an integrated system of a tree-based model with structural alerts. Environ Health Perspect. 2002 Jan;110(1):29–36. doi: 10.1289/ehp.0211029. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Kulkarni A., Hopfinger A. J., Osborne R., Bruner L. H., Thompson E. D. Prediction of eye irritation from organic chemicals using membrane-interaction QSAR analysis. Toxicol Sci. 2001 Feb;59(2):335–345. doi: 10.1093/toxsci/59.2.335. [DOI] [PubMed] [Google Scholar]
  13. Linusson A., Gottfries J., Lindgren F., Wold S. Statistical molecular design of building blocks for combinatorial chemistry. J Med Chem. 2000 Apr 6;43(7):1320–1328. doi: 10.1021/jm991118x. [DOI] [PubMed] [Google Scholar]
  14. Livingstone DJ. The characterization of chemical structures using molecular properties. A survey. J Chem Inf Comput Sci. 2000 Mar;40(2):195–209. doi: 10.1021/ci990162i. [DOI] [PubMed] [Google Scholar]
  15. Martin Y. C., Lin C. T., Hetti C., DeLazzer J. PLS analysis of distance matrices to detect nonlinear relationships between biological potency and molecular properties. J Med Chem. 1995 Aug 4;38(16):3009–3015. doi: 10.1021/jm00016a003. [DOI] [PubMed] [Google Scholar]
  16. McDowell R. M., Jaworska J. S. Bayesian analysis and inference from QSAR predictive model results. SAR QSAR Environ Res. 2002 Mar;13(1):111–125. doi: 10.1080/10629360290002280. [DOI] [PubMed] [Google Scholar]
  17. Shi L., Tong W., Fang H., Xie Q., Hong H., Perkins R., Wu J., Tu M., Blair R. M., Branham W. S. An integrated "4-phase" approach for setting endocrine disruption screening priorities--phase I and II predictions of estrogen receptor binding affinity. SAR QSAR Environ Res. 2002 Mar;13(1):69–88. doi: 10.1080/10629360290002235. [DOI] [PubMed] [Google Scholar]
  18. Topliss J. G., Edwards R. P. Chance factors in studies of quantitative structure-activity relationships. J Med Chem. 1979 Oct;22(10):1238–1244. doi: 10.1021/jm00196a017. [DOI] [PubMed] [Google Scholar]
  19. Worth A. P., Balls M. The importance of the prediction model in the validation of alternative tests. Altern Lab Anim. 2001 Mar-Apr;29(2):135–144. doi: 10.1177/026119290102900210. [DOI] [PubMed] [Google Scholar]

Articles from Environmental Health Perspectives are provided here courtesy of National Institute of Environmental Health Sciences

RESOURCES