Abstract
The present regulatory climate has led to increasing demands for scientists to attest to the predictive reliability of numerical simulation models used to help set public policy, a process frequently referred to as model validation. But while model validation may reveal useful information, this paper argues that it is not possible to demonstrate the predictive reliability of any model of a complex natural system in advance of its actual use. All models embed uncertainties, and these uncertainties can and frequently do undermine predictive reliability. In the case of lead in the environment, we may categorize model uncertainties as theoretical, empirical, parametrical, and temporal. Theoretical uncertainties are aspects of the system that are not fully understood, such as the biokinetic pathways of lead metabolism. Empirical uncertainties are aspects of the system that are difficult (or impossible) to measure, such as actual lead ingestion by an individual child. Parametrical uncertainties arise when complexities in the system are simplified to provide manageable model input, such as representing longitudinal lead exposure by cross-sectional measurements. Temporal uncertainties arise from the assumption that systems are stable in time. A model may also be conceptually flawed. The Ptolemaic system of astronomy is a historical example of a model that was empirically adequate but based on a wrong conceptualization. Yet had it been computerized--and had the word then existed--its users would have had every right to call it validated. Thus, rather than talking about strategies for validation, we should be talking about means of evaluation. That is not to say that language alone will solve our problems or that the problems of model evaluation are primarily linguistic. The uncertainties inherent in large, complex models will not go away simply because we change the way we talk about them. But this is precisely the point: calling a model validated does not make it valid. Modelers and policymakers must continue to work toward finding effective ways to evaluate and judge the quality of their models, and to develop appropriate terminology to communicate these judgments to the public whose health and safety may be at stake.
Full text
PDF
Selected References
These references are in PubMed. This may not be the complete list of references from this article.
- Davis J. M., Svendsgaard D. J. Lead and child development. Nature. 1987 Sep 24;329(6137):297–300. doi: 10.1038/329297a0. [DOI] [PubMed] [Google Scholar]
- Ducoffre G., Claeys F., Bruaux P. Lowering time trend of blood lead levels in Belgium since 1978. Environ Res. 1990 Feb;51(1):25–34. doi: 10.1016/s0013-9351(05)80180-x. [DOI] [PubMed] [Google Scholar]
- Jasanoff S. Science, politics, and the renegotiation of expertise at EPA. Osiris. 1992;7:195–217. [PubMed] [Google Scholar]
- Leggett R. W. An age-specific kinetic model of lead metabolism in humans. Environ Health Perspect. 1993 Dec;101(7):598–616. doi: 10.1289/ehp.93101598. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Marcus A. H., Elias R. W. Some useful statistical methods for model validation. Environ Health Perspect. 1998 Dec;106 (Suppl 6):1541–1550. doi: 10.1289/ehp.98106s61541. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Miller G. D., Massaro T. F., Massaro E. J. Interactions between lead and essential elements: a review. Neurotoxicology. 1990 Spring;11(1):99–119. [PubMed] [Google Scholar]
- Mushak P. Defining lead as the premiere environmental health issue for children in America: criteria and their quantitative application. Environ Res. 1992 Dec;59(2):281–309. doi: 10.1016/s0013-9351(05)80036-2. [DOI] [PubMed] [Google Scholar]
- O'Flaherty E. J. Physiologically based models for bone-seeking elements. IV. Kinetics of lead disposition in humans. Toxicol Appl Pharmacol. 1993 Jan;118(1):16–29. doi: 10.1006/taap.1993.1004. [DOI] [PubMed] [Google Scholar]
- O'Flaherty E. J. Physiologically based models for bone-seeking elements. V. Lead absorption and disposition in childhood. Toxicol Appl Pharmacol. 1995 Apr;131(2):297–308. doi: 10.1006/taap.1995.1072. [DOI] [PubMed] [Google Scholar]
- Oreskes N., Shrader-Frechette K., Belitz K. Verification, validation, and confirmation of numerical models in the Earth sciences. Science. 1994 Feb 4;263(5147):641–646. doi: 10.1126/science.263.5147.641. [DOI] [PubMed] [Google Scholar]
- Rosner D., Markowitz G. A 'gift of God'?: The public health controversy over leaded gasoline during the 1920s. Am J Public Health. 1985 Apr;75(4):344–352. doi: 10.2105/ajph.75.4.344. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rykiel E. J., Jr The meaning of models. Science. 1994 Apr 15;264(5157):330–331. doi: 10.1126/science.264.5157.330. [DOI] [PubMed] [Google Scholar]
- Van Damme K., Casteleyn L., Heseltine E., Huici A., Sorsa M., van Larebeke N., Vineis P. Individual susceptibility and prevention of occupational diseases: scientific and ethical issues. J Occup Environ Med. 1995 Jan;37(1):91–99. doi: 10.1097/00043764-199501000-00013. [DOI] [PubMed] [Google Scholar]