Abstract
The principle of parsimony, also known as ‘Occam's razor’, is a heuristic dictum that is thoroughly familiar to virtually all practitioners of science: Aristotle, Newton, and many others have enunciated it in some form or other. Even though the principle is not difficult to comprehend as a general heuristic guideline, it has proved surprisingly resistant to being put on a rigorous footing – a difficulty that has become more pressing and topical with the ‘big data’ explosion. We review the significance of Occam's razor in the philosophical and theological writings of William of Ockham, and survey modern developments of parsimony in data science.
Keywords: Occam's razor, William of Ockham, principle of parsimony, large deviations, Kolmogorov complexity, Bayesian inference
Full Text
The Full Text of this article is available as a PDF (309.0 KB).
8. References
- 1.Baake M. (1997) Solution of coincidence problem in dimensions d ≤4. In: Moody R.V. (ed.), The mathematics of long-range aperiodic order, pp. 9–44. Kluwer, Dordrecht. [Google Scholar]
- 2.Feyerabend P.K. (1975) Against method: outline of an anarchist theory of knowledge. New Left Books, New York. [Google Scholar]
- 3.Barnes J. (1984) The complete works of Aristotle, revised Oxford translation, vol. 2. Princeton University Press, Princeton. [Google Scholar]
- 4.Crombie A.C. (1953) Robert Grosseteste and the origins of experimental science 1100–1700. Oxford University Press, Oxford. [Google Scholar]
- 5.Copleston F.C. (1972) A history of medieval philosophy. University of Notre Dame Press, Notre Dame. [Google Scholar]
- 6.Russell B. (1945) A history of Western philosophy. George Allen & Unwin, London. [Google Scholar]
- 7.Quine W.V.O. (1955) Mind, 64, 145–159. [Google Scholar]
- 8.Newton I. (1713) Philosophiæ naturalis principia mathematica, 2nd edition. Phil. Trans. Roy. Soc.Cambridge University Press, Cambridge. http://www.lib.cam.ac.uk/exhibitions/Footprints_of_the_Lion/duty.pdf [accessed 26 June 2018]. [Google Scholar]
- 9.Dorlas T.C. (1999) Statistical mechanics: fundamentals and model solutions. CRC Press, Boca Raton. [Google Scholar]
- 10.den Hollander F. (2000) Large deviations. Fields Institute Monographs, vol. 14. American Mathematical Society, Providence, RI. [Google Scholar]
- 11.Boettinger C., and Hastings A. (2013) Proc. R. Soc. B, 280, 20131372. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Zint N., Baake E., and den Hollander F. (2008) J. Math. Biol., 57, 841–861. [DOI] [PubMed] [Google Scholar]
- 13.Penrose R. (2004) The road to reality. Jonathan Cape, London. [Google Scholar]
- 14.Li M., and Vitányi P. (2008) An introduction to Kolmogorov complexity and its applications. Springer, Berlin. [Google Scholar]
- 15.Gödel K. (1936) Ergebnisse eines mathematischen Kolloquiums, 7, 23–24. [Google Scholar]
- 16.Hofstadter D.R., and Bennet D.C. (1992) The mind's I: fantasies and reflections on self and soul. Bantam Books, New York. [Google Scholar]
- 17.Bain L.J., and Engelhardt M. (2000) Introduction to probability and mathematical statistics. Cengage Learning, Boston. [Google Scholar]
- 18.Gilles D. (2000) Philosophical theories of probability. Routledge, London. [Google Scholar]
- 19.Peñaranda J.J.L. (2013) Philos. Stud., 162, 683–696. [Google Scholar]