Abstract
We describe a hierarchical, generative model that can be viewed as a nonlinear generalization of factor analysis and can be implemented in a neural network. The model uses bottom-up, top-down and lateral connections to perform Bayesian perceptual inference correctly. Once perceptual inference has been performed the connection strengths can be updated using a very simple learning rule that only requires locally available information. We demonstrate that the network learns to extract sparse, distributed, hierarchical representations.
Full Text
The Full Text of this article is available as a PDF (1.4 MB).
Selected References
These references are in PubMed. This may not be the complete list of references from this article.
- Becker S., Hinton G. E. Self-organizing neural network that discovers surfaces in random-dot stereograms. Nature. 1992 Jan 9;355(6356):161–163. doi: 10.1038/355161a0. [DOI] [PubMed] [Google Scholar]
- Durbin R., Willshaw D. An analogue approach to the travelling salesman problem using an elastic net method. Nature. 1987 Apr 16;326(6114):689–691. doi: 10.1038/326689a0. [DOI] [PubMed] [Google Scholar]
- Hinton G. E., Dayan P., Frey B. J., Neal R. M. The "wake-sleep" algorithm for unsupervised neural networks. Science. 1995 May 26;268(5214):1158–1161. doi: 10.1126/science.7761831. [DOI] [PubMed] [Google Scholar]
- Olshausen B. A., Field D. J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature. 1996 Jun 13;381(6583):607–609. doi: 10.1038/381607a0. [DOI] [PubMed] [Google Scholar]