The remarkable structural homogeneity of isocortex strongly suggests a canonical cortical algorithm that performs the same essential function in all regions [1]. That function is widely construed/modeled as probabilistic inference, i.e., the ability, given an input, to retrieve the best-matching memory (or, most likely hypothesis) stored in memory. In [2], I described a cortical model for which both storage (learning) of new items into memory and probabilistic inference are constant time operations, which is a level of performance not present in any other published information processing system. This efficiency depends critically on: a) representing inputs with sparse distributed representations (SDRs), i.e., relatively small sets of binary units chosen from a large pool; and on b) choosing (learning) new SDRs so that more similar inputs are mapped to more highly intersecting SDRs. The macrocolumn (specifically, its pool of L2/3 pyramidals) was proposed as the large pool, with its minicolumns acting in winner-take-all fashion, ensuring that macrocolumnar codes consist of one winner per minicolumn. Here, I present results of large hierarchical model instances, having many levels and hundreds of macrocolumns, performing: a) single-trial learning of sets of sequences derived from natural video; and b) immediate (i.e., no search) retrieval of best-matching stored sequences. Figure 1 shows the major shift in going from the localist coding scheme present in most hierarchical cortical models, e.g., [3], to SDR coding. In Figure 1A, each feature in a coding module (red rectangle) is represented by a single unit, whereas in Figure 1B, each feature in a coding module (red hexagon) is represented by a set of co-active units, one per minicolumn. Yellow call-outs show a sample suggesting the potentially large number of other features stored in a macrocolumn. This change has a potentially large impact on explaining the storage capacity of cortex, but more importantly on explaining the speed and other characteristics of probabilistic/approximate reasoning possessed by biological brains.
Figure 1.
Comparison of localist (A) and SDR-based (B) versions of visual feature hierarchies.
References
- Douglas RJ, Martin KA, Witteridge D. A canonical microcircuit for neocortex. Neural Computation. 1989;1(4):480–488. doi: 10.1162/neco.1989.1.4.480. [DOI] [Google Scholar]
- Rinkus GJ. A cortical sparse distributed coding model linking mini- and macrocolumn-scale functionality. Frontiers in Neuroanatomy. 2010;4(17) doi: 10.3389/fnana.2010.00017. doi:10.3389/fnana.2010.00017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Giese MA, Poggio T. Neural Mechanisms for the Recognition of Biological Movements. Nature Reviews Neuroscience. 2003;4(3):179–192. doi: 10.1038/nrn1057. [DOI] [PubMed] [Google Scholar]