Adaptive Relational Networks - A Detailed Model for Effective Cortical Computation
We desire to understand the principles upon which our brains are able to memorize, recall, reason, and interact with their environment. The neural computations underlying these tasks are fundamentally different from how computers work. The electro-chemical foundation of leaky integrate-and-fire neurons is relatively well understood, but it is much less clear how populations of these neurons collectively cooperate so as to be able to learn and reason with the patterns of spiking activity that they are faced with.
In the anatomy of biological neural networks in the neocortex, we find many stereotyped patterns of connectivity between layers and between areas. One relatively uniform feature is the local structure, generally consisting of about four times as many excitatory neurons as inhibitory neurons, with most of the synaptic connections between neurons being extremely local. This results in a local circuit which we refer to as a recurrent competitive network (RCN). Already at the small scale of RCNs, there are many open questions regarding population behavior.
During a recent pilot project we found an unexpectedly close link between the emergent behavior of physiologically realistic RCN models and soft relational computation primitives. This opens the door to progress on the goals stated in the first few sentences of this summary, via the learning and inference capabilities of a form of graphical model known as relational networks. It is interesting to note that behavioral studies have also found that working memory usage in humans correlates with the relational complexity of the task at hand.
This proposal aims to develop and strengthen this nascent connection between biologically faithful neural network simulations and these learning and reasoning abilities. Our plan directly targets the expansion of our understanding of how to construct neurally realistic simulations of what we call adaptive relational networks, which exhibit the learning and reasoning characteristics necessary for eventually scaling up to large-scale inference systems. In the course of this project, we will take the initial concrete steps necessary to use adaptive relational networks to lay a foundation for the development of scalable self-organizing, self-taught, reasoning neural systems.