Cortical hierarchies, sleep, and the extraction of knowledge from memory

Odin the Allfather had in his service two great ravens. These ravens' names were Hugin (Thought) and Munin (Memory) and every morning at dawn they would fly off over Midgard (the world) in search of news and information to learn more about humans and their activities. At sundown, they would return to Odin where they would perch one on each of Odin's shoulders, and whisper into his ears all that they had seen and heard. Experience, stored in the brain as memory, is the raw material for intelligence and thought. It has been suggested that at sundown (i.e., during sleep) the brain adjusts its own synaptic matrix to enable adaptive responses to future events by a process of gradient descent optimization, involving repeated reactivations of recent and older memories and gradual adjustment of the synaptic weights. Memory retrieval, thought, and the generation of adaptive behavioral responses involve globally coordinated trajectories through the neuronal state-space, mediated by appropriate synaptic linkages. Artificial neural networks designed to implement even the most rudimentary forms of memory and knowledge extraction and adaptive behavior incorporate massively and symmetrically interconnected nodes; yet, in the cerebral cortex, the probability of a synaptic connection between any two arbitrarily chosen cells is on the order of 10(-6), i.e., so close to zero that a naive modeler might neglect this parameter altogether. The probability of a symmetric connection is even smaller (10(-12)). How then, are thought and memory even possible? The solution appears to have been in the evolution of a modular, hierarchical cortical architecture, in which the modules are internally highly connected but only weakly interconnected with other modules. Appropriate inter-modular linkages are mediated indirectly via common linkages with higher level modules collectively known as association cortex. The hippocampal formation in the temporal lobe is the highest level of association cortex. It generates sequentially coupled patterns unique to the location and content of experience, but which do not contain the actual stored data. Rather, the patterns serve as pointers or 'links' to the data. Spontaneous reactivation of these linking patterns during sleep may enable the retrieval of recent sequences of experience stored in the lower levels of the cortex and the gradual extraction of knowledge from them. In this essay I explore these ideas, their implications, and the neuroscientific evidence for them. (C) 2009 Published by Elsevier B.V.