A Visit From Jeff Hawkins: Hierarchical Temporal Memory < Beckman Blog.
Recently we were lucky enough at the Beckman Institute to get a visit from Jeff Hawkins, the cofounder of Numenta Inc. and founder of the Redwood Center for Neuroscience. Jeff Hawkins had an early interest in the brain and biological intelligence since college, but took a break in order to invent the palm pilot, the grandfather of all our modern handheld computer systems. Since starting Numenta Inc., Jeff has focused his efforts at creating a biologically realistic model of the cortex that can be applied to information processing problems; a framework he calls Hierarchical Temporal Memory (HTM). The main feature of this artificial intelligence system is a hierarchically interconnected web of identical neuron elements, each simulating what can be thought of as a cortical column.
At each level of this hierarchy, incoming information is represented in a sparse distributed fashion, such that only a minimal number of “neurons” are active. These neurons then can pass this representation to the next level, which will transform it in turn to its own sparse distribution, with abstraction and consolidation increasing hierarchically. One of the most important aspects of this HTM framework is the temporal aspect. After each layer gets its input, it automatically make a prediction about what the next input will be. Thus these HTM networks can come to learn increasingly complex patterns even if they are extended in time. Given their extraordinary ability to acquire new dynamic patterns of information, as well as the robustness of the sparse representations to system or input noise, Jeff sees great promise for the application of this technology. He stressed very much that this may be the ground level of a paradigmatic change occurring in our technological innovation, and encouraged us young researchers to consider these possible applications and how they might come to shape our future world.
I also had the chance of having breakfast with Jeff, along with a number of other post-doc and graduate students from the Beckman Institute. The students ranged from cognitive neuroscientists like me, who study awake behaving human brains in action, to systems level neuroscientists, to artificial intelligence researchers. This diverse group provided Jeff with a great indication of the interdisciplinary nature of the Institute, and provided for an exciting range of breakfast discussion. I was very interested in Jeff’s thoughts on how attention can be instantiated in his HTM networks. He proposed modeling attention as a gatekeeper of information as it passes between layers of the network, with a role very much like the thalamus seems to play in an actual brain. Given my personal interest in our awareness and conscious experience, I had to ask Jeff what his thoughts were on how the brain creates our vivid experiential world. He didn’t consider it much of an issue, assuming that any ability to reflect on the past states of the network could be construed as a conscious, reflective experience. I am not sure I am convinced by this answer. I think there is more to it than that…