Feedback Loops and Emergent Behavior

(This post is part of my blog archiving project. This post appeared on blog.mattgauger.com on January 5, 2010.)

This post started out as a review of two books. My friend Zac Witte introduced me to the work of Jeff Hawkins, previously of Palm and now working at a company called Numenta, which is working on a method of creating artificial intelligence. Hawkins wrote a book called On Intelligence in 2005 detailing his research into artificial intelligence up to that point. Zac recommended the book while we were hacking on code at a coffee shop here in Milwaukee.

I countered by recommending the work of Steve Grand, who is responsible for one of my childhood videogame addictions: Creatures. Creatures was an artificial life simulator that was deceptively simple and childlike. Yet behind the scenes, complex biological processes were being simulated and multi-lobed brains were allowing the Norns, the Creatures of the game, to think. At least, think in a rudimentary, simulated way.

Steve Grand wrote a book called Creation: Life and How to Make It detailing his experience in coding the game. It quickly runs through several thought experiments in how one might simulate physics, matter, and chemical reactions. (And indeed, what these simulations say about the real world.) Grand likes to link various disciplines together, and the book easily jumps from transistors and electrical engineering to neurons to chemical reactions and on to more abstract concepts. Grand’s work can be said to be of the field of artificial life, or A-life, and not necessarily artificial intelligence.

Emergent behavior theory suggests that complex behavior emerges from simple behavior. Or rather, that the whole is smarter than the sum of its parts. Think of an ant hill, which does not really have a leader yet manages to survive and adapt through simple actions of the ants. Emergent behavior found its popularity among the MIT Mobile Robot Lab, among other places. Simple behaviors telling a robot how to react to stimuli, for example, to back up its right wheel when the front left bumper is pressed or vice-versa, can produce seemingly intelligent behavior.

Jeff Hawkins began to study AI by reading as many scientific articles as possible. But, he was frustrated by the seeming lack of progress and unwillingness of the AI field to study real neuroscience. Hawkins wanted to study the human brain and use that for inspiration into how to create a general artificial intelligence. Traditional academic approaches to artificial intelligence at that point completely ignored neuroscience. Hawkins does not want to fully simulate the brain at some chemical level, either. Instead, Hawkins studied the neocortex, in his view the most important key to understanding the human brain. His research led him to an architecture he calls Hierarchical Temporal Memory, or HTM, based on how the brain processes patterns and fragments across time rather than whole memories instantaneously.

Hawkins discusses some of the shortcomings of traditional neural network research. Most neural networks, from the 70’s to the present, were simply three stages of neurons: an input stage, a “black box” stage, and an output stage. Neural networks were trained with sample data, in which connections between neurons could strengthen or weaken. But after training, the neurons’ connections are “locked” and no further training occurs. Hawkins likens three stage neural network research to reverse engineering the transistors in a modern computer and being able to build simple amplifier circuits, or perhaps transistor radios. There is interesting behavior in an amplifier circuit, but it will never be a digital computer.

So why are these two books together in one review? Grand is an artifical life video game creator, and not at all interested in the same kind of artificial intelligence that Hawkins hopes to create. Their research seems to be opposed, or at least completely different.

The concept that these two differing authors agree on is the importance of feedback loops. Those traditional three stage neural networks rarely included simulating feedback loops in the neural connection, while the human brain’s neocortex is largely feedback loops. Further, the number of connections coming in from the senses to the brain is far outweighed by the number of connections going back out to the sensory organs, in a nearly 10-to-1 ratio. This is extraordinary if you think that the brain simply processes incoming data passively. It is actively responding and anticipating sensory data.

You remember music in bits and and fragments rather than whole songs. There is a time element to these memories. You can’t remember the whole song all at once, or hear every note at the same time. The same can be said for all memories. Hawkins points out that being able to predict the world led to the brain being organized this way. After all, prediction is a powerful survival skill.

Grand writes:

Living beings are high-order persistent phenomena, which endure through intelligent interaction with their environment. This intelligence is a product of multiple layers of feedback. An organism is therefore a localized network of feedback loops that ensures its own continuation.

We can find feedback loops around us everywhere in nature. One of my favorite examples comes from the book Sync. Fireflies in Thailand will swarm in tall grass, flashing randomly. But they synchronize with each flash, until the whole swarm flashes in unison. How do they do it? There is no leader or conductor telling the fireflies when to flash. They change the rate of their flashing slightly, faster or slower, until they all match and the flashing syncs. Surprising complexity and cooperation can therefore occur from simple, basic behavior.

One of the places where a feedback loop exists in technology is in the TCP/IP protocol. In a simplified view, one could say that the rate of packets being sent is regulated based on signal loss. The machines at both ends of a network connection don’t know anything about the connection between them or how good it is. They simply use the degradation of the signal - whether packets make it through and are responded to - to modulate whether to send more packets or the same packet over. This feedback loop has a speed regulating effect. The feedback loop ensures that the message will get delivered, even over a bad connection.

Feedback loops and emergent behavior are both fields of research that I’m actively interested in learning more about. These books have fueled my thoughts for further exploration of both concepts. While feedback loops and emergent behavior may not have applications in all endeavors, they can be powerful solutions to the right problem.

The research of both authors is interesting and their books are highly readable. They both provide interesting thinking on the brain and the natural world, and I’d like to follow up with more books in these areas. Both books are relatively short are definitely recommended reads. Too many ‘pop science’ books seem to fall to the side of including too little real science, or paraphrasing studies that don’t really have a neuroscience background. I’ve had that issue with recent books on the human brain and learning marketed towards computer programmers: much of it was based on nonscientific surveys or speculation, which was highly disappointing.