Comp. Intro. NeuroSci. -- Ch. 2
July 19, 2005
2
Computational Neuroscience and You
Based on: Lytton, From Computer to Brain, ch.2
Computational neuroscience aims at providing theories and models (mathematical, computational) as to how the brain works. It is a new field whose essential paradigms are still the subject of debate. Today, the digital computer, or sometimes the Internet, is cited as a model for brain function. Do these modern mechanisms hold greater promise than prior brain metaphors (a hydraulic system, with pressurized signals coursing in and out; a post office, with information packets being exchanged; or a telephone switchboard, with multiple connecting wires to be variously assorted) for helping us understand our most intimate organ? In many ways, the brain is not much like the standard digital computer. Yet, both as a direct model of certain aspects of brain functioning and as a tool for exploring brain function, the computer enjoys many advantages over previous models.
A.P. Papli ski n
21
Intro. Comp. NeuroSci. -- Ch. 2
July 19, 2005
When we liken the brain to a computer, we mean several things. First, we mean that several definable computer actions are analogues of things that the brain appears to do. Such computer actions include memory, input/output, and representation. Second, we mean that computers have been used to do a variety of tasks that were previously believed to be exclusively the province of human intelligence: playing chess, reading books aloud, recognizing simple objects, performing logical and mathematical symbol manipulations. Finally, although no machine has yet passed the Turing test (a machine passes if it fools a conversation partner into thinking that it is a person), those who work intensively with computers develop a distinct sense of communicating or even communing with the machine.
A.P. Papli ski n
22
Intro. Comp. NeuroSci. -- Ch. 2
July 19, 2005
Intellectual landscape of the Computational Neuroscience according to T. P. Trappenberg
Quantitative models (mathematical) Nonlinear dynamics Information theory
D. S. Touretzky
Refinement feedback
Psychology Psychology Psychology
Experimental facts Facts Experimental predictions Predictions
Psychology Psychology Psychology
Neurophysiology Neurophysiology Neurophysiology
Computational Computational neuroscience Neuroscience
Neurophysiology Neurophysiology Neurophysiology
Neurobiology Neurobiology Neurobiology
Neurobiology Neurobiology Neurobiology
Applications
Refinement feedback
A.P. Papli ski n
23
Intro. Comp. NeuroSci. -- Ch. 2
July 19, 2005
Modeling is the work and play of computational neuroscience, as it is for much of physics, engineering, business, and applied mathematics. To learn something about the thing being modeled, we need to reduce the model to the essentials. If we reduce too far, however, we may miss a critical component that is responsible for interesting properties. We typically aim at modelling the structure or function or behaviour. The brain is difficult to model: it performs many tasks simultaneously and using hidden processes to do them. Therefore, we can model a brain function, such as chess playing, and yet gain little or no insight into how the brain plays chess. The brain is utilizing unconscious properties that we are not aware of when we play chess. We consider various brain features and wonder whether or not these are critical features for information process, for memory, or for thought. Certainly, many aspects of brain design are not critical for brain information processing but are there for other, purposes: metabolism, growth and differentiation, cell repair, and general maintenance. Using the computer as a model to understand the brain raises questions about similarities both in detail and in function.
A.P. Papli ski n
24
Intro. Comp. NeuroSci. -- Ch. 2
July 19, 2005
Both brains and computers process information, but information processing may not be central to the process of thinking. Therefore, we will wish to explore not only differences from the bottom, differences in materials and design principles, but also differences from the top, differences in capacity and capability. Starting with the manufacturing side, there are already a variety of differences that can be explored. Computers are made of sand and metal, while brains are made of water, salt, protein, and fat. The computer chip is built onto a two-dimensional matrix, while the brain fills three dimensions with its wiring. The time and size scales involved are also believed to be vastly different. Computers are made of transistors the brain consists of neurons. With this comparison, the time scales are about 1 ms for the neuron vs. 1 ns for the transistor. The spatial scale is about 1 mm for the largest neuron vs. less than 1 m for a modern CMOS transistor. Thus the neuron is much bigger and much slower. Brains take hints; computers are remarkably stupid if given a slightly misspelled command or incomplete information. The digital computer has a general-purpose architecture that is designed to run many different programs.
A.P. Papli ski n
25
Intro. Comp. NeuroSci. -- Ch. 2
July 19, 2005
The brain, on the other hand, has dedicated, special-purpose circuits that provide great efficiency at solving particular problems quickly. Calculations on a digital computer are done serially, calculating step by step in a cookbook fashion from the beginning to the end of the calculation. The brain, on the other hand, performs many calculations simultaneously, using parallel processing. Digital computers use binary values. The brain seems to work with frequency coded continuous (aka analog) values.
A.P. Papli ski n
26
Intro. Comp. NeuroSci. -- Ch. 2
July 19, 2005
2.1
Origins of computer science and neuroscience
Neuroscience and computer science came into being at about the same time and influenced each other heavily in their formative stages. Over time, the fields have diverged widely and have developed very different notions of seemingly shared concepts such as memory, cognition, and intelligence. D.O. Hebb proposed over 40 years ago that a particular type of use-dependent modification of the connection strength of synapses might underlie learning in the nervous system. The Hebb rule predicts that synaptic strength increases when both the presynaptic and postsynaptic neurons are active simultaneously. Recent explorations of the physiological properties of neuronal connections have revealed the existence of long-term potentiation, a sustained state of increased synaptic efficacy consequent to intense synaptic activity. The conditions that Hebb predicted would lead to changes in synaptic strength have now been found to cause long-term potentiation in some neurons of the hippocampus and other brain areas. As we will see, similar conditions for changing synaptic strength are used in many neural models of learning and memory.
A.P. Papli ski n
27
Intro. Comp. NeuroSci. -- Ch. 2
July 19, 2005
One difference between the neuroscience and computer science view-points has to do with the necessary adoption of a big-picture approach by computer scientists and a reductionist approach by many neuroscientists. These two approaches are typically called top-down and bottom-up, respectively. The top-down approach arises from an engineering perspective: design a machine to perform a particular task. If you're interested in intelligence, then design an artificial intelligence machine. The bottom-up perspective is the province of the phenomenologist or the taxonomist: collect data and organize it. An essential element of biology is the discovery of facts. Hypotheses are then designed to fit these facts together. We concern ourselves with many ideas that have been promulgated for understanding higher levels of nervous system function such as memory. Much of the data on real nervous systems has been gathered from either the peripheral nervous systems of higher animals or from the nervous systems of invertebrates such as worms, leeches, and horseshoe crabs. The low-level source of much of our knowledge of the nervous system contrasts sharply with the ambition to understand the highest levels of mental functioning, and helps explain why some of the topics to be discussed may seem quite remote from human neural function, while other subjects will be very relevant but highly speculative.
A.P. Papli ski n
28
Intro. Comp. NeuroSci. -- Ch. 2
July 19, 2005
2.2
Levels
The notions of bottom-up and top-down approaches to the problem of nervous system function, and the corresponding contrast between acknowledged facts at the lower level and uncertain hypotheses at the higher, lead naturally to hierarchical divisions. Two such divisions that are commonly used are called the levels of organization and levels of investigation. Each of these divisions into levels creates a hierarchy for brain research that leads between the reductionist bottom and the speculative top. The levels-of-investigation analysis was historically a product of top-down thinking. This approach, pioneered by computationalists, starts at the top with the big-picture problem of brain function and drips down to the implementation in neurons or silicon. The levels-of-organization analysis was in part a reaction to this. By putting all of its levels on an equal footing, the levels-of-organization approach invited the investigator to start anywhere and either build up or hypothesize down.
A.P. Papli ski n
29
Intro. Comp. NeuroSci. -- Ch. 2
July 19, 2005
Levels of organization
Levels of organization is fundamentally a bottom-up perspective. The basic observation that leads to this division of the knowledge comes from the "grand synthesis" that connected the physical with the vital world. Modern biology explains genetics and physiology in terms of the interactions of molecules. This allows connections to be made all the way from physics to physiology. The basic concepts of biology can be understood from the concepts of physics, while the converse is not the case. However, understanding biology directly from physics would be a hopeless task for two reasons. First, there is no way one can predict what would occur in a biological system using knowledge of atoms and electron orbits. Second, the conceptual leap from physics to biology is simply too great to be made without interposed models from other fields.
A.P. Papli ski n
210
Intro. Comp. NeuroSci. -- Ch. 2
July 19, 2005
Specifically, much of biology can be understood from cell biology, which can be understood from molecular biology, which can be understood from biochemistry, which can be understood from organic chemistry, which can be understood from physical chemistry, which can be understood from physics. In comparison with this known hierarchy of knowledge, the levels of organization of the nervous system remain tentative. Any hierarchy will likely embody a fundamental, trivial law: big things are built out of smaller things. Following the scheme of others, we can build a hierarchy of levels of organization and levels of study. From smallest to largest: object of study study method physics ions chemistry transmitters and receptors cell biology neurons computer science networks neurology systems psychology behavior or thought
A.P. Papli ski n
211
Intro. Comp. NeuroSci. -- Ch. 2
July 19, 2005
Some levels of organization in the central nervous system on different scales:
from Trappenberg: Fundamentals of Computational Neuroscience
1m
CNS
PFC PMC
An outline of the brain A system-level model of working memory A Self-Organizing Map Speculation about the circuit behind orientation-sensitive neurons in V1 by Hubel and Wiesel A compartmental model of neuron
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + +
10 cm
System
HCMP
1 cm
Maps
1 mm
Networks
100 m
Neurons
1 m
H H2 N C R O C OH
Synapses
+ + + + + +
+ + + +
A chemical synapse An amino acid molecule
1A
Molecules
A.P. Papli ski n
212
Intro. Comp. NeuroSci. -- Ch. 2
July 19, 2005
Although the general order of dependencies in the nervous system can be assumed to be based on size and the simple inclusion of one structure within another, the exact structures that are of functional importance are not clear. At the top, one can choose to regard either behavior or internal mental representation as the highest level suitable for scientific investigation. Behavioralists believe that since physical movement is the only measurable evidence of nervous system function, this is the only appropriate area of high-level functional study. Other psychologists believe that putative representations internal of the external world are also suitable subjects of investigation, even though these cannot be measured directly. Computational approaches generally make the latter assumption, not only postulating internal representations but often making them the central question for further study. At the small end of the organizational scale, most investigators would consider the concentrations of ions and neurotransmitters and their channels and receptors to be the smallest pieces of nervous system that are worth paying any attention to. To go back to the physics-to-biology spectrum described above, the conceptual jump from the concepts of physics to the concepts of organic chemistry would not be possible without the intermediate concepts developed by physical chemistry. This is because the representations of electron orbitals and chemical bonds used in physical chemistry provide conceptual links between the detailed equations describing electron orbitals used in physics,
A.P. Papli ski n
213
Intro. Comp. NeuroSci. -- Ch. 2
July 19, 2005
and the schematic stick diagrams used for bonds in organic chemistry. Similarly, the neuroscience levels of organization suggests that neurons can be adequately described by taking account of properties at the level of transmitters and receptors. That's probably not going to turn out to be true. It's likely that intermediate-sized ultrastructural components of the neuron such as spines, dendrites, and synapses may have their own critical properties that cannot be understood without independent study of these structures in themselves. As we move up the scale, higher levels of neural organization are less well understood and can be farmed out somewhat arbitrarily to various interested specialty areas. Much study of networks has come out of computer science, but the organization of networks is also studied in mathematics by geometry and topology. The level of cortical columns is not shown in this diagram. It is unclear whether this level would go below or above the level of the network. I gave systems to neurology, a clinical field that subdivides brain function into motor, sensory, and various cognitive systems based on changes seen with brain damage. Engineers mean something different when they study systems in the field called "signals and systems." Systems neuroscience has yet another connotation, referring to neurophysiological techniques related to investigating the origins of perception and behavior.
A.P. Papli ski n
214
Intro. Comp. NeuroSci. -- Ch. 2
July 19, 2005
Levels of investigation
The levels-of-investigation approach comes from David Marr, a computationalist who produced some very influential early models of different brain areas. This viewpoint is from the top down. The top level is the level of problem definition (this was called the computational-theoretic level by Marr). Marr suggested that understanding any particular brain function requires that we first understand what problem the brain is solving. Problem in hand, we can deduce what additional information the brain would need to solve it. The next level is that of algorithm definition. An algorithm is like a cookbook recipe, defining a step-by-step approach to accomplish some task. The third and final level is the level of implementation, where the algorithm is finally translated into machinery, whether neural or silicon, that can actually perform the task. Marr's three levels of problem, algorithm, and implementation are the current approach a software engineer would take in designing a big program (e.g., a word processor or a Net browser) using a modern computer language. This Marr trinity of problem, algorithm, and implementation can be collapsed into the familiar concepts of software and hardware. A problem is provided. Algorithms are written into the software. The software is compiled so as to run on a computer -- the physical implementation level.
A.P. Papli ski n
215
Intro. Comp. NeuroSci. -- Ch. 2
July 19, 2005
2.3
The neural code
The brain denies the philosopher's, the mathematical modeler's, and the guy-on-the-street's desire for clarity and simplicity. Since there is no single overarching task for the brain to do, different facets of brain function must be studied separately. This does not necessarily mean that there are no unifying principles. There may well be basic neural codes that are used throughout the animal kingdom. However, the discovery of neural codes will no more free us from the need for further research into different brain areas than the discovery of the genetic code revealed all the functions of all enzymes and structural proteins. The analogy between the search for the genetic code and the search for a neural code has been highlighted by Francis Crick, discoverer of the former and pursuer of the latter. To doubters, he points out that the quest for a simple genetic code seemed quixotic to anyone who considered that the complexity of the natural design encompasses enzymes and organs, growth and development. Of course, the discovery of a simple genetic code did not in any way provide an understanding of all the things that are coded for. It did, however, provide a powerful new tool for exploring these things.
A.P. Papli ski n
216
Intro. Comp. NeuroSci. -- Ch. 2
July 19, 2005
Similarly, the discovery of neural code or codes will not tell us how any part of the brain works, but will enable us to start to understand what we see when we amplify electrical signals from different parts of the brain. Several neural signals are well established. However, some of these signals probably carry no information at all, while other signals carry information that is not used by the brain or body. For example, the electroencephalogram (EEG) is a very well studied signal that is emitted by the brain. There is information in the EEG that permits an outside observer to determine whether a brain is awake or asleep or even, after some signal processing, whether the brain is hearing clicks or seeing a flashing checkerboard. These field signals are generally an epiphenomenon, a side effect that has no functional relevance. These signals are not used within the brain under normal circumstances, and are too weak to be used for telepathy, no matter how close you put the two heads together. There are some cases where the field is used or misused. Some neurons in the goldfish communicate internally with such field effects. Field effects are used to communicate between individuals in the weakly-electric fish (the strongly electric fish use their fields to stun prey). Field effects are also responsible for pathological signaling in cases of epilepsy and multiple sclerosis. However, in general the EEG can be considered an information-carrying neural signal that is not used internally as a neural code.
A.P. Papli ski n
217
Intro. Comp. NeuroSci. -- Ch. 2
July 19, 2005
Various signals are used directly by the brain and therefore can be considered to be codes. For example, the rate of spiking of neurons carries information that determines how powerfully a muscle will contract. This is a code that has been cracked: the nerve tells the muscle "squeeze . . . squeeze harder." It appears likely that similar rate coding is also used in the central nervous system. Rate coding has also been suggested to be the primary code in parts of sensory cortex. Neurons in visual cortex spike fastest when presented with oriented bars of a certain configuration, and auditory cortex neurons will spike faster in response to particular sou...