scholars from neurobiology, neuroinformatics, and robotics. To study interfaces between perception and action, in one experimental platform a modular structured ANN simulating toad vision (Fingerling et al. 1993)—in connection with a CCD-camera—instructed a robot to select and pick out differently shaped work pieces from a conveyor belt (see also Further Reading, Movie A1).
Visual Perception in Primate Cortex: Dedicated, Modifiable, Crossmodal, and Multifunctional Properties in Concert
Roughly comparable to toads and other vertebrates, sensory information processing in primates proceeds in a parallel-distributed and interactive fashion. Tremendous complexity arises from cortical neuronal circuits with regard to feature analysis, plasticity, multisensory integration, and sensory substitution (Kaas 1991). Combining and binding of features is a common task (Singer 1995). There is no one single place for perception at the “top” of a sensory system. All processing levels contribute to the resulting picture (Damasio 1990).
In primate vision, Ungerleider and Mishkin (1982) showed that two neural processing streams are involved to answer the questions “what kind of object?” and “where is the object?” (see also Hubel & Livingstone 1987).
Ventral processing stream answering “what”
This processing stream originates in the small-celled system of the retina, passes the related structures of the diencephalic lateral geniculate nucleus, LGN, and reaches—via corresponding cortical areas V1-4—the integration and association fields of the inferior temporal cortex, ITC (Figure 2.11).
Figure 2.11 Visual processing streams in the primate cortex. Simplified diagram; bidirectional pathways not shown. LGN, lateral geniculate nucleus; V1-V5, visual cortical areas; ITC, inferior temporal cortex; PPC, posterior parietal cortex. (Compiled after data by Ungerleider & Mishkin 1982; Hubel & Livingstone 1987).
In area V1 the different orientations of contrast borders in terms of lines (|/\—) are determined by certain neurons arranged in columns as prerequisite for the analysis of combinations of lines and angles between them (L V ∧ T) explored in areas V2 and V3 (Hubel & Wiesel 1977). Shape and color are analyzed separately in layers of area V2 and are combined in V4, thus allowing assignments like “yellow banana.” Associations depend on connections with the hippocampus.
The ITC is involved in the recognition of gestures and postures, e.g., suitable for social communication. Neuronal responses selective to faces were discovered by Perrett and Rolls (1983) (Chapter 5). Comparable face-selective neurons were recorded in the temporal cortex of Dalesbred sheep: some neuron types preferring a conspecific’s face, others responding selectively to a German shepherd dog’s face (Kendrick 1994). It is suggested that an assembly of differently face-tuned neurons code for the recognition of an individual face (cf. Cohen & Tong 2001).
Dorsal processing stream answering “where”
“Where is the object?” deals with “how should it be responded to?” This requires spatial vision in connection with analyses of object motion and depth: starting in the large-celled retina, continuing in related structures of LGN and processing—via corresponding areas V1-3, V5—in the posterior parietal cortex, PPC (Figure 2.11).
The PPC contains neurons responsible for target-oriented reaching or grasping involving arm, hand, and fingers. Such neurons fulfill integrative tasks. Motivation plays an essential role. If a satiated monkey was offered a banana, its visual fixation neurons failed to respond or discharged sluggishly and the animal ignored the banana (Mountcastle et al. 1975), in reminiscence of a comparable situation observed in toads.
The “what” and “where/how” processing streams are not completely segregated. A patient with damage to the “what” stream was able to reach for an object; however, if the object’s shape required an appropriate grasping pattern, the patient failed to grasp it.
Selective attention: what an individual does not like to see, it may not see
Animals, including humans, may guide their perception toward interesting parts of a scene and suppress uninteresting ones. In a behavioral experiment monkeys were trained to draw their attention either to a red or a green stripe (Barinaga 1997). In the neurophysiological experiment both stripes were presented in the excitatory visual receptive field of a red-sensitive neuron of area V4. The neuron responded to the red stimulus. If both stimuli were presented and the monkey was prompted to focus on the red one, the neuron fired as expected. However, requested to focus on the green stripe, the red-sensitive neuron was silent albeit the red stripe, too, was present in its excitatory receptive field.
Studies applying functional neuroimaging technologies in humans offer a look at the activity pattern in cortical visual areas. The regional neural activity depends on which property of an object the test-person shows interest. If a person is asked to focus on an object’s motion, an area corresponding to V5 is mainly activated. Paying attention to the color, an area corresponding to V4 is principally responsive. When shape is the focus, greatest activation is elsewhere along the “what” processing stream. In another task, photos of faces in frames were shown. Asking whether the faces looked different, ITC was strongly activated. Requesting whether a face was positioned symmetrically in the frame, activation shifted to the PPC.
Does imagination reactivate processes involved in visual perception?
Studies from neuroimaging in humans suggest that during visual imagination of an object, a “protocol” of the neurons operating during visual perception of that object is reactivated. Ishai and Sagi (1995) discuss common mechanisms of visual perception and visual imagination. This explains, for example, why humans with lesions in an area corresponding to V4 are unable both to recognize and imagine colors.
A different phenomenon of imagination is sensing a stimulus that actually affects another person. You see a child touching a hot stove and you will feel pain, caused by activity in appropriate structures of your brain. This kind of compassion is mediated by so-called mirror neurons (for details see Chapter 5).
Sensory maps shrink or expand depending on supply and demand
Sensory space is represented in the brain by topographic maps. From lesion and regeneration studies in amphibians it is known that the visual map of the retina in the optic tectum can be either compressed or expanded depending on the available tectal room and on the disposable retinal input. For example, if in a frog half of the retina of one eye is destroyed, a blown-up map of the remaining retina will regenerate in the entire contralateral tectum (Udin 1977).
Although such regeneration capability is not conferable to mammals, sensory cortical maps, too, may expand or shrink depending on the available cortical space and the need for perceptual skills. The phenomenon is called functional remodeling. Studies applying neuroimaging showed in the somatosensory cortex of string instrument players an expansion of the representation of the active digits to the detriment of the less active thumb (Elbert et al. 1995).
Universal potential of neural networks allows sensory substitution
The fact that underemployed cortical regions take over functions of overemployed regions is documented by neuroimaging in people blind from early age. Their visual cortex is activated by tactually reading Braille or embossed Roman letters or by other tactile discrimination tasks. Evidence of this sensory substitution was provided by transient disruption of the visual cortex by means of transcranial magnetic stimulation TMS. This induced errors